Experimental Evaluation of Software and Systems in Computer Science

We received 6 nominations as a result of this mail.

Date: March 12, 2013
To: (OOPSLA'13 PC members, OOPSLA'13 PC chair)
Subject: Nominate a student to the OOPSLA Artifact Evaluation Committee

Dear OOPSLA PC Member,

We are creating an Artifact Evaluation Committee (AEC) for OOPSLA 2013. The purpose of this board is to slowly move our discipline towards repeatability and reproducibility of scientific results [1]. The idea was developed by Shriram Krishnamurthi and Carlo Ghezzi for ESEC/FSE'11 [2]. It has been designed to be as lightweight and unthreatening as possible, while still making an essential step in the right direction. Basically, we are going to let authors of *accepted* papers submit artifacts (code, data, proofs,...) to a committee whose role is to validate that artifacts are consistent with the paper, as complete as possible, well documented, and easy to reuse, facilitating further research. We will report the result of the validation effort at the conference. Very importantly: this is voluntary, it does not affect acceptance of the paper, and does not require sharing the artifact broadly (details of the rules are here [3]).

[1] http://www.cs.purdue.edu/homes/jv/pubs/r3.pdf
[2] http://www.cs.brown.edu/~sk/Memos/Conference-Artifact-Evaluation/
[3] http://splashcon.org/2013/cfp/due-june-01-2013/665-oopsla-artifacts

Following the example of ECOOP’13, we intend to populate the AEC with senior PhD students. This is for two reasons: to have people who are familiar with contemporary software tools and methods, and to get the next generation of researchers thinking about the artifact evaluation process.

We are contacting you, as research leaders, to each nominate one student. We would like students with established records in both publication and artifact creation. We expect you to mentor your students to be responsible citizens of the Committee. Essentially, please think of it as similar to nominating a student for a program committee.

The workload depends on how many artifacts are submitted, and of what nature. We will work hard to keep the work manageable. We will consider enlarging the board if it appears necessary.

Committee work will take place in the window from June 1 to July 29, 2013, with the work trailing off after July 20, so students should be available during this period.

If you wish to nominate someone:

* Please nominate only ONE student. Please make sure it's a relatively senior student who
has experience both with publication and with artifacts (code, data, proofs,...). Reply
to this email with the name of the student, one sentence stating why the student would
be a good candidate for the Committee, and a pointer to their Web page, before March 22,
2013.

* Please do ***NOT*** inform the student. We are contacting more research groups than we
have slots; our goal is to gather all nominations, then find the group of students who
give us the best coverage. Therefore, we cannot guarantee that your student will be put
on the Committee. Please do not be offended if this happens -- hopefully you can
understand our constraints!

We look forward to your nomination.

Thanks,
Matthias Hauswirth and Steve Blackburn
(OOPSLA'13 AEC co-chairs)