Home › Artifact Evaluation › The AE Artifact › Announcements

Artifact Evaluation Artifact - AEC Welcome Letter


Date: May 8, 2013
To: (all 24 AEC members)
Subject: OOPSLA AEC: Timeline, Contact Info

Dear OOPSLA'13 AEC Member,

Thanks again for agreeing to serve on OOPSLA's first ever Artifact Evaluation Committee.

Could you please reply to this email (only to Steve and Matthias) by the end of this week with your answers to the following two questions:

1. Could you provide your contact information (full name, affiliation, and homepage URL) so we can publish it on the OOPSLA AEC call site (http://splashcon.org/2013/cfp/due-june-01-2013/665-oopsla-artifacts)?

2. Could you check the timeline for the artifact evaluation process below and confirm that you will be able to meet these deadlines? (note: we expect three artifacts per committee member, and we plan to enlarge the committee if needed).

Thanks a lot for your prompt response!

Best regards,
Matthias & Steve

Artifacts submitted: Saturday, June 1
AEC members bid for artifacts, identifying their preferences and expertise

Bids due: Monday, June 3 (23:59, Samoa time)
AEC members ‘kick the tires’ of each of their assigned submissions by working through the “Getting Started” guide provided by the authors. Reviewers evaluate this phase with a boolean ‘satisfactory’ / ‘unsatisfactory’ and provide short feedback and questions.

Kicking-the-tires due: Monday, June 17
AEC members review each of their assigned submissions, reading the associated paper.

Reviews due: Monday, July 15
AEC members engage in online discussion with other members who reviewed the same artifacts, calibrating their scores and identifying and resolving points of disagreement where possible. The PC chairs will identify ‘discussion leaders’ for each submission who will prepare a one-paragraph statement on the submission for consumption by the rest of the AEC.

Deadline of one-paragraph statement: Monday, July 22
AEC members will hold a virtual PC meeting where final decisions will be made on each submission. AEC members will read summary paragraphs for each submission.

Notification: Monday, July 29