Experimental Evaluation of Software and Systems in Computer Science

The ability to conduct consistent, controlled and repeatable large-scale experiments in all areas of computer science related to parallel, large-scale or distributed computing and networking is critical to the future and development of Computer Science. Yet conducting such experiments is still too often a challenge for researchers, students and practitioners due to unavailability of dedicated resources, inability to create controlled experimental conditions, and variability in software. Availability, repeatability, and open sharing of electronic products are all still a challenge. This workshop will bring together scientists involved in building and operating two infrastructures dedicated to supporting Computer Science experiments, Grid 5000 in France and Future Grid in the United States, to discuss challenges and solutions in this space. Our objectives are to share experiences and knowledge related to supporting large-scale experiments conducted on experimental infrastructures, solicit requirements, and discuss methodologies and opportunities created by emerging technologies.

Workshop at SC'11, organized by Kate Keahey (Argonne National Laboratory) and Frederic Desprez (INRIA).