%0 Conference Paper %B Proceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing %D 2005 %T Pitfalls in parallel job scheduling evaluation %A Frachtenberg, Eitan %A Dror G. Feitelson %C Berlin, Heidelberg %I Springer-Verlag %K dynamic workload %K experimental methodology %K parallel job scheduling %K performance evaluation %K simulation %K static workload %P 257–282 %R 10.1007/11605300_13 %S JSSPP'05 %U http://www.cs.huji.ac.il/%7Efeit/parsched/jsspp05/p-05-13.pdf %X There are many choices to make when evaluating the performance of a complex system. In the context of parallel job scheduling, one must decide what workload to use and what measurements to take. These decisions sometimes have subtle implications that are easy to overlook. In this paper we document numerous pitfalls one may fall into, with the hope of providing at least some help in avoiding them. Along the way, we also identify topics that could benefit from additional research. %@ 3-540-31024-X, 978-3-540-31024-2