A microbenchmark case study and lessons learned

TitleA microbenchmark case study and lessons learned
Publication TypeConference Paper
Year of Publication2011
AuthorsGil, J Y, Lenz K, Shimron Y
Conference NameProceedings of the compilation of the co-located workshops on DSM'11, TMC'11, AGERE!'11, AOOPES'11, NEAT'11, &\#38; VMIL'11
PublisherACM
Conference LocationNew York, NY, USA
ISBN Number978-1-4503-1183-0
Keywordsbenchmark, measurements, steady-state
Abstract

The extra abstraction layer posed by the virtual machine, the JIT compilation cycles and the asynchronous garbage collection are the main reasons that make the benchmarking of Java code a delicate task. The primary weapon in battling these is replication: "billions and billions of runs", is phrase sometimes used by practitioners. This paper describes a case study, which consumed hundreds of hours of CPU time, and tries to characterize the inconsistencies in the results we encountered.

URLhttp://doi.acm.org/10.1145/2095050.2095100
DOI10.1145/2095050.2095100