<?xml version="1.0" encoding="UTF-8"?><xml><records><record><source-app name="Biblio" version="6.x">Drupal-Biblio</source-app><ref-type>27</ref-type><contributors><authors><author><style face="normal" font="default" size="100%">Christopher Small</style></author><author><style face="normal" font="default" size="100%">Narendra Ghosh</style></author><author><style face="normal" font="default" size="100%">Hany Saleeb</style></author><author><style face="normal" font="default" size="100%">Margo Seltzer</style></author><author><style face="normal" font="default" size="100%">Keith Smith</style></author></authors></contributors><titles><title><style face="normal" font="default" size="100%">Does Systems Research Measure Up?</style></title></titles><dates><year><style  face="normal" font="default" size="100%">1997</style></year></dates><urls><web-urls><url><style face="normal" font="default" size="100%">ftp://ftp.deas.harvard.edu/techreports/tr-16-97.ps.gz</style></url></web-urls></urls><publisher><style face="normal" font="default" size="100%">Harvard University</style></publisher><abstract><style face="normal" font="default" size="100%">We surveyed more than two hundred systems research papers published in the last six years, and found that, in experiment after experiment, systems researchers measure the same things, but in the majority of cases the reported results are not reproducible, comparable, or statistically rigorous. In this paper we present data describing the state of systems experimentation and suggest guidelines for structuring commonly run experiments, so that results from work by different researchers can be compared more easily. We conclude with recommendations on how to improve the rigor of published computer systems research.</style></abstract></record></records></xml>