Experimental Evaluation of Software and Systems in Computer Science

Wieringa et al.'s "How to Write and Read a Scientific Evaluation Paper" provides a nice and crisp definition of evaluation papers:

Scientific evaluation papers evaluate existing problem situations or validate or refute proposed solutions by means of scientific research, ranging from formal, mathematical analysis to empirical research.

Observational studies belong to the first category: the evaluation of existing problem situations.

Prior work on observational studies

• Tichy 1998 Should Computer Scientists Experiment More?
  • "Observation and experimentation can lead to new, useful, and unexpected insights and open whole new areas of investigation."
  • "If scientists neglect experiment and observation, they'll have difficulties discovering new and interesting phenomena worthy of better theories"
• Feitelson 2006 Experimental Computer Science: The Need for a Cultural Change
• Wieringa et al. 2009 How to Write and Read a Scientific Evaluation Paper

Check out all papers in the bibliography classified under "observational studies".