Evaluating Scientific Research Projects: The Units of Science in the Making

Original research is of course what scientists are expected to do. Therefore the research project is in many ways the unit of science in the making: it is the center of the professional life of the individual scientist and his coworkers. It is also the means towards the culmination of their specific activities: the original publication they hope to contribute to the scientific literature. The scientific project should therefore be of central interest to all the students of science, particularly the philosophers and sociologists of science. We shall focus on the preliminary evaluation of research projects—the specific task of referees—and will emphasize the problem of their scientificity—the chief concern of scientific gatekeepers. In the past such an examination aimed only at protecting the taxpayer from swindlers and incompetent amateurs, such as the inventors of continuous motion artifacts. In recent times a similar issue has resurfaced with regard to some of the most prestigious and most handsomely funded projects, namely work on string theory and many-worlds cosmology. Indeed, some of their faithful have claimed that these theories are so elegant, and so full of high-grade mathematics, that they should be exempted from empirical tests. This claim provoked the spirited rebuttal of the well-known cosmologists Ellis and Silk (Nature 516:321–323, 2014), which the present paper is intended to reinforce. Indeed, we shall try to show why empirical testability is necessary though insufficient for a piece of work to qualify as scientific. Finally, the present paper may also be regarded as an indirect contribution to the current debate over the reliability of quantitative indicators of scientific worth, such as the h-index of scientific productivity (e.g., Wilsdon in Nature 523:129, 2015). But we shall touch only tangentially on the sociological, political, and economics of research teams: our focus will be the acquisition of new scientific knowledge.