An international perspective of monitoring educational research quality: Commonalities and differences

This chapter considers the notion of educational research quality and evaluation from an international perspective. We consider how and why these approaches differ from the US-based Gold Standard design (i.e., research based on randomized controlled trials [RCTs] mimicking third-stage drug trials; see Shelley, Yore, & Hand, Chap. 1). The Gold Standard is based on the assumption that RCT design alone, regardless of other factors, provides the desired quality. We suggest here that the notion of quality in research and the mechanisms used to evaluate research quality are highly dependent on the overarching aim of education. To illustrate, the governments of many countries see education, especially science education, as a key component in economic progress and as a means of delivering on social services (Coll & Taylor, 2008). Hence, there are a number of reasons why we need to evaluate research quality in science education. We need to provide evidence that our science education regimes (and vocational education and training) do in fact produce outputs in terms of qualified people needed to drive economic success. There is then the notion of accountability; the expenditure of taxpayer monies by government-especially in the area of education-is subject to much public scrutiny and often to criticism. There also is accountability to specific legalization in which governments require the education sector to deliver on education aims, such as scientific literacy. In each of these examples, we need to be as sure as we can that the research findings are trustworthy-to use Guba and Lincoln's (1989) term-or believable. New curricula and teaching and learning approaches often prove highly controversial (e.g., Bell, Jones, & Carr, 1995; Coll & Taylor; Matthews, 1994), and education stakeholders-including government-naturally want to see convincing evidence that costly educational interventions actually work.