ERIC Identifier: ED474304
Publication Date: 2003-04-00
Author: Beghetto, Ron
Source: ERIC Clearinghouse on Educational Management Eugene OR.
Scientifically Based Research. ERIC Digest.
Increasingly, there has been an emphasis on evidence-based practice in education. The most recent incarnation of this focus is "scientifically based research" (SBR), a phrase often associated with the No Child Left Behind (NCLB) Act of 2001.
The NCLB Act, the most sweeping reform of the Elementary and Secondary Education Act (ESEA) since it was enacted in 1965, redefines the federal role in K-12 education. This legislation stipulates that federally funded programs and practices must be grounded in "scientifically based" research. School leaders who depend on federal funding are now required to be aware of the nature of the research that guides their programs and practices. In particular, federal funding decisions will be determined in part by whether programs and practices have a basis in "scientific research."
The U.S. Department of Education, among others, is developing guidelines to assist school leaders in assessing the scientific basis of programs and practices. Ultimately, the difficult decisions of selecting programs and practices will rely on the professional discretion and expertise of school leaders. This Digest describes the nature and implications of SBR.
WHAT IS THE HISTORY BEHIND SBR IN EDUCATION?
The phrase scientifically based is gaining prominence in discussions about the nature of educational research. However, the question of whether education has a scientific basis is controversial and has deep and contentious historical roots. Debate can be traced back to at least the early 1890s, when Josiah Royce examined the issue in an essay titled Is There a Science of Education? (Lagemann 2000). Although debate persists, efforts to transform education into a scientifically based field have continued.
The federal government's involvement in these efforts can be traced from the Cooperative Research Act of 1954 to the creation of the National Institute of Education in the early 1970s, which was later subsumed by the Office of Educational Research and Improvement (OERI). In the latest reorganization of the U.S. Department of Education, OERI was replaced by the Institute of Educational Sciences (IES), which established the What Works Clearinghouse. In time, after more research studies have been completed, it is hoped the What Works Clearinghouse will provide education leaders with scientific evidence about the efficacy of specific programs and practices and their effect on achievement, dropout rates, and other student outcomes.
The implications of SBR for education are the subject of heated debate among education researchers and academicians. Proponents of SBR see it as a necessary step in moving education to an evidence-based field. On the other hand, some scholars are concerned that SBR is an attempt to determine "what works" without considering all the implications, financial and otherwise.
WHAT IS SBR?
The NCLB Act (2002) defines scientifically based research as "research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs."
While there is not universal agreement regarding all the nuances associated with SBR, defining characteristics have appeared across numerous sources (for example, the Coalition for Evidence-Based Policy 2002, Comprehensive School Reform Program Office 2002, No Child Left Behind Act 2002, National Research Council 2002, Raudenbush 2002). Incorporating defining characteristics that appear in these sources, scientifically based research can be defined as follows:
Persuasive research that empirically examines important questions using appropriate methods that ensure reproducible and applicable findings.
Persuasive. This attribute refers to research that is moving from "tentative knowledge claims generated at local research sites to become stabilized and transformed into widely accepted facts" (Smith and others 2002). Appropriate research design, methods, and techniques; logic and reasoning; and replicable results can all help to establish persuasiveness.
A critical element in persuasiveness is the peer-review process, in which researchers who have been trained in research methodology review and critique each other's work to help ensure that the methods used match the research questions and conclusions. Research findings published in a peer-reviewed journal can be assumed to have undergone careful scrutiny, been considered in light of alternative explanations, and deemed sufficiently "persuasive" by a panel of individuals with expertise in research methods.
Empirical. Research that is empirical is based on measurement or observation, that is, experienced "through the senses" (NRC 2002). For example, research that measures or observes the impact of school vouchers on student achievement would be considered empirical. However, there are certain questions that cannot be addressed by empirical investigations (NRC ), such as "Should school vouchers programs be enacted in my state?" Questions involving "should" are typically addressed through means other than observation and measurement.
Important Questions. This refers to questions addressed by research that build upon, add to, fill a void in, or otherwise clarify what is known and practiced. The NRC explains that the importance of a question is often determined by its relationship to prior research, theory, and relevance to policy and practice.
Appropriate Methods. This refers to the use of designs, methods, and techniques that fit the nature of the question the study is attempting to answer. However no research design, method, or analytic technique on its own makes a study or program of research scientific (NRC). If the question pertains to "Does it work?," then randomized experiments or quasi-experiments are most appropriate (Raudenbush 2002, Coalition for Evidence-Based Policy).
Simply stated, randomized experiments involve randomly assigning individuals, schools, or districts to a group that receives a particular intervention (such as class-size reduction) and to a group that does not. In contrast, if the question pertains to "What was the 'it'?," then qualitative methods (such as the case study) are most appropriate (Erickson and Gutierrez 2002). Among other things, qualitative methods provide "up-close descriptions" of what is, or is not, working; how interventions are working; and what might be facilitating or impeding the effectiveness of a particular intervention (Raudenbush).
Replicable and Applicable Findings. In general, this attribute refers to consistent, meaningful findings. The research presents sufficient detail to allow for "replication or, at a minimum, ... the opportunity to build systematically on their findings" (NCLB 2002).
Such findings are understandable, accessible, and applicable to a wide audience (Comprehensive School Reform Program Office). For example, a program of research should be designed and conducted to ensure that school leaders across the nation have a solid sense of whether they can expect to see similar results from implementing a school-reform program that has demonstrated increased student learning in another state.
WHAT ARE ANTICIPATED IMPLICATIONS OF SBR?
Important issues in education tend to have proponents, opponents, and everything in between. This is certainly the case when it comes to the concept of scientifically based research. Although there likely are as many perspectives on the issue as there are people talking about it, several assertions regarding the implications of SBR have emerged.
Advocates assert that SBR will replace the "folk wisdom of education" with a scientific knowledge base (Whitehurst 2002). Scientific research is seen as a means for improving education and developing a knowledge-base for "what works." Whitehurst claims that "there is every reason to believe that, if we invest in the education sciences and develop mechanisms to encourage evidence-based practices, we will see progress and transformation... of the same order and magnitude as we have seen in medicine and agriculture."
Alternatively, some scholars fear focusing on "what works" may blind educators to important aspects and potentially harmful unanticipated outcomes of education processes (Erickson and Gutierrez 2002). To illustrate this point, Erickson and Gutierrez use an example from medicine that "worked" but had disastrous side effects-the use of thalidomide to treat morning sickness. St. Pierre (2002) expresses her concern that SBR represents too narrow a focus. She argues for keeping research in education open to diverse traditions of scholarship and inquiry so as not to obstruct "the proliferation of knowledge."
WHAT ARE THE IMPLICATIONS FOR SCHOOL LEADERS?
For school leaders, the implications of SBR are still emerging. At the very least, those relying on federal funding are required to select programs and practices supported by scientifically based research. One of the Institute of Education Sciences' first projects is to develop a database of "what works." The What Works Clearinghouse (available online at www.w-w-c.org/about.html) is designed to serve as a resource for educational decision-makers involved in selecting scientifically based programs and practices.
A comprehensive, or even adequate, database of "what works" is far from complete. In the meantime, general guidelines and recommendations are being generated by the U.S. Department of Education and other organizations to assist school leaders in selecting and implementing new programs and practices.
For example, the Comprehensive School Reform Program Office published a report in August 2002 entitled "Scientifically Based Research and the Comprehensive School Reform (CSR) Program." This report is intended to help school leaders develop an understanding of SBR and assess the scientific basis of programs and practices.
Another example is the ERIC Clearinghouse on Educational Management, which is developing guidelines for both authors and users of its products that will help clarify the nature of evidence cited. Such efforts are intended to assist readers in making informed decisions and drawing reasonable conclusions about the findings.
School leaders play a significant role in determining the implications of SBR for education. Keeping abreast of issues and becoming part of the conversation of how SBR is implemented in policy and practice will help shape the role of SBR in education. What remains unchanged is that school leaders still must rely on their professional judgment and the best information available in making decisions about the selection, implementation, and management of programs and practices in their schools.
Coalition for Evidence-Based Policy. Bringing Evidence-Driven Progress to Education: A Recommended Strategy for the U.S. Department of Education, 2002.
Comprehensive School Reform Program Office, Office of Elementary and Secondary Education, U.S. Department of Education. Scientifically Based Research and the Comprehensive School Reform (CSR) Program. Washington, D.C.: Author, 2002. Available online at http://www.ed.gov/offices/OESE/compreform/appendc.pdf%20
Erickson, F., and K. Gutierrez. "Culture, Rigor, and Science in Educational Research." Educational Researcher 31, 8 (2002): 21-24.
Lagemann, E. An Elusive Science: The Troubling History of Educational Research. Chicago: University of Chicago Press, 2000. 302 pages. ED 457 075.
National Research Council. Scientific Research in Education, edited by R. J. Shavelson and L. Towne. Committee on Scientific Principles for Educational Research. Washington, D.C.: National Academy Press, 2002.
No Child Left Behind Act of 2001, Pub. I, No. 107-110, 115 Stat. 1425 (2002).
Raudenbush, S. "Scientifically-Based Research." U.S. Department of Education Seminar on Scientifically-Based Research. Washington, D.C.: U.S. Department of Education, 2002. Available online at http://www.ed.gov/offices/OESE/esea/research/%20
Smith, L. D.; L. A. Best; A. Stubbs; A. B. Archibald; and R. Roberson-Nay. "Constructing Knowledge." American Psychologist 57, 10 (2002): 749-61.
St. Pierre, E. A. " 'Science' Rejects Postmodernism." Educational Researcher 31, 8 (2002): 25-27.
Whitehurst, G. J. "Statement of Grover J. Whitehurst, Assistant Secretary for Research and Improvement, before the Senate Committee on Health, Education, Labor and Pensions." Washington, D.C.: U.S. Department of Education, 2002.Available online at http://www.ed.gov/offices/IES/speeches%20
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.