ERIC Identifier: ED474304
Publication Date: 2003-04-00
Author: Beghetto, Ron
Source: ERIC Clearinghouse on
Educational Management Eugene OR.
Scientifically Based Research. ERIC Digest.
Increasingly, there has been an emphasis on evidence-based practice in
education. The most recent incarnation of this focus is "scientifically based
research" (SBR), a phrase often associated with the No Child Left Behind (NCLB)
Act of 2001.
The NCLB Act, the most sweeping reform of the Elementary and Secondary
Education Act (ESEA) since it was enacted in 1965, redefines the federal role in
K-12 education. This legislation stipulates that federally funded programs and
practices must be grounded in "scientifically based" research. School leaders
who depend on federal funding are now required to be aware of the nature of the
research that guides their programs and practices. In particular, federal
funding decisions will be determined in part by whether programs and practices
have a basis in "scientific research."
The U.S. Department of Education, among others, is developing guidelines to
assist school leaders in assessing the scientific basis of programs and
practices. Ultimately, the difficult decisions of selecting programs and
practices will rely on the professional discretion and expertise of school
leaders. This Digest describes the nature and implications of SBR.
WHAT IS THE HISTORY BEHIND SBR IN EDUCATION?
scientifically based is gaining prominence in discussions about the nature of
educational research. However, the question of whether education has a
scientific basis is controversial and has deep and contentious historical roots.
Debate can be traced back to at least the early 1890s, when Josiah Royce
examined the issue in an essay titled Is There a Science of Education? (Lagemann
2000). Although debate persists, efforts to transform education into a
scientifically based field have continued.
The federal government's involvement in these efforts can be traced from the
Cooperative Research Act of 1954 to the creation of the National Institute of
Education in the early 1970s, which was later subsumed by the Office of
Educational Research and Improvement (OERI). In the latest reorganization of the
U.S. Department of Education, OERI was replaced by the Institute of Educational
Sciences (IES), which established the What Works Clearinghouse. In time, after
more research studies have been completed, it is hoped the What Works
Clearinghouse will provide education leaders with scientific evidence about the
efficacy of specific programs and practices and their effect on achievement,
dropout rates, and other student outcomes.
The implications of SBR for education are the subject of heated debate among
education researchers and academicians. Proponents of SBR see it as a necessary
step in moving education to an evidence-based field. On the other hand, some
scholars are concerned that SBR is an attempt to determine "what works" without
considering all the implications, financial and otherwise.
WHAT IS SBR?
The NCLB Act (2002) defines scientifically
based research as "research that involves the application of rigorous,
systematic, and objective procedures to obtain reliable and valid knowledge
relevant to education activities and programs."
While there is not universal agreement regarding all the nuances associated
with SBR, defining characteristics have appeared across numerous sources (for
example, the Coalition for Evidence-Based Policy 2002, Comprehensive School
Reform Program Office 2002, No Child Left Behind Act 2002, National Research
Council 2002, Raudenbush 2002). Incorporating defining characteristics that
appear in these sources, scientifically based research can be defined as
Persuasive research that empirically examines important questions using
appropriate methods that ensure reproducible and applicable findings.
Persuasive. This attribute refers to research that is moving from "tentative
knowledge claims generated at local research sites to become stabilized and
transformed into widely accepted facts" (Smith and others 2002). Appropriate
research design, methods, and techniques; logic and reasoning; and replicable
results can all help to establish persuasiveness.
A critical element in persuasiveness is the peer-review process, in which
researchers who have been trained in research methodology review and critique
each other's work to help ensure that the methods used match the research
questions and conclusions. Research findings published in a peer-reviewed
journal can be assumed to have undergone careful scrutiny, been considered in
light of alternative explanations, and deemed sufficiently "persuasive" by a
panel of individuals with expertise in research methods.
Empirical. Research that is empirical is based on measurement or observation,
that is, experienced "through the senses" (NRC 2002). For example, research that
measures or observes the impact of school vouchers on student achievement would
be considered empirical. However, there are certain questions that cannot be
addressed by empirical investigations (NRC ), such as "Should school vouchers
programs be enacted in my state?" Questions involving "should" are typically
addressed through means other than observation and measurement.
Important Questions. This refers to questions addressed by research that
build upon, add to, fill a void in, or otherwise clarify what is known and
practiced. The NRC explains that the importance of a question is often
determined by its relationship to prior research, theory, and relevance to
policy and practice.
Appropriate Methods. This refers to the use of designs, methods, and
techniques that fit the nature of the question the study is attempting to
answer. However no research design, method, or analytic technique on its own
makes a study or program of research scientific (NRC). If the question pertains
to "Does it work?," then randomized experiments or quasi-experiments are most
appropriate (Raudenbush 2002, Coalition for Evidence-Based Policy).
Simply stated, randomized experiments involve randomly assigning individuals,
schools, or districts to a group that receives a particular intervention (such
as class-size reduction) and to a group that does not. In contrast, if the
question pertains to "What was the 'it'?," then qualitative methods (such as the
case study) are most appropriate (Erickson and Gutierrez 2002). Among other
things, qualitative methods provide "up-close descriptions" of what is, or is
not, working; how interventions are working; and what might be facilitating or
impeding the effectiveness of a particular intervention (Raudenbush).
Replicable and Applicable Findings. In general, this attribute refers to
consistent, meaningful findings. The research presents sufficient detail to
allow for "replication or, at a minimum, ... the opportunity to build
systematically on their findings" (NCLB 2002).
Such findings are understandable, accessible, and applicable to a wide
audience (Comprehensive School Reform Program Office). For example, a program of
research should be designed and conducted to ensure that school leaders across
the nation have a solid sense of whether they can expect to see similar results
from implementing a school-reform program that has demonstrated increased
student learning in another state.
WHAT ARE ANTICIPATED IMPLICATIONS OF SBR?
in education tend to have proponents, opponents, and everything in between. This
is certainly the case when it comes to the concept of scientifically based
research. Although there likely are as many perspectives on the issue as there
are people talking about it, several assertions regarding the implications of
SBR have emerged.
Advocates assert that SBR will replace the "folk wisdom of education" with a
scientific knowledge base (Whitehurst 2002). Scientific research is seen as a
means for improving education and developing a knowledge-base for "what works."
Whitehurst claims that "there is every reason to believe that, if we invest in
the education sciences and develop mechanisms to encourage evidence-based
practices, we will see progress and transformation... of the same order and
magnitude as we have seen in medicine and agriculture."
Alternatively, some scholars fear focusing on "what works" may blind
educators to important aspects and potentially harmful unanticipated outcomes of
education processes (Erickson and Gutierrez 2002). To illustrate this point,
Erickson and Gutierrez use an example from medicine that "worked" but had
disastrous side effects-the use of thalidomide to treat morning sickness. St.
Pierre (2002) expresses her concern that SBR represents too narrow a focus. She
argues for keeping research in education open to diverse traditions of
scholarship and inquiry so as not to obstruct "the proliferation of knowledge."
WHAT ARE THE IMPLICATIONS FOR SCHOOL LEADERS?
leaders, the implications of SBR are still emerging. At the very least, those
relying on federal funding are required to select programs and practices
supported by scientifically based research. One of the Institute of Education
Sciences' first projects is to develop a database of "what works." The What
Works Clearinghouse (available online at www.w-w-c.org/about.html) is designed
to serve as a resource for educational decision-makers involved in selecting
scientifically based programs and practices.
A comprehensive, or even adequate, database of "what works" is far from
complete. In the meantime, general guidelines and recommendations are being
generated by the U.S. Department of Education and other organizations to assist
school leaders in selecting and implementing new programs and practices.
For example, the Comprehensive School Reform Program Office published a
report in August 2002 entitled "Scientifically Based Research and the
Comprehensive School Reform (CSR) Program." This report is intended to help
school leaders develop an understanding of SBR and assess the scientific basis
of programs and practices.
Another example is the ERIC Clearinghouse on Educational Management, which is
developing guidelines for both authors and users of its products that will help
clarify the nature of evidence cited. Such efforts are intended to assist
readers in making informed decisions and drawing reasonable conclusions about
School leaders play a significant role in determining the implications of SBR
for education. Keeping abreast of issues and becoming part of the conversation
of how SBR is implemented in policy and practice will help shape the role of SBR
in education. What remains unchanged is that school leaders still must rely on
their professional judgment and the best information available in making
decisions about the selection, implementation, and management of programs and
practices in their schools.
Coalition for Evidence-Based Policy. Bringing
Evidence-Driven Progress to Education: A Recommended Strategy for the U.S.
Department of Education, 2002.
Comprehensive School Reform Program Office, Office of Elementary and
Secondary Education, U.S. Department of Education. Scientifically Based Research
and the Comprehensive School Reform (CSR) Program. Washington, D.C.: Author,
2002. Available online at http://www.ed.gov/offices/OESE/compreform/appendc.pdf%20
Erickson, F., and K. Gutierrez. "Culture, Rigor, and Science in Educational
Research." Educational Researcher 31, 8 (2002): 21-24.
Lagemann, E. An Elusive Science: The Troubling History of Educational
Research. Chicago: University of Chicago Press, 2000. 302 pages. ED 457 075.
National Research Council. Scientific Research in Education, edited by R. J.
Shavelson and L. Towne. Committee on Scientific Principles for Educational
Research. Washington, D.C.: National Academy Press, 2002.
No Child Left Behind Act of 2001, Pub. I, No. 107-110, 115 Stat. 1425 (2002).
Raudenbush, S. "Scientifically-Based Research." U.S. Department of Education
Seminar on Scientifically-Based Research. Washington, D.C.: U.S. Department of
Education, 2002. Available online at
Smith, L. D.; L. A. Best; A. Stubbs; A. B. Archibald; and R. Roberson-Nay.
"Constructing Knowledge." American Psychologist 57, 10 (2002): 749-61.
St. Pierre, E. A. " 'Science' Rejects Postmodernism." Educational Researcher
31, 8 (2002): 25-27.
Whitehurst, G. J. "Statement of Grover J. Whitehurst, Assistant Secretary for
Research and Improvement, before the Senate Committee on Health, Education,
Labor and Pensions." Washington, D.C.: U.S. Department of Education,
2002.Available online at http://www.ed.gov/offices/IES/speeches%20