National Center on Secondary Education and Transition
Increasing Rates of School Completion
Moving From Policy and Research to Practice
A Manual for Policymakers, Administrators, and Educators
Part II: How Were Sample Intervention Programs Selected?
The Need for Examples of Effective Interventions
The need for examples of effective interventions that can be used as
guides to assist educators, administrators, and policymakers in implementing
school-based dropout interventions is clear. While there have been many
reports describing promising dropout interventions, only a small portion
of these offer conclusions that are based on methodologically sound analysis.
It is from this list that examples of interventions were selected for
inclusion in this document. It is important to note that this
is not a comprehensive list of effective interventions.
The interventions described serve as examples that show some evidence
of effectiveness and meet criteria (described below) for inclusion in
this document. Given the nature of applied research and the number of
variables that cannot be controlled, exact replication of any program
is difficult. The reader is encouraged to carefully consider the population
for whom each intervention was successful and the contextual variables
that may have influenced effectiveness. Each of the examples must be considered
in relation to the needs, demographics, resources, and other circumstances
of local schools or districts. Rather than rate the merits of each intervention,
information is intended to facilitate critical review of the intervention
programs/strategies by those implementing such programs. References and
contact information are provided for those wanting more detailed information.
Search Process & Initial Criteria
The identification process for strategies and programs to include as
examples in this NCSET Essential Tool occurred in two stages.
First, an initial list of programs to review was generated from a search
that relied on information from four sources. Programs were included in
the initial pool if they
- Focused on dropout prevention;
- Appeared to include evidence of effectiveness using a research or
evaluation design; and
- Results were published (or the study was completed) between 1988
The sources used to conduct the initial search are described below.
- Integrative review of prevention and intervention studies addressing
dropout described in professional journals (Lehr et al., 2003).
Authors of this review conducted computerized searches of an in-house
database of more than 600 documents and online databases including Educational
Resources Information Center (ERIC), Psychological Abstracts, and Education
Abstracts. Search terms linked to dropout and school completion included
“dropout prevention” along with the descriptors “school
engagement,” “school completion,” “achievement,”
and “disabilities.” A list of more than 300 unduplicated
citations from 1980 to 2001 was compiled. The list of articles was further
refined using the following criteria: (a) was published in a professional
journal, (b) focused on a dropout prevention or intervention program,
and (c) included impact data on the described program. Studies with
an intervention focus and an emphasis on impacting truancy or dropout
were included. General summaries of the literature, nonexperimental
studies, and policy reports were omitted. Additional articles were collected
through a review of references cited in rejected articles and a computer
search (using the same key terms) to identify articles published during
the coding period. Forty-five studies were included in the final review.
- Literature search conducted by What Works Transition Research
Synthesis Project. This current project (funded by OSEP) is engaged
in a review and synthesis of research conducted in the past 20 years.
The project, based at the University of Colorado, works in collaboration
with the Evidence for Policy and Practice Information Coordination Centre
at the University of London and the Campbell Collaboration at the University
of Pennsylvania. What Works plans to produce several research-based
syntheses in the context of transition for youth with disabilities including
academic outcomes, dropout prevention (“risk and resiliency”),
and transition outcomes. After conducting a systematic search of the
literature using ERIC, Psych Info, and Medline databases for studies
on dropout prevention and students with disabilities, What Works
provided a list of 232 references and associated abstracts. A review
of the abstracts from this list yielded 10 articles that were selected
for further review.
- Web-based search for documents on dropout intervention and prevention
produced by the Office of Special Education Programs (OSEP), Employment
and Training Administration (ETA), and the Institute of Education Sciences
(IES, formerly the Office of Educational Research and Improvement, or
OERI). Project staff conducted a Web-based search for dropout intervention
and prevention programs funded by OSEP, ETA, and IES. The key terms
“dropout intervention,” “dropout prevention,”
and “dropout program” were employed to search each of these
federal government Web sites. This search collected additional relevant
documents describing empirical studies of dropout intervention. Titles
and/or abstracts of nearly 200 documents were reviewed to determine
their relevance for further review. Abstracts referring to intervention
programs and data on effectiveness were obtained for further examination.
- A search of ERIC documents for reviews of dropout interventions.
The authors of this document also conducted a search of ERIC documents
for comprehensive reviews of dropout intervention and prevention programs
completed between 1988 and 2003. A search was conducted using the key
terms “dropout program,” “review,” and “data,”
yielding 69 documents. The abstracts of these reviews were carefully
considered. Several promising reviews that appeared relevant were obtained.
These documents were then examined to determine if they included evidence
of effectiveness and, if so, were added to the list of initial references
for further consideration.
Raising the Bar
After generating a pool of interventions that met initial selection
criteria, additional criteria for inclusion in this document were applied.
Use of these additional criteria “raises the bar” by selecting
only those interventions clearly supported by empirical evidence (see
Table 3 for desirable components). Many publications
describe and recommend effective or model programs. However, upon closer
examination, many of these interventions were not supported by research
or evaluation data and/or did not measure enrollment status. Furthermore,
quality research must be conducted in an objective manner, and results
should be tested by other researchers.
Policies directed at implementing large scale programming with significant
associated costs ought to be based on research that is conceptually and
methodologically sound. Sometimes, programs are promoted despite a lack
of supporting data. This project’s review of the literature yielded,
for example, that a program was mandated despite the fact that data showed
an increase in truancy for high school students who were in the
program and no evidence of improved graduation rates (http://www.edpriorities.org/Pubs/Opinion/Letters95/Let95_Lernfare.html).
It is the responsibility of educators, administrators, and policymakers
to require that claims of success be supported by adequate research before
adopting those strategies or interventions. We used rigorous criteria
to select interventions for inclusion in this document because the implications
of the decisions being made to address school completion for students
at risk of school failure are important. The stringent criteria excluded
some promising (but not proven) programs. The selection process
may also have unintentionally excluded some programs that would have met
the criteria because the available documents did not include the necessary
Final Parameters for Selection
The second stage of the identification process further refined the list
of programs to be included as examples of intervention programs. Final
criteria for selection are listed below.
- Focus on dropout prevention/intervention (manipulation of independent
- Measurement of an outcome variable tied to enrollment status (e.g.,
graduation, current attendance in school, dropout rate).
- Evidence of effectiveness using a research design (preferably randomized
control or trial—but not exclusively) that yields statistically
significant results in favor of the intervention on one or more dependent
variables, including enrollment status.
- Results published in a professional journal or comprehensive government
report (to ensure critical review of the intervention and its accessibility
to interested users).
Abstracts: Coding & Definitions
The categories for describing interventions were selected based on a
review of coding manuals including the Procedural and Coding Manual
for Identification of Evidence-based Interventions (Kratochwill et
al., 2002), and guidelines for examining the conceptual and methodological
quality of intervention (Crane, 1998). In addition, compilations of other
evidence-based programs (e.g., Safe and Sound: An Educational Leader’s
Guide to Evidence-Based Social and Emotional Learning Programs, CASEL,
2002) were examined. Coding areas from these sources were considered and
used to derive categories suitable for this document. The categories selected
and subsequent coding is intended to be descriptive rather than evaluative.
This document includes an abstract of each sample intervention and a
chart summarizing information across interventions. The contact person
listed for each intervention reviewed the chart and relevant abstract
for accuracy. Table 3 lists the information that is included in the abstract
and/or the summary chart.
Table 3: Abstracts: Coding & Definitions
Intervention Program or Strategy. The
title of the program is listed. In cases where the intervention
did not have a formal title, the type of program is listed using
Background. History and purpose of
the intervention is briefly described.
Intervention Description. The intervention
is briefly described.
Outcome Variables. Outcome variables
that were significantly impacted are listed.
Population and Setting. The population
intended for the intervention is briefly described with regard to
grade level and any targeted subgroups (e.g., students with disabilities).
In addition, the setting in which the intervention has been implemented
is described (e.g., urban).
Evidence of Effectiveness. Research
showing evidence of effectiveness is described. The availability
of the following information is noted.
- Research design. Three types of research design are
identified. These include: (a) studies incorporating random selection
and assignment of participants to two or more groups; (b) studies
incorporating random assignment of participants (no random selection)
to treatment or control groups; and (c) pre-post studies including
measures and comparisons of outcome data before and after an intervention
with a single group.
- Statistical significance. A mathematical determination
that indicates the presence of an effect that is unlikely to have
resulted from chance alone. When key outcomes are statistically
significant, the intervention/program is assumed to have had an
- Effect size. This indicator of effectiveness measures
the amount of impact attributed to the program or intervention,
and is not influenced by sample size.
- Durability of effects. Evidence indicating program
effects persisted after the termination of the intervention.
- Treatment integrity. Information indicating whether
the treatment or intervention was implemented as specified.
- Sample size. The sample size used in the research
studies is specified.
- Use of an external evaluator. The use of an evaluator
external to program development and implementation is noted.
- Multiple sites or studies. Implementation of the program
in more than one setting or more than one evaluative study is
Implementation Considerations. Practical
considerations for implementation are described in terms of support
personnel required, training, additional resources, and the estimated
duration of the intervention. In addition, information about the
availability of a manual or training is provided. When available,
information about cost is also included.
Contact Information. Names and contact
information of individuals associated with the interventions (and
in some cases Web addresses) are listed.
References. Additional references regarding
the interventions are included.
1.19MB, 84 pages
Citation: Lehr, C. A., Johnson, D.
R., Bremer, C. D., Cosio, A., & Thompson, M. (2004). Essential
tools: Increasing rates of school completion: Moving from policy and research
to practice. Minneapolis, MN: University of Minnesota, Institute
on Community Integration, National Center on Secondary Education and Transition.
Permission is granted to duplicate this publication in its entirety or
portions thereof. Upon request, this publication will be made available
in alternative formats. For additional copies of this publication, or
to request an alternate format, please contact: Institute
on Community Integration Publications Office, 109 Pattee Hall, 150
Pillsbury Drive SE, Minneapolis, MN 55455, (612) 624-4512, firstname.lastname@example.org.
This document was published by the National
Center on Secondary Education and Transition (NCSET). NCSET is supported
through a cooperative agreement #H326J000005 with the U.S. Department
of Education, Office of Special Education Programs. Opinions expressed
herein do not necessarily reflect the policy or position of the U.S. Department
of Education Programs, and no official endorsement should be inferred.
The University of Minnesota, the U.S. Department of Education, and the
National Center on Secondary Education and Transition are equal opportunity
employers and educators.