Article Review Structure Quantitative
Introduction: How did the author(s) introduce the content area? How was the topic situated within a larger view of early childhood education? What is the purpose? Was the need for this research discussed? (i.e. Did the author(s) build a case for a need for this research?) If so, what was the reasoning? Were related studies discussed in terms of the present research? What are the research questions?
Methods: What is the setting? Who are the participants? What are the data sources? How was the data collected? How was the data analyzed?
Findings: What were the findings from this study? How were the findings supported? (examples from transcripts, statistical evidence, etc…)
Discussion: How were the findings interpreted? How were the findings discussed in terms of the research questions? How were the findings discussed in terms of the larger problem within early childhood education? Did the explanation of the findings make sense to you?
Limitations: What were the limitations stated by the author(s)? Did you find any other limitations existed?
Future directions: What were the future directions stated by the author? Would you use this research for any other future research not mentioned?
Reflection: What did you think of this article? Do you believe there was a need for this research within the field of early childhood education? Were there any foundational components missing? If you could change portions of this research, how would you change it?
LSHSS
Language and Literacy Curriculum
Supplement for Preschoolers Who
Are Academically At Risk:
A Feasibility Study
Laura M. Justice
The Ohio State University, Columbus
Anita S. McGinty
Sonia Q. Cabell
Carolyn R. Kilday
University of Virginia, Charlottesville
Kathy Knighton
Ginger Huffman
West Virginia Department of Education, Charleston
A n unprecedented number of 3- to 5-year-old children currently attend publicly funded preschool and pre-kindergarten (pre-K) programs (National Institute for
Early Education Research, 2004). Many of these children exhibit
elevated risks for later reading problems due to poverty and its well-
established impact on developmental precursors to reading achieve-
ment. Consequently, increased attention is being directed toward
ensuring that children who attend publicly funded preschool programs
ABSTRACT: Purpose: The potential benefit that a low-cost scripted
language and literacy curriculum supplement titled “Read It Again!”
(RIA; L. M. Justice, A. S. McGinty, A. R. Beckman, & C. R. Kilday,
2006) may have on preschool-age children’s skills was explored. RIA
was developed to meet the needs of preschool educators who
may not have access to current commercially available high-cost
language and literacy curricula, which often require ongoing
intensive professional development. RIA involves implementing
60 large-group lessons over a 30-week period that feature repeated
use of 15 commercial storybooks.
Method: Using a quasi-experimental pre–post research design,
11 preschool teachers implemented RIA in their classrooms for an
academic year, and 9 teachers working in comparable preschool
have adequate opportunities to develop such critical reading pre-
cursors as vocabulary knowledge, narrative ability, phonological
awareness, and print knowledge (see Barnett, 2001; Dickinson &
Brady, 2006; Justice & Kaderavek, 2004). Early Reading First pro-
vides an apt example: This federally funded program was designed
to create preschool “centers of excellence” in language and emer-
gent literacy throughout the United States to serve as model instruc-
tional programs (Institute of Education Sciences, 2007). The potential
programs served as comparisons. Language and literacy measures
were collected in the fall and spring of the year.
Results: Children whose teachers implemented RIA had higher scores
in the spring on measures of language (i.e., grammar and vocabulary)
and measures of literacy (i.e., rhyme, alliteration, and print). Effect-
size estimates were consistent with medium- to large-size effects.
Conclusions: RIA may be a viable means of enhancing the language
and literacy instruction that is delivered within preschool classrooms
and, therefore, a means of enhancing children’s language and literacy
learning. Future directions for continued evaluation of RIA are discussed.
KEY WORDS: emergent literacy, preschool programs,
at-risk children, curriculum
LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010 * American Speech-Language-Hearing Association 161
for such large-scale prevention-focused initiatives to reduce dispa-
rities in reading achievement between children with economic dis-
advantages and their more advantaged peers is the key catalyst behind
such initiatives. Toward this goal, the current study investigated the
pilot effects of a classroom-based, prevention-focused language and
emergent literacy curriculum supplement on the language and emer-
gent literacy skills of preschoolers who qualify for targeted publicly
funded preschool programs based on family income and, in some
cases, additional demographic risk factors (e.g., family instability).
Designing early childhood programs that are centers of excel-
lence for promoting young children’s language and literacy develop-
ment is a complex endeavor that likely involves efforts at many
layers within the system of early childhood education. There is a
wide range of options currently being considered for how to best
build classroom-based support for children’s language and literacy
development, ranging from preservice teacher training to ongoing
professional development and support (see Justice & Vukelich, 2008;
Zaslow & Martinez-Beck, 2006). Specifically, the use of scientifically
validated language and literacy curricular tools is regarded as an
important mechanism for ensuring that children receive high-quality
language and emergent literacy instruction in the classroom (Christie,
Vukelich, & Enz, 2007). In fact, the federal government recently
made a significant investment in evaluating 14 promising preschool
curricula involving language and literacy skills (as well as mathe-
matical skills) through the Preschool Curriculum Evaluation Research
Initiative (Preschool Curriculum Evaluation Research Consortium,
2008). Further, the federal government is actively compiling reviews
of the evidentiary basis for specific early childhood language and
literacy curricula.1 The goal of these efforts is to enhance the ability of
early childhood practitioners to use evidence-based curricular tools to
enhance their classroom instructional practices.
Findings regarding the benefits of preschool language and liter-
acy curricula, however, are mixed and raise questions regarding the
effectiveness of existing curricular tools for promoting high-quality
classroom language and literacy instruction. For example, many
targeted interventions conducted within preschool classrooms have
had notably large effects on young children’s language and emergent
literacy skills (e.g., Justice & Ezell, 2004; van Kleeck, Vander
Woude, & Hammett, 2006; Whitehurst et al., 1994). Further, there are
a number of efficacy and effectiveness studies providing empirical
support for the positive impacts of language and literacy curricula
within preschool classroom settings (e.g., Assel, Landry, Swank, &
Gunnewig, 2007; DeBaryshe & Gorecki, 2007; Fischel et al., 2007).
DeBaryshe and Gorecki (2007), for instance, evaluated child out-
comes for the “Learning Connections” (LC) curriculum, a curric-
ulum that was designed by the authors to improve teacher instruction
in oral language, phonological and phonemic awareness, alphabet
knowledge and print conventions, and emergent writing. Children
who received LC showed greater end-of-year outcomes on measures
of phonemic awareness and emergent writing relative to children
who received the prevailing curriculum, although LC did not accel-
erate children’s growth in vocabulary or emergent reading. Simi-
larly, Assel et al. (2007) evaluated child outcomes for two commercial
curricula, both of which provide teachers with an explicit scope,
sequence, and instructional activities focused on alphabet knowl-
edge, phonological awareness, and oral language. In general,
1For descriptions and reviews of many of the mentioned curricula, see the What Works
Clearinghouse at http://ies.ed.gov/ncee/wwc/.
children who received either of the experimental curricula showed
improved outcomes on measures of prereading, vocabulary, and
listening comprehension relative to children in control classrooms
in which no specified curricular scope and sequence was used. How-
ever, it is important to note that effects of these curricula were also
moderated by program type, where the strongest effects were seen in
Head Start classrooms, as opposed to Title I and state-funded pre-K.
In contrast to these positive findings, however, the federal pre-
school curriculum evaluation showed that only one of the 14 cur-
ricula demonstrated significant effects on both children’s language
and emergent literacy skills (effect sizes ranged from –.38 to .68
across all curricula). Interestingly, the federal initiative failed to find
significant effects for the two curricula that were found to benefit
children’s skill development in an independent evaluation by Assel
et al. (2007). The variability in findings regarding the benefit of
preschool language and literacy curricula suggests that curricula ef-
fects, even when evident, may not be stable across samples or imple-
mentation efforts. The divergent findings regarding language and
literacy curriculum effects suggest that more needs to be learned
regarding how to best design and implement curricula so they can
effectively support young children’s skills development.
One possible drawback to many available language and literacy
curricula is that they do not appear to be easily implemented well
by teachers, which may reduce the benefit of such tools when they
are considered at scale (Assel et al., 2007; Justice, Mashburn, Hamre,
& Pianta, 2008; Pence, Justice, & Wiggins, 2008; Wasik, Bond,
& Hindman, 2006). For example, teachers implementing structured
language and literacy curricula may show immediate fidelity to
structural aspects of the curricula (e.g., using specified materials,
following lesson plans), but their implementation of more process-
oriented elements of the curricula is typically much more variable
(e.g., using recasts and expansions during conversations; see Justice
et al., 2008; Pence et al., 2008). In fact, evidence suggests that
many teachers require sustained, distributed support if they are to use
many of the prevailing language and literacy curricula (Assel et al.,
2007; Wasik et al., 2006; for a discussion, see Dickinson & Brady,
2006). As such, research-based implementation efforts often re-
quire intensive and ongoing professional development provided by
university-led experts and teams to ensure efficacious implementa-
tion of language and literacy programs (e.g., DeBaryshe & Gorecki,
2007; Jackson et al., 2006; Wasik et al., 2006). Drawing from an
earlier example, DeBaryshe and Gorecki (2007) reported providing
teachers with two 1.5-hr in-service workshops across the year,
weekly coaching by a research assistant, and meetings every third
week with the lead researcher to assist teachers in planning and imple-
mentation. Similarly, Assel et al. (2007) reported providing teachers
with a 4-day institute focused on the curricula to which they were
assigned and twice-monthly in-class mentorship over the course of
the year to support implementation. Other studies report similarly
intensive models of professional development, including specially de-
signed credit-bearing college or university courses and distributed learn-
ing models using distance technologies (Dickinson & Brady, 2006).
Although sustained professional development efforts can be effec-
tive in supporting teacher implementation of language and emergent
literacy curricula (Fukkink & Lont, 2007), these coaching models
can be expensive, time consuming, and difficult to access, if not
prohibitive, for many preschool programs. Thus, a gap exists be-
tween the conditions under which many language and literacy
curricula are shown to be efficacious or effective in research studies
and the conditions under which preschool programs can readily
162 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010
http://ies.ed.gov/ncee/wwc
adopt language and literacy curricula in “business-as-usual” condi-
tions. Indeed, outside of a research study context, some preschool
programs do not have the resources—both money and personnel—
to provide preschool educators with the necessary intensive supports
they might need to implement evidence-based language and literacy
techniques and curricula. Plagued with systemic challenges such
as teacher turnover rates five times that of the K–12 school system
(Barnett, 2003; Bellam, Burton, Whitebook, Broatch, & Young,
2002) and state-per-pupil spending falling below that of the K–12
system in 80% of states (Bryant et al., 2002), preschools need access
to empirically validated tools and programs that can be easily used
by large numbers of professionals at relatively low costs.
The present study examined the feasibility of implementing a
language and literacy curriculum supplement, “Read It Again!”2
(RIA; Justice, McGinty, Beckman, & Kilday, 2006), that can be used
to effectively enhance young children’s language and emergent liter-
acy skills but requires minimal material costs or ongoing profes-
sional development supports. RIA was developed in collaboration
with the West Virginia Department of Education for the purpose of
providing early childhood programs across the state with a scien-
tifically based supplement that (a) could be used effectively by edu-
cators with little training, (b) was compatible with a variety of
program structures and curricula, and (c) could be implemented
relatively inexpensively. As is occurring in many states across the
nation, West Virginia is moving toward universal access to pre-
school education; specifically, Policy 2525 (Universal Access to a
Quality Early Education System, 2002) stipulates that all 4-year-
olds in the state will have access to preschool by 2012–2013. The
policy emphasizes not only access, but also quality and consistency of
the education offered to children within a variety of program typologies
(e.g., Head Start, public pre-K).
RIA was designed to provide early childhood educators with a
curriculum supplement that may be layered into existing classroom
instruction to explicitly address language and literacy goals within their
programs. RIA was developed in a 3-year collaborative process that
featured an iterative, interactive design process whereby researchers
designing the tool actively sought practice-oriented feedback from
end users, including teachers, speech-language pathologists, teaching
assistants, and administrators. The 3-year collaborative process
featured (a) a series of planning and design meetings to identify
key characteristics of an effective supplement, to include an initial
draft of the tool (Year 1); (b) a 15-week pilot implementation in-
volving 18 early childhood educators and their school-based col-
leagues in two counties to study feasibility of use by professionals
working in a variety of settings (Year 2); and (c) a 30-week effec-
tiveness study involving 20 teachers, 11 of whom implemented RIA
for an entire academic year (Year 3). This “bidirectional model of
reciprocal influence (i.e., research 6 practice)” provides a potentially
important solution to closing the research-to-practice gap (Schaughency
& Ervin, 2006, p. 159).
A particularly important feature of RIA’s development was its
adherence to a priori design principles dually focused on using avail-
able scientific knowledge of language and literacy instruction for
young children and maximizing ease of implementation for teachers.
The first design principle was systematicity, whereby RIA system-
atically addresses a predefined scope and sequence of language and
literacy learning targets that are critically linked to early and later
2The most current version of this supplement, “Read It Again—PreK!,” is available as
a free download at www.myreaditagain.com.
reading achievement. Although research findings have supported the
efficacy or effectiveness of a range of intervention techniques and
targets to bring about developmental change, a commonality among
approaches is explicit goal setting within a systematic scope and
sequence of language and literacy instruction (e.g., DeBaryshe
& Gorecki, 2007; O’Connor, Notari-Syverson, & Vadasy, 1996;
van Kleeck, Gillam, & McFadden, 1998). The development team
researched a range of materials to develop a scope and sequence, to
include the meta-analyses of the National Early Literacy Panel
(2004), Hammill (2004), and Scarborough (1998); a comprehen-
sive synthesis of states’ early reading and writing standards (Bodrova,
Leong, Paynter, & Semenov, 2000); prevailing preschool language
and literacy curricula (e.g., Bunce, 1995; O’Connor, Notari-Syverson,
& Vadasy, 2005); and research articles examining predictive relations
between early achievements in language and literacy and children’s
later reading outcomes (e.g., Connor, Morrison, & Slominski, 2006;
Lomax & McGee, 1987; Storch & Whitehurst, 2002). The eventual
scope encompassed two domains of language (vocabulary and
narrative) and two domains of literacy (print knowledge and pho-
nological awareness); a total of 24 objectives aligned to these four
domains comprised the sequence of instruction, with approximately
six specific objectives per domain, as presented in Appendix A.
The second design principle was applicability, whereby RIAwas
designed for widespread use (a) in a variety of programs (e.g., Head
Start, private preschools, public pre-K, home-based child care)
and (b) by a wide range of professionals with diverse levels of back-
ground knowledge and experience. In early childhood settings,
program structure and workforce heterogeneity is substantial. Pro-
grams may be full or part time and may be located in widely different
spaces (e.g., homes, elementary schools; Brandon & Martinez-Beck,
2006). Layered on this setting-level variability are early childhood
professionals whose qualifications can range from a high school
diploma to a master’s degree in early childhood education (Adminis-
tration for Children and Families, 2006; Bellam et al., 2002). In
the design of the RIA curriculum, we needed to contend with the
heterogeneity at the setting and user levels, as these can have marked
implications for the effectiveness of instruction (see Schaughency &
Ervin, 2006).
For the setting-level variability, we organized the curriculum to
provide two brief lessons each week, requiring an estimated 40 min
of instructional time per week; thus, we made RIA accessible to
programs serving children only 2 days a week and those that operate
on a half-day schedule. Given user-level (e.g., teacher-level) hete-
rogeneity and our recognition that some users may have little if any
formal knowledge of language and literacy instruction, RIA features
a relatively scripted design, consisting of 60 lesson plans with ex-
plicit objectives, materials, activity sequences, and even suggested
language for instruction (see Appendix B). Curricula can range from
codified frameworks that specify philosophies of learning to highly
scripted manuals that specify session-by-session activities (Lonigan,
Elbert, & Johnson, 1998). In scripted approaches, the pace, se-
quence, and activities are governed by the curriculum itself rather
than by the professional. The use of scripted versus individualized
lessons is controversial (e.g., MacGillivray, Ardell, Curwen, &
Palma, 2004), and some researchers have suggested that scripted
instruction is less effective than nonscripted instruction (Moustafa
& Land, 2002). Yet, studies have shown comprehensive scripted
instruction to increase children’s academic achievement (Borman
et al., 2005a, 2005b), and manualized lesson plans have been asso-
ciated with efficacious preschool language and literacy instruction
Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 163
www.myreaditagain.com
(e.g., Byrne & Fielding-Barnsley, 1993; Justice, Chow, Capellini,
Flanigan, & Colton, 2003; O’Connor, Jenkins, Leicester, & Slocum,
1993; van Kleeck et al., 2006). Further, sequencing of instructional
objectives is aligned with best practice, as it serves to makes the
instructional path transparent to both children and teachers (Scott-
Little, Kagan, & Frelow, 2003; Wixson & Dutro, 1999) and provides
teachers with a level of specificity regarding what children should
know and be able to do (Roskos, Rosemary, & Varner, 2006; Scott-
Little et al., 2003).
The RIA program was also designed to be applicable to children
in early childhood education classrooms regardless of their hetero-
geneous skill levels and/or potentially wide range of risk factors (e.g.,
poverty, developmental difficulties). One particular area of child
heterogeneity, namely, incoming language skills, may be an espe-
cially important consideration in relation to RIA effects. Research
has suggested that children’s developmental language character-
istics may moderate the benefit that a particular language or literacy
experience or intervention offers (e.g., Justice et al., 2003, Penno,
Wilkinson, & Moore, 2002; Torgesen et al., 1999; van Kleeck et al.,
1998; Vellutino et al., 1996). Thus, a particular intervention may
show group mean benefits but may also be highly variable in its
effect on individual children or groups of children based on their skill
in language (as well as other relevant developmental areas; e.g.,
Bierman, Nix, Greenberg, Blair, & Domitrovich, 2008). RIA is de-
signed to provide a balance of instructional approaches to address the
potentially varied learning needs of young children with a range
of language skills. For example, RIA instructional techniques such
as teacher-directed instruction and explicit skills focus have dem-
onstrated importance for increasing language and emergent literacy
learning for children with low language or literacy abilities (e.g.,
Connor et al., 2006; Penno et al., 2002; Ukrainetz, Cooney, Dyer,
Kysar, & Harris, 2000). Additionally, RIA allows for significant
skills practice, repetition of material, and integration across skills,
which are also considered important components of learning for
children with low language abilities (Coyne, Kame’enui, & Carnine,
2007; Fazio, 1997). Yet, a central component of RIA is an interac-
tive book reading instructional approach, where teachers actively
involve children through questions. This type of interactive book
reading session has been shown to benefit the language and literacy
skills of a variety of children, including children whose language
abilities are already in the average range (e.g., van Kleeck, Gillam,
Hamilton, & McGrath, 1997; Whitehurst et al., 1994). Understanding
how a curricular tool such as RIA may vary in its benefit as a function
of children’s language skill is informative in considering the appli-
cability of this program to meet children’s heterogeneous learning
needs. This study directly addresses this point by considering not
only whether RIA appears to benefit children’s language and liter-
acy development, but also whether RIA is more or less beneficial to
children based on their initial language skill.
The third design principle was feasibility, whereby RIA was de-
signed for ready implementation with relatively few material re-
sources, costs, or ongoing professional development required of its
users. Although a number of well-designed and empirically vali-
dated early language and literacy curricula are available for use by
the early childhood community, their use may be prohibitive to educa-
tors or administrators with limited resources. The material costs
associated with one popular early language and literacy curriculum
involve not only the cost of the curriculum kit, but also the cost
of a boxed set of several hundred manipulatives and materials that
need to be purchased separately. A center seeking to implement this
curriculum in six classrooms would need to spend approximately
$20,000 to do so, which does not include professional development
of its teachers. RIAwas designed as an alternative to those programs
or curricula. Consequently, in the design of RIA, the development
team worked within a set of constraints requiring the final product to
be a self-contained manual that could be placed on the Internet free
of charge. Additionally, the team’s intent was for required supple-
mentary materials to cost no more than $100 to $200, to be easily
accessible (i.e., not obscure items), and to be high yield (i.e., to be
functional for other purposes). With these goals in mind, the devel-
opment team opted for the use of storybooks as a relatively in-
expensive, accessible, and high-yield material that should be heavily
featured in the curriculum. A set of 20 storybooks (estimated cost
$100) was selected for repeated use in the 60 RIA lessons; the story-
books are used as a way to both organize instruction using a before
reading, during reading, and after reading sequence, and embed
code- and meaning-based instruction in a context that is familiar to
both teachers and children. The sample lesson presented in Appen-
dix B illustrates the lesson format.
In terms of professional development, RIAwas created to be min-
imally dependent on ongoing professional development or coaching.
In this study, participating teachers had had a semester of practice
with some of the RIA lessons before this study and also received
training in language and literacy (1 day) and training on the RIA
program (4 hr) before the study. At the study’s inception, teachers
received an additional half-day of refresher training on the RIA
program, which included a discussion of the changes that had been
made to the program. In total, professional development for this
curriculum supplement involved approximately 16 hr of workshop
training delivered at two points during teachers’ participation, as
well as weekly use of the materials for 45 weeks (30 weeks during
this study and 15 weeks of “practice” before implementation). It is
important to note, however, that during the teachers’ use of the
program, there was a minimal amount of discussion with the research
team and almost no feedback on implementation. Thus, the RIA
model is specifically designed for feasibility, where professional
development is in the form of workshops (a format that is familiar
and is widely used by many districts and early childhood programs)
and where materials are designed to be easily and independently
accessible to teachers, requiring minimal ongoing feedback or coach-
ing regarding their use.
In the present article, we report findings from a study that was
designed to explore the feasibility of implementing a language and
literacy supplement that is beneficial to children and easily imple-
mented by teachers. Campbell et al. (2000) illustrated a model of
progressive, cyclical research as a means of determining how to
design and implement effective interventions, especially “complex”
interventions involving many pieces and personnel. In such a cy-
clical approach, parameters of an intervention (e.g., intensity) may
be explored and rigorously tested. Yet, additional exploratory and
efficacy studies may be needed to determine which design features
and model of implementation may be most beneficial to the most
people. Although the field of early education has explored instruc-
tional parameters of language and literacy curricula in relation to
children’s skills (e.g., different teaching techniques, different instruc-
tional targets), less attention has been given to designing interventions
that are cost effective and accessible to the end user (e.g., teachers).
This feasibility study directly explores this research-to-practice pa-
rameter of early language and literacy curricular interventions by
considering the potential benefit that RIA may provide to accelerating
164 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010
children’s language and literacy development during their pre-K year
when it is implemented with minimal coaching or ongoing profes-
sional support.
The current study was exploratory in nature and, consistent with
pilot- or feasibility-study designs, represents a precursor to more
rigorously designed, randomized controlled trials that seek to estab-
lish the causal effects of a given intervention. Feasibility studies are
an especially important initial step when conducting classroom-
based research (van Teijlingen & Hundley, 2001). That is, when
conducting research on interventions that are implemented to
groups of children clustered in classrooms, randomization should
occur at the level of the classroom or teacher, resulting in a signif-
icant investment of resources to recruit and retain the number of
classrooms needed for a sufficiently powerful design. Feasibility
research outcomes, such as that reported here, inform the planning of
later studies as they can provide details on the number of clusters
required in larger scale trials, including presumed effect sizes and
intraclass correlations (Campbell, Mollison, & Grimshaw, 2001). As
a feasibility study, the proposed study has several key design limi-
tations, including use of a quasi-experimental research design, that
preclude us from making definitive statements regarding causal
impacts of the intervention. Nonetheless, our intent was to consider
whether RIA appears to exhibit promise as a curricular tool, thus
suggesting the need to further consider RIA as a potentially effective,
low-cost, and accessible language and literacy curricular tool. The
study involved two research questions:
& Do children who receive RIA for a 30-week period in their
preschool classrooms exhibit better language and literacy skills
in the spring of the year relative to children in comparison
classrooms? The careful design of RIA focuses on targets and
teaching techniques supported by empirical research; therefore,
we hypothesized that children who were enrolled in classrooms
in which teachers implemented RIA would exhibit better
language and literacy skills at the end of the academic year
relative to children in classrooms in which teachers maintained
business-as-usual instruction.
& To what extent is the effect of RIA on children’s outcomes
moderated by children’s language ability at the start of the
preschool year? We hypothesized that RIA would have equal
impacts for children, regardless of their initial language ability.
Our rationale for this hypothesis is based on the careful design
of the RIA curriculum, as discussed earlier, and its inclusion
of a variety of instructional approaches that may be amenable
to children of varying language skill levels. As such, we
expect RIA to meet the diverse learning needs of children with
varying linguistic ability levels.
METHOD
Participants
Twenty preschool teachers and 137 children attending 14 schools
in four districts participated in this program evaluation. Of the
20 teachers, 11 teachers served as an experimental group who imple-
mented the preschool curriculum supplement, RIA, and 9 teachers
served as a control group who used their standard educational prac-
tices. All classrooms were designated as full-day, full-time preschool
programs (i.e., 5–8 hr of instruction per day, meeting 4 or 5 days
per week), and all classrooms received public funding to prioritize
enrollment of economically and/or instructionally at-risk children.
All 20 classrooms were located within eight rural, largely Appalachian
counties in two contiguous states. The 11 RIA classrooms were
located in two counties in which the median household incomes
were $27,743 and $29,323, according to the 2004 U.S. Census, and
the percentage of persons living below the poverty line was 19%
and 18%, respectively. The 9 control/comparison classrooms were
located in two different counties where the median household in-
comes were $25,549 and $26,900 and the percentage of persons
living below the poverty line was 21% and 20%, respectively.
Demographically, the contexts in which the two sets of classrooms
were located were highly similar.
Teachers
Experimental teachers (n = 11) in two districts were informed
about the planned assessment of the RIA program by their adminis-
trators and were provided the opportunity to participate if they so
desired. The 11 experimental teachers in this study were part of the
pool of 18 teachers who had implemented and evaluated draft RIA
materials in the previous year. All 18 of these teachers provided
informed consent to participate in the current 30-week study of RIA;
however, project resources mandated that child data could be col-
lected from only 11 classrooms. The 11 experimental teachers in-
cluded in the study were, therefore, randomly selected from the
18 teachers who were implementing and evaluating the draft RIA
materials during the year prior to this current study.
The 11 experimental teachers were Caucasian, non-Hispanic fe-
males ranging in age from 35 to 61 years (M = 47.5 years, SD = 8.02)
and had, on average, 14–15 years of teaching experience (range =
5–27 years; SD = 5.56). Approximately half of the teachers had an
associate’s degree as their highest educational degree, and 54% had
higher education degrees. All teachers reported implementing “The
Creative Curriculum for Preschoolers” (Creative Curriculum; Dodge,
Colker, & Heroman, 2002) as their prevailing instructional ap-
proach. The experimental teachers provided informed consent to
participate, and each agreed to implement RIA for a 30-week period
as a supplement to Creative Curriculum.
The 9 comparison teachers were enrolled in a separate study that
involved examining preschool teachers’ classroom practices and
their effects on children. The 9 teachers were among those who had
been assigned to a comparison group as part of a separate study and,
as a result, were asked to maintain their established instructional meth-
ods for the length of their involvement in the study. The 9 teachers
represented all of those in the comparison group who taught in two
counties that were identified as highly similar to those of the RIA
teachers.
The 9 comparison teachers were female and ranged in age from
37 to 59 years (M = 48.5 years, SD = 7.89). The majority of these
teachers were Caucasian (63%), although other race/ethnicities were
represented (1 teacher was African American, 1 was Native Amer-
ican, and 1 was multiracial). All had an associate’s degree as their
highest educational level, had an average of 13–14 years of teaching
experience (range = 8–18 years, SD = 3.73), and reported using
Creative Curriculum as their prevailing instructional approach.
In comparing the two groups of teachers, there were some sim-
ilarities between the experimental and comparison groups as well
as some differences. In terms of similarities, all teachers in both
groups had at least an associate’s degree, and the two groups were
Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 165
statistically similar in their average age and their average years of
teaching experience ( p > .10). The two groups of teachers were dif-
ferent in terms of race/ethnicity, with the comparison group demon-
strating statistically higher numbers of teachers who were African
American, Native American, and multiracial (based on chi-square
tests, p < .001). Further, more of the teachers in the experimental
group had educational experience beyond an associate’s degree
( p < .001).
Students
Of the 137 children participating in the study, 66 participated in
experimental classrooms and 71 participated in comparison class-
rooms. These children represented a subset of children who were
enrolled in these classrooms; that is, rather than testing all of the
children in each classroom to estimate intervention effects, a small
sample was selected from each class using random selection, similar
to procedures used in large-scale studies of preschool quality (e.g.,
Early et al., 2005). In the RIA classrooms, 6 children, on average, were
randomly selected from the class roster by the research team. Children
(a) who were unable to be tested due to gross sensory, motor, or
cognitive reasons; (b) who were absent on the day of testing; or
(c) for whom parental assent was not provided to the district were
excluded from the study. For the comparison condition, 8–9 children
per classroom (on average) were randomly selected from the pool
of children for whom parental written consent had been provided
to the research team. Exclusionary criteria were the same as those
employed in the RIA classrooms.
Children in the RIA classrooms (n = 66) ranged in age from 39 to
66 months (3;3 [years;months] to 5;6) and included 33 boys and
33 girls. In terms of race/ethnicity, all were White/Caucasian. Teacher
report indicated that all children spoke English as their primary lan-
guage, and 7.6% (n = 4) of the children had an individualized educa-
tion program (IEP). Children in the comparison classrooms (n = 71)
ranged in age from 42 to 60 months (3;6 to 5;0) and included 27 boys
and 35 girls (gender not recorded for 9 children). In terms of race/
ethnicity, 42.3% (n = 30) were White/Caucasian, 48.8% (n = 29)
were Black/African American, and data were not available for
16.9% (n = 12) of the children. Teacher report indicated that 11.3%
(n = 8) of the children had an IEP, and parent report indicated that
all of the children (for whom data was available; n = 38) spoke English
as the primary language in the home.
Missing data. In order to maximize power and minimize the im-
pact of missing data estimates of RIA effects, cases were not deleted
list-wise as a result of missing data; all participants with relevant
data within a specific analytic model were included. Levels of missing
data for the fall and spring time points for all child-level measures
showed almost no missing data within the experimental group at the
fall time point. In contrast, within the comparison group, missing data
levels across each of the fall measures ranged from 12% to 18%.
In the spring, data were missing for 19% of the children within the
experimental group and for 18% to 21% of the children in the com-
parison group (varying across spring outcome measures). Indepen-
dent t tests were examined to determine if children within the
comparison group who had missing fall data were statistically dif-
ferent from those with no missing fall data on age or spring out-
comes. Results indicated no significant differences ( p < .05) of age,
spring language, or emergent literacy scores ( p < .05) between
children in the comparison group with missing versus full fall data.
Similarly, independent t tests indicated no significant differences
on children’s initial language or literacy abilities for children with
missing spring data as compared to children with full spring data.
These patterns suggest that children’s data were likely missing at
random and that missing data should not have biased the findings in
any systematic way.
PROCEDURES
This study used a quasi-experimental pre–post comparison group
design to evaluate child language and literacy outcomes associated
with receipt of the 30-week, 60-lesson RIA preschool supplement.
Both the experimental (i.e., RIA) and comparison classrooms par-
ticipated during the 2006–2007 school year, and teachers in both
groups received professional development at the beginning of the
year as part of their participation. For their participation, teachers
in the experimental condition received a binder of the RIA materials
and all 15 related storybooks for their classroom. Teachers in the
comparison condition were also participants in a larger, randomized
controlled trial; for participating in that study, they received $150
in an educational fund, $150 in a materials account to purchase re-
sources for their classroom, and equipment needed to conduct video-
tapings of small-group activities within their classroom (classroom
videotapes were relevant to the larger randomized controlled trial,
not the current study).
In order to be considered as active participants in the current
study, all of the teachers were required to submit information to the
research team on a monthly basis. Teachers in the experimental
condition submitted implementation logs regarding their use of RIA
to the research team on a monthly basis using self-addressed, prepaid
envelopes. Teachers in the comparison condition videotaped a
specified small-group activity and sent this tape to the research team
biweekly in self-addressed, prepaid envelopes. Teachers in both
conditions were aware that the research team was interested in how
preschool may help children learn language and literacy skills and that
data were being collected on their classroom and from individual
children in their classroom. No teachers were aware of the study’s
specific research questions. Teachers’ compliance in sending these
materials to the research team was not reported to administrators.
All teachers involved in this study reported using the same
general curriculum to establish child learning goals and instructional
practices in their classrooms (i.e., Creative Curriculum); thus, RIA
effects can be considered as influencing children’s skills above and
beyond that curriculum.
Professional Development for RIA Teachers
RIA teachers participated in two phases of training. The first
phase occurred during the semester before the current study’s incep-
tion. During this semester, teachers attended a 1½-day workshop—
1 day (8 hr) on language and literacy development and ½ day (4 hr)
on the RIA materials specifically. Participants in this workshop in-
cluded not only teachers who would be using RIA, but also assistants
in these classrooms, district speech-language pathologists (SLPs),
and program administrators. Following training, teachers imple-
mented a pilot version of RIA for a 15-week period, submitted weekly
notes on implementation, and had telephone contact with research
personnel twice regarding their use of RIA. This telephone contact
involved the research team asking predesigned questions about the
166 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010
RIA program in order to glean feedback on the materials’ usability.
No coaching or feedback was given to teachers regarding their
implementation and, if questions were initiated by teachers, the
research team provided generic feedback that guided them to follow
the lesson plans as closely as possible and document any changes
they made on their implementation notes. The second phase of
training was conducted at the start of the current feasibility study;
this included a half-day workshop (4 hr) for teachers that focused
primarily on expectations for implementation; that is, how long RIA
should be used (30 weeks) and how often lessons should be imple-
mented (twice weekly). Classroom assistants, district SLPs, and
program administrators also participated in this workshop, per the
request of administrators in the West Virginia Department of Educa-
tion. However, only lead teachers (e.g., study participants) imple-
mented RIA, and other attendees of the workshop were not formally
consented into the study. Given that our intent in this feasibility study
was to explore whether RIA could be used with little ongoing pro-
fessional development resources, no coaching or feedback was pro-
vided to teachers during the 30-week period of implementation
with the exception of periodic calls (three times in total) to the
2 teachers who did not consistently submit fidelity documents (as
discussed below). No incentives were provided to teachers for
participation other than the materials needed for implementation of
RIA and the set of corresponding storybooks for their classroom.
In agreeing to participate, the teachers were agreeing to evaluate and
use the RIA curricular tool and to allow us access to their classroom
in order to assess children whose parents had provided consent
and who were selected into our study.
RIA Description
RIA is a 30-week language and literacy curriculum supplement
consisting of 60 separate lesson plans. Each lesson includes three
sets of activities organized around a whole-class group storybook
reading interaction: before reading, during reading, and after read-
ing. Each lesson addresses two of the four instructional domains in
the RIA scope—vocabulary, narrative, print knowledge, and pho-
nological awareness—such that each domain is addressed once
within a given week of instruction. Implementation of RIA lesson
plans is sequenced over a 30-week period such that each domain’s
instructional objectives increase in complexity and difficulty over
time and lessons build on prior knowledge. Materials needed to use
the supplement include common classroom tools (e.g., whiteboard,
marker, paper) with the exception of a set of 15 storybook titles that are
used repeatedly as a content of instruction. Teachers were provided
with the set of storybooks at the start of the academic year but received
no additional materials.
Each lesson is designed to last approximately 20 to 30 min. Im-
plementation recommendations suggest that teachers deliver two
lessons per week, although when the lessons are delivered within the
classroom schedule is based on teacher preference. Both lessons
for a week correspond to the same storybook, thus requiring the
teacher to read the week’s designated book two times. The same set
of storybooks is used repeatedly during the 30-week period of in-
struction; typically, a storybook is used every 4 to 5 weeks as a ve-
hicle for instruction. Each lesson provides a scripted outline that
specifies a step-by-step sequence of instruction as well as suggested
language that teachers can use to support children’s learning during
each activity. A sample lesson is presented in Appendix B. The RIA
lessons can be conducted in large-group, small-group, or one-on-one
settings; however, teachers in this study implemented all lessons
with the full class and only lead teachers conducted RIA lessons.
Fidelity of Implementation to RIA
To track adherence to the RIA program and its schedule of imple-
mentation, the experimental teachers were required to complete
an implementation log after every lesson. This log required teachers
to indicate the date of the lesson, that the lesson was conducted as
a whole group, the length of each activity in the lesson, any chal-
lenges with the lesson’s activities, the general performance of the
children, and any extensions or modifications that occurred. Teachers
were required to submit these logs to research personnel on a monthly
basis and received stamped addressed mailers for this purpose at the
start of the project. As long as teachers indicated that the lessons
were conducted according to the designated schedule and that all
activities in the lesson were conducted, teachers were considered as
having fidelity to that lesson. Previous studies have found that teach-
ers are reliable reporters of their classroom practices, as seen by
high correlations between reported and observed behaviors (r = .7;
Rimm-Kaufman & Chiu, 2007).
Of the 11 RIA teachers, 1 teacher submitted no implementation
logs, and another teacher logged only 3 weeks of lessons out of the
possible 30 weeks of instruction. These 2 teachers were sent four
letters and received three phone calls regarding fidelity, yet their
fidelity logs were never submitted. It is therefore impossible to know
with certainty whether the program was being implemented in these
classrooms. (These same teachers had participated in the prior year
of pilot implementation, and their implementation logs from that
period provided evidence of their cooperation and fidelity during a
15-week period of implementation.) Averaging fidelity across all
11 teachers, the percentage of lessons implemented was 66%, with
an average of 40 lessons implemented according to teacher report.
Averaging fidelity across the 9 teachers who submitted fidelity logs on
a regular basis, the percentage of lessons implemented was 80%, with
an average of 48 lessons implemented over the 30-week period.
Professional Development
for Comparison Teachers
At the start of the school year, teachers in the comparison class-
rooms attended a 2-day workshop addressing topics applicable to
preschool educators but not directly related to language and literacy
(e.g., behavior management, enhancing learning in the block center).
Comparison teachers were asked to maintain business-as-usual
classroom practices. No specific feedback on instruction was pro-
vided to these teachers over the course of the year, although the
teachers had periodic communications with project staff regarding
various project deliverables.
Measures
Two sets of measures are of relevance to this study: measures of
child language and measures of child emergent literacy skill. These
measures were collected in the fall and spring of the year as part
of a larger battery that was administered individually to each child.
All measures were administered by research assistants (e.g., under-
graduate and graduate students, part- and full-time project staff ),
all of whom received standardized training on each measure and
Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 167
passed a test on each measure’s administration procedures. Children
were tested outside of their classrooms in a relatively quiet area of
the school, and tests were typically administered in two short (ap-
proximately 15 min) testing sessions. After test administrations were
completed, scoring of measures was completed by undergraduate
research assistants who had received training on the scoring of each
measure and who were blinded to the study conditions of the par-
ticipants. All measures were double scored by two research assis-
tants working independently of one another using paper copies of
administered protocols; any discrepancies were resolved in order
to achieve 100% agreement by both scorers.
The language measures used consisted of three subtests of the
standardized norm-referenced Clinical Evaluation of Language
Fundamentals Preschool—Second Edition (CELF Preschool–2;
Wiig, Secord, & Semel, 2004), namely, Sentence Structure, Word
Structure, and Expressive Vocabulary. Collectively, these subtests
require approximately15 min to administer. For data analyses, we
used individual subtest raw scores as outcome measures. To con-
sider the association of general language ability in response to
treatment, we used the combined standardized composite score of the
three subtests, the core language composite (M = 100, SD = 15).
Adequate reliability and validity are well established for this measure
(see Wiig et al., 2004). The three specific subtests used in this study
provide estimates of children’s grammar (Sentence Structure), mor-
phology (Word Structure), and vocabulary (Expressive Vocabulary).
The emergent literacy measures used included tests of phono-
logical awareness (i.e., rhyme and alliteration) and print knowledge
(i.e., alphabet knowledge and print concepts) that collectively re-
quire approximately 15 min to administer. The Rhyming Individual
Growth and Development Indicator (Rhyming IGDI; Early Child-
hood Research Institute on Measuring Growth and Development,
2000b) was used to measure children’s ability to identify as many
rhyming pairs as possible in a 2-min period. The Alliteration Indi-
vidual Growth and Development Indicator (Alliteration IGDI; Early
Childhood Research Institute on Measuring Growth and Develop-
ment, 2000a) was used to measure children’s ability to identify word
pairs that share an initial sound during a 2-min period. Children’s
alphabet knowledge was measured using the Phonological Awareness
Literacy Screening: Preschool (PALS–PreK) Upper-Case Alphabet
Recognition task (Invernizzi, Sullivan, Meier, & Swank, 2004). In
this task, children are asked to name each of the 26 individual, up-
percase letters as they are presented in random order on a single
printed sheet. Scores range from 0 to 26. The Preschool Word and
Print Awareness Test (PWPA; Justice & Ezell, 2001; see also Justice,
Bowles, & Skibbe, 2006) was used to study children’s knowledge
of 14 print concepts (e.g., print directionality, print forms, meaning
of print) in the context of a shared book reading. Raw scores were
converted to standard scores based on M = 100 and SD = 15.
RESULTS
Analytic Approach
In our data analyses, we used an intent-to-treat framework
whereby children attending treatment classrooms were included in
our models regardless of the classroom teacher’s fidelity to the
program. In an intent-to-treat framework, all individuals assigned to
a particular condition are maintained in analyses irrespective of
compliance, withdrawal, or deviations in implementation (Peduzzi,
Detre, Wittes, & Holford, 1991).
The data adhere to a multilevel structure in which 6 to 8 children
are nested within 20 classrooms. Given the nonindependence of
multilevel data, it is possible that Type I error rate is inflated, such
that a statistical test failing to account for nesting effects may be
liberal (Raudenbush & Bryk, 2002). Thus, it may be the case that
children in the same classroom would perform more similarly than
children in different classrooms due to classroom-level influences.
To examine the effects of RIA, hierarchical linear modeling (HLM;
Raudenbush & Bryk, 2002) was employed, accounting for non-
independence and allowing examination of both child- and classroom-
level components. Through the use of HLM6 software (Version 6.06;
Raudenbush, Bryk, Cheong, & Congdon, 2006), full maximum
likelihood estimation procedures were employed, with all continu-
ous variables entered grand-mean centered for ease of interpretation.
To maximize power and to minimize the influence of missing data,
we used the largest sample size possible per analysis by deleting
missing data while running the analyses; thus, sample size varied for
different models. Further, it is important to note that all models were
also rerun excluding any children with IEPs (4 in RIA classrooms,
8 in comparison classrooms), and no differences in statistical findings
were noted. The results presented include all possible children (in-
cluding children with IEPs) who have data for the given model.
Seven HLM models were examined, one for each area of lan-
guage and emergent literacy skill. The dependent variables with
regard to language were grammar, morphology, and vocabulary. The
dependent variables with regard to literacy were rhyme, alliteration,
alphabet knowledge, and print concepts. We first estimated the variance
components associated with the unconditional model (i.e., without
predictors), examining the magnitude of the intraclass correlations
(ICCs). The ICCs ranged from .17 to .51 (with 4 of the 7 models > .25),
indicating that substantial nesting effects were present in the data.
To address our first research aim investigating the impact of RIA
on children’s language and literacy skills, we examined children’s
spring scores of language and emergent literacy skills while con-
trolling for fall scores; this approach allows for a consideration of
performance after controlling for pre-intervention scores, which is
important because initial between-group differences are a common
artifact of the quasi-experimental research design that relies on in-
tact groups. Raw scores were used in this set of analyses, and we
used the following base model (Equations 1a and 1b). For each
outcome indicator, a child’s performance (Equation 1a; Yij) is a func-
tion of the mean spring score for the classroom (b0j) after adjusting
for the influence of child age (b1j) and fall score (b2j).
Yij ¼ b þ b1jðageÞ þ b ðfall scoreÞ þ rij ð1aÞ0j 2j
The estimate of the classroom mean (b0; Equation 1b) is a function of
the grand mean of all classrooms (l00), the experimental condition
of the classroom (l01) dummy coded as 1 (treatment) and 0 (com-
parison), and the estimation error.
b ¼ l00 þ l01ðconditionÞ þ m0j ð1bÞ0
b
b
1
¼ l10
2
¼ l20
We computed effect sizes for each model by multiplying the pre-
dictor coefficient with its standard deviation and dividing by the
outcome’s standard deviation (see Mashburn et al., 2008).
168 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010
To address our second research aim investigating whether RIA
would impact children equally regardless of their initial language
ability, we evaluated the extent to which the effects of RIA were
moderated by children’s initial language ability through a second set
of seven HLM models. Building on the base model, we examined the
cross-level interaction between children’s initial language ability
(i.e., fall language composite) and condition (Equations 2a and 2b).
The fall language composite is a standard score based on M = 100
and SD = 15.
Yij ¼ b0j þ b1jðageÞ þ b2jðfall scoreÞ
þ b3jðfall language compositeÞ þ rij ð2aÞ
b0 ¼ l00 þ l01ðconditionÞ þ m0j ð2bÞ
b1 ¼ l10
b2 ¼ l20
b3 ¼ l30 þ l31ðconditionÞ
RESULTS
Table 1 presents the means and standard deviations for the
full sample on all measures of language and literacy. The data in
Table 1 also identify the percentage of children in the sample who
received scores ≤ the 16th percentile (≤ –1 SD of the mean) and
≤ the 25th percentile on the norm-referenced measures at the fall
time point. These results show that a higher percentage than ex-
pected of the full sample exhibited depressed language abilities with
respect to normative references. That is, based on a normal curve, we
would expect approximately 16% of the children to fall ≤1 SD;
in our sample, one quarter to one third of all children fell within this
risk category before the start of intervention. Table 2 presents the
means and standard deviations for the children in the RIA (n = 66)
and comparison classrooms (n = 71) for both fall and spring assess-
ment time points.
Impact of RIA Curriculum on Children’s Language
and Literacy Skills
The first aim of the present study was to examine the extent to
which participation in RIA impacted end-of-year performance on
children’s language and emergent literacy skills relative to partici-
pation in the prevailing preschool curriculum. To evaluate differ-
ences between the two groups on end-of-year language abilities, we
considered differences in three dependent variables of spring per-
formance (grammar, morphology, vocabulary) using raw scores while
controlling for child age and fall performance. The results from the
three HLM models are presented in Table 3. Children who participated
in the supplemental RIA curriculum demonstrated significantly higher
spring language performance than those in the comparison condi-
tion in all areas: grammar (outscoring by 2.08 points, p = .02, effect
size = .24); morphology (3.20 points, p < .01, effect size = .35); and
vocabulary (2.68 points, p = .04, effect size = .17).
To evaluate children’s performance on end-of-year measures of
emergent literacy, we examined differences by condition for chil-
dren’s spring scores when controlling for child age and fall scores.
The results from the four HLM models presented in Table 4 indicate
that children in the RIA condition outperformed children in the com-
parison condition on three of the four skill areas: rhyme (outscor-
ing by 4.70 points, p < .01, effect size = .41), alliteration (2.09 points,
p < .01, effect size = .30), and print concepts (3.57 points, p < .01,
effect size = .44). Between-group differences on alphabet knowledge
were not significant (1.02 points, p = .51, effect size = .05).
Initial Language Ability as a Moderator
of RIA Effects
To address our second research aim investigating whether RIA
would impact children equally regardless of their initial language
ability, we examined whether the effects of RIA were moderated by
children’s language ability before intervention. Condition × Language
interactions were not significant when examining language out-
comes. In other words, children’s initial language ability did not
Table 1. Means and standard deviations for the full sample (N = 137 children) on all measures of language and literacy.
Fall Spring
n M SD n M SD ≤16th percentile ≤25th percentile
Language
Grammar 124 12.23 4.53 110 15.67 4.36 26.3 % 38.7 %
Morphology 125 12.13 5.05 111 16.04 4.73 30.7 % 43.8 %
Vocabulary 124 17.41 8.03 110 24.06 8.05 28.5 % 37.2 %
Literacy
Rhyme 128 2.98 4.00 109 5.25 5.87 — —
Alliteration 130 1.59 2.58 110 2.48 3.50 — —
Alphabet knowledge 128 7.67 8.82 109 13.97 9.89 — —
Print concepts 123 6.01 3.52 112 9.85 4.14 — —
Note. Grammar, morphology, and vocabulary scores are from the Clinical Evaluation of Language Fundamentals Preschool—Second Edition
(CELF Preschool–2; Wiig, Secord, & Semel, 2004), maximum = 22, 24, 40, respectively; rhyme and alliteration scores are from the Rhyming
Individual Growth and Development Indicator (Rhyming IGDI; Early Childhood Research Institute on Measuring Growth and Development,
2000b), no maximum score; alphabet knowledge scores are from the Phonological Awareness Literacy Screening for Preschool (PALS–PreK)
Upper-Case Alphabet Recognition task (Invernizzi, Sullivan, Meier, & Swank, 2004), maximum = 26; print concepts scores are from the Preschool
Word and Print Awareness Test (PWPA; Justice & Ezell, 2001), maximum = 17.
Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 169
Table 2. Means and standard deviations for the children in the “Read It Again!“ (RIA; Justice, McGinty, Beckman, & Kilday, 2006; n = 66)
classrooms and the comparison classrooms (n = 71).
RIA Comparison
Fall Spring Fall Spring
n M SD n M SD n M SD n M SD
Language
Grammar 66 13.29 4.37 53 17.42 3.07 58 11.02 4.45 57 14.05 4.76
Morphology 66 13.85 4.75 53 18.42 3.44 59 10.20 4.70 58 13.86 4.72
Vocabulary 66 20.53 7.34 53 28.09 6.50 58 13.86 7.33 57 20.32 7.56
Fall composite 66 97.55 14.63 58 86.72 14.
16
Literacy
Rhyme 66 4.14 4.40 54 8.24 6.40 62 1.76 3.12 55 2.31 3.30
Alliteration 65 1.74 2.73 54 3.70 4.11 65 1.45 2.44 56 1.30 2.2
7
Alphabet knowledge 66 7.77 8.76 53 15.02 9.43 62 7.56 8.96 56 12.98 10.30
Print concepts 66 6.82 3.81 54 12.22 3.73 57 5.07 2.93 58 7.64 3.
18
Note. Grammar, morphology, and vocabulary scores are from the CELF Preschool–2, maximum = 22, 24, 40, respectively; rhyme and alliteration
scores are from the Rhyming IGDI, no maximum score; alphabet knowledge scores are from the PALS–PreK Upper-Case Alphabet Recognition
task, maximum = 26; print concepts scores are from the PWPA, maximum = 17.
moderate the effects of RIA for language outcomes (see Table 3). In
contrast, for three of the four literacy measures, Condition × Language
interactions were significant. Specifically, initial language ability
moderated the effects of RIA for alliteration, alphabet knowledge,
and print concepts (see Table 4). Figures 1, 2, and 3 depict the nature
of the resulting interactions. For the sake of illustration, we present
figures illustrating spring scores for children with low (standard score
of 85), middle (standard score of 95), and high (standard score of 105)
initial language ability. These graphs illustrate that, in general, as
children’s language ability increased, the positive benefit of RIA to the
emergent literacy skills of alphabet knowledge, alliteration, and print
concepts also increased.
DISCUSSION
In the current climate, which emphasizes not only accountabil-
ity but also the arrival of “evidence-based progress to education”
(Stanovich & Stanovich, 2003), preschool teachers are expected to
meet children’s needs and even equalize disparities among students
who enter school from remarkably diverse ethnic, linguistic, racial,
and economic backgrounds. This is particularly true in the area of
language and emergent literacy achievement, both of which serve
as critical determinants of children’s successful adjustment to the
kindergarten milieu and as consistent predictors of later outcomes in
word recognition and reading comprehension (e.g., Storch & Whitehurst,
2002). The use of evidence-based programs and practices for ac-
celerating children’s language and literacy achievement within rural
preschool programs is an ideal that remains out of reach for many
preschool educators today, whose programs may not have the finan-
cial resources to access empirically validated programs or to provide
teachers with the ongoing specialized mentorship or course work
needed to implement such programs. Although there is evidence that
available, comprehensive language and literacy curricula may ef-
fectively enhance young children’s language and literacy skills,
many of these programs are complex, involve high-cost materials,
and require intensive mentorship for successful implementation. The
design of these programs may be prohibitive to many preschool class-
rooms; thus, their design may preclude their short-term scalability.
Table 3. Hierarchical linear modeling results demonstrating the impact of RIA on children’s language outcomes.
Grammar (n = 100 children
in 20 classrooms)
Coeff. SE dF
Morphology (n = 102
children in 20 classrooms)
Coeff. SE dF
Vocabulary (n = 100
children in 20 classrooms)
Coeff. SE dF
Base model
Spring intercept, g00
Child-level condition
Age, g10
Fall score, g20
Classroom-level condition, g01
Interaction model Condition × Language, g31
*p < .05, **p < .01.
14.60**
0.13*
0.43**
2.08*
–.02
0.56
0.07
0.08
0.79
0.0
5
18
96
96
18
9
4
14.46**
–0.006
0.47**
3.20**
–0.02
0.61
0.06
0.07
0.86
0.04
18
98
98
18
95
22.44**
0.07
0.78**
2.68*
–0.04
0.88
0.07
0.05
1.23
0.05
18
96
96
18
94
170 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010
5
4.5
4
C
3.5
0 3 :;::;
I.!! 2.5
Cl)
~ 2
< 1.5
1
0.5
0
Low Medium High
Initial Language Ability
□ Treatment
ii Comparison
16
& 15 I,!———-,
“C
Cl)
] 14
C
~
13
ai
i 12
.c
0.
10 Initial Language Ability Ill Comparison
Table 4. Hierarchical linear modeling results demonstrating the impact of RIA on children’s literacy outcomes.
Rhyme (n = 103 Alliteration (n = 105 Alphabet knowledge Print concepts Fixed effects Coeff. SE dF Coeff. SE dF Coeff. SE dF Coeff. SE dF
Base model Classroom-level condition, g01 4.70** 1.45 18 2.09** 0.66 18 1.02 1.53 18 3.57** 0.82 17 *p < .05, **p < .01.
The unique contribution of the present study resides in our intent to Specifically, the first finding of this study was that children in Figure 1. Condition × Language interaction for spring alliteration the result of some other aspects of the experimental classrooms. Effect sizes, in particular, suggest the potential magnitude of Figure 2. Condition × Language interaction for spring alphabet Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 171 13 12 a 11 ,,,_ ___ …. g 10 7 6 Initial Language Ability Figure 3. Condition × Language interaction for spring print concepts print knowledge (d = .37) when implementing a language and literacy Nonetheless, the effect-size findings from this study suggest the early childhood education programs outside of the research context As a means of exploring the extent to which RIA may be ap- 172 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010 In sum, RIA is a promising curricular tool for which benefits diverse learners.
Limitations and Conclusions
Several limitations of this study warrant mention and suggest A second limitation of this study is its generalizability of findings It is also important to consider that the teachers in this study were In sum, intervention efforts that seek to bridge the research-to- ACKNOWLEDGMENT
This study was supported in part by the West Virginia Department of REFERENCES
Administration for Children and Families. (2006). FACES findings: New Assel, M. A., Landry, S. H., & Swank, L. (2008). Are early childhood Assel, M. A., Landry, S. H., Swank, P. R., & Gunnewig, S. (2007). An Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 173 https://acf.hhs.gov/programs/opre/hs/faces/reports/faces_findings_06/faces http://www Barnett, S. (2001). Preschool education for economically disadvantaged Barnett, S. (2003). Low wages = low quality: Solving the real preschool Bellam, D., Burton, A., Whitebook, M., Broatch, L., & Young, M. (2002). Bierman, K. L., Nix, R. L., Greenberg, M. T., Blair, C., & Domitrovich, Bodrova, E., Leong, D. J., Paynter, D. E., & Semenov, D. (2000). A Borman, G. D., Slavin, R. E., Cheung, A. C. K., Chamberlain, A. M., Borman, G. D., Slavin, R. E., Cheung, A., Chamberlain, A. M., Madden, Brandon, R. N., & Martinez-Beck, I. (2006). Estimating the size and Bryant, D., Clifford, R. M., Saluja, G., Pianta, R. C., Early, D. M., Bunce, B. H. (1995). Building a language-focused curriculum for the Byrne, B., & Fielding-Barnsley, R. (1993). Evaluation of a program to Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Campbell, M. K., Mollison, J., & Grimshaw, J. M. (2001). Cluster trials in Christie, J. F., Vukelich, C., & Enz, B. J. (2007). Teaching language and Connor, C. M., Morrison, F. J., & Slominski, L. (2006). Preschool in- Coyne, M., Kame’enui, E. J., & Carnine, D. W. (2007). Effective teaching DeBaryshe, B. D., & Gorecki, D. M. (2007). An experimental validation of Dickinson, D., & Brady, J. P. (2006). Toward effective support for lan- Dodge, D. T., Colker, L., & Heroman, C. (2002). The Creative Curriculum Early, D., Barbarin, O., Bryant, D., Burchinal, M., Chang, F., Clifford, Early Childhood Research Institute on Measuring Growth and Early Childhood Research Institute on Measuring Growth and Fan, X. (2001). Statistical significance and effect size: Two sides of a coin. Fazio, B. B. (1997). Learning a new poem memory for connected speech Fischel, J. E., Bracken, S. S., Fuchs-Eisenberg, A., Spira, E. G., Katz, S., Fukkink, R. G., & Lont, A. (2007). Does training matter? A meta-analysis Fullan, M. (2000). The return of large-scale reform. Journal of Educational Hammill, D. D. (2004). What we know about correlates of reading. Excep- Hamre, B., LoCasale-Crouch, J., & Pianta, R. C. (2008). Formative as- Institute of Education Sciences. (2007). National evaluation of early Invernizzi, M. A., Sullivan, A., Meier, J. D., & Swank, L. (2004). Phono- Jackson, B., Larzelere, R., St. Clair, L., Corr, M., Fichter, C., & Egertson, H. Justice, L. M., Bowles, R. P., & Skibbe, L. E. (2006). Measuring preschool Justice, L., Chow, S., Capellini, C., Flanigan, K., & Colton, S. (2003). 174 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010 http://ies.ed.gov http://ggg.umn.edu http://ggg.umn.edu http://www.fpg.unc.edu/(ncedl/pdfs http://www.ccw.org/pubs/ccw_pre-k_10 effects of two approaches. American Journal of Speech-Language Pathology, Justice, L. M., & Ezell, H. K. (2001). Word and print awareness in 4-year- Justice, L. M., & Ezell, H. K. (2004). Print referencing: An emergent liter- Justice, L., & Kaderavek, J. (2004). Embedded–explicit emergent literacy Justice, L. M., Mashburn, A., Hamre, B., & Pianta, R. (2008). Quality of Justice, L. M., McGinty, A. S., Beckman, A. R., & Kilday, C. R. (2006). Justice, L., Meier, J., & Walpole, S. (2005). Learning new words from Justice, L. M., & Vukelich, C. (2008). Achieving excellence in preschool Landry, S. H., Swank, P. R., Smith, K. E., Assel, M. A., & Gunnewig, Lomax, R. G., & McGee, L. M. (1987). Young children’s concept about Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically sup- MacGillivray, L., Ardell, A. L., Curwen, M. S., & Palma, J. (2004). Mashburn, A. J., Pianta, R. C., Hamre, B. K., Downer, J. T., Barbarin, Maxwell, K. L., Field, C. C., & Clifford, R. (2006). Defining and mea- Moustafa, M., & Land, R. (2002). The reading achievement of econom- National Early Literacy Panel. (2004, November). The National Early National Institute for Early Education Research. (2004). The state of O’Connor, R., Notari-Syverson, A., & Vadasy, P. F. (1996). Ladders to O’Connor, R., Notari-Syverson, A., & Vadasy, P. F. (2005). Ladders to O’Connor, R. E., Jenkins, J. R., Leicester, N., & Slocum, T. A. (1993). Peduzzi, P., Detre, K., Wittes, J., & Holford, T. (1991). Intent-to-treat Pence, K., Justice, L. M., & Wiggins, A. (2008). Preschool teachers’ Penno, J. F., Wilkinson, I. A. G., & Moore, D. W. (2002). Vocabulary Pianta, R. C. (2006). Teacher–child relationships and early literacy. In D. K. Preschool Curriculum Evaluation Research Consortium. (2008). Effects Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Raudenbush, S., Bryk, A., Cheong, Y., & Congdon, R. (2006). HLM6 Rimm-Kaufman, S. E., & Chiu, Y. J. I. (2007). Promoting social and Roskos, K., Rosemary, C. A., & Varner, M. H. (2006). Alignment Scarborough, H. S. (1998). Early identification of children at risk for Schaughency, E., & Ervin, R. (2006). Building capacity to implement and Scott-Little, C., Kagan, S. L., & Frelow, V. S. (2003). Creating the con- Stanovich, P. J., & Stanovich, K. E. (2003). Using research and reason in Storch, S. A., & Whitehurst, G. J. (2002). Oral language and code-related Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Rose, E., Lindamood, P., Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 175 http://ecrp.uiuc http://ncer.ed.gov http://nieer Ukrainetz, T. A., Cooney, M. H., Dyer, S. K., Kysar, A. J., & Harris, T. J. United States Census. (2004). State and County QuickFacts. Retrieved Universal Access to a Quality Early Education System, W. Va. Code van Kleeck, A., Gillam, R. B., Hamilton, L., & McGrath, C. (1997). The van Kleeck, A., Gillam, R. B., & McFadden, T. (1998). A study of van Kleeck, A., Vander Woude, J., & Hammett, L. (2006). Fostering van Teijlingen, E. R., & Hundley, V. (2001). The importance of pilot Vellutino, F., Scanlon, D., Sipay, E. R., Small, S. G., Pratt, A., Chen, R., Walpole, S., & Meyer, C. K. (2008). Models for coaching: Making them Wasik, B. A., Bond, M. A., & Hindman, A. (2006). The effects of a lan- Whitehurst, G., Arnold, D. S., Epstein, J. N., Angell, A. L., Smith, M., & Wiig, E. H., Secord, W. A., & Semel, E. (2004). Clinical Evaluation of Wixson, K. K., & Dutro, E. (1999). Standards for primary-grade reading: Yoshikawa, H., & Shinn, M. (2002). Facilitating change: Where and how Zaslow, M., & Martinez-Beck, I. (2006). Critical issues in early childhood Received May 26, 2008 Contact author: Laura Justice, Arps Hall, School of Teaching and 176 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010 mailto:justice.57@osu.edu http://quickfacts.census.gov/qfd/index.html APPENDIX A. “READ IT AGAIN!” SCOPE AND SEQUENCE OF INSTRUCTION
Instructional Domain 1: Vocabulary 1. To understand and use words for the names of unfamiliar objects (nouns) and actions (verbs) and that describe things and actions (adjectives and adverbs) Instructional Domain 2: Narrative 1. To identify and describe the setting and characters of a story Instructional Domain 3: Print Knowledge 1. To recognize that print carries meaning and to distinguish print from pictures Instructional Domain 4: Phonological Awareness 1. To identify when two words or sounds are the same (e.g., dog–dog, b–b) and when they are different (e.g., dog–man, d–m) Justice et al.: A Feasibility Study of a Preschool Curriculum Supplement 177 Weekl Before Reading: Phonological Awareness
Learning Objective 1: To identify when two words share a rhyming 1. Introduce the activity by saying: We are going to look at some pictures that 2. Show each of the “OG” picture cards (dog, frog, hog, and log), and have the
children name each card. Tell the children: All these words rhyme; tf:ey sound 3. Make some rhymes with the “OG” cards, and discuss these rhymes with the
children, as in: This picture is dog (show card) and it rhymes with frog (show 4. Hold all four cards in your hand, and allow children to select two cards from
your hand and say the two words on them. Then ask the whole group: Do During and After Reading: Narrative
Leaming Objective 2: To identify and describe the setting and 1. Read the book Clifford Goes to Dog School with the children. Stop reading 2. After reading the book, place the large paper where all children can see it. At 3. Review each of the key characters in the story. You could say: In our book 4. Go around the group of children and ask each child to tell you his/her favor Don’t forget to take a look for ideas about adapting
the Phonological diverse learners. Materials
• Book: Clifford Goes to • OG Picture Cards: • Large paper and
marker
Lesson Pla ns Read It Again-PreK! II myreaditagain.com
[ 23 ]
APPENDIX B. “READ IT AGAIN!” SAMPLE LESSON PLAN
178 LANGUAGE, SPEECH, AND HEARING SERVICES IN SCHOOLS � Vol. 41 � 161–178 � April 2010 Copyright of Language, Speech, & Hearing Services in Schools is the property of American Speech-Language-
Hearing Association and its content may not be copied or emailed to multiple sites or posted to a listserv
without the copyright holder’s express written permission. However, users may print, download, or email
articles for individual use.
Low Medium High
□ Treatment
children in 20 children in 20 (n = 103 children (n = 101 children
classrooms) classrooms) in 20 classrooms) in 19 classrooms)
Spring intercept, g00 2.86* 1.05 18 1.43** 0.46 18 13.17** 1.10 18 8.17** 0.61 17
Child-level condition
Age, g10 0.10 0.09 99 0.04 0.06 101 0.04 0.11 99 0.15** 0.06 97
Fall score, g20 0.42** 0.12 99 0.26* 0.12 101 0.82** 0.07 99 0.42** 0.08 97
Interaction model Condition × Language, g31 0.08 0.06 93 0.14** 0.04 93 0.17* 0.08 92 0.08* 0.04 89
provide a curriculum supplement that may be applied at scale with
relatively low costs and require minimal professional supports to
teachers for its effective use. Replication of this study under more
rigorous conditions is needed to establish the causal impacts that RIA
may have on young children’s language and literacy development.
However, the two major findings suggest that RIA may be a promising
curricular tool that is beneficial to children and yet easily accessible to
teachers and early childhood programs.
classrooms using the RIA curriculum supplement demonstrated signifi-
cantly higher end-of-year scores in language and emergent literacy
skills than did children in classrooms implementing business-as-usual
curricula, with significant effects ranging from d = .37 to d = 1.21
across skills examined. In interpreting these findings, it is important to
consider that this study relied on intact groups for its estimation of
program impacts and is a quasi-experimental, rather than experi-
mental, evaluation. In any quasi-experimental design, there are
potentially systematic differences across experimental and compari-
son groups (observed and unobserved) that may impact the results.
Thus, the positive effects of RIA on children’s gains in language
and literacy skills cannot be definitely attributed to RIA and could be
scores.
Despite this caution, consideration of the statistical differences be-
tween children in the RIA classrooms and comparison classrooms,
as well as the effect sizes of these differences, is informative for
determining whether RIA may warrant further investigation as a po-
tentially effective, low-cost, accessible alternative to existing lan-
guage and literacy curricula.
influence that RIA may have had on children’s skills and provide a
means of comparing findings across curricular intervention studies.
In fact, RIA achieved effect sizes equal to or larger in both magnitude
and scope to those reported for other curricular interventions, many of
which are more intensive and expensive in nature than RIA. For ex-
ample, Wasik et al. (2006) reported moderately large vocabulary
effects (d = .44) when implementing an experimental curricula fea-
turing 22 prop boxes and professional development in the form of
2-hr monthly workshops and 2-hr monthly in-class mentoring during
the implementation year. Assel et al. (2007) found smaller effects
(d = .36) when implementing an approximately $3,000-per-kit com-
mercial language and literacy curriculum and providing teachers
with a 4-day workshop coupled with 1.5-hr monthly in-class men-
toring. DeBaryshe and Gorecki (2007) found moderate effects on
knowledge scores.
Cl)
0
(.) 9 -C
~ 8
Low Medium High
□ Treatment
Ill Comparison
scores.
curriculum ($3,000 per kit) with weekly and monthly individual
teacher coaching. RIA matched or exceeded these effects when
teachers used the 30-week scope and sequence of lessons (commer-
cially costing approximately $100 in material) with little ongoing
professional support (e.g., 2 days in training, a semester of inde-
pendent lesson exposure/practice). When considering the compa-
rability of these findings, it is important to note that all effect-size
estimates are based on experimental or quasi-experimental group-
based intervention studies, and that effect size estimates standardized
measures of outcome, thus providing a statistically viable way of
comparing findings across studies and samples (Fan, 2001). We must
note, however, that it is unclear whether the effect-size contrasts
demonstrated in this study as well as others have meaningful and
functional impacts to children. Longitudinal research is needed to
show that relatively modest increases in children’s language and
literacy skills as we observed provide long-term advantages to fos-
tering academic and reading success in children.
possibility that curricular tools such as RIA, which has a focus and
scope based on research but a design that is pragmatic and amenable to
teachers, may offer a reasonable means for influencing teachers’ in-
structional practices and children’s skill development. Importantly,
these effect sizes suggest the potential for designing curricular tools
that may enhance classroom language and literacy instruction in
preschool classrooms (and thereby children’s skills within these class-
rooms) without the need for costly ongoing professional development
and coaching components emphasized in many current professional
development models (e.g., Assel, Landry, & Swank, 2008; Dickinson
& Brady, 2006; Hamre, LoCasale-Crouch, & Pianta, 2008; Pianta,
2006; Walpole & Meyer, 2008). That is, although models of coach-
ing teachers in language and literacy instructional practices have had
positive effects on teachers and students (e.g., Assel et al., 2007;
Wasik et al., 2006), this option may not be feasible or desirable for all
early childhood programs. Notably, the gap between the ideal of
intensive, ongoing professional development and the constraints on
the field are evident, with research studies reporting high attrition
and poor recruiting rates for professional development programs,
poor sustainability of curricular programs requiring significant pro-
fessional development, and a lack of widespread use of coaching in
(Assel et al., 2007; Dickinson & Brady, 2006; Jackson et al., 2006;
Maxwell, Field, & Clifford, 2006). Further, not all studies have
found coaching models to provide added benefit to teacher practices
and children’s outcomes when implemented in conjunction with
other forms of professional development (Assel et al., 2007; Jackson
et al., 2006). The findings of this study suggest that highly suppor-
tive curricular materials that are scripted, sequenced, self-contained
(i.e., a manual and books), and limited in scope (i.e., limited to a single
context, namely, book reading) may represent a valid option for
improving the language and literacy instruction of preschool class-
rooms. The potential of RIA as a low-cost, high-yield alternative to
higher cost curricula and professional development models (e.g.,
in-class coaches) suggests the need to continue to investigate RIA to
determine whether findings from this study replicate under more
tightly controlled, experimental conditions. Additionally, researchers
should consider how RIA—implemented with and without ongoing
professional development supports—might influence teacher and child
outcomes relative to other presently available practices and curricula.
plicable to the diverse learning needs of young children in early
childhood programs, this study also considered whether RIA effects
varied by children’s language skill at the beginning of the preschool
year. Understanding whether and how RIA may have varied in its
effect on children is important to understanding how well a single
curricular program such as RIA may sufficiently meet the diverse
instructional needs of young children. Our findings indicated that
RIA equally benefitted children’s vocabulary and syntactic growth,
regardless of their initial language ability, but that RIA had a more
variable influence on children’s emergent literacy skills growth as a
function of their initial language. Specifically, RIA had a positive
effect on children’s rhyme development, regardless of their initial
language skill, but was more beneficial to the print concepts devel-
opment of children with higher language skills as compared to lower
language skills. Further, the effect of RIA on children’s alliteration
and alphabet knowledge was notably different based on children’s
initial language ability. Findings suggest that RIA may have had a
positive effect on the alliteration development only of children
with relatively average or high initial language abilities and may
have had a positive effect on the alphabet knowledge only of
children with relatively high initial language abilities. Our findings
that children with weaker language skill may have benefitted less
comprehensively or strongly from RIA is consistent with studies
showing that children with weak language demonstrate slower and
more variable growth in language and literacy interventions when
compared to children with more average skills (O’Connor et al.,
1996; Penno et al., 2002; Ukrainetz et al., 2000, but see Justice,
Meier, & Walpole, 2005). However, it is also important to note that
for many skills (e.g., vocabulary, grammar/morphology, and rhyme),
RIA had an equal effect on children’s growth, regardless of variation
in children’s language ability. Collectively, our findings point to the
importance of considering factors related to individual variability in
children’s intervention responsiveness and emphasize the point that
no one curricular tool or approach may be sufficient for building
a classroom that effectively supports all of the learning needs of all
of the children. Further, our data suggest that it is unlikely that any
particular group of children may benefit or fail to benefit from an
intervention, but that it is important to consider the “match” between
a particular curricular tool, the outcomes of interest, and the char-
acteristics of the children in the classroom.
appear to extend across many skill areas for children with diverse
language abilities. Nonetheless, its use failed to significantly affect
the literacy skills (alliteration, alphabet knowledge, print concepts)
of children with very weak language skills. Exploring whether RIA
may more comprehensively reach the needs of more vulnerable chil-
dren (such as children with very weak language skills) when imple-
mented in a different intensity level or in a different instructional
context is an important direction for future research in order to inform
classroom practices regarding the use of RIA for
directions for future research. First, as we have noted, the current
study employed a quasi-experimental design to study the potential
impacts of the RIA curriculum supplement on the language and liter-
acy skills of children, providing a weaker form of causal evidence
relative to true effect studies due to lack of random assignment and
other internal controls. Of particular concern, and one that relates
to the quasi-experimental nature of this study, is the difference in the
educational levels of the teachers who implemented RIA as com-
pared to those in the comparison classrooms. Descriptive data showed
that RIA teachers were better educated than the comparison teachers,
and it is possible that improvements in child outcomes that we are
attributing to RIA impacts may, in fact, reflect the higher educational
status of the teachers. Consequently, the results from the current study
require further replication in more rigorous research designs and
within a range of real-world contexts to increase confidence. Use of
a fully randomized design that can therefore control for any effects
attributable to teacher education is an important next step. Until such
replication occurs, RIA must not be considered a fully validated em-
pirical curricular tool, but a potentially promising curricular tool
that balances the language and emergent literacy learning needs of
young children with the needs of early educators to have a low-cost
and easily accessible curricular option.
given the relatively small sample size and limited scope of teacher
and child heterogeneity. The children in this study were not repre-
sentative of the fairly ethnically diverse preschoolers who are served
by many preschool programs. For example, our sample did not in-
clude any children who were English language learners. Replicating
this study’s findings within samples that allow for consideration
of additional child characteristics such as English language learner
status is an important step in building the field’s knowledge of op-
timally designed early intervention programs suited to the hetero-
geneous needs of preschool-age children.
part of the development process of the RIA program. All teachers in
this study used a pilot version of the RIA program for 15 weeks in
the year previous to this study. Although the final RIA program was
different from the pilot program, the teachers in this study could
be considered as having prior experience with RIA. Intervention
studies have not found consistent differences between teachers who
were second-year implementers of a program versus teachers who
were participating in a program for the first time (Fischel et al., 2007;
Landry, Swank, Smith, Assel, & Gunnewig, 2006). However, whether
the success of RIA was related to teachers having some implementation
experience gained during their use of the pilot version of the program
is a question to be addressed in future program evaluation studies.
practice gap must be “adaptive” to the context in which implemen-
tation occurs and must be cognizant of the capacity of the current
system so as to achieve incremental change that is sustainable
(Fullan, 2000; Yoshikawa & Shinn, 2002). Presently, the field of
early education is in a state of substantial capacity building with the
aim of promoting higher quality and quantity of language and liter-
acy instruction within preschool classrooms. We contributed to
this capacity building by studying a specific curriculum that may
effectively support children and yet also effectively bridge the
research-to-practice gap by being easily accessible to preschool
teachers. Specifically, we examined the effectiveness of a practical
tool that was designed to meet the needs of early educators who work
in challenging contexts that preclude the short-term scalability of
some currently available high-cost programs and practices requiring
intensive mentorship for successful implementation. RIA offers
preschool educators a potentially high-impact instructional tool for
facilitating children’s growth in language and literacy. Although
building globally high-quality teachers likely requires a sustained
and multipronged approach (Zaslow & Martinez-Beck, 2006), our
findings suggest that improvement of a specific aspect of the class-
room, namely, language and literacy instruction, may be aided in
the short term by a stand-alone curriculum supplement. In particular,
RIA offers a model of a curriculum supplement that is responsive to
children’s learning needs and is cognizant of the needs and con-
straints of the teachers and early childhood programs in which they
work. The promising findings of the benefits of RIA to children
suggest the need to consider the value of designing curricular tools
with issues of scalability in mind and of the need to evaluate RIA
as such a tool, specifically.
Education and in part by the U.S. Department of Education, Institutes for
Education Sciences Award Grant R305F05124. These data were collected
while the second author was on a Predoctoral Training Fellowship awarded to
the University of Virginia Interdisciplinary Doctoral Training Program in
Education Sciences, supported by the Institute of Education Sciences U.S.
Department of Education Award R305B040049. The authors are grateful to
the administrators, teachers, pupils, and research assistants who participated
in this study.
research on Head Start outcomes and program quality. Washington, DC:
Department of Health and Human Services. Retrieved from http://www.
acf.hhs.gov/programs/opre/hs/faces/reports/faces_findings_06/faces_
findings_bw .
classrooms preparing children to be school ready? The CIRCLE Teacher
Behavior Rating Scale. In L. M. Justice & C. Vukelich (Eds.), Achieving
excellence in preschool literacy instruction (pp. 120–135). New York,
NY: Guilford Press.
evaluation of curriculum, setting, and mentoring on the performance of
children enrolled in pre-kindergarten. Reading and Writing, 20, 463–494.
children: Effects on reading achievement and related outcomes. In S. B.
Neuman & D. K. Dickinson (Eds.), Handbook of early literacy research
(pp. 421–443). New York, NY: Guilford Press.
teacher crisis (Preschool Policy Brief No. 3). New Brunswick, NJ:
National Institute for Early Education Research, Rutgers University.
Inside the pre-K classroom: A study of staffing and stability in state-
funded prekindergarten programs. Washington, DC: Center for the Child
Care WorkforceRetrieved from http://www.ccw.org/pubs/ccw_pre-k_10.
4.02 .
C. E. (2008). Executive functions and school readiness intervention:
Impact, moderation, and mediation in the Head Start REDI program.
Development and Psychopathology, 20(3), 821–843.
framework for early literacy instruction: Aligning standards to develop-
mental accomplishments and student behaviors. Pre-K through kinder-
garten (rev. ed.). Aurora, CO: Mid-continent Research for Education and
Learning Lab. (ERIC Document Reproduction Service No. ED465183)
Madden, N. A., & Chambers, B. (2005a). The national randomized field
trial of Success for All: Second-year outcomes. American Educational
Research Journal, 42, 673–696.
N. A., & Chambers, B. (2005b). Success for All: First-year results from
the national randomized field trial. Educational Evaluation and Policy
Analysis, 27(1), 1–22.
characteristics of the United States early care and education workforce. In
M. Zaslow & I. Martinez-Beck (Eds.), Critical issues in early childhood
professional development (pp. 49–76). Baltimore, MD: Brookes.
Barbarin, O., . . . Burchinal, M. (2002). Diversity and directions in state
pre-kindergarten programs. Chapel Hill, NC: The University of North
Carolina, FPG Child Development Institute, NCEDL.
preschool classroom (Vol. 2). Baltimore, MD: Brookes.
teach phonemic awareness to young children: A 1-year follow-up.
Journal of Educational Psychology, 85, 104–111.
Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evalu-
ation of complex interventions to improve health. British Medical
Journal, 321, 694–696.
implementation research: Estimation of intracluster correlation coeffi-
cients and sample size. Statistics in Medicine, 20, 391–399.
literacy: Preschool through the elementary grades. New York, NY: Allyn
& Bacon.
struction and children’s emergent literacy growth. Journal of Educational
Psychology, 98, 665–689.
strategies that accommodate diverse learners (3rd ed.). Columbus, OH:
Pearson Merrill Prentice Hall.
a preschool emergent literacy curriculum. Early Education and Devel-
opment, 18, 93–110.
guage and literacy through professional development. In M. Zaslow &
I. Martinez-Beck (Eds.), Critical issues in early childhood professional
development (pp. 141–170). Baltimore, MD: Brookes.
for Preschoolers. Washington, DC: Teaching Strategies.
R., . . . Barnett, W. S. (2005). NCEDL working paper pre-kindergarten
in eleven states: NCEDL’s multi-state study of pre-kindergarten & study
of state-wide early education programs (SWEEP) preliminary descrip-
tive report. Retrieved from http://www.fpg.unc.edu/(ncedl/pdfs/
SWEEP_MS_summary_final .
Development. (2000a). Individual Growth and Development Indicators
for Preschool Children: Alliteration/Early Literacy. Minneapolis, MN:
Center for Early Education and Development, University of Minnesota.
(Available online at http://ggg.umn.edu).
Development. (2000b). Individual Growth and Development Indicators
for Preschool Children: Rhyming/Early Literacy. Minneapolis, MN:
Center for Early Education and Development, University of Minnesota
(Available online at http://ggg.umn.edu).
Journal of Educational Research, 94(5), 275–282.
and phonological awareness in low-income children with and without
specific language impairment. Journal of Speech, Language, and
Hearing Research, 40, 1285–1297.
& Shaller, G. (2007). Evaluation of curricular approaches to enhance
preschool early literacy skills. Journal of Literacy Research, 39, 471–501.
and review of caregiver training studies. Early Childhood Research
Quarterly, 22, 294–311.
Change, 1, 5–27.
tional Children, 70, 453–468.
sessment of classrooms. In L. M. Justice & C. Vukelich (Eds.), Achieving
excellence in preschool literacy instruction (pp. 102–119). New York,
NY: Guilford Press.
reading first: Final report to Congress (NCEE 2007-4007). Washington,
DC: U.S. Department of Education National Center for Education
Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/
ncee/pdf/20074007 .
logical Awareness Literacy Screening: Preschool. Charlottesville, VA:
University of Virginia.
(2006). The impact of HeadsUp! Reading on early childhood educators’
literacy practices and preschool children’s literacy skills. Early Child-
hood Research Quarterly, 21, 213–226.
attainment of print-concept knowledge: A study of typical and at-risk
3- to 5-year-old children using item response theory. Language, Speech,
and Hearing Services in Schools, 37, 224–235.
Emergent literacy intervention for vulnerable preschoolers: Relative
12, 320–332.
old children. Child Language Teaching and Therapy, 13, 207–225.
acy enhancement strategy and its clinical implications. Language, Speech,
and Hearing Services in Schools, 35, 185–193.
intervention I: Background and description of approach. Language, Speech,
and Hearing Services in Schools, 35, 201–211.
language and literacy instruction in preschool classrooms serving at-risk
pupils. Early Childhood Research Quarterly, 23, 51–68.
Read It Again! Language and literacy supplement for preschool pro-
grams. Charlottesville, VA: Preschool Language and Literacy Lab, Center
for Advanced Study of Teaching and Learning, University of Virginia.
storybooks: An efficacy study with at-risk kindergartners. Language,
Speech, and Hearing Services in Schools, 36, 17–32.
literacy instruction. New York, NY: Guilford Press.
S. B. (2006). Enhancing early literacy skills for preschool children:
Bringing a professional development model to scale. Journal of Learning
Disabilities, 39, 306–324.
print and reading: Toward a model of word reading acquisition. Reading
Research Quarterly, 22, 237–256.
ported psychosocial interventions for children: An overview. Journal of
Clinical Child Psychology, 27, 138–145.
Colonized teachers: Examining the implementation of a scripted reading
program. Teaching Education, 15(2), 131–144.
O. A., Bryant, D., . . . Howes, C. (2008). Measures of classroom
quality in prekindergarten and children’s development of academic,
language, and social skills. Child Development, 79(3), 732–749.
suring professional development in early childhood research. In M. Zaslow
& I. Martinez-Beck (Eds.), Critical issues in early childhood professional
development (pp. 21–44). Baltimore, MD: Brookes.
ically disadvantaged children in urban schools using Open Court. In Year-
book of the Urban Learning, Teaching, and Research Special Interest
Group of the American Educational Research Association (pp. 44–53).
New York, NY: McGraw-Hill.
Literacy Panel: A research synthesis on early literacy development. Paper
presented at the National Association of Early Childhood Specialists,
Anaheim, CA.
preschool: 2004 state preschool yearbook. Retrieved from http://nieer.
org/yearbook/pdf/yearbook #page=6.
Literacy: The effects of teacher-led phonological activities for kinder-
garten children with and without disabilities. Exceptional Children, 63,
117–130.
Literacy: A kindergarten activity book (2nd ed.). Baltimore, MD: Brookes.
Teaching phonological awareness to young children with learning dis-
abilities. Exceptional Children, 59, 532–546.
analysis and the problem of crossovers. An example from the Veterans
Administration coronary bypass surgery study. The Journal of Thoracic
and Cardiovascular Surgery, 101, 481–487.
fidelity of implementation for a language-rich preschool curriculum.
Language, Speech, and Hearing Services in Schools, 39, 1–14.
acquisition from teacher explanation and repeated listening to stories: Do
they overcome the Matthew effect? Journal of Educational Psychology,
94, 23–33.
Dickinson & S. B. Neuman (Eds.), Handbook of early literacy research
(Vol. 2, pp. 149–162). New York, NY: Guilford Press.
of preschool curriculum programs on school readiness (NCER 2008-2009).
Washington DC: National Center for Education Research, Institute of
Education Sciences, U.S. Department of EducationRetrieved from
http://ncer.ed.gov.
Applications and data analysis methods. Thousand Oaks, CA: Sage
Publications.
[Computer software]. Lincolnwood, IL: Scientific Software International.
academic competence in the classroom: An intervention study examining
the contribution of the “Responsive Classroom” approach. Psychology in
the Schools, 44(4), 397–413.
in educator preparation for early and beginning literacy instruction.
In M. Zaslow & I. Martinez-Beck (Eds.), Critical issues in early
childhood professional development (pp. 255–279). Baltimore,
MD: Brookes.
reading disabilities: Phonological awareness and some other promising
predictors. In B. K. Shapiro, P. J. Accardo, & A. J. Capute (Eds.), Specific
reading disabilities: A view of the spectrum (pp. 75–119). Timonium,
MD: York Press.
sustain effective practices to better serve children. School Psychology
Review, 35(2), 155–166.
ditions for success with early learning standards: Results from a national
study of state-level standards for children’s learning prior to kindergarten.
Early Childhood Research & Practice, 5. Retrieved from http://ecrp.uiuc.
edu/v5n2/little.html.
education: How teachers can use scientifically based research to make
curricular & instructional decisions. Washington, DC: The Partnership
for Reading, National Institute for Literacy, RMC Research Corporation.
(ERIC Document Reproduction Service No. ED482973)
precursors to reading: Evidence from a longitudinal structural model.
Developmental Psychology, 38, 934–947.
Conway, T., . . . Garvan, C. (1999). Preventing reading failure in
young children with phonological processing disabilities: Group and
individual responses to instruction. Journal of Educational Psychology,
91, 579–593.
(2000). An investigation into teaching phonemic awareness through
shared reading and writing. Early Childhood Research Quarterly, 15,
331–355.
from http://quickfacts.census.gov/qfd/index.html.
§18-5-44 (2002).
relationship between middle-class parents’ book-sharing discussion and
their preschoolers’ abstract language development. Journal of Speech,
Language, and Hearing Research, 40(6), 1261–1271.
classroom-based phonological awareness training for preschoolers with
speech and/or language disorders. American Journal of Speech-Language
Pathology, 7(3), 65–76.
literal and inferential language skills in Head Start preschoolers with
language impairment using scripted book-sharing discussions. American
Journal of Speech-Language Pathology, 15, 85–95.
studies. Social Research Update, 35, 1–5.
. . . Denckla, M. (1996). Cognitive profiles of difficult-to-remediate and
readily remediated poor readers: Early intervention as a vehicle for
distinguishing between cognitive and experiential deficits as basic causes
of specific reading disability. Journal of Educational Psychology, 88,
601–638.
work for preschools. In L. M. Justice & C. Vukelich (Eds.), Achieving
excellence in preschool literacy instruction (pp. 69–83). New York, NY:
Guilford Press.
guage and literacy intervention on Head Start children and teachers.
Journal of Educational Psychology, 98, 63–74.
Fischel, J. E. (1994). A picture book reading intervention in day care and
home for children from low-income families. Developmental Psychology,
30, 679–689.
Language Fundamentals Preschool (2nd ed.). San Antonio, TX:
Harcourt Assessment.
An analysis of state frameworks. The Elementary School Journal, 100(2),
89–110.
should community psychology intervene. In T. A. Revenson, D. Livert,
A. R. D’Augelli, E. Seidman, S. E. French, M. Shinn, . . . H. Yoshikawa
(Eds.), A quarter century of community psychology: Readings from the
American Journal of Community Psychology (pp. 33–49). New York,
NY: Kluwer/Plenum.
professional development. Baltimore, MD: Brookes.
Revision received November 7, 2008
Accepted February 2, 2009
DOI: 10.1044/0161-1461(2009/08-0058)
Learning, The Ohio State University, Columbus, OH 43210.
E-mail: justice.57@osu.edu.
Objectives
2. To understand and use new words representing spatial concepts (e.g., over, under, above)
3. To understand and use new words representing time concepts (e.g., before, after, then)
4. To talk about the meaning of new words, including how words can have more than one meaning
5. To understand and use new words representing feelings (e.g., embarrassed, sad, joyful)
6. To understand and use new words representing thinking processes (e.g., believe, imagine)
Objectives
2. To identify and describe one or more major action(s) or event(s) in a story
3. To order three or more major events in a story
4. To produce a fictional story that has a setting and characters
5. To produce a fictional or personal story that has a clear beginning, middle, and end
6. To share feelings, ideas, or experiences in a single story that is precise and understandable
Objectives
2. To recognize the left-to-right and top-to-bottom directionality of print
3. To identify some uppercase letters, including those in own names and those of some friends or family members
4. To understand and use new words describing aspects of books (e.g., illustrator, author, cover, title page) and print (e.g., word, letter, spell, read, write)
5. To recognize the difference between letters and words
6. To recognize some common sight words, including environmental print
Objectives
2. To identify when two words share a rhyming pattern
3. To produce words that share a rhyming pattern
4. To segment words into syllables and to blend syllables into words
5. To identify when two words share the same first sound
6. To produce a word starting with a specific first sound
Lesson 2: Which words sound the same?
Book: Clifford Goes to Dog School by Norman Bridwell
pattern.
rhyme. I’ll say the name of the picture and you say it after me.
the same at the end. See how my mouth is the same at the end?
card). My mouth does the same thing at the end: frog, dog. Continue this process
for other pairs (dog-log, dog-hog).
(word) and (word) sound the same? Does your mouth do the same thing?
characters of a story.
periodically to higf1light the character and the setting in the book. Ask chil
dren open-ended questions about the characters, such as: Why did he do that?
What will he do next? Also, describe any changes that happen in the setting,
such as: Clifford was outside.
the top write the word: Characters.
we met Clifford, Emily Elizabeth, and Auntie. Write each of the names on the
sheet, leaving lots of space between names.
ite character and dictate wiry. Record children’s answers below the character
name. Allow children’s answers to guide what you write, but extend their
answers. If the child says, Clifford is good, you could extend this answer, as
in: Alex said he likes Clifford the best because he is a good dog and was
looking out for Emily Elizabeth.
at the Learners’ Ladder
Awareness activity to
Dog School, by
Norman Bridwell
dog, frog, hog, log