31st Century Community Learning Centers: Evaluation and Implementation Issues
CRS Report for Congress
Received through t he CRS W e b
st
21 Century Community Learning Centers:
Evaluation and Implementation Issues
December16,2003
GailMcCallion
Specialist in Labor Economics
Domestic Social Policy Division
Congressional Research Service ˜ The Library of Congress
Evaluation and Implementation Issues
Summary
The 2 1 st Century C ommunity Learning Centers (21st C C LC) program, as
reauthoriz ed in the No C hild Left Behind Act o f 2001 (P.L. 107-110), and sign ed into
law on J anuary 8, 2002, is structured as formula grants to states, with local grants
awarded competitivel y by states t o eligible local entities for a period of 3 to 5 years.
Recipients of 21st CCLC grants are to use t hem for out of school time activities t hat
advance s t udent academ i c achi evem ent . T he program ’s focus i s now ex cl usi v el y o n
thes e activities for children and youth and on educational activities for thei r families.
In creasing public and congressional attention h as been focused o n t he program,
i n part because t h e 2 1st CCLC p rogram h a s g r o w n rapidly — from $750,000 in
funding in its first year, FY1995, to $1 billion i n FY2004. Public support for after
school programs is very strong; however, a recent evaluation o f t he 21st CCLC
program b ased on data from t he 2000-2001 school year, conducted f o r t h e U .S.
Depart m ent of Educat i on, di d not fi nd si gn i fi cant i m p rovem ent s i n m ost academ i c
outcomes.
The reaut hori z ed 21 st C C LC p rogram requi res s t at e and l ocal reci pi ent s t o use
funds to help students acquire the s kills they need in order t o m eet s t a t e academic
achi evem ent st andards, em pl oyi n g s ci ent i fi cal l y based research where rel evant ; and
t o dem ons t rat e t he success o f t hei r program s i n achi evi ng academ i c i m p rovem ent .
Several i ssues have arisen regarding implementation of t he reauthoriz ed program.
Some of the hurdles that states and p rogram implementers m ust confront in achieving
academ i c go a l s i ncl ude: t he di ffi cul t y i n securi ng part i ci p at i o n b y s t udent s
(particularly middle s chooler s ) in a p rogram focused o n academics; the relatively
limited amount o f p r o g r a m time per s tudent on average, in which t o m ake a
difference i n academic achievement; and the v ast d ivergence i n after school program
content and go als.
Although t here is limited research to date on the 2 1st C C LC p rogr am i t s el f, t h ere
i s a n ex tensive and growing body of research on after s chool progra m s m o r e
generally. This res earch has provided m any ex amples of promising practices in after
school programs that focus o n academic improvement. S ome o f t he elements found
prom i s i n g for program s uccess are wel l qual i fi ed and ex t ensi v el y t rai n ed st aff, a
connection t o t he school day curriculum, opportunities for one on one tutoring, and
a s tructure design ed to achieve clear go als. This report will not be updated.
Contents
In troduction ..................................................1
Background ..................................................1
Accountability Requirements .....................................3
EvaluationIssues ..............................................4
TheMathematicaStudy .....................................5
ImplementationIssues ..........................................7
AcademicImprovement .....................................7
Program S ustainability ......................................9
Directions fortheFuture .......................................10
ListofTables
Table 1 . 2 1 st Century Learning Centers: Funding History ..................3
st
21 Century C ommunity Learning Centers:
Evaluation and Implementation Issues
Introduction
The 2 1 st CCLC program was origi nal l y authorized as Part I of Title X, of the
Elementary and S econdary Education A ct (ESEA), as amended. The amendment
authoriz ing t he 21st CCLC p rogram was i ncluded as p art o f t he Im proving America’sst
Schools Act of 1994, P.L. 103-382. The 2 1 CCLC program was i nitially authorized
for 5 years, FY1995-FY1999. The 2 1st C C LC p rogram was not reaut hori z ed i n t h eth
106 Congress, and consequently its authorization (but not its funding) ex p ired at the
end o f FY2000. This program was reauthoriz ed as part of the reauthoriz ation o f t he
ESEA by the No C hild Left Behind Act o f 2001, P.L. 107-110, and was sign ed into
law o n J anuary 8, 2002.
In creasing public and congressional attention h as been focused o n t he program,st
in part bec a u s e t h e 2 1 CCLC p rogram has grown rapidly — from $750,000 in
funding in its first year, FY1995, to $1 billion i n FY2004 (See Table 1 for t he
program’s funding history). P ublic support for after s chool programs is very strong,
but fi rst year resul t s from a recent eval u at i o n o f t he 21st CCLC p rogram did not find
si gn i fi cant i m p act s o n m ost academ i c out com es o r o n n u m b ers of l at chkey k i d s
(children i n s el f-care). This report discusses implementation of t he reauthorized 21 st
C C LC p rogram , and t h e recent eval u at i o n o f t he program and i t s i m p l i cat i ons.
Background
Originally, t he 21st CCLC program was a competitive grant program with
grant ees sel ect ed by t h e U.S . Depart m ent of Educat i o n ( E D ) . A w ards were for up
to 3 years and were required t o i n c l u d e at l eas t 4 out of 13 potential activities
intended t o serve th e l o c al community.1 Funds were required t o b e equitably
distributed among urban and rural areas across t he nation, among the s tates, and
among rural and urban areas within states . The program s hifted in emphasis as t he
amount appropriated for the p rogram increased. The original authoriz ing l angu age
required t he Secretary o f ED t o give p riority to those 2 1st C C LC p rojects t hat “offer
a b road sel ect i o n o f s ervi ces whi ch address t he needs o f t he com m uni t y.” 2 Begi nning
w i t h t h e p r o gr a m ’ s s i gn i f i c a n t e x p ansion in FY1998, a n a d d itional p riority was added
1 For a list of t hese 13 activities s ee 20 U.S.C. 8245.
2 20 U.S.C. 8243.
for: “activities t hat o ffer ex p anded l earni ng opportunities for children and youth i n
the community and t hat contribute t o reduced drug use and violence.”3
In contrast, t he 21st CCLC p ro gr a m a s reauthoriz ed in P.L. 107-110, is
s t ruct ured as formula grants t o states, with local grants awarded competitivel y b y
states to local entities for a p eri o d o f 3 t o 5 years. States are allocated funds in
proportion t o t he awards they received under S ubpart 2 o f Title I-A o f t he ESEA for
t h e p recedi n g year.
State e d u c ational agencies (SEAs) m ust award at least 95% of their s tate
allotment to eligible local entities (defined as l ocal educational a gen c i es (LEAs),
community based organizations (CBOs), other public or privat e entities, or consortia
of one or more of the above). To the ex t e n t p o s s i ble, SEAs are t o d istribute funds
equitably a m o ng geographic areas within the s tate, i ncluding urban and rural
communities. SEAs may only award grants to eligible en t ities t hat will be serving
students who attend schools eligible for schoolwide programs under S ection 1114 of
th e E S EA, or schools t hat s erve a h igh p ercentage of students from l ow income
families, and t he families of t hese students. 4 Stat es are t o give priority to applications
t h a t propose t o t arget s ervices to students who attend schools t hat h ave b e e n
identified as i n n eed of improvement under S ection 1116 of the ESEA (schools t hat
fail to make adequate yearly progress for 2 consecutive years by s tate meas ures ) and
are s ubmitted j ointly by an LEA and a C BO or other public or private entity. 5
Recipients of 21st CCLC grants are to use t hese funds for b efore and after s chool
activities t hat advance s tudent academic achievement.6 The p rogram’s focus i s now
ex clusivel y o n t h e s e before and after school activities for children and youth, and
educational activities for thei r families. The s tated purposes o f t h e p rogram as
reaut hori z ed, are t h reefol d:
! Provide opportunities for academic enrichment to help students
(particularly those attending low-performing s chools) to meet state
and l ocal st udent academ i c achi evem ent st andards;
! Offer s tudents a wide variety o f additional s ervices, p rograms and
activities i ntended t o rei nforce and complem ent t heir regular
academ i c program ; and
3 For ongoing updated i nforma tion on t he status of the progr am see CRS Report RL31240,
21 st Century Community Learning Centers i n P.L. 107-110: Background and Funding,by
GailMcCallion.
4 In ge neral, schools e ligible for s choolwide programs under Section 1114 of the ESEA, are
those public schools where 40% or more of the s tudents are from l ow income families.
5 An exception i s made i f an LEA demonstrates that it is unable t o partner with a CBO of
sufficient quality and r easonable geogr aphic proximity.
6 T hroughout this report, before and after school activities and out of school time activities
are used s ynonymously. 21st CCLC f unds can be used for activities before and after s chool,
during s chool breaks a nd during t he summe r.
! Offer families o f s t u dents s erved an opportunity for literacy and
related educational d evelopment.7
Table 1. 21st Century L earning Centers: Funding History
P resident’s budget
FY request ( i n $) Appropri at i on ( i n $)
19950750,000
19960750,000
199701,000,000
1998 50,000,000 40,000,000
1999 200,000,000 200,000,000
2002 845,614,000 1,000,000,000
a. T his amount includ es a r e s c i ssi o n of FY2000 discretio nary budget authority required by the
FY2000 appropriatio ns act (P.L. 106-113).
b. T his amount includes an across the board rescission of FY2001 appropriatio ns adopted in the
Miscellaneous Appropriatio ns Ac t ( H.R. 5666) enacted into law b y T he Co nsolid ated
Appropriatio ns Act for FY2001 (P.L. 106-554).
c. T his amount includ es an across the board reductio n p er P.L. 108-7.
d. T his amount has b een approved by conferees, b ut has not yet b een finalized. I t may be subj ect to
an across the board reductio n o f 0 .59%.
Accountability Requirements
The reaut hori z ed 21 st C C LC p rogram requi res s t at e and l ocal reci pi ent s t o use
funds to help students acquire the s k ills they need in order to meet state standards,8
em pl oyi n g s ci ent i fi cal l y based research; and t o d em onst rat e t he success o f t hei r
program s i n achi evi ng academ i c i m p rovem ent .
States are required t o conduct comprehen sive evaluat i o n s o f local programs’
effect i v eness u si ng perform ance i ndi cat ors and m easures t h ey have d e v e l o p ed for
t h at purpose. To m eet t h e p ri nci p l es o f effect i v eness, a p rogram m u st :
7 For a list of t hese activities s ee 20 U.S.C. 7175.
8 Scientifically based r esearch is defined i n 20 U.S.C. 7801.
(A) b e b ased upon an assessment o f obj ective d ata regarding the n eed for
before and after school programs (including during s ummer recess p eriods)
and activities i n t he schools and communities;
(B) b e b ased upon an established s et of performance meas u r e s aimed at
ensuring the availability of high -quality academic enrichment
opportunities; and
(C) i f appropriate, b e b ased upon scientifi cal l y based research t h at provi des
evidence that the program or activity will help students m eet the S tate and
l o cal st udent academ i c achi evem ent st andards.9 Each l o cal p rogram i s
required t o p articipate i n p eriodic evaluations based o n t hese principles of
effect i v eness “t o assess i t s progress toward achieving its go al of providing
high quality opportunities for academic enrichment”; and t he evaluation’s
resultsmustbe:
(i) u sed t o re f i n e, improve, and strengthen the p rogram or
activity, and to refine the performance meas ures ; and (ii) made
available t o t he public upon request, with public notice of s uch
availability provided.10
Evaluation Issues
Most of the research into after s chool programs has ex amined after s chool
programs in general, rather than 21 st C C LC p rogram s i n p art i cul ar, i n p a r t b ecause
of the rel ative newness of the 21 st CCLC p r o gr a m . T he research often focuses on
individual p rograms t hat p rovide high quality out of school time services to a group
of children or youth over a n e x t en d e d period of time. In general , res earch that
employs random assign ment methods is considered th e m o s t rigorous. However,
becau s e o f t h e r ari t y of such research desi gn s i n t he fi el d, m any surveys o f aft er
school progr a m s d iscuss at least s ome p rograms t hat d id not employ random
assignment in order t o be able t o ex amine evidence from t he field o n what elements
in programs are m ost p romising. There h ave b e e n many surveys o f t his quasi-
ex perimental and cas e s tudy research that have indicat ed positive effect s from after
school programs on a v ariety of outcomes.11
9 20 U.S.C. 7175. Also, U.S. Department of Education, 21st Century Community Learning
Centers Non-Regul atory Guidance , Feb. 2003.
10 20 U.S.C. 7175. Local gr antees are a lso r equired t o s ubmit Annual Performance Reports
that provide detailed i nformation about the p rogr am. Information, including APR f orms is
a va i l a bl e o n t he De pa r t me nt of Educ a t i on’ s we b s i t e [ h t t p : / / www.e d.gov] . S e e a l s ost
Evaluation of 21 Century Community Learning Ce nter Programs A Guide f or S t ate
Education Agencies , H arvard Family Research Proj ect, Harvard Graduate School of
Education,2003.
11 A Review of Out-of-School Ti me Program Quasi - E x p e ri mental and Experimental
Evaluation Results, Harva rd Family Research Proj ect, Harva rd Gr a d u a t e School of
Education, 2003. T his discussion is only illustrative of t he extensive literature evaluating
after s chool and youth deve l o pme nt programs . For revi ews of progr ams l ooking a t t he
impact on youth development s ee Ri chard Catalano, et al., Positive Y outh Development i n
the United States: Research Fi ndings on Evaluations of Positive Y o u t h D e v elopment
(continued...)
O n e e x ample of this research is published by t he Harvard Family Research
Project (HFR P). The HFRP maintains a database of evaluations on the impact of out
of school time programs on outcomes related to academic achievement, p revention,
and yout h d evel opm ent . Drawi n g o n i t s dat abase i n a recent report , t h e HFR P
sel ect ed 27 eval uat i ons t h at em pl oyed ei t h er ex peri m ent al or quasi - e x p e ri m ent al
design s for part or all o f t heir re s e arch. Twenty-five of these evaluations assessed
academic outcomes. The d ata f r o m t hese evaluations indicated that out-of-school
time programs resulted i n:
! Better attitudes t oward s chool and h igher educational aspirations.
! Better p erformance in school, as m easured by achievement test
scores and grades.
! High er school attendance (as measur ed by attendance and tardiness).
! Less disciplinary action (e.g. , s uspension).12
The Mathematica S tudy. The U.S. Department of Educat i o n contractedst
with Mathem atica for both an implementation and an i m pact study of 21 CCLC
after-school programs. T he Mathematica s tudy is the m ost rigorous s t u d y t o d atest
speci fi cal l y focused o n t he 21 CCLC program . The Mathem atica study is a m ulti-
year study based o n b aseline and follow u p d ata coll e c t e d on 4,400 middle s chool
students and 1,000 (in t he first year) element ary s chool students. Data are collected
from s tudents, teachers, principals, p rogram staff m embers and s chool records. The
first results from t he evaluation were published i n February 2003. Based o n 1 year
of data for t he 2000-2001 school year, t he eva l u a t i o n d id not find sign ificantst
improvements from 2 1 C C LC p rogram s i n m ost academ i c out com es o r i n t he
numbers of latchkey k ids (children i n s elf- care). T he study was d esigned t o focus onst
out com es o f t yp i cal 21 CCLC program s, rather than of programs implementing bes t
pract i ces. 13
The middle s chool sample was b ased on data for 4,400 students. Because these
middle s chool programs were undersubscribed, random sampling was not used.
11 (...continued)
Programs , U.S. Department of Health and Human Services and t he National Institute for
Child Health and Human Deve lopment, 1998; and J acq u e l yn n e Eccles and J ennifer
Appleton G o o t ma n, Eds., Community Programs t o Promote Y outh Development
(Washington, D.C.: Nationa l Academy Press, 2002), 411 p.
12 T he HFRP r eport cautions that it is not possib l e t o link s pecific out of school time
activities with specific outcomes because the outcomes are reported as attributable to the
combined effects of progr am components; and HFRP also i ndicated that their i nformation
is based on data provi ded i n public reports authored by the progr am evaluators and l eaders.
Harvard Family Research Proj ect, A Review,p.2.
13 T he s tudy authors i ndicated that the comparison design employed: “offers a r igorous
assessment of t he impacts of after-s chool programs on mi ddle s chool students. T he design
used for t he assessment, however, was d i ctated by the l ack of oversubscription f or most
mi ddle s chool programs . T he findings l ack the s ame h i gh degree of i nternal validity of
random-assignment designs.” M athematica and Decision Information Resources, Wh e nst
Schools Stay Open Late: The National Evaluation of t he 21 Century Community Learning
Centers Program, U.S. Department of Education, 2003, p. 10.
In stead, t he study used a comparison-group model, where s imilar non-participating
students are compared with those who par ticipated. After school programs in middle
schools t yp ically had 4 5 minutes to an hour of academic programmin g ( s u c h as
homework assistance), and then mi ddle-schoolers would usually be able to choose
an activity in which t o participat e (typically recreational and/or enrichment and
interpersonal activities).
Positi v e findings from t he middle s chool centers were: s mall increases in
school attendance, math grades, and parent involvement for s tudents wh o
part i ci p at ed. T here was n o s i gni fi cant d i fference b et ween t h e t reat m ent and cont rol
groups in English, science, or history grades, or in classroom performance or social
development. Small n egative f i n dings were found regarding victimization outside
of the classroom and engaging i n negative behaviors. On average, s tudents attended
middle s chool programs for 3 2 d ays during t he year.14
Mino ri t y students benefitted t he most: African American students who
participat ed had positive res ults in cl assroom effort, m at h grades and reduced
lateness. Hispanic students who part i ci p at ed al so di dbetterin mathgradesand
reduced lateness.
The elementary s cho o l s a m p l e was b ased on the random assign ment of 1,000
elementary school children t o either a t r eatment group (participatin g i n t h e 21st
CCLC) or a control group (not participating). An additional 1,600 c h i l dren were
added i n t he 2001-2002 school year. M athematica considers t hese first year results
for t he elementary school sample preliminary; it anticipates that first year results
from t he complete elementary sample will be available i n t he winter of 2003-2004.
Random sampling was facilitated b y t he fact that many elementary school 21 st CCLC
programs were unable t o s erve all t he students who would have liked to participat e.
The elementary s chools i n t h e initial s am ple h ad a h igher p ercentage of minority
students, especially African Americans, and h ad high er poverty l evels, than o t her
elementary centers overall, that are i n t he same cohort. Nevertheless, the s tudy’s
authors b elieve: “t h e e lementary s chool findings h ave s trong internal validity for
attributing s tudent outcome differences to the 21 st Century p rogram.”15
A t yp ical elementary center h ad 45 mi nutes to an hour for s nack and homework,
an academic activity for an hour, and recreational o r o ther activity for t he remaining
1 t o 2 hours. However, some elementary centers had p rograms t hat were s tructured
differently. S ome focused on skill building for st at e assessm ent t est s , for ex am pl e. 16
Overall, positive effect s for children participating i n t he el em entary programs,
compared to control group children, were found for s ocial s tudies grades and s ome
measures of parent involvement. T here was n o s i g n i f i c ant difference b etween the
treatm e n t a n d control groups in reading scores, homework completion, reading
14 Mathematica and Decision Information Resources, When Schools Stay Open Late, pp. 53
and58.
15 Ibid., p. 13.
16 Ibid., p. 87.
confidence, interpersonal s kills, p ercei ved s afety during after-school hours, or
behavior problems. On average, students attended t he elementary programs for 5 8
days during t he year. M athematica’s s tudy results for t he 2000-2001 school year also
indicated that participation i n p rogr ams, by both middle and elementary school
students, declined over time.
The i ssuance of the first year results from t he Mathem atica s tudy has prompted
considera b l e d i scussion and controvers y. Part of the controversy regarding the
Mathem atica evaluation i s due to the fact that the s tudy results were linked t o a
sign ificant reduction i n FY20 0 4 spending for t he 21 st CCLC p rogram proposed by
the Administration i n its budget request.
Critiques of t he Mathem atica s tudy have noted that the results are only first year
results, and are for the 2000-2001 school year, b efore t he requirements o f t he newly
reauthoriz ed program were i n p lace. For ex ample, i n a 2003 P o licy C ommentary, the
non-governmental Fo rum for Youth Investment states:
At least a dozen o t her s tudies that employed experimental and quasi-
experimental designs offer different , more positive f indings about after-school
p r o gr a ms . Drastic cuts are r ecommended based on the f indings of one stu d y,
using one year of data collected on programs that are only a few years old, when
positive f indings from s trong studies a bound. Such decisions should be i nforme d17
by an accumulation of documented knowledge.
Additionally, s ome h ave argued t hat p rograms t hat are relatively n ew and
undergoing changes s hould not be evaluated for fundin g decisions at that stage o f
their d evelo p m ent. Others have criticized some aspects o f t he methodology18
employed in the s tudy.
Implementation Issues
H o w will after s chool programs and s tates m eet the 2 1st CCLC pro gram
accountability requirements? What are t he hurdles program implement er s might
encounter in achi e v i ng academic improvement go als? Data from t he first year
Mathem atica s tudy have provided i nsight into some of the implementat ion issues that
have been encountered by 21 st CCLC programs.
Academic Improvement. The r e authoriz ed 21 st C C LC p rogram requires
t h at program s focus o n academ i c assi st ance and enri chm en t ; a n d t hat s t at es and
program implementers evaluate t he success o f funded p rograms. S o me of the hurdles
17 T he Forum for Youth Investment, Out of School Time Policy Co mmentary # 3: Reflections
On System Building: Lessons from the After-School M ovement (Washington, D.C.: T he
Forum f or Youth Investment, Impact Strategi es, Inc., 2003), 7 p.
18 J oseph Mahoney a nd Edward Zi gl er, The National Evaluation of t h e 2 1 st C e n t ury
Community Learning Centers: A Critical Analysis of Fi rst-Year Findings ( Ne w Ha ve n, Ya l e
University: 2003), 43 p. See also J oan Bissell, Christopher Cross, K aren M app, Elizabeth
Reisner, Deborah Lowe V a ndell, Constancia Warren, Richard W e i s s bourd, Statementst
Released by the M embers of the Scientific Advisory Board f or the 21 Century Community
Learning Center Evaluation, M ay 10, 2003. Available on t he Web.
t h at st at es and p rogram i m p l em e n t e r s m u s t confront i n achi evi ng academ i c go al s
incl ude: firstly, t he difficulty in securing participation by s tudents (part i c u l arly
middle s choolers) in a p rogram focused on ac a d e m i c improvement; s econdly, the
relatively limited amount of program time per s tudent on average, in which t o m ake
a d ifference i n academic achievement; and fi nally, t he vast divergence in after s chool
program content and go als.
Fi rst l y, b ecause program s are u sual l y vol unt ary, st udent s m ust fi n d t h e aft er
school programs appealing. The M athematica s tudy found that on average, middle
school students a t t e n d e d a center only 3 2 d ays a year, and elementary school
students, 58 days a year. And, attendance was found t o d r o p o ff as t he year
progressed. Thus, l ow attendance rat es and t h e decline i n participation over time
m ay b e a si gn i fi cant i m p edi m ent t o academ i c i m p rovem ent . Because part i ci p at i o n
is voluntary, it may o ccur o n a regu lar o r drop i n basis. For middle s chool students,
the dec i s ion of whether or not to participat e i s generally made by the s tudent.
Programs must be enticing t o potential p ar t i cipants b y not making them too m uch
like s chool, and by including recreational and other activities t hat appeal to middle
schoolers. Mathematica i nterviewed students who chose not to participate t o ex p lore
thisphenomenon:
Students who had chosen not to participate ( surveyed in six s elected programs )
said th a t t h e y would r ather ‘ hang out’ after school, were i nvolved i n other
organized activities after school, or were not interested in the activities. Al most
half of the s tudents t hought the centers were ‘mostly a place ki ds go when their
parents a re at work,’ and a quarter considered them ‘j ust f or ki ds who need help19
in school.’ Participants who had stopped attending echoed t hese sentiments.
S econdly, student success i n p rograms focusing o n academic achievement has
been found to be strongly correlated with more time when learning actually occurs20
( t ime on task or academic learning time). However, most after s chool pro g r a m s
have limi t e d “time on task” for academic activities. The after school programs
e v a l uated b y M athematica m ost o ften provided homework assistance as the m ain
t ype of academ i c assi st ance, and i n m any cases t h e assi st ance was m i n i m al . In
middle s chools visited, for ex ample:
Site vi sitors observed t hat most homework s essions resemb l e d s tudy halls in
which s tudents were expected to know thei r assignments, bring t heir materials,
and work i ndependently. T hese sessions typically consisted of about 20 students
monitored by 2 staff members (usually certif i e d t eachers or a certified t eacher
and a paraprofessional). Although having t eachers from t he host s chool oversee
homework s essions offered a potentially fruitful path for helping students after21
school, t he caliber of homework a ssistance was l ow ....
19 Mathematica and Decision Information Resources, When Schools Stay Open Late,p.xvii.
20 Testimony of Dr . Gary Estes, i n U.S. Congress, Senate Committee on Health, Education,
L a b or a nd Pensions, Hearing on I lliterate Gr aduates: Social Promotion and Retention,th st
21 Mathematica and Decision Information Resources, When Schools Stay Open Late,p.21.
These d at a i ndi cat e t hat s om e ex i st i n g p rograms m ay have difficulty providing
academic assistance of su f f i cient duration, and with sufficient time on task, t o
si gn i fi cant l y i m p rove academ i c out com es. There i s al s o evi dence, however, t hat
more attendance alone may not be sufficient.22 These i ssues have caused s om e t o ask
whet her an ex p ect at i o n o f s i gni fi cant l y i m p roved academ i c out com es i s reasonabl e
for t hese after s chool programs:
Holding after-s chool programs accountable for i mprovi ng standardized test
scores at a t i me w h e n s chools are struggling under t he weight of this demand
makes little sense. At best, policy makers c o u l d p u s h f or proportionate j oint
accountability and encourage s c h o ols and out of school programs to work23
together to achieve this long-t erm goal.
Fi nal l y, p rogram s recei vi ng 21 st CCLC funds have traditionally been a d iverse
group with different go als, content, and s tructures. As a consequence, monitoring,
setting goals, and eval uating t hese diverse program s can be difficult:
ma ny of which a re not connected by an overarching s ys tem a t t he local or state
level. T here i s no guarantee that gr antees are part of networks t hat set common24
standards or create a shared sense of accountability.
Because of t h i s di versi t y and l ack of coordi nat i on, wi despread di ssem i n at i o n o f
promising practices , and thei r implementation across program s, is made morest
difficult. Among the goals of the reauthoriz ed 21 CCLC are providing a c l e arer
focus for programs and facilitating t he dissemination of promising practices .25
Program Sustainability. The 2 1 st C C LC p r o gr a m requires p rospective
grantees to submit an application t o s tates t hat i ncludes (am ong o t h e r t h i n gs): “a
description of a preliminary plan for how the community learning center will
continue after funding under t his p art ends.”26
22 First year results from M athematica did not find more positive outcomes f or mi ddle s chool
students who participated for more days du r i n g t h e year, however, i t did find that these
students were more disadvantaged, and more motivated, on average, t han t h o s e w ho
parti c i p a t ed less. Mathematica and Decision Information Resources, When Schools Stay
OpenLate,p.79.
23 T he Forum for Youth Investment, Out-Of-School-Time Policy Commentary # 1: Out-of-
School Research Meets After-School Policy (Washington, D . C . : T h e Forum f or Youth
Investment, Impact Strategi es, Inc., 2003), p. 5.
24 T he Fo r um for Youth Investment, Out-Of-School-Time Policy Commentary # 3:
Reflections on System B u ilding: Lesson f rom t he After-School Movement (Washington,
D.C.: T he Forum f or Youth Investment, Impact Strategi es, Inc., 2003), p. 4.
25 ED is conducting a 3-year evaluation of effective academic materials t hat f acilitate
successful academic interventions that could be used by after-s chool programs . Department
of Education, Fiscal Y ear 2004 Justifications of Appropriation Estimates to the Congress ,
p. C-34. ED has a lso s et up a c learinghouse o n what works at [http://www.w-w-c.org] .
26 20. U.S.C. 7174.
However, the first year results from M at hematica i n d i cat e t hat s ustainability
planning is limited i n m any program s. Program s taff indicat ed to Mathem atica t heir
desire to c o n t i n u e programs after 2 1st CCLC funding ex pired, but few h ad
successfully lined up sufficient outside funding:
... site vi sitors observed f ew concrete actions leading t oward sustainability. At
t h e time of t he vi sits, about one-t hird of gr antees had made no plans and t aken
no actions to sustain t heir program; half had developed s ome plans but had not
ye t t aken any action. In surveys of c enter c oordinator s a n d host s chool
principals, only 10% of principals and 12% of coordinators reported t hat f unding27
sources had been identified or s ecured.
Mathem atica cited t hree fact ors t hat potentially impede program sustai nability:
(1) p roject directors were o ften responsible for a host o f t asks, m any times i ncluding
non-21 st CCLC t as ks, and as a consequence had little time for planning for t he future
and s eeking potential funding sources; (2) project directors i n remote o r poor areas
ex pressed concerns to interviewers about the l ack of availability in their area o f
potential s ources of future funding; and (3) t h e l a c k of a m atching requirement for
local grantees may have delayed planning for future funding. Additionally,
foundation, community, and l o cal funding for t hese programs is limited and the 3-
year grant p eriod p rovided a relatively s hort time frame to conduct future p lanning. 28st
The reaut hori z ed 21 CCLC program has i ncreas ed the time frame for grants (they
may now be up to 5 years); and permits states to require matching funds.
Di r ecti ons for t he Futur e
W h at have researchers found that may b e h elpful i n c r eating effective after
school programs? As noted above, research on after s chool programs is by and l argest
not based o n 2 1 C C LC p rogram s b ecause of i t s rel at i v e n ewness, but on ot her
individual after school programs and l argely focuses o n p romising p ractices gl eaned29
from h igh quality after s chool programs. Fo r b roadly focu s e d a f t er school
p rograms, one researcher has s tated t hat effective p rograms m ust h ave s upportive
staff, positive p eer relations, a n d opportun ities for sustained p articipation i n
meaningful activities (which m ay be s ports, arts, music, or academics). 30 Fo r a ft er
schoo l p r o gram s w i t h an academ i c focus, som e research i ndi cat es t h at successful
programs need well qualified and ex tensively t rained staff, a connection t o t he school
day curriculum, opportunities for one on one tutoring, and clear go als with a well l aid
out structure t o achieve their goals.31
27 Mathematica and Decision Information Resources, When Schools Stay Open Late,p.46.
28 Ibid., p. 48.
29 For an extensive list of r es o u r c e s see U.S. Department of Education, 21st Century
Community Learning Centers Non-Regulatory Guidance, Feb. 2003.
30 Deborah V a ndell, Forum f or Youth Investment, Out-of- School Research Meets After-
School Policy ,p.3.
31 Fashola Olatokunbo, Review of Extended Da y and After-School Programs and Their
Effectiveness , Center f or Research on the Education of Students Placed a t Risk, Oct. 1998,
(continued...)
The North Central R egional Educational Laboratory (NCREL) has published
several resources on after s chool programs, i ncluding the Beyond the Bell Toolkit,
which i s a comprehensive guide intended t o provide advice on setting up an effective
after s chool program. N C R E L c i t e s five elements t hat i t b elieves are essential for
effect i v e p rogram s:
! Fi rst, programs need to be designed based o n i n d i vidual s tudents’
academic needs revealed by the s chool ’ s student assessments and
teacher reports .... the regular classroom teacher s hould regularly
share t he specific n eeds o f i ndividual s tudents — skills that should
be learned m ore completely — with after-school staff.
! Second, staff n eed specific conten t knowl e d ge and instructional
strategi es to facilitate learning .... It is not enough t o h ave s taff that
simply supervise homework completion.
! Third, class s iz es need to be small .... Generally, a ratio of 1:15 or
lower for younger students s eems t o b e i deal.
! Fourth, t here needs t o b e consistent, formal, and s pecific
communication b etween ex tended an d regu lar d ay staff .... Daily
pl anners and academ i c com m uni cat i o n l ogs can serve as v ehi cl es for
student-led conferencing among students, staff and parents.
! Fi nal l y, p rogram s n eed t o be eval uat ed for t h ei r effect on rai s i n g
student achievement . T h i s m eans collecting p re- and post-
assessment d ata an d c o n ducting l ongitudinal s tudies on the effects
of ex tended academic support.32
Additionally, ED, Arnold S chwarz enegger, and t he C.S. M o t t Fo undation,
hosted an after school summit in the s ummer o f 2003 to ex plore ways t o improve and
measure t he performance of 21st CCLC a f t e r school programs. The attendees
developed an ex t ensive list o f s tudent perfo rm ance i ndi cat ors i n s everal m aj o r areas
— academ i c, s oci al , ski l l bui l d i n g, h e a l t h , and com m uni t y — t hat were v i ewed as
important in indicating s uccess i n after school programs. In addition, a beginning list
of eval uation m eas ures and program el em ents essential for attaining improvement in
these s tudent performance indicators was developed b y t he attendees. 33
As st at es and t he l o cal grant ees work t o adapt t o t he changes requi red b y t he 21st
CCLC p rogram as reauthoriz ed by P.L. 107-110, they c o n f r o n t a v ariety of
difficulties i n implementing s uccessful programs, s ome o f which are d iscussed h ere.
Howev er, they will also have increasing access t o a growing body of research and
eval uation, incl uding the bes t practice literature, t o hel p guide the evolution of t hese
programs.
31 (...continued)
p.55.
32 N C REL, V i ewpoints No. 10, After-School Learning and Beyond ,LearningPoint
Associates, 2003.
33 U.S. Departme nt of Education a nd the C.S. M ott Foundation, After Sch ool Summit
Summary Report , 2003.