Formative Report on the DC 21st Century Community Learning Center Summer Program

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

This report was prepared for DC Agenda by Jacqueline Raphael and Duncan Chaplin of The Urban Institute, with assistance from Zakia Redd, Alissa Anderson, and Jane Hannaway. We also gratefully acknowledge the assistance of the Evaluation Group for the DC 21^st Century Community Learning Center program, including Howard Brown, Sandy Handy, and Joyce Jamison of the DC Public Schools; Connie Spinner and Keith Watson of the DC Youth Investment Trust Corporation; Carrie Thornhill of DC Agenda; Eric Bruns of Johns Hopkins University School of Public Health; and Janet Reingold of Reingold Associates. Funding for this work was provided by the Lasalle Adams Family Fund through DC Agenda, and by the U.S. Department of Education, through the DC Public Schools and DC Agenda.

This report is available in its entirety in the Portable Document Format (PDF), which many find convenient when printing.

The purpose of this report is to summarize key issues that emerged from The Urban Institute's visits to the five sites of the District of Columbia's 21^st Century Community Learning Center program (DC 21^st CCLC) in July 2000. These sites served students, ages 4 through 13, from nine junior high/middle schools and their feeder schools.¹

During the site visits, information was gathered to answer the following questions:

Organizational Issues

How satisfied were school staff (those implementing the program at the schools) with the support provided to them by the District of Columbia Public School (DCPS) staff who operate the program², including training, facilities, equipment, etc.? What suggestions were made by school staff to improve the assistance and support provided by DCPS?

Potential Student Outcomes

What types and quality of activities were provided to students in the summer program? What potential student outcomes were observed by researchers and reported by parents and/or guardians and could be explored in a summative evaluation of the program?

During site visits, Urban Institute researchers undertook the following activities:

• Review of school site proposals and DCPS documents detailing professional development activities for staff.
• Short interviews with nine "Assistant Principals" ("APs," or program managers) and twelve "facilitators" (instructors) from all but one school program.
• Observation of 30- to 60-minute segments of 22 program activities, including at least two activities at each school program.
• Interviews with DCPS staff who operate the DC 21^st CCLC program.

I. Background of the DC 21^st CCLC Program

The DC 21^st Century Learning Centers (DC 21^st CCLC) program began in 1999 as part of a broader initiative called the DC Children and Youth Investment Partnership (DC CYIP), which is designed to help coordinate the many different types of youth activities provided for children during non-school hours in the District of Columbia. A brief discussion of DC CYIP and earlier initiatives is provided in our formative report on the after-school component of the DC 21^st CCLC program (Raphael & Chaplin, 2000).

The DC 21^st CCLC program is run by the District of Columbia Public School System (DCPS) and includes after-school, summer, Saturday, and adult evening sub-programs. The after-school program was established in 10 middle/junior high schools with high need for such services in November 1999.³ This report focuses on the summer sub-program, which ran in 9 of the 10 original schools, from June 26 to August 4, 2000.

In addition to being a subset of the DC CYIP initiatives, the DC 21^st CCLC is a subset of the out-of-school-time programs operated by DCPS on behalf of the District of Columbia Department of Human Services (DHS), which were implemented at forty-seven (47) other DCPS sites in addition to the DC 21^st CCLC school sites. Each of these programs is funded through Temporary Assistance for Needy Families (TANF) funds. The summer operation is collectively referred to as the Summer Education/Arts/Sports (SEAS) program, which started in 1998 with 30 schools.

II. Program Structure

Each DC 21^st CCLC school's summer program was managed by an Assistant Principal (AP), who supervised a staff of instructors (facilitators). The AP was responsible for recruiting facilitators and students and for organizing special events. Most APs hired teachers from their schools to work as facilitators.⁴

The DC 21^st CCLC program is implemented through a collaborative program planning process that is overseen by DCPS staff. Through workshops and written materials, APs receive guidance on how to structure their programs, particularly regarding the types of activities to be offered. These included activities in:

• Technofluency (typing, E-mail, and Internet skills);
• Sports/health (physical and mental well-being);
• Arts (visual, performing, and other types); and
• Community service.

During spring 2000, each AP wrote a proposal (including a budget) for his or her school site's summer program. DCPS 21^st CCLC administrative staff reviewed these proposals, with assistance from outsiders with relevant experience, and made suggestions for revisions that led to final approval of the plans and the implementation of the program. (A similar procedure was used for the after-school component of DC 21^st CCLC.)

The DCPS program provided training to APs and reviewed program implementation in monthly meetings and special workshops. During the four weeks of this summer program, APs attended nightly meetings from 7 to 9 p.m. to review the summer program⁵ and to prepare for the adult component of the program, scheduled to start in the fall of 2000. In addition, DCPS staff indicated that they encourage APs to contact them at any time to discuss program implementation issues.

DCPS staff also provided training for facilitators. Some of the athletics facilitators, for example, reported that they attended a pre-summer program orientation session with Frank Briscoe, who organized all summer athletic programs being run and operated by DC schools. According to Mr. Briscoe, this training covered the philosophy of the program, different activities to be conducted, and assistance to be provided to sports coordinators. In addition, DCPS staff reported that they provided four days of training to the facilitators who had been identified by that time during the week of June 11 (before the program began).

III. Organizational Issues

This section identifies key implementation issues that may be of interest to DCPS staff for future program implementation. These issues include the capacity of the program to serve the target population, school/community issues, program planning, staffing, and the extent and quality of support provided to school staff.

Several of the issues discussed in this section may represent opportunities for program improvement (e.g., better communication between DCPS and program staff). Providing such feedback is an important purpose of this formative report. This discussion does not imply, however, that the program has suffered major implementation problems. In fact, we found that program staff have responded with flexibility to common implementation challenges, and that the program has been implemented generally as planned in nine of the ten intended sites.⁶

Capacity to Serve the Target Audience

While the DC 21^st CCLC program has received a great deal of support from DCPS, some implementation issues arose that influenced the program's ability to serve its target population. These include: the grades served, the site locations, and the location of complementary programs.

Grades Served

The original plan of the DC 21^st CCLC program was to serve 2,000 youth in grades 6 through 8 during the school year and 2,000 during the summer (DCPS, 1999). However, DCPS staff revised their goals for the summer program in two ways. First, in the proposals for the summer programs, they reduced the total goal to between 1,000 and 1,400 students. Second, they expanded the program to include students from all elementary schools that feed into the original middle and junior high schools.⁷ They have come quite close to meeting the revised goal, serving about 1,000 students regularly.

Including students from any elementary school allowed DC 21^st CCLC to serve more students than it most likely would have served otherwise, given projections based on enrollment in the after-school program.⁸ It also changed the nature of the program somewhat without detracting from achievement of its goals and objectives. Most of the time, students engaged in program activities with similar-age peers, but they also interacted with students of varying ages during field trips, athletic competitions, and certain other activities. Although serving a wide age range posed challenges for some facilitators (most of whom were teachers at junior high/middle schools), when asked, most facilitators reported that they enjoyed working with a range of student ages and were successful with them. Some parents, too, said that they appreciated the efficiency and potential familial benefits of having their various children in the same program together.

In one case, however, facilitators may have been confused about the target population. Four facilitators — three at one site appear to have been given the initial impression, perhaps by their AP, that they would be able to choose the age groups with which they would work. We did not receive any reports that this had actually occurred at any program site. The three facilitators at this particular site indicated that they were less comfortable serving younger students. The technology facilitator, for example, said it was more difficult for him to engage these students, particularly because this site had longer activity periods than others.

Site Location

The second factor influencing the program's capacity to serve targeted students is site location. In the initial design of the summer component, each of the junior high/middle school sites was supposed to implement its own summer program. However, in January 2000, the Superintendent of DCPS alerted the district that 10 percent of its schools would not operate programs in the summer to facilitate needed repairs. DCPS staff operating the DC 21^st CCLC program learned in March that three of its schools would be closed.⁹ In response, DCPS staff combined four school programs and operated them out of one school site, and combined two other school programs at another site. In total, nine school programs were operated at five school sites.

The "relocation" of some school programs to sites other than the "home" school meant that some students had to travel somewhat significant distances to attend the program. Several APs indicated that the relocation limited student enrollment, as parents were either unable to transport their children to the alternative site or unwilling to let their children (particularly younger children) use public transportation to get to the site.

Complementary Program Location

DCPS also runs an academic program during the summer (STARS) that complements the SEAS programs (which include the DC 21^st CCLC programs). The STARS programs run in the mornings, ending at 12:30 p.m., which is when the SEAS programs begin. Thus, students attending both programs are provided with full weekdays of activities during the six weeks of program operation. Unfortunately, three schools had their DC 21^st CCLC programs held in different locations than their STARS programs.¹⁰ The decision to hold these programs in different locations was made by DCPS administration and was viewed as a challenge by the DCPS staff that managed the DC 21^st CCLC program. Parents of children whose programs were held in the same location expressed satisfaction at being able to drop off their children in the morning and know that they were supervised until 6:30 in the evening. But in those several school programs where the programs were held in different locations, parents were inconvenienced. This problem may have affected the enrollment at the affected sites, according to APs.

These three factors may have affected the program's ability to serve its target population. The perseverance of the staff, in spite of these issues, suggests a healthy ability to flexibly implement the program and to deal with future challenges they are likely to face.

School/Community Issues

Although most of our discussions with program staff concerned the summer component of the DC 21^st CCLC program, APs discussed two challenges to implementation of the after-school program: other neighborhood programs and safety. We discuss these issues here because they are relevant to the summer program as well.

Other Neighborhood Programs

The first issue concerns the effect of other programs, either within the host school or at other sites in the neighborhood, on the DC 21^st CCLC program. Three APs specifically mentioned that other programs "competed" with DC 21^st CCLC for students. One AP felt strongly that the District ought to "bring [all] community-level youth programming under the same umbrella" to reduce such competition. Another AP spoke to us about coordinating her summer program schedule and activities with those of surrounding neighborhood programs, indicating that it was challenging but not prohibitively so. In addition, some APs anticipated having difficulty recruiting the projected number of students because of the competing programs, a challenge discussed in our previous report.¹¹

Safety

One AP expressed continued concerns about the after-school program students going home after dark. This issue was also discussed in The Urban Institute's July 2000 report on the after-school component of the DC 21^st CCLC program. This AP felt program enrollment suffers because parents do not want to have their children out when it is dark. She thought that the DC 21^st CCLC program may need to demonstrate a more realistic perspective about the neighborhoods it serves — although she did not make specific suggestions about how to do so. She also suspected that parents might not feel comfortable participating in the adult component (scheduled to start in Fall 2000), which is scheduled to run until 9:30 p.m. Finally, she suggested that the program may want to consider providing an on-site babysitter for the adult component to encourage parents to attend.

Program Planning

Although most of the APs and facilitators were satisfied that they had been able to implement their school programs as planned, they discussed ways to improve programming through more intensive planning at an earlier period.

Some APs and facilitators felt that more planning would have improved the program. For example, two APs and two facilitators felt that they needed more time to plan the summer component together. One of these facilitators, who conducted athletic activities, said the program planning process needs to start three to four weeks in advance of the program's launch date. He also felt that the facilitators, at least in athletics, needed more than one planning meeting, particularly if they want to plan and offer "clinics" for students, with professionals who could share their expertise with students.

Similarly, a third AP felt that her school staff needed additional general training, starting in advance of the program. Specifically, she felt that facilitators who are new to DCPS need more training. One "new" facilitator at another program agreed that he could have benefited from more training about program procedures, who to contact for problems, and other issues. Presumably these new facilitators were not able to attend the 4-day training provided by DCPS because they arrived too late.¹²

Overall, we had the impression that some staff (APs and facilitators) felt somewhat "rushed" to start up the summer component. Two APs said they needed more time to make the transition from the after-school to the summer component. One of these APs felt that the start date of June 26^th gave APs too little time (sixteen days) to wrap up the after-school program, and complete their proposals (including revisions requested by DCPS) in preparation for the summer program. Related to program planning, parents in our focus groups often suggested that the sites begin publicizing the summer program earlier in the school year. Some parents reported not learning of the program's existence until after it had already started. On the other hand, starting the program later would mean that the children were left without access to this program for a longer period during the time when the regular school is not in session.

In general, however, APs did not indicate that their programs were seriously compromised by these issues.

DCPS Staffing Issues

Several staffing issues related to the DCPS staff came up in our discussions with APs. These related to the number of staff at each site, the training and support they received, and the schedules for the APs.

Number of Staff

Nearly all APs and facilitators felt that they had hired a sufficient number of staff to serve the students in their program. However, two APs said that it would have helped them if DCPS staff had postponed making decisions about the number of facilitators to be hired¹³ until the second or third week of the summer program instead of during week one. At one site, many students delayed enrolling in the program after the first week, but the AP had already "cut" program staff to meet DCPS requirements for staff-to-student ratios. Staff had to be rehired, delaying program implementation somewhat.

Staff Training and Support

Most APs and facilitators were satisfied with the training and support they received to help them implement this program — although it was our impression that the training seemed somewhat unimportant to most of the staff we interviewed. Four of the nine APs reported that the professional development provided by DCPS helped them establish direction for their programs and serve as better program managers. The other five APs felt that the professional development had a minimal effect on their job performance, maintaining that they were "already" well qualified to do their jobs. One AP did suggest that DCPS staff consider having APs give more input on what type of professional development was offered to determine if these sessions could be more useful.

In general, APs felt that they were given adequate opportunity to talk with DCPS staff about issues in the program. However, two APs reported tensions with DCPS staff.

Most, though not all, facilitators in the summer program reported receiving two-hour or half-day orientations to the program from DCPS. Apparently they either arrived too late for the 4-day training or did not view it as "training." In either case, facilitators had little to say about this training. One facilitator, however, said he would have liked a more thorough orientation to the program, particularly because he was not a teacher in DCPS. Athletic facilitators reported that the training they received from DCPS was useful.

Schedule for APs

Some APs expressed concern about how their days were scheduled. The APs were supposed to arrive at 12:30 p.m. each day, just before the children who would arrive between 12:30 and 1:00 p.m. Two APs said that they did not feel comfortable arriving at the same time as their students because they needed to make sure that the site was well prepared each day before they arrived. They could have arrived earlier, but this would have been difficult given their late hours in the evening. After the program ends at 6:30 p.m., APs were to participate in group meetings at Shaw Middle School, from 7 to 9 p.m., to discuss/review progress in the summer program and to plan the adult program, scheduled to begin in the fall. One AP suggested that DCPS try to avoid the evening meetings, as APs were too tired after working with their students earlier in the day and needed more time to work on preparing for and managing the summer program.¹⁴

School-Level Support

We discussed issues related to facilities, equipment, program supplies, and staff support with the site-level staff.

Facilities, Equipment, and Program Supplies

Nearly all APs and facilitators reported that they had adequate facilities, equipment, and program supplies to implement their programs. There were two exceptions. First, two sites lacked DC 21^st CCLC computers. At one of these sites, students would walk across the street to the high school to use computers. At the other site, the procurement process was delayed until the start of the school year and students did not participate in technology activities. Second, two APs reported that they were unable to write checks for supplies at the start of the program. In one case, this was a school-specific issue, and in the other, it was not clear why funds were not available to the AP.

Staff Support at Site

Nearly all facilitators reported that they received adequate support from their APs. They noted that their APs dealt quickly with requests for materials and facilities for activities, provided assistance with payroll issues, and "backed up" facilitators when disciplinary problems arose with students. One facilitator valued the fact that the AP voiced facilitators' concerns to the DCPS staff who run the DC 21^st CCLC program. One facilitator did not feel well-supported by her AP but had not discussed her concerns with the AP because she started working in the program late and was not able to participate in the program planning process.

Facilitators were assisted by aides in most activities. Some of these aides were Americorps volunteers, some were students (age 14-15) and a few were parents. Student aides were present but appeared to be underutilized in some activities. For example, we observed student aides sitting on the bleachers during aerobic movement/dance and art activities, and sitting alone in a classroom, with no students present, while the entire student body competed in a sports event in the gym. Although there may be limits to what these student aides can or should be doing in a program, it seemed to us that they could be trained to take on a greater role in the activities. However, many of the student aides either attended the same school or lived in the same neighborhood as the student participants. This familiarity may make it difficult to use them as supervisors or mentors. In addition, student aides were provided with an important job experience through DC 21^st CCLC.¹⁵

The Americorps aides appeared to take a more active role. The technology activities at one site (the site that combined four school programs) utilized more aides, and in a greater instructional capacity, than other activities. Many of these aides were Americorps volunteers, which may account for the difference. We observed the Americorps volunteers providing a good deal of structured support to facilitators and students in the technology and art components. The Americorps volunteers were older and more experienced than the student aides, and their role in the program may have differed because of this. Facilitators occasionally relied heavily on the Americorps aides to teach and monitor the children. Indeed, DCPS staff told us that the Americorps volunteers were primarily responsible for the delivery of instruction in the technology components.

In any case, all of the aides were highly valued by facilitators and APs. A technology facilitator mentioned that having aides was particularly important with the younger children, who need more help with computers than the older students. Several parents also spoke very highly of the Americorps volunteers in particular, saying that their children developed meaningful relationships with these individuals.

IV. Observed Features of Student Activities

Our site visits included observations of 30- to 60-minute segments of 22 student activities conducted as part of the DC 21^st CCLC program (at least two activities at each school program). Appendix A provides the dates, times, staffing, and number of students in these observed activities.

We begin by providing some common features we observed in student activities at the sites, followed by a discussion of the structure of the activities.

Common Features of Activities

• Observed student activities were implemented with sufficient staff supervision and in adequate spaces, with sufficient time and equipment. Most of the activities we observed involved small numbers of students, typically 12, with a range of 4 to 20, with an average of around 13 (not including gym activities). Most activities were led by one facilitator who was assisted by some number of aides (usually between one and four), depending on the type of activity and the size of the group. Most of these activities were conducted in a sufficiently sized space such as a classroom or in an area of an "open air" school. Materials and equipment were adequate. The computer facilities, in particular, were quite sophisticated at some sites, with computer desks that used a glass screen and special housing to protect the computers from damage. Activity periods generally were one to one-and-a-half hours, with one site offering activities for three-hour periods of time.
• Activities were well planned. Even when activities involved "free" time for students to use the Internet or play sports, the students had a routine to follow and did not stay idle for substantial amounts of time. There were exceptions, of course, but these were few.
• Nearly all of the activities involved hands-on "work" rather than listening to an instructor or passively watching a presentation. There were exceptions, but in general students were doing various things, including dancing, singing, doing artwork, using computers, and playing educational games. They frequently took responsibility for leading one another in a task, such as leading peers in a series of athletic stretching exercises.
• In general, facilitators and aides maintained discipline and kept students on task. However, as expected in any program, some facilitators exercised better "classroom" management than others. In two activities we observed what appeared to be poor classroom management. Both of these activities involved the older students in junior high/middle school, suggesting that what appeared to be poor classroom management may have been caused, at least in part, by difficulties related to managing older students.

Structure of Student Activity Groups

In most cases, the groups in which students participated in activities were of two types, either small or large. Most groups were small, with students of a specific age group, and were monitored closely by one facilitator and several aides. As mentioned, "class" sizes (except in gym) ranged from 4 to 20 students. In these small groups, students did creative, athletic, and/or educational activities, such as weightlifting, dancing, creating artwork, playing educational games, using technology, etc. Students worked independently in many activities, including technology activities, arts and crafts, and weightlifting. In other small "classes," students worked in even smaller groups playing educational games, competing or playing in sports, singing, and dancing. In several of the observed activities, students were rehearsing performances (such as dances and fashion shows) that would be performed for other students and their families at the end of the summer.

In contrast, other groups — particularly for athletic events — included larger numbers of students and received less direct supervision by facilitators and/or aides. In some athletic events held in school gymnasiums, for example, students were given access to a variety of sports equipment (such as jump ropes, basketballs, and lacrosse and boxing equipment) and were permitted to use this equipment appropriately on their own. For example, we observed over 25 students in one activity at one site and about 60 at another. In both cases the students were of various ages and were playing different sports. They obtained the equipment themselves and used it while being supervised by facilitators. However, the facilitators did not monitor the students so much as maintain order. In other athletic activities, students in large groups competed against one another, as in a large "horseshoe tag" game involving approximately 40 students, ages 8-13.¹⁶ In this case, one facilitator "presided" over the game, providing encouragement and promoting good sportsmanship and teamwork through verbal feedback to teams and individual students.

Although students were typically grouped in these two ways, several features of service delivery enhanced the potential for learning and interacting. These features include the following:

1. In many activities of both types, students were given choices about what they would do during the activity period. For example, students in one activity could play chess or checkers, use the computer to play educational games (using the Learnstar program, in this case), or sit quietly and color. Students were able to start with and switch to whatever specific activity they wished. Significantly, we observed them persisting in each activity rather than switching frequently.
2. In a significant number of activities, particularly technology, students were permitted to work individually and test their own limits in various tasks. For example, facilitators assigned students a particular lesson from the Almena typing tutorial but did not appear to monitor students' progress continually. Occasionally, students were asked to take a timed typing test that gave them feedback on their progress. In the weightlifting activity, the facilitator emphasized that students should select the appropriate weights for each exercise, based on their individual capacity.
3. Students were also encouraged to compete with each other in a number of areas. Sports competitions were organized by Mr. Briscoe, as discussed earlier. Students played competitive games, such as chess, checkers, and tic-tac-toe. In addition, results from the Almena tests and from the stock-market game were stored electronically and students were told how they were doing both absolutely and relative to each other.
4. Some activities were structured to satisfy the special interests/needs of specific groups of students. For example, a weightlifting activity was geared toward a group of male students who wished to participate in the school-year football team. Their workout routine was more rigorous than that of other coed groups, according to the facilitator. In another example, students with a strong interest in participating in a fashion show were given this option, whereas other students did not participate in this activity. Students who were particularly interested in mural-making were observed being pulled out of other activities to complete a wall mural at one of the school sites.
5. Often, students were permitted to interact freely with their peers. Many activities were structured to give students independent learning opportunities but also the freedom to interact freely with one another. For example, in one art activity, 7- and 8-year-old students freely observed one another's projects and gave feedback to each other.

V. Potential Program Outcomes

One of the purposes of our visits to the school sites was to gather information about possible outcomes of the program, particularly for students. The information supplied here is not designed to show program effects. Instead, we provide this information to stimulate further investigation. The information is based on our observations and comments made by parents and/or guardians and facilitators about the program. A rigorous evaluation would attempt to demonstrate whether these outcomes differed between students with access to the program and those in an appropriate comparison group.

Short-Term Outcomes

These are outcomes that we believe could be observed as immediate benefits of the program. Many of these may also affect student outcomes in the longer term. For instance, increased engagement may improve student learning.

Student interest and engagement in activities. Although students were clearly more enthusiastic about some activities than others, they appeared to enjoy nearly all of the activities we observed. Students demonstrated high levels of engagement in these activities and were spending most of their activity time discussing the tasks at hand with one another. Parents and/or guardians also provided numerous examples of their children's positive reactions to the program, including being prepared to go early, telling their parents not to pick them up before the program ended, and discussing specific program activities at home with their parents. Some parents expressed surprise that their children had remained interested in the program throughout its duration, indicating that other summer programs had not held their children's interest for so long.

Based on the reports from parents in the focus groups, it appears that students would demonstrate high participant satisfaction using surveys or interviews (as was found in focus groups with students in the after-school site visits).

Providing free, high-quality care for school-aged children. Many parents and/or guardians indicated that by providing free child care, the summer program had some benefits for families. Some parents said they would have had to pay (e.g., $50 per week) to enroll their children in other summer programs. Parents also appreciated having some time away from their children, which reduced stress in their homes. In some cases children from the same family were able to be together in the program, which parents viewed as convenient and beneficial for the family.

Students' comfort level with technology. Students appeared quite comfortable using computer technology in the activities we observed. They started working on their assignments quickly, with the facilitator helping them with equipment only if needed (e.g., setting up a headset for the typing CD). Clearly, a routine had been established by the facilitators, and students followed this routine well.

Longer-Term Outcomes

Longer-term outcomes are those that may benefit students in the long run, though many may also have short-run benefits. For instance, learning about new activities (the first potential outcome) can be fun and can also open up more options for future activities for these youth to participate in.

Learning about new activities. Although we did not survey students about their previous experiences, we are fairly certain that many had not been previously exposed to many of the activities available in the DC 21^st CCLC program, such as lacrosse, ballet, and various forms of "community service." (Students in focus groups in the after-school program frequently indicated that program activities were new to them.) Student participants were given the opportunity to try new activities and to learn about them. For example, one athletics facilitator said he doubted many of his students had access to weight machines. He hoped that by emphasizing the concepts underlying weight training, he had helped his students understand the health benefits of weightlifting.

Developing collaborative and cooperative skills. We were impressed with the high level of collaboration that occurred in many student activities, particularly among the older students, such as in developing an original dance to be performed for students and their families. In addition to collaboration, students frequently supported one another, as when young children helped each other clap out a rhythm in a music activity or helped one another use the computers. In technology activities, we observed a good deal of spontaneous helpful behavior between students. For example, during free time devoted to "surfing the Net," students shared information and tips about how to navigate the Web. And as discussed in the previous section, students were observed exercising conflict resolution skills while playing games with one another.

Learning social skills. Parents and/or guardians were pleased with the potential for good socialization that is part of the program. They appreciated that their children were spending time with students from different schools and of different ages (students of different ages are combined during field trips, snacks, and other large-group activities) and from different schools. Some said their children were around a "better group of students", and had the opportunity to establish "more positive" friendships in the program. Some parents felt their children might not engage with other children if not enrolled in a program such as DC 21^st CCLC.

The program also provided a great deal of time for the children to interact with adults in a low-pressure environment. In nearly all activities in the smaller groups—and frequently in the larger groups—students received individual attention from adults. We observed facilitators providing "instruction" in every activity, as well as encouragement, advice, and constructive feedback. Several parents felt that the Americorps volunteers were good role models for their children.

In addition to teaching them to get along with others, the DC 21^st CCLC program provided an opportunity for the youth to work to help others. The community involvement component of the DC 21^st CCLC program encouraged the youth to do community-oriented activities such as cleaning up the school area and visiting homes of the elderly. Such activities may increase the chances that these youth continue to work to develop their communities in the long run, thereby benefiting both themselves (because of the opportunity to socialize) and those with whom they share these communities.

Increasing self-confidence. Some parents reported that during their participation in the program, their children had begun displaying increased confidence at home. They were "coming out of their shells," by dancing, singing, and expressing more of their opinions at home. These parents could not be certain that these changes would not have occurred in the absence of the program, however.

Increasing ability to focus. Several parents felt that their children were learning to be calmer and possibly more focused by pursuing enjoyable activities while in the program. These parents had children with attention-deficit disorders and/or behavioral difficulties.

Developing academic skills. In technology and other activities, we observed students engaged in the development of basic academic skills, including reading, writing, using mathematics, and problem-solving. While these are primarily the focus of the schools, approximately one-fourth of DC 21^st CCLC program time is devoted to development of academic skills. A particular strength of the DC 21^st CCLC program could be the high level of use of up-to-date technology. Specific skills covered included becoming computer literate, typing, using the Internet to search for information, purchasing stocks,¹⁷ and using e-mail. More detailed observations of this feature of the program are provided in Appendix B.

VI. Conclusion

The DC 21^st CCLC program has continued its progress toward full implementation in the District of Columbia during the summer of 2000. Activities were provided for about 1,000 children attending nine middle and junior high schools and their feeder elementary schools in the areas of education (reading, math, and technofluency), sports, arts, and community service. These activities built on past after-school and summer school programs by helping to develop a "seamless web" of activities. When fully implemented, this web should help to ensure that students are not left wanting for productive things to do during the summertime when their schools close, and when many youth drop behind academically.

The success in implementing the program was accomplished in spite of a number of significant challenges. In particular, a number of schools were closed for the summer and the other major summer program for youth (the STARS program) could not be co-located with the DC 21^st CCLC programs in three schools. It is very likely that the enrollment figures would have been higher in the absence of these challenges.

Planning, staff support, and staff development all appeared to be proceeding smoothly. We observed evidence of adequate preparation and support, and staff reported being generally satisfied, though some staff requested more staff development, or a different schedule for staff development, suggesting that there may still be room for improvement.

Parent reviews of the program (as provided in focus groups) were enthusiastic. They were particularly thankful for the fact that the program was provided at no cost and was providing their children with structured activities. Many parents reported observing positive changes in their children's behavior that they attributed to the program.

Our site visits revealed that the student activities held promise for producing positive outcomes in students. We observed students engaged in a wide range of learning opportunities and interacting socially in a low-stress environment that was free from disruptive conflict. In addition, the staff generally appeared active and well-utilized. A particularly striking feature of the program that we noted was the use of technology. Students at all but one site had access to computers where they could develop their general computer, typing, Internet, and e-mail skills, all of which are crucial for most good jobs given the current economic environment.

Of course, we cannot describe what sorts of experiences these youth would have had in the absence of the program since we had no data on comparison sites. This points to the importance of a carefully designed impact evaluation, which is currently being designed. In addition, as stated in our earlier report on the after-school component of the DC 21^st CCLC program, continued self-monitoring can help provide useful information to the DCPS staff who run the program, to the rest of the DC Children and Youth Investment Partnership, and to individuals involved in helping to design and manage similar programs in other cities.

Notes

1. In April-May, 2000, Urban Institute researchers conducted site visits to these same nine junior high/middle schools when they were implementing the after-school component of DC 21^st CCLC.

2. The Director of the DC 21^st CCLC is Howard M. Brown. The Program Manager is Saundra Handy.

3. Need was determined based on factors such as poverty and test scores of students at these schools.

4. In addition, the APs were invited by DCPS to meet and hire artists for their summer programs from a pre-selected roster during a specially arranged meeting at Gallaudet University in May of 2000.

5. We were told by DCPS that during these sessions APs were provided with information about the technology and sports component of the program, how to document their activities, how to manage and train their staff, and related information.

6. MacFarland did not have a program for several reasons: the original coordinator resigned, the school was slated to be closed for more than a year, and another summer program for kids (the STARS program described below) for MacFarland was located at Lincoln, which is far from the MacFarland neighborhood.

7. This decision was made by the DCPS staff running the DC 21^st CCLC program and is reflected in the individual school proposals for the summer programs.

8. The program fell somewhat short of its school-year attendance goal (Raphael & Chaplin, 2000).

9. Francis, Garnet-Patterson, and Terrell.

10. The Sousa DC 21st CCLC program was moved to Kramer, Francis was combined with that of Shaw, and MacFarland was scheduled to have its STARS and DC 21^st CCLC programs in different locations.

11. Raphael & Chaplin (2000).

12. At least one facilitator specifically told us this.

13. This decision is based on student enrollment.

14. DCPS administrative staff pointed out that the APs were hired with the full knowledge that their schedules would involve the hours they served during the summer.

15. This was arranged in cooperation with the DC Department of Employment Services.

16. Another 40 were watching.

17. The children did not purchase real stocks. Rather, they played a computer game that simulated stock market transactions using the true market prices.

Assistant Principal Interview Protocol