Case Study 02

NYC Dept. of Education, Office of Community Schools

ACR conducted our first program evaluation for the NYC Office of Community Schools (OCS) in 2016, and has since been awarded seven contracts (and counting) from OCS and partnering agencies. The following is an example of an evaluation for OCS’s CareerCLUE initiative.

CareerCLUE Program Evaluation

Project Background

In the summer of 2016, OCS piloted CareerCLUE, a summer youth employment program that combines STEAM-based classroom curricula, project based-learning experiences, service learning, and field trips to relevant businesses. The goal of CareerCLUE is to expose younger youth (aged 14-15) to a targeted workforce sector, while advancing their academic learning and connecting this learning to their future in the workforce.

ACR was contracted by OCS to measure the fidelity with which CareerCLUE was implemented; assess evidence of program impact; and provide actionable recommendations for program expansion. A major challenge we had to overcome was timing: the contract was awarded four weeks before the program ended.

ACR’s Approach

ACR rapidly revised our proposed research plan with input from partner agencies, and created and cleared all research instruments and consents through an expedited IRB review in time to conduct student focus groups, staff interviews, and student/staff surveys in the final week of the program.  We successfully conducted a rigorous and informative study within this short timeframe because of our:

• Previous experience working with the NYC DOE IRB that we applied when submitting a clear and thorough protocol that was approved within two days;
• Deep understanding of NYC schools that enabled us to effectively communicate with schools to organize research activities, recruit student focus group participants, and obtain parental consent;
• Small team structure, which allowed us to make informed decisions quickly while adhering to the project objectives.

The Impact

We developed two reports and a presentation for partner agencies and external funders with actionable recommendations for improving program implementation, better aligning the program to the mission and objectives of partnering agencies, refining the program’s theory of change and program outcomes for future evaluation efforts, and revising the structure of professional development.

Our reports led to program revisions and policy changes that helped CareerCLUE expand to 9 schools in 2017, 14 schools in 2018, and 19 schools in 2019. ACR was also contracted by OCS for follow up evaluations in 2018 and 2019.