The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
The text below is an excerpt from the complete document. Building Evaluation Capacity Guide 1: Designing a Cross-Project Evaluation and Guide 2: Collecting and Using Data in Cross-Project Evaluations are available in PDF format.
This two-guide set for evaluators and others interested in evaluation grew out of a National Science Foundation funded effort to improve cross project evaluations. Guide 1, Designing a Cross-Project Evaluation, focuses on evaluation design including identification and operationalization of program goals, building of logic models, and selection of indicators and appropriate measures for these indicators. Guide 2, Collecting and Using Data in Cross-Project Evaluation, lays out multiple issues involved in data collection, strengths and weaknesses of different data collection formats, and methods for ensuring data quality, confidentiality, and the protection of human subjects.
The national trend toward increasing accountability of the public sector, which gave
rise to the enactment in 1993 of the Government Performance and Results Act
(GPRA), has placed added emphasis on the ability of public agencies to report accurately
and consistently on performance and results. The more recent No Child Left Behind Act,
with its calls for scientific evidence in education research, and the current Campbell
Collaborative, which seeks to improve social science research through a focus on random
assignment in studies in social, behavioral, and educational arenas, indicate this trend is
growing. Evaluation capacity building (ECB), as a system for enabling organizations and
agencies—both public and private—to develop the mechanisms and structure to facilitate
evaluation to meet accountability requirements, is a concept whose time has come.
One important way ECB differs from mainstream evaluation is that ECB is continuous
and sustained rather than episodic. Other characteristics are that it is contextdependent;
operates on multiple levels; is flexible in responding to multiple purposes,
requiring continuous adjustments and refinements; and requires a variety of evaluation
approaches and methodologies (Stockdill, Baizerman, and Compton 2002).
The goal of this project was to develop a model to build evaluation capacity in three
organizations: the National Science Foundation (NSF), the National Institutes of Health
(NIH), and the GE Foundation. More specifically, the project’s intent was to test the feasibility
of developing models to facilitate the collection of cross-project evaluation data
for programs within these organizations that focus on increasing the diversity of the
STEM workforce. To facilitate the standardization of the model, we chose programs with
a similar goal: increasing the participation and success of underrepresented groups1 in
STEM fields of study and careers. A description of each program used to test the model
appears in appendix A.
(End of excerpt. Building Evaluation Capacity Guide 1: Designing a Cross-Project Evaluation and Guide 2: Collecting and Using Data in Cross-Project Evaluations are available in PDF format.)
Usage and reprints: Most publications may be downloaded free of charge from the web site and may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact firstname.lastname@example.org.
If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.
Disclaimer: The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute.