Evaluation of the Sectoral Employment Demonstration Program

Nancy M. Pindus, Carolyn T. O'Brien, Maureen Conway, Conaway Haskins, Ida Rademacher

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Note: This report is available in its entirety in the Portable Document Format (PDF).

---

Executive Summary

In Program Year 2000, the Employment and Training Administration (ETA) of the U.S. Department of Labor (DOL) funded 39 local Workforce Investment Boards (WIBs) to participate in the sectoral employment demonstration (SED).¹ The demonstration was conducted to help DOL determine whether sector-based strategies could be adopted and used effectively by local WIBs. In the context of this demonstration, sectoral initiatives primarily seek to identify workforce needs and opportunities within a local or regional industry or cross-industry occupational group while also focusing on economic performance and competitiveness. These efforts target a specific population—such as low-income workers, dislocated workers, or new entrants to the job market—to meet both industry needs for a competitive workforce and the needs of those targeted for improved job opportunities. Sectoral initiatives are typically long-term efforts that, over time, attempt to improve the functioning of their regional labor markets, benefiting both workers and employers.

DOL was particularly interested in examining the potential for successful involvement in sectoral initiatives by local WIBs, chiefly because the goals of the se activities paralleled the role envisioned for the boards under the Workforce Investment Act of 1998 (WIA). These roles include developing employer linkages, coordinating with economic development, promoting private sector involvement in the workforce investment system, and developing workforce investment plans as part of a comprehensive workforce investment system focusing on individual employment goals and the needs of firms for skilled workers.

This report documents the range of projects undertaken and the outcomes achieved by the 38 WIBs that participated in the demonstration. The demonstration included 26 formation grantees that received one-year grants (some with three-month extensions) to form stakeholder groups, collect and analyze labor market data, analyze industry opportunities and training capacity, and develop a strategic plan for the selected industry sector. Twelve organizations were awarded 15-month implementation grants (some with 3- or 6-month extensions) to undertake specific sectoral interventions, such as outreach, recruitment, curriculum development, training, and designing career paths for the benefit of individual participants within the context of specific industry/occupational sector needs.

The evaluation ran from September 2002 through December 2003, taking place toward the end of the grant period. Evaluation activities consisted of (1) a review of grant files, (2) telephone discussions with all grantees, (3) site visits to 12 grantees (6 formation and 6 implementation), and (4) cross-site analysis across qualitative variables associated with project organization and process, as well as outcome indicators, where available.

Key Findings

Sectoral Grantees Focused on Business Goals

• Over 80 percent of the grantees chose a target industry that was experiencing a labor shortage.
• Over half of the grantees stated that encouraging growth in the targeted sector was one of the reasons for their choice of industry.
• Other reasons reported for choosing a sector were to improve industry competitiveness, to retain the industry as an employer in the community, and to prepare an industry for demographic or technological changes.

Meeting the Needs of the Labor Force

• Most grantees defined their target population broadly and included more than one group, such as incumbent workers and displaced/unemployed workers.
• Recruiting participants was a challenge for forty percent of the grantees. Outreach techniques included direct advertising; working with social service agencies, community-based organizations, and other stakeholders; and using workplace resources.
• A number of grantees changed or expanded their target populations over the course of their projects. Three major reasons given for target population changes were (1) an overall expansion of the target group to compensate for lower than anticipated participation, (2) a decision to include groups with fewer barriers to employment, and (3) an increased focus on incumbent workers in order to create entry-level openings as these workers advance.
• Because of the limited time frames and funding associated with the sectoral employment grants, many grantee organizations focused primarily on business and industry needs and then defaulted to working with a target population that could most easily be prepared to meet those needs. While these projects may have been quite useful to the workers involved, they were less likely to involve hard-to-serve populations. Such strategies are in contrast to those that initially focus equally on industry and selected target populations.

Meeting the Needs of the Sector

• Goals associated with improving industry competitiveness were reported by 11 of the demonstration grantees. Specific issues addressed included improving the image of the industry to attract workers and facilitating relationships between employers and public training providers, such as community colleges.
• Some sectoral grantees worked with industries that were having difficulty meeting a higher skill need. Grantees designed incumbent worker training programs to develop more advanced skills needed by the industry to remain competitive.
• Most grantees (31 of 38, or 80 percent) chose a target industry that was experiencing a labor shortage. Although a downturn in the labor market during the grant period reduced workforce needs, shortages remained in the health care field. Many grantees cited labor shortages as an important reason for their choice of sector.

Employer/Industry Involvement

• Three distinct approaches were used to designate a target industry and set of occupations: (1) targeting the firms of an identified industry, (2) targeting a specific occupational area that crosses a range of industries, and (3) working with a single large employer. The first of these was the most common approach, taken by 29 of the grantees.
• Most of the grantees said they built on previous efforts undertaken in the selected sector.
• Grantees used labor market data, employer surveys, and focus groups, and worked with industry associations to gain an understanding of employer needs.
• Challenges faced by sectoral grantees in working with employers included the WIB's limited knowledge about the industry sector, employer mistrust or negative impressions about government workforce development initiatives, and employer reluctance to share information with other employers. These challenges were successfully addressed by many grantees.
• Nine of the 38 grantees attracted direct employer financial contributions, while 13 received in-kind contributions from employers.

Sector Programs and the WIB System

• WIBs in a variety of institutional settings were able to engage in sector work. Effective WIB roles ranged from a high level of WIB member involvement in attracting support from the business community, to day-to-day involvement of One-Stop staff in recruiting and orienting participants, to primarily conducting contract monitoring activities and overseeing implementation by another community organization. Different organizational settings can offer varying sets of resources—there was clearly no one "best" organizational structure for initiating and operating a sector project.
• For about one-third of the WIBs, the project was particularly energizing and led to such outcomes as rethinking strategic plans or reshaping operations around sector-based work.
• Conversely, another one-third of the WIBs noted that board involvement in the project was relatively low. Reasons for this low level of interest included the small size of the grant relative to the overall WIB budget and no perceived need for the WIB to take on more than an oversight role with respect to the project.
• Implementing agencies combined the resources of in- house staff with the expertise of consultants and outside agencies in various ways. No particular staffing pattern or patterns emerged as the best way to staff a sector project.
• Respondents from 21 sites stated that the sector initiative helped improve the WIB's relationship with employers. The same number stated that they plan to continue with sector work and expand the approach to other sectors.
• By the end of 2002, 11 grantees had obtained additional WIA funding for their sector initiatives, and 8 grantees had received additional grants from state agencies, foundations, or DOL to continue their initiatives. In response to a follow-up request in March 2004, one additional grantee reported receiving WIA funds and two additional grantees reported receiving grants from other sources.

Outcomes/Accomplishments: Formation Grantees

• Grantees formed broad stakeholder groups that included employers, community colleges, unions, and industry associations.
• Almost 70 percent of the formation grantees found that the stakeholder groups encouraged employers to work together.
• In addition to the required strategic plan, more than three-quarters of the formation grantees established ongoing coalitions that will continue to meet. Almost half of the formation grantees (12) responded to a follow-up request almost two years after their grants expired, and all indicated that they are continuing or planning to do sector work.
• Sixty-five percent of the formation grantees leveraged additional resources to continue their planning efforts and stakeholder meetings, and/or to begin implementing their plans.

Outcomes/Accomplishments: Implementation Grantees

• Ten of the 12 implementation grantees increased training options available in their community/region.
• Seven of the 12 implementation grantees facilitated career ladders/advancement opportunities, and the same number established new pipelines for workers in an industry, most often targeting youth.
• All but one of the implementation grantees undertook projects that enrolled students in training, serving from 24 to 578 individuals. Eight of the 12 implementation grantees reported successful job placements for trainees, and 2 grantees reported average wage gains of 18 percent or more.

Lessons for Program Operators

• WIBs can play many roles in sector projects, depending on their interest in and knowledge of the sector, staff capacity, and ability to serve the target populations. Successful WIBs used their assessment of these factors to inform decisions about the roles and responsibilities that would be appropriate to take on directly and those that would be best to delegate.
• Sector projects need to recruit and maintain the right mix of stakeholders. This is a dynamic process, since the roles and responsibilities and relative importance of the various stakeholders can change over time. For example, some grantees found that it was important to have the buy-in of chief executive officers (CEOs) of target businesses early in the project and that supervisory or human resources staff involvement was critical at later stages of a project.
• Successful sector projects build in frequent opportunities to solicit feedback from employers and program participants, and they are willing and able to make midcourse corrections as needed.
• Among the sectoral employment demonstration grantees, promising practices were identified in stakeholder collaboration; leveraging training resources and building training capacity; coordinating resources to address target population needs; employer involvement for continuous improvement; and developing and supporting career ladders, retention, and advancement.

Implications for Policy

The following factors that pertain to the sectoral employment demonstration grants have broader implications for policy and future DOL demonstration grant initiatives.

• Balancing employer and worker needs. The sectoral employment demonstration reflects a policy focus of the WIA to develop a stronger link between the public workforce system and employer needs in the business/workplace environment. However, the reality of balancing employer and workforce needs can present challenges to serving those most in need. In the economic downturn of 2001 in which many sector projects found themselves, a reduced demand for workers resulted in some projects refocusing their target populations to those who could gain skills quickly, usually better educated or incumbent workers.
• Time frame. Demonstration grants are, by definition, time-limited, but the time frames depend on the goals of the grant. For the formation grants, whether or not a one-year grant period was reasonable depended on the extent to which a core stakeholder group existed for the targeted industry.² Although most of the grantees were able to prepare a strategic plan in the allotted time and most coalitions intended to continue to meet after their grants ended, grantees said that a longer grant period (such as 18 months) would have been better for developing collaborative relationships and planning. Implementation grantees faced different challenges with their 15-month grants.³ Start-up activities took longer than anticipated for some grantees. These activities included developing curricula, identifying training providers, recruiting training participants, and making improvements to the physical site. Finally, the short time frames of the grants limit the ability to report outcomes. Some implementation grantees established or supported training programs and enrolled participants, but the grant ended before participants completed training. For other grantees, participants completed training, but it was too soon to report employment outcomes or, for incumbent workers, job advancement or wage gains. Still other grantees addressed career ladders and long-term human resources practices in an industry, such as benefits, employer-supported training, and advancement. It is too soon to determine whether these sector projects will have a lasting influence on employer practices. Effecting such changes will require that the grantees, or their partners, stay with the initiatives for the long haul.
• Accountability. All sectoral employment demonstration grantees (formation and implementation) were required to identify specific measured outcomes pertaining to their project activities. Since grantees pursued a wide range of activities, outcome measures and the methods for documenting outcomes varied considerably. It is important to consider both readily quantifiable outcomes (such as number completing a training program) and more qualitative outcomes (such as whether the project promoted collaboration between employers and training providers). Since most of the sectoral demonstration grants were formation grants aimed at convening stakeholder groups and planning, the more qualitative outcomes are critical to understanding the accomplishments of the grantees. Such outcomes include bringing new partners together, creating stakeholder groups that cross traditional divides of professions and disciplines, and crossing geographic boundaries. These activities, while difficult to quantify, help build the infrastructure needed to maintain sector projects.
• Resources. The sectoral employment demonstration grants were small and generally comprised a very small portion of a WIB's budget. Nevertheless, WIBs found that the grants provided resources that would otherwise not be available for collaborative activities and that the award of the grant helped to engage partners and staff. Grantees had discretion to use grant funds in ways that best supported the objectives of the grant, including staff, contracted services, and purchase of curriculum materials. This flexibility was welcomed by grantees. Grantee strategies and accomplishments were necessarily tailored to the small size and short-term nature of the grants. While grantees responded positively to the flexible funding, formation grantees were disappointed in the lack of follow-on implementation funding and had varied success in sustaining their projects.
• Capacity building. The sectoral employment demonstration grant program included some efforts to bring grantees together to share ideas and experiences. Several grantees noted that this was a particularly helpful benefit of participating in the demonstration. Some grantees are well integrated in the existing and growing network of groups involved in sectoral initiatives, while others are very new to this field and seek more information and guidance. Continued interaction among the grantees and other organizations involved in sectoral initiatives will build capacity as they share best practices and establish a network of peers for ongoing consultations. Organizations involved in sector projects can then use this information to benchmark their own progress, identify effective strategies, and build support (both public and private) for their activities. This support is especially important because sectoral projects must continue beyond the grant period if they are to achieve success.

Notes from this section

1 The evaluation includes 38 grantees. One grantee withdrew from the demonstration.

2 Half of the formation grantees requested and received extensions of one to three months, with one receiving a fivemonth extension.

3 Eight of the 12 implementation grantees requested and received extensions of three months; 2 other grantees received extensions of six months.

Note: This report is available in its entirety in the Portable Document Format (PDF).

---

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Usage, posting and reprint of materials on the UI web site:

Most publications may be downloaded free of charge from the web site in PDF format. This information may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required.

Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact paffairs@urban.org.

If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.