Workforce Development and Community College Outcomes
by: F. E. “Skip” Gillum, Vice President, Casper College; Rosanne Crossen, Director of Institutional Research, Casper College; and Sara Saulcy, Economist, Research & Planning
"This partnership...has resulted in information being developed for the first time that documents the way that Casper College interacts with Wyoming’s job market."
February 2000, Casper College and the Wyoming Department of Employment, Research
& Planning (R&P) have worked together to design a process for describing the
interaction between Casper College graduates and the Wyoming labor market. A primary
goal of our work is meeting the training provider requirements for performance
measurement in the Federal Workforce Investment Act of 1998
(WIA).1 Our partnership also meets the needs
of Casper College: supporting educational improvement and the accreditation process.
These needs meet the criteria for data sharing between Casper College and R&P as
established by the Family Educational Rights and Privacy Act (FERPA).2 To meet the statutory requirements of WIA, we
developed a pilot employer questionnaire to obtain information not available through
other sources. Research & Planning’s administrative databases contain information
about employers and workers, which permits us to identify the work histories of college
completers but that in other ways is incomplete. To fill these gaps, we collected
additional information directly from employers by mail questionnaire. In this article
we present background information on performance measurement, and discuss users who
benefit from this information. The research design and data collection techniques,
as well as limitations of the data, are also explained. In a complementary article
(see "Implementing the Workforce Investment Act..."), summary
statistics are provided.
Purpose of Performance Measurement
Little direct information about the outcomes of training programs is available. Educated guesses about the impact of education by measuring earnings before and after training are possible. We also know, for example, that on average a physician earns more than a mechanic. It is difficult to measure how training programs compare for occupations having less clear-cut outcomes. The incomplete, inequitable, or untimely provision of career information may lead to outcomes for employers and employees that are undesirable or unacceptable.
The performance measurement process as described in WIA is an attempt to provide consumers of training programs useful and objective comparisons. Students are the direct consumers of training programs through coursework and hands-on instruction. Employers are also consumers in that they apply the skills, knowledge, and experience that labor acquire through training programs to produce goods and services. The benefits of the consumption of educational services accrue to society at large as well. Neil Bruce points out that, “It is difficult to imagine how an advanced industrial economy could function if the population were illiterate and ignorant….”3
Society has recognized the value of a skilled and knowledgeable workforce, and publicly funds education from pre-school through post-graduate studies. However, in return for expending funds for postsecondary training programs, accountability has been introduced through WIA to provide for objective measurement of education outcomes. Ideally, labor market performance measures lead to
• program offerings more consistent with positive labor market
• an increased ability by training providers to learn about and
adapt to the changing needs of both employers and students;
• support for the accreditation process through better
Community colleges are often viewed as homogenous training organizations. In fact they differ, sometimes greatly, in the quality and type of training they provide, and in the role they play in the labor market. As part of their mission, Wyoming’s community colleges function as dynamic institutions serving the sometimes-competing demands of student aspirations, local employer needs, and the changing economic and industrial conditions of the local, state, and regional labor market. For many students, they also serve as stepping stones to the University of Wyoming (UW) or other institutions of higher education outside of Wyoming.
Students from colleges have typically been viewed as new labor supply. In reality, many are returning adult learners who already have workforce experience, (see "Implementing the Workforce Act...", Figure 4). Furthermore, students often work while attending school, sometimes by choice, but often because their personal circumstances dictate their doing so.
New and incumbent workers use the training provider system for a variety of purposes. Some individuals may truly be new labor supply. Others represent a part of the workforce seeking skills upgrades for their existing jobs. Students may also be looking to improve their skills or acquire new skills so they can change jobs, or are acquiring education in order to move on to another institution. Still others may have been absent from the labor force for some time and are seeking skills to improve their chances of acquiring meaningful employment upon returning to the labor force.
WIA Requirements and the Role of R&P
Training providers, whose Wyoming clients receive services under WIA, are required to furnish labor market outcomes reports to the Wyoming Department of Employment (DOE). Under Section 122d of WIA, training providers must make available to the public information about
• program completion rates;
• the percentage of individuals participating in a training
program who obtain unsubsidized employment, including
those who obtain employment in an occupation related to their
chosen training program;
• the percentage of individuals who completed the applicable
• the retention rate six months after hire of program
participants who completed the applicable program;
• rates of licensure or certification and other credentials, or
attainment of other skills;
• wages earned at initial placement in employment by all
program participants; and
• wages earned by training program participants who
completed the program six months after hire.
The goal for R&P was to develop a template, using one college, to demonstrate how to meet the requirements of one provider of training services. Ideally the pilot process becomes the vehicle for answering practical questions about producing the required information, and identifies the resources needed to produce consumer reports and performance measures for training providers in general. The initial information derived from the questionnaire responses gives us a baseline for how training program participants, and therefore training providers, are performing. Once we have this information, training providers and other participants in the workforce development system can experiment with different labor market interventions (i.e., adjusting career development services, expanding internship offerings, identifying appropriate training programs). Ultimately the goal of labor market interventions is to achieve desirable outcomes in a timely manner. Such outcomes include obtaining employment in a preferred career, attaining employment with a desired compensation package, or meeting career or job goals within a certain time frame.
Beneficiaries of Performance Measurement Information
The information derived from the pilot survey of employers of Casper College graduates is potentially beneficial to a variety of groups. Among them are Casper College itself, industry, job seekers and job changers (traditional and nontraditional students), and policy makers.
Casper College benefits by utilizing the information that is developed over time to improve existing programs and to develop new program offerings, which will continue to meet the changing needs of the Wyoming labor market. The data also document the earning power of program completers who attended Casper College.
Employers who require post high school education of their workers also benefit from access to local labor supply information. For example, an employer might want to know if industries employing welders are satisfied with the skills individuals bring to their jobs once they have completed or participated in a related technical program at Casper College. Positive feedback from local peers could encourage that employer to send his/her employees to Casper College for additional training. As another example, an employer could review the wage results of the survey to determine if the compensation package he/she is offering is competitive, and if not, adjust the compensation package offered accordingly.
Job seekers and job changers benefit by having access to information that identifies the average salary for employment fields in the state, and assists in identifying the most suitable program of study. Among those who benefit are students, parents of students, career counselors, dislocated workers, and other individuals currently outside of the workforce (such as stay-at-home parents or spouses). All have access to the objective information that they need to make decisions consistent with their goals and interests.
Policy makers benefit by having documented evidence of the long-term return on investment in higher education in the State of Wyoming. The information also helps to describe how Casper College interacts with the state job market. Among the policy makers who can benefit from this project are legislators, state and local workforce investment boards, and others involved in workforce policy.
Advantages and Disadvantages of the Casper College/R&P Partnership
This partnership has resulted in a win-win undertaking for all concerned. It has resulted in information being developed for the first time that documents the way that Casper College interacts with Wyoming’s job market. It provides reliable information concerning the earning power of graduates and allows for conclusions to be drawn concerning the impact that the college has on the earning power of those individuals. Through subsequent employer surveys conducted by DOE, the college will have reliable data concerning the quality of their programs of study. Performance measurement also provides information to the college about areas where employers feel their employees need additional training, thus allowing the college to respond to those training needs.
The resulting data, however, are not without limitations. Wage Records data only account for those individuals working in Wyoming and are not all-inclusive. Not found in these records are individuals working for the Federal Government, persons in the military, agricultural workers, and persons who are self-employed. Despite the limitations of the survey conducted jointly between Casper College and R&P, the information is better than the self-reported data traditionally gained by Casper College through the use of graduate surveys.
Pilot Survey Process
The pilot survey process was driven by a number of factors:
• the availability of student data;
• the timing of downloads from the Unemployment Insurance
• a goal of publishing draft consumer reports by summer 2001
to provide information to Casper College prior to the next
academic year, enabling administrators to make decisions
based on the information.
Availability of student data, and the timing of the download of Unemployment Insurance (UI) Wage Records4 were the primary determinants guiding the employer follow-up survey. The Wage Records database consists of all employers submitting UI tax records to the DOE, and contains detailed work behavior information on individuals working for employers who are required to pay Unemployment taxes.5 Student data are generally available shortly after the end of a semester. In contrast, the time delay for Wage Records from the end of a reporting quarter to its availability for downloading is two quarters. For example, data for the first quarter of 2000 (2000Q1) was available at the beginning of the third quarter of 2000 (2000Q3). In order to meet the requirements of Section 122 of the WIA that we obtain information about graduates at the time of hire and six months later, student data were matched with employment data from the Wage Records database two quarters after the date of graduation. The number of employers of graduates who were included in the survey are shown on Table 1 (see Implementing the Workforce Investment Act...). Because the employer survey was a pilot, mailing questionnaires six months after graduation (as opposed to six months after hire) served as a proxy time frame for considering graduates six months after hire. By doing so we were able to consider how using administrative databases in concert with student data works in practice, even though we do not precisely conform to the six-month time frame outlined by WIA.
Graduates of the May 2000 class represent the population under consideration. The class had a total of 303 graduates. In the fourth quarter of 2000 (the time that Wage Records are available from the second quarter of 2000), employer, student, and Wage Records were matched to form the subset of the population we were evaluating (employed graduates). Student records were matched with employer Wage Records using Social Security Numbers (SSNs) as the matching criterion. The student SSNs and program enrollment data were obtained under a Memorandum of Understanding between DOE and Casper College. The data sharing agreement is consistent with the Family Educational Rights and Privacy Act regulations describing the sharing of information for purposes of improving educational programs, and support of the accreditation process.6
Based on the match, 145 out of 303 graduates (47.9%) were found to be working in Wyoming. Although only 47.9 percent were found to be working in Wyoming, this does not necessarily mean that the remaining 52.1 percent are unemployed. Some may be working in non-UI covered employment, such as Federal Government, railroads, or Agriculture, or are self-employed. Others may have left Wyoming and are not tracked by Wyoming’s UI Wage Records. The possibility also exists that they have moved on to the University of Wyoming or another institution of higher education. Individuals attending UW or other schools may appear in Wage Records as either full-time or part-time employees, or may not appear at all because they are attending school only and not working.
Because individuals occasionally work for multiple employers, there were a handful of instances of multiple employers for a single graduate. Similarly, many individuals may work for a single employer. Overall, 183 matches between student and employer were identified in Wage Records.
Research & Planning distributed the survey to Wyoming employers of completers of Casper College programs; employers of program non-completers were not sent a questionnaire. We collected information from firms who employed one of two different types of Casper College graduates (see Table 1):
• graduates employed in April, May, and June of 2000 (the
quarter of graduation, denoted as 2000Q2) and the quarter
following graduation (third quarter of 2000, denoted as
• graduates not employed at the quarter of graduation, but
employed the quarter following graduation (2000Q3).
The questionnaire was mailed in late January 2001. A follow-up letter with the questionnaire was mailed in mid-February to employers who had not responded to the first questionnaire by early February. Employer questionnaires that were returned because of inadequate addressing were re-mailed to an updated address for the employer.
Depending on the employment status of the graduate at the time of graduation, an employer was sent one of two questionnaires.7 For graduates employed only in 2000Q3 as identified in wage records, we asked employers what they paid when the graduates were first hired. For graduates employed in both 2000Q2 and 2000Q3 (as identified in wage records), we asked the employer what the highest rate of pay was in April, May, and June of 2000. The questionnaire consisted of ten questions on the following topics:
• whether or not the person was still employed as of January
12, 2001 (six months after graduation);
• wage rates at first hire, or in April, May and June of 2000,
and wage rates as of January 12, 2001;
• number of hours worked per week;
• benefits or other non-monetary compensation received by the
• whether or not the employer received a public subsidy for the
• the occupation and primary activities of the person;
• training or education required for the job;
• satisfaction with the employee’s work and work habits; and
• whether the employer considered the supply of labor for the
occupation to be sufficient and skilled.
January 12, 2001 was the date we requested information from because it is the UI reporting date for wages paid by employers at that point in time in the first quarter. The goal was to ease the reporting burden on employers by permitting them to draw information on employees from reports they were already required to submit.
Responses to questions by employers were a reflection of whether or not graduates worked for the same employers as of January 12, 2001. If an individual was still employed, we asked the employer to complete the entire questionnaire. If the individual was not working for the same employer as of January 12, 2001, the employer was asked to complete questions regarding wage rate at first hire, or wage rate for April, May and June of 2000, employer satisfaction with the employee’s work, and to comment on the available supply and skill of labor for the occupation.
A total of 181 questionnaires were mailed to employers. Two employers of graduates were not sent questionnaires because of incomplete UI Wage Records for the employers. Of the 181 questionnaires, 158 were returned. Five of the 158 survey responses were invalid.8 The bulk of responses were tallied as reported. Three exceptions were wages, childcare (as a benefit), and employer opinion of the labor supply. Employers were given the option of reporting wages as hourly, weekly, biweekly or monthly. Since the majority of wages were reported as hourly, R&P converted all wages to hourly to aid the comparison of wages. Where employers reported wages as weekly, biweekly, or monthly, and when hours normally worked were provided, R&P converted those wages directly to hourly wages. In cases where the employer reported the wage but not the number of hours, R&P calculated hourly wages assuming a 40-hour workweek.9 When a conversion resulted in a value inconsistent with other information provided, the value was entered as missing.
Employer opinion of labor supply was evaluated using content analysis. The analysis yielded four groups of responses:
• employees were not willing or committed to work;
• the labor supply is not sufficiently skilled;
• there is a lack of labor supply in general; and
• the labor supply is sufficient and skilled.
Data Limitations and Remedies
Although we received 158 of 181 questionnaires sent (an 87.3% return rate), our success is somewhat muted. One problem involved the wording of the benefits question. We asked if employees received (as opposed to simply being offered) any of the benefits10 listed on the survey. Several employers stated that the employee was offered the particular benefit, but did not necessarily receive it. Consequently employer-provided benefits may appear lower than is actually the case. For example, the initial response for child care benefits was that none of the employees received them. Based on other survey programs conducted within R&P, we knew that certain employers offered subsidized child care. As confirmation, we contacted certain employers for further information. From the telephone contact with the employers, we revised the number of employees receiving child care benefits. Another error was in omitting asking employers about retirement benefits employees received. Employers had the option of writing in other benefits that employees received, but none elected to write in retirement.
Another problem involved employer non-response. Despite assurances that information would remain confidential under state and federal law, some employers declined to answer certain questions, citing company policy. This occurred most frequently for questions regarding salary and satisfaction. Non-responses also occurred because the employer would only report whether or not the company still employed the graduate, and would not answer any of the other questions as requested. Some employers responded that they did not have knowledge of the particular question being asked. Larger employers more frequently had this problem, where the individual filling out the survey may have had little direct contact with the employee. As a result, some employers were unable to provide answers in such areas as satisfaction with the employee’s work or work habits.
Contributing to the problem of non-response were employers who did not answer certain questions (even though they were requested to do so), and did not report their reason. In some cases, employees were temporary, seasonal, or volunteer, and therefore do not ordinarily receive standard salaries and benefits. These yielded a small number of non-responses for certain questions.
Basic statistical problems are also part of the data set. First, we are unable to generalize to other populations (e.g., graduates of Casper College in general, graduates of other community colleges) using the data from the questionnaire regarding May 2000 graduates. The small number of responses to various questions also means that R&P is unable to do much in the way of statistical modeling (i.e., wage or occupational predictions). Because of the small number of observations, we will need more than one year’s worth of information to establish any trends in the data. In addition, the small number of responses precludes us from providing more detailed occupational and industry data.
The most immediate plan for R&P is to revise the survey. One revision is a change in the benefits question to read, “Is the employee offered [as opposed to receiving] any of the following job benefits?” Another possible revision is to the question on the employer’s opinion of labor supply. The question was open-ended to allow employers to express their opinion of the current status of labor supply and skills. While R&P made attempts to code responses using content analysis, we are somewhat uncertain about whether or not we have accurately depicted employers’ sense of labor supply and skills. We expect to format the question in a way that will elicit a more precise response. We also will evaluate other wording possibilities to encourage employers to respond.
The most important development is that Northwest College (NWC) in Powell and Laramie County Community College (LCCC) in Cheyenne have entered into agreements with R&P to assist in their completion of the WIA requirements. Our objective is to build on the pilot that R&P and Casper College have developed.
The outgrowth of the performance measurement process is consumer reports. Consumer reports will integrate information from performance measurement and administrative databases. Through the provision of information, we hope to reduce the level of uncertainty for those in the market for educational or training providers. Consumer reports will aid in achieving the goals of labor market participants.
As our article shows, measuring the performance of graduates of training programs in the labor market yields many benefits beyond the requirements of the Workforce Investment Act. The partnership between Casper College and Research & Planning is a mutually beneficial relationship, allowing our institutions to draw on the resources with which we are familiar, and to access alternative resources where we have either less knowledge or do not have the tools, such as Wage Records. The American Association of Community Colleges advocates the use of UI wage data for performance measurement tasks.11 While there are limitations to the data, we are encouraged by the initial results of the Casper College pilot survey. Now that we have conducted the pilot survey, we look forward to improving our methods and our results. For further information, see the related article on the summary statistics in this issue, “Implementing the Workforce Investment Act: Results from an Employer Survey Follow-up of Casper College Graduates.”
1 105th Congress, “Workforce Investment Act of 1998,” n.d..
2 “Family Educational Rights and Privacy Act (FERPA),” n.d..
3 Neil Bruce, Public Finance and the American Economy, 1998, p. 354.
4 The UI Wage Records database consists of all employers submitting UI tax records to the Wyoming Department of Employment (DOE), and contains detailed work behavior information on individuals working for employers that are required to pay Unemployment taxes. See Tony Glover, “Enhancing the Quality of Wage Records for Analysis Through Imputation: Part One,” Wyoming Labor Force Trends, April 2001.
5 Tony Glover, “Enhancing the Quality of Wage Records for Analysis Through Imputation: Part One,” Wyoming Labor Force Trends, April 2001.
6 See Endnote 2.
7 The survey instruments are available on Research & Planning’s Education Links website.
8 Four employers reported they did not have the person on record as an employee. Potential sources of this problem include incomplete or incorrect records on the part of the employer, lack of knowledge of the employee by the person filling out the questionnaire, unwillingness of the employer to participate in the survey, or incomplete or incorrect UI Wage Records. A fifth response was invalid because the employer reported once that the employee had worked for them, but then responded a second time that they had no record of the employee. The first questionnaire sent to the employer was received after response deadlines had passed. A second survey was sent to the employer to encourage a response. A comparison of the two surveys for the same employer and graduate yielded the conflicting responses. Consequently responses for the employer for the particular graduate were entered as missing.
9 The Alien Labor Certification program uses a 40-hour work week to convert wages reported in any units other than hourly. See General Administration Letter No. 4-95, n.d., for further discussion.
10 The benefits question was asked in a manner similar to the collection of information for our employee benefits survey (see Wyoming Department of Employment, Research & Planning, Employee Benefits in Wyoming 2000).
11 Robert T. Mundhenk, “Institutional Effectiveness and Unemployment Insurance Data,” Fall 2000, American Association of Community Colleges White Paper.
Table of Contents | Labor Market Information | Wyoming Job Network | Send Us Mail
These pages designed by
Gayle C. Edlin.
Last modified on by Valerie A. Davis.