© Copyright 2004 by the Wyoming Department of Employment, Research & Planning

An Examination of Short-Term Employment Projections Accuracy

by: Douglas W. Leonard, Research Analyst

It is often useful to review past employment projections to understand where errors occurred and how to improve results. With the availability of Wyoming covered employment data from second quarter 2003, such assessments can be made using the short-term (eight-quarter) projections developed during the 2001-2002 projections cycle. The results indicate that the statewide mean absolute projection error for all jobs covered by unemployment insurance was 1.1 percent. However, projection accuracy varied greatly by industry.

The Research & Planning (R&P) section of the Wyoming Department of Employment is required to produce both short- and long-term employment projections under contract with the Employment and Training Administration of the U.S. Department of Labor. Short-term employment projections forecast jobs by industry eight quarters into the future. In this article we present comparisons between short-term industry employment projections for second quarter 2003 (2003Q2) and actual employment for the same quarter with a primary focus on industries having 5,000 or more jobs.

Background

Each quarter, employers who are liable for Unemployment Insurance (UI) report the number of jobs and employee wages in each firm to the Department of Employment through UI tax reports. These data are then compiled into a data set known as the Quarterly Census of Employment and Wages (QCEW; U.S. Department of Labor, 2004). QCEW data represent approximately 95 percent of all jobs worked in Wyoming exclusive of the self-employed, unpaid family workers, some federal government workers, and most railroad employees; and thus provide an accurate picture of employment levels and changes within the state. These data sets form a time series of UI covered employment used to develop short- and long-term projections. 

In 2001-2002, R&P produced statewide short-term employment projections at the 2-digit Standard Industrial Classification (SIC) level for 75 industries (Leonard, 2002). We used 2001Q2 as the base period and projected the 2003Q2 employment.

Methodology

Projection values and actual values from 2003Q2 were compared using three basic measures of forecast accuracy: absolute percentage error, bias, and directional analysis. Absolute percentage error is calculated by summing the absolute value of error for each 2-digit SIC code and dividing it by the total actual employment during the projected period. In simple terms, the absolute value of error represents how far off, in employment (either more or less), the projections were. Absolute percentage error is simply this figure expressed as a percentage of total employment. The second accuracy measurement utilized in this analysis is bias. Bias is defined as the average amount of projection error observed in each industry (i.e., the number of jobs). In other words, how far off, on average, were the employment projections across industries. Although bias and absolute percentage error provide valuable insights into projection accuracy, it is also important that the anticipated changes [either positive or negative] in employment in both actual and projected values are in the same direction. A matched projection is one where the projected and actual employment changes occurred in the same direction (i.e., employment was projected to grow and actually grew), while an unmatched projection moves in the opposite direction of actual changes (i.e., employment was projected to grow but actually fell).

Findings

Table 1 displays projection data for Wyoming and 17 selected industries based on employment and relative importance in the state's economy. Employment in these industries represented 69.1 percent (165,011 jobs) of UI covered employment in 2001Q2 (238,753 jobs) and 69.3 percent (168,860 jobs) of UI covered employment in 2003Q2 (243,798 jobs). When all 2-digit SIC projections were summed and compared to the actual 2-digit SIC total, the difference was 1.1 percent or 2,721 jobs. Accuracy, as measured by absolute percentage error, varied between 0.1 percent (eating & drinking places) to 12.3 percent (general building contractors). Absolute projection errors (measured in jobs) varied from a low of 9 jobs in eating & drinking places to a high of 1,015 jobs in oil & gas extraction. Table 2 shows projection results when the industry data are aggregated to the 10 major SIC industries. Absolute percentage errors ranged from a low of 0.2 percent in Services to a high of 6.0 percent in Manufacturing. Absolute errors (in jobs) ranged from a low of 126 in Wholesale Trade to a high of 1,278 in Retail Trade.

A summary of projection accuracy measures is shown in Table 3. Here the accuracy measures discussed previously are displayed at the 2-digit SIC level and at the major SIC industry level. The absolute percentage error at the 2-digit SIC level was 5.0 percent, while the absolute percentage error at the major industry level was 2.1 percent. The projection bias at the major industry level was 272.1 jobs, while the bias at the 2-digit level was much lower at 36.3 jobs. 

Tables 4 and 5 display the results of directional analysis. Table 4 shows that of the 75 industries projected, 52 (69.3%) had matched projections. The matched projections group of industries tended to be underprojected (bias of -26), while the absolute percentage error was lower than the 2-digit SIC result (3.8% in Table 4 versus 5.0% in Table 3). Dissimilar results were seen for the unmatched group as its bias statistic was a positive 177.0 (overprojected) and its absolute percentage error was 9.5 percent. Table 5 shows a breakdown of matched and unmatched forecasts by major industry. The lowest proportions of matched forecasts were in Agriculture (25.0%), Retail Trade (50.0%), and Finance, Insurance, & Real Estate (FIRE; 57.1%) and the highest proportions of matched forecasts were in Wholesale Trade (100.0%) and Public Administration (100.0%).

Implications

The variation in projection accuracy measures can be influenced by a variety of factors. In some industries, such as Mining, commodity prices and their inherent volatility play a significant role in expected profit levels and corresponding employment. Other factors affecting projection accuracy may include analyst biases, time series stability (data follows a steady trend versus exhibiting large changes in short time spans), and unforeseen events. As Leonard stated in Employment Outlook: 2000-2010 "…projections represent the midrange of possible outcomes based on past and current trends, and assume [sic] that the current policies will remain the same" (2003, p. 37). However, one can reasonably expect that projection accuracy should improve over time as knowledge increases and projection technical capabilities advance. The accuracy of subsequent rounds of short-term projections will be compared to past rounds to see if projection error decreases over time. However, the detail at which accuracy useful for assessing accuracy may be limited by the change from the SIC system to the North American Industry Classification System (NAICS). Once R&P completes two projection rounds using NAICS, detailed comparisons will once again be possible.

References

Leonard, D.W. (2002). Statewide short-term employment projections [unpublished]. Casper, WY: Wyoming Department of Employment, Research & Planning.

Leonard, D.W. (2003). Wyoming's labor market employment: Historical and projected. In Gallagher, T., Harris, M., Leonard, D.W., Liu, W., & McVeigh, B. (2003). Employment Outlook: 2010 (p. 37). Casper, WY: Wyoming Department of Employment, Research & Planning.

U.S. Department of Labor, Bureau of Labor Statistics. (n.d.). Quarterly Census of Employment and Wages. Retrieved March 16, 2004, from http://www.bls.gov/cew/home.htm 

Table of Contents | Labor Market Information | Wyoming Job Network | Send Us Mail

These pages designed by Julie Barnish.
Last modified on by Susan J. Murray.