• Vol. 48, No. 7. © July 2011
• Table of Contents
• Back Issues
• Contact us
• Subscribe

Training for What? Part 4

Skills Needs in Manufacturing

by: Sara Saulcy, Senior Economist

Questionnaires using closed-ended questions are in many ways ideal for both respondents and researchers because the tasks of providing and compiling information are simplified. However, they limit the information provided by respondents. Open-ended questions give respondents an opportunity to expand on questionnaire topics and introduce new topics. These questions can provide researchers with richer information than otherwise might be obtained from closed-ended questions. This article explores employer responses to an open-ended question about skills needed by newly hired employees in the manufacturing industry.

In the spring of 2010, the Research & Planning (R&P) Section of the Wyoming Department of Workforce Services, along with several other state Labor Market Information offices, received research funding to study newly hired employees and, as a subset, environmental jobs: those that increase energy efficiency, use or develop renewable energy resources, or preserve or restore the environment; see http://doe.state.wy.us/LMI/energy.htm for more research related to this subject.

R&P designed a mail questionnaire that contained a question intended to measure the degree to which a job was involved in any of these environmental activities, and to capture and assess the types of skills needed in Wyoming; this questionnaire is available online. Skills were assessed using open-ended and closed-ended questions (see Definitions list). This article explores skills needs for jobs in the manufacturing industry using text mining, an automated process of examining text for themes that can then be quantified.

For the purpose of this study, R&P was interested in sampling from only those employees that were designated hires excluding those that fell in the both hire and exit category. Specifically, only employees that were considered a new hire during the quarter of interest were included. Rehires were excluded to control for the confounding effects of seasonal re-hiring and to eliminate circumstances where employers and employees based hiring decisions on prior joint human capital and business investment. Finally, R&P was most interested in including new hires that were retained by the same employer for at least two quarters. These jobs were more likely to require a training or educational investment by the employer. This was so R&P could also track what kinds of jobs employers were hiring for and the skills required for those positions (see related article).

The questionnaire was composed of two types of questions: closed-ended questions and open-ended questions. To assess employer skills needs, R&P first asked employers closed-ended questions about five types of skills: service orientation, critical thinking, reading comprehension, technology design, and operation and control (see Table 1). These were selected after cognitive interviews were conducted by the Wyoming Survey & Analysis Center (WYSAC, 2010). Cognitive testing helps to determine if a questionnaire is serving its intended purpose. The five skills on the original questionnaire were chosen from the most frequently occurring skills for environmental jobs. Cognitive testing revealed that the five original skills were measuring the same concept. The questionnaire was revised to include skills that contrasted with one another such that the five skills would be measuring the importance of different skills.

R&P then asked employers to answer an open-ended question about which skill they considered most important for the job. The question indicated that it could be one of the five skills previously mentioned or another skill. Although the questionnaire asked employers to provide a single skill, several employers reported two or more skills having equal importance.

To evaluate respondents' answers to the open-ended question, R&P used text mining, a useful tool for evaluating and quantifying responses to open-ended questions. The process helps to identify themes that cannot otherwise be determined from closed-ended questions. The purpose of identifying themes is to capture information based on what respondents consider important, not what researchers consider important. For large surveys (more than 10,000 responses in this case), text mining by hand is impractical. R&P used text mining software to expedite the process of capturing common themes reported by employers about skills needed to be successful in jobs for which employees were newly hired. The process was then supplemented by reviewing concepts in individual records.

Methodology

R&P used PASW Text Analytics for Surveys 4 text mining software from SPSS Inc. Data originated from the New Hires Survey for fourth quarter 2009 through third quarter 2010 and were entered into a SQL server database using a form in Visual Basic.

Responses were imported into the text mining software and concepts were extracted from the text. The 5 skills from questions 6 through 10 of the questionnaire were entered into a library. The 5 skills were:

• Service orientation – actively looking for ways to help people.
• Critical thinking – using logic and reasoning to identify the strengths and weaknesses of alternative solutions, conclusions or approaches to problems.
• Reading comprehension – understanding written sentences and paragraphs in work-related documents.
• Technology design – generating or adapting equipment and technology to serve user needs.
• Operation and control – controlling operations of equipment or systems.

These skills were also entered as categories. In addition to the 5 skills, R&P added 19 other skills listed in O*NET OnLine (http://online.onetcenter.org/find/descriptor/browse/Skills/) as categories for a total of 24 skills. Selected skills were combined because of content overlap (see Table 1 for a complete list of skills, definitions, and skills that were combined). R&P used O*NET to describe skills in a nationally known context. O*NET skills are used by employers, job seekers, career counselors, and researchers to help assess skills.

To get a broader sense of the skills employers considered important, R&P combined responses to the questions about the five skills with answers to the open-ended question about which skill or skills employers considered most important for the job. All of the responses indicated as "important" on the survey were included with answers to the open-ended question for text mining analysis.

In the first pass through the data, R&P took the extraction results that were synonymous placements and grouped them into the 24 categories. For example, if an employer reported that critical thinking was the most important skill for the job, then the response was placed into the critical thinking category. The software did not always correctly determine into which categories to place concepts. R&P then reviewed the data to determine which categories to place records. "Common sense" is an example of a concept which the text mining software did not automatically place in the category of critical thinking. R&P then took this concept and placed it into the critical thinking category. In some instances concepts did not appear in the list of extraction results to be placed into categories. Records were reviewed individually and, where necessary, records were forced into categories. For example, one respondent indicated that "knowledge of the game and the ability to describe the action to the audience" was the most important skill. Because the skill is unique, R&P did not find it necessary to create a rule to categorize the skill, and thus the skill was forced into the category of speaking. Alternatively some records were placed by the software in categories where they did not belong. These records were forced out. In one such instance, "knowledge of good cleaning skills" was misplaced by the software into the service orientation category so R&P forced the skill out of the service orientation category. Although the software expedites the process of categorizing concepts through rules, libraries, and other resources, the process of determining the categories in which concepts belong is a subjective process (SPSS, Inc., 2010, p. 2).

Results

Table 2 shows the employer-reported importance of selected skills by Wyoming manufacturing employers and all employers. The first five skills were from the survey and were made into categories. Remaining skills were determined from responses to the open-ended survey question and categorized according to the O*NET skills (for a complete table showing all of the skills, go to http://doe.state.wy.us/LMI/green.htm).

Regarding skills that were part of the survey, operation and control had a greater number of manufacturing employers who reported the skill as important relative to all employers. Nearly 80% of manufacturing employers indicated operation and control to be an important skill. By comparison, 63.0% of employers across all industries described operation and control as important. Critical thinking and technology design were skills that also had a greater percentage of manufacturing employers reporting them as important compared to the total. Whereas 85.1% of manufacturing employers detailed critical thinking as an important skill, 79.4% of all employers reported the skill as important. Just over half (51.3%) of manufacturing employers noted technology design as an important skill, while 39.5% of all employers reported the skill to be important. Nearly equal percentages of manufacturing employers and total employers indicated reading comprehension as an important skill (68.1% and 68.5%, respectively). Service orientation was a skill in which 72.7% of total employers reported the skill as important compared to 51.6% of manufacturing employers.

Of the employer-reported skills, more manufacturing employers described installation as important than did employers generally. Of the 565 manufacturing employers, installation was indicated to be an important skill for 10.4% of jobs. By comparison, 4.6% of all employers reported installation skills as important. A greater percentage of manufacturing employers also detailed active listening as an important skill (5.8% compared to 4.3% of all employers).

The Figure shows satisfaction with employees' skills for manufacturing employers and all employers, respectively. About 10% of manufacturing and all responding employers did not respond to the question. A slightly higher percentage of manufacturing employers were satisfied with employees' skills (67.7%) than all employers (65.9%). A similar percentage of employers in both categories were neither satisfied nor unsatisfied with employees' work skills (15.5% of manufacturing employers, 15.0% of all employers). Slightly more of all employers were dissatisfied with employees' work skills (8.1%) than manufacturing employers (7.0%).

How did satisfaction with employees' skills factor into the importance of employees' various skills? Table 3 shows employer-reported importance of the five skills from the survey in a cross-tabulation of employer satisfaction with employees' skills for manufacturing and all industries. Note that satisfaction was with regard to employees' skills generally, not satisfaction with employees' aptitude for a specific skill. Across all five skills, a greater percentage of manufacturing employers who reported the skills as important were also satisfied with employees' skills relative to employers generally. However the difference for each skill was no more than one to two percentage points. For example, for the skill operation and control, the 80.1% of employees with whom manufacturing employers indicated they were satisfied was only 0.4% higher than for all employers (67.9%). Even though the differences between employers in manufacturing and all employers are relatively small with regard to satisfaction and skills, the consistently greater percentages for manufacturing employers suggests that, overall, manufacturing employers are more satisfied with their employees and their employees' skills than are employers generally.

The New Hires Survey found that most manufacturing employers were satisfied with their newly hired employees' skills; this was consistent with the results of the Manufacturing Training Needs Survey recently conducted by R&P (see related article). Results from the manufacturing survey of 221 employers found that 38.5% considered skills shortages to be a very important factor in deciding whether or not to expand their businesses. For the most part, Wyoming manufacturing employers were satisfied with employees' skills and those skills were not a major factor for most businesses in deciding whether to expand.
Wyoming's manufacturing industry is small relative to other industries in the state, but it is growing (see the June 2011 issue of Wyoming Labor Force Trends for a more complete discussion). Skills that employers reported as important include operation and control, critical thinking, and installation. In order to be successful in the industry, employees can acquire the skills that employers demand either on their own or through employer-provided training opportunities.

References

SPSS, Inc. (2010). IBM SPSS Text Analytics for Surveys 4.0 User's Guide.

WYSAC. (2010). Cognitive Interviews for the Wyoming Department of Employment: Testing a Job Skills Questionnaire, by T. Furgeson & M. Dorssom. (WYSAC Technical Report No. SRC-1014). Laramie, WY: Wyoming Survey & Analysis Center, University of Wyoming.

Senior Economist Sara Saulcy can be reached at (307) 473-3819 or sara.saulcy@wyo.gov.

Last modified by Phil Ellsworth.

Table of Contents | Labor Market Information | Wyoming At Work | Contact Us