1. Create a self-scoring, online self assessment instrument that measures the work values identified by the Theory of Work Adjustment (Dawis & Lofquist, 1984). The Work Values Assessment test is based on previously developed measures of work values, the Minnesota Importance Questionnaire (MIQ; Rounds, Henly, Dawis, Lofquist, & Weiss, 1981) and O*NET Work Importance Locator (WIL; O*NET, 1999). However, both the MIQ and the original version of the WIL are not well suited for self-assessment. In particular, they were pen-and-paper instruments that employed complex manual scoring systems that typically required the assistance of a counselor or teacher. Steps were taken to design the Work Values Assessment test in a manner that allows users to take the test independently and have their scores tabulated automatically by the system without the aid of a counselor or teacher.
   
   
2. Develop an instrument that reliably and accurately measures the work values identified by the Theory of Work Adjustment (Dawis & Lofquist, 1984). While the Work Values Assessment test is based on the MIQ and the WIL, modifications to the items and procedures were necessary. Steps were taken to ensure and verify that such alterations did not compromise the instrument’s psychometric characteristics, including the test-retest reliability, internal consistency, and accuracy of its scores. The following sections summarize the development and evaluation of the WIL and the Work Values Assessment test that followed.
   
   
3. Develop a system and methodology for profiling Work Needs and Work Values for all the careers found in the Dictionary of Occupational Titles (DOT; U.S. Department of Labor) and subsequently in the O*NET database of careers. These career profiles would need to characterize the content of work (e.g., authority, creativity) and conditions of the work environment (e.g., compensation, advancement potential) for 1,122 different occupations by way of a scale that supported the Theory of Work Adjustment and its basic needs statements. Two separate approaches were employed – one empirical and one rational in nature – to develop reliable and comprehensive Work Value/Needs profiles for all 1,122 different careers. These studies proved instrumental in the development the CareerMatch™ scoring system.

The Test. The Work Values Assessment test uses a card-sorting task that is similar to the Q-sort technique (Stephenson, 1953). Users complete the test by sorting 20 cards, each containing a description of a need statement, in terms of their relative importance in the user’s ideal job. The following are examples of need statements:

Pilot Studies. Two pilot studies were conducted with the modified 5-column instrument. A pen-and-paper version of the instrument was administered to a total of 69 employment service clients in North Carolina and Utah. Users were asked to complete the test unaided and give feedback on the clarity of instructions, easy-of-use, and other factors.

Most participants completed the test without difficulty. Some expressed frustration with having to assign low ratings to some of the needs statements, considering them all important. Some were confused by the meaning of some of the cards. Some had suggestions for improved clarity in the instructions. Based on this feedback, new versions of the instrument were developed.

Main Study. In this Main Study, three versions of the test were administered to employment center clients and junior college students. Respondents in the Main Study took two of the three instruments so that information on the same respondents taking different measures would be available. The design attempted to balance the pairing and ordering of measures taken by respondents so that all pairs and orders would be represented. The overall sample size was 1,199 respondents. The Main Study allowed for a quantitative evaluation of the WIL’s psychometric characteristics. Results from the Main Study are detailed in the section on Evaluation & Measurement.

Introduction. The research design for generating work values score profiles for 1,122 different careers involved obtaining score profiles from two sources: a) regression equations that produced estimated work values scores, and b) an SME (Subject Matter Expert) study in which work values scores were derived from expert judgments of occupational analysts.

Regression Study. The regression study met with some success, yielding equations that accurately estimated scores for three of the six work values. Specifically, the procedure was to a) obtain a sample of jobs having work values scores, b) regress these scores on variables that measured important characteristics of the jobs, and c) evaluate the degree of relationship between the predicted and actual scores. Variables included GATB Aptitude Ratings, Temperament Ratings, and DOT data on environmental conditions and physical demand of various occupations for a total of 61 input variables.

The first step was to reduce the number of variables while retaining as much of the variance present in the scores as possible. A principal components analysis of the 61 variables yielded a 13-component solution.

The next step was to regress each of the six work values dimensions on the 13 principal components scores. 50 samples containing a randomly drawn 60 percent of the occupations (here, 108 occupations) were created. For each of the 50 samples, a regression equation was calculated for each of the six work values dimensions. Thus, for a given work values dimension, there were 50 estimates of the coefficient of determination (R²), and each component score had 50 estimates of its dimension-specific regression coefficient. The mean R² values were respectable, with three of the values scores being predicted quite well.

SME Study. The SME study comprised three phases: a) initial development of materials for rating the work values score profiles, b) a Pilot Study to refine the materials and determine whether non-incumbent raters could provide reliable results, and c) a Main Study in which work values score profiles for all 1,122 careers in the database were created based on their ratings on the 21 need statements.

During the SME study, occupationally anchored rating scales were developed for judges to rate the extent to which the 21 needs measured by the test are reinforced by various occupations. A Pilot Study in which nine non-incumbent raters (I/O psychology graduate students) rated 30 occupations on the 21 needs demonstrated that (a) raters using the rating scales could provide reliable ratings, and (b) the judges provided work values profiles that correlated .50 or higher with profiles for the same occupations published in 1986 (Stewart, 1986) by incumbent raters.

In the Main Study, 17 occupational analysts and I/O psychology graduate students each rated the work values score profiles for half of the 1,122 careers. Raters provided profiles of adequate and acceptable reliability, with mean and median interrater reliabilities in the .80s. The average correlation between need profiles from incumbents and from SMEs was .37, with incumbents rating their jobs higher on the needs. For 33 percent of the occupations, the top two needs were identified by SMEs and incumbents.

Summary. The work values scoring project produced reliable, valid measures of work values and occupation profiles of value scores. Two methods of producing profiles (one empirical, one rational) were investigated. The data obtained from these investigations supported the use of the SME ratings of work values scores for the careers that appear in the CareerMatch™ system. For example, profiles generated by the SMEs evidenced appreciable reliability, respectable correlation with profiles obtained by job incumbents, and reasonable patterns of work values scores across occupations.