By Lisa Harpe and Michael Aamodt
A number of DCI blogs and resources have summarized recent executive actions across the employment landscape. One theme across these actions is an emphasis on “merit-based” decision making. Although not explicitly defined, we assume based on context, that “merit-based” means worthy of reward and not due to illegal discrimination. In other words, employers should hire, promote, and pay employees based on merit and not demographics.
Most employers would agree with this concept. But how does a company ‘prove’ that employment decisions are merit-based? Within the field of Industrial-Organizational (I-O) Psychology, the concept of merit-based decisions is not new. I-O psychologists study jobs and conduct validity studies of selection procedures to demonstrate their job-relatedness. This research process can be considered merit verification. Validity studies evaluate the job-relatedness of selection procedures and the extent to which evidence supports proposed uses. These studies include an understanding of the job (i.e., required knowledge, skills and abilities (KSAs)) and an examination of the relationship between the job and the selection procedure.
The two most common approaches to validating selection procedures are content and criterion validation. Content validation examines the relationship between the content of the job and the content of the selection procedure (e.g., a data entry assessment for a data entry specialist job). Criterion validation examines the statistical relationship between selection procedure scores and measures of job performance or other important work outcomes (e.g., data entry assessment scores significantly correlate with data entry metrics tracked on the job).
Role of Demographic Information in Merit Verification Analyses
Intuitively, employers benefit from confirming that their employment processes are job related and based on merit. However, conducting this research can be complex, difficult, and potentially infeasible. For example, financial, human, and data availability may limit the likelihood of validating all selection procedures used by an employer. How can an employer determine which employment procedures should be prioritized for validation studies? Disparity analyses of employment decisions across protected groups can provide information on the likelihood that non-job-related factors (e.g., race or gender) may influence employment decisions.
These analyses are valuable tools in merit verification, and on their face allow for an evaluation of the likelihood that non-merit factors could be influencing decisions.
For example, an employer may undertake an analysis of all hiring decisions over a certain period (e.g., a year) and identify that men and women have very similar hiring rates for a particular job. This finding suggests that factors other than demographics (e.g., merit) influence decisions. However, results showing women with a significantly higher likelihood of being hired relative to men suggest the possibility that decisions were not based on job-related factors and instead could be based on demographics.
Because a final hiring decision usually results from multiple decisions in the selection process (e.g., basic qualification review, resume review, assessment, manager interview), the employer can conduct additional analyses to identify the discrete decision point(s) that may drive the overall disparity. Perhaps women and men pass the basic qualification screen, the resume review, and the interview at comparable rates, but women pass the customer service representative assessment at a significantly higher rate than men. After determining that the difference in hiring rates can be attributed to only one of the hiring components, the employer can focus on the job-relatedness of the customer service representative assessment.
When Can a Merit Verification Analysis Help?
The employer may have already conducted a validity study during the process of identifying and implementing the assessment and thus feels comfortable about the job-relatedness of the assessment. If a validity study has not been conducted, however, the employer from the previous example may decide to conduct one to ensure that the assessment is job related and that the gender differences in hiring rates are justified and due to merit. Thus, the analysis of hiring decisions allows the employer to scrutinize specific selection procedures and ensure they are merit based.
As another example, companies may conduct pay studies to ensure that pay is merit-based. A simple evaluation of base pay differences by race may show that the average pay for Asian software engineers is significantly higher than White software engineers. However, after using multiple regression analysis to control for merit factors such as prior experience, company service years, geographic differences, and skills, the company may find that this difference in pay is no longer statistically significant. That is, the initial difference in pay is explained by merit factors, not by race.
Alternatively, an employer may examine discrete employment decisions that underlie base pay such as starting pay, merit increases, or promotional increases. Where race or gender decisions appear, the employer should evaluate the factors that influence those decisions to ensure their job-relatedness.
Moving Forward
Although employers are waiting for further guidance from the federal government, evaluating merit is something that I-O psychologists have been doing for many years. The recipe for merit verification is well known and employers should continue to analyze gender and race differences in employment outcomes and validate processes used to make those decisions. These approaches are persuasive and evidence-based methods for verifying merit in employment decision making.
