By: Don Lustenberger and Lisa Harpe
Our summer blog series contrasting “expert” versus “robot” approaches to pay equity continues this week as we discuss what happens after compensation data have been analyzed: investigating the results of the analysis and making pay adjustments. Our previous installments reviewed phases that lay important groundwork for these steps: project kickoff & data cleaning and analysis planning & execution.
Investigating Analysis Results
Once compensation data have been analyzed, the process of reviewing and investigating the results begins. In its simplest form, this involves understanding which analysis groups (recall the term “similarly situated employee groups” or SSEGs) show evidence of significant sex, race, or ethnicity disparities in pay — or as we may say, “which groups flagged.” From there, the approaches taken to investigate these results may differ markedly between an expert and a robot, and these differences can have crucial consequences for what an organization ultimately ends up doing with the study results.
There are several things that an expert consultant will do once the results of a pay-equity study are ready to be reviewed. Among them include the following:
- Identifying trends among the results across SSEGs
- Reviewing and interpreting statistics and model diagnostics to inform how to evaluate the results for different SSEGs
- Assisting the compensation team with understanding why an SSEG might be flagging and what to do about it—before considering making pay adjustments.
As important as it is to understand which SSEGs may be flagging, it’s also important to understand the scope of these flags and whether there are any patterns in the results. An expert will be able to summarize the results by sharing information with an organization’s compensation team about details such as (a) the percentage of SSEGs flagged with a disparity, (b) which demographic subgroups, if any, may repeatedly be favored and disfavored (which is, of course, not a good thing), and, for external experts, (c) how the incidence of flags might compare with respect to that of other organizations. An expert can also help contextualize this information: For example, if an organization has very large or broad SSEGs, it might expect to see more of its groups flag. And some of this information can help legal counsel assess risk for the organization.
Robots may very well have the capacity to tabulate flags and present them in an aggregate form. They may even be able to help benchmark how one organization’s set of flags stacks up against some norms. And they may be able to do this efficiently and with dashing visuals. However, without an expert, it’s much harder for a robot approach to contextualize the results for an organization’s compensation team. And the information that a robot presents related to these points may not matter at all — or, worse, be misleading — if it can’t do the other investigative work that we detail below.
A critically important step in investigating the results of a pay-equity study involves really digging into the results and… investigating them. An expert can scrutinize the statistics and output from results for both SSEGs that flag and those that don’t. Experts can identify anomalies in results that may warrant additional attention, and they can verify that the assumptions of the statistical models being run have been met. They can review diagnostic statistics (recall the discussion of R-squared in our previous installment), and they can run additional follow-up analyses as needed to better understand results. For example, if an organization insists that performance drives pay, but performance ratings are not predictive of pay for a particular SSEG, an expert can dive into the details and find out why (for example, perhaps it’s because everyone in the SSEG earned the same performance rating last year, or perhaps it’s because performance is highly related to another pay factor that was included in a regression model).
The process of investigating the results of a pay-equity study can be quite complicated. An expert brings both extensive experience and a sharp set of statistical tools to help understand details within the results of a study. This is not an area where a robot is likely to excel. One reason for this is that if there are issues with the results, there’s often some critical thinking that needs to be involved in exploring the issues and then deciding what plan of action to take in response. At best, a robot may be able to present some diagnostics or highlight anomalies among the results, and maybe it can offer basic suggestions for what to do considering those. But a robot is going to be very limited in terms of its ability to synthesize information about the results and recommend specific courses of action that are unique to the analysis for a specific organization. At worst, a robot approach is not even going to address any of these issues; it may just run the analysis and present the results under the potentially false supposition that everything is copacetic. And that’s not a situation in which you want to rely on a robot to make decisions involving organizational legal risk.
In conjunction with exploring the results of pay-equity analyses further, an expert can and should provide education and guidance to an organization’s compensation team to help them investigate the results of the study. This can include having conversations with the team, for example, about whether other important pay factors are available and make sense to add those statistical models where SSEGs have flagged.
This can also include encouraging the compensation team to look at individual employees who may be identified by the analysis as statistical outliers: individuals whose pay is much greater or lower than would be expected based on the legitimate pay factors included in the model. Statistical models will only sometimes include all the variables or HRIS fields that explain differences in pay (e.g., relevant prior experience or education). There may be other legitimate, business-related reasons why employees are paid differently, and sometimes a manual review of the results of individual employees within SSEGs that flag is necessary. In fact, we view a statistical flag as only an initial marker of a pay disparity between two demographic subgroups; it’s an indicator to explore the SSEG and the pay of employees in it further. It’s not necessarily a definitive signal of a pay disparity in need of remediation.
Sometimes, a pay disparity or statistical flag may be driven by a small number of employees in one demographic subgroup with relatively high pay (i.e., high outliers). In other case, there may be no statistical outliers but a large difference in average pay between males and females, for example. How or whether these situations would be addressed might differ substantially. So, having an expert guide a compensation team in reviewing pay-equity results, and exploring individual employees in SSEGs that flag can be critically important when it comes to interpreting flags and deciding what to do with them.
Perhaps some robot approaches can help organizations identify employees to investigate further or indicate where to direct their attention during this investigation stage of the study. But, again, the strength of the robot approach is not going to be offering qualitative guidance and education to a compensation team working to interpret the results of the study. On the contrary, a robot approach may offer no guidance and leave an organization to assume that the way to handle all statistical flags is essentially to throw money at them via pay adjustments. This approach is certainly concerning.
Pay Adjustments
Let’s talk about pay adjustments. When there is a significant pay disparity within an SSEG, adjusting the pay of some employees is often a practical and acceptable way to address the issue. The process for adjusting pay typically involves identifying which employees for whom an adjustment may be warranted, considering options for adjustments in conjunction with total costs, and then implementing those pay adjustments. It sounds simple, but that’s often not the case.
An expert can work with a compensation team during the investigation phase to first and foremost determine whether an SSEG that flags is a candidate for pay adjustments. There are many situations where pay adjustments may not be appropriate. Oftentimes, organizations may be interested not only in minimizing risk associated with pay disparities under Title VII of the Civil Rights Act of 1964, but they also have a genuine interest in ensuring their employees are paid fairly in general given what they bring to the table in terms of experience, skills, and performance. An expert can help develop pay adjustment strategies that allow organizations to meet both goals. It can be a tricky prospect, however, because adjusting the pay, for example, of some individuals flagged as low outliers, those who appear underpaid in SSEGs that don’t flag, can unintentionally introduce significant disparities. An expert can also provide adjustments of different sizes depending on an organization’s budget and legal counsel’s guidance for mitigating risk (more on this in our next installment). Aside from pay adjustments, an expert can also provide guidance in those other situations where pay adjustments may not be appropriate.
When it comes to making pay adjustments, there are certain areas where robot approaches may, in fact, be advantageous. They can quickly and efficiently compute adjustments and provide a compensation team with information on the total costs of those adjustments. They may allow users to divvy adjustments to disfavored employees quite flexibly and immediately observe what happens to statistical flags were those adjustments to be applied. They may even have the ability to sync those adjustments with an HRIS. And if there are changes in the workforce, it’s easier to accommodate those in conjunction with making pay adjustments. However, these benefits can only be realized if an organization is planning pay adjustments for employees under the right conditions. And a robot is not going to be as skilled in identifying and communicating those situations. An expert is going to have an edge in that area.
When it comes to spending a limited pool of money budgeted for making pay adjustments, we guarantee that organizations are going to want to be spending that money wisely to enhance pay equity and reduce legal risk. There are numerous decisions made over the course of a pay-equity study — during planning, while cleaning and analyzing data, and in interpreting results — that without proper, astute guidance, can lead to an organization with the best of intentions to spending money on pay adjustments in ways that may do no good, make existing disparities worse, or create other disparities among other protected classes. Pay equity is complicated business, and the sheen and user-friendliness of new robot approaches may belie that complexity.
It's difficult to overemphasize the importance of having an expert provide guidance to organizations conducting pay-equity studies. Another important ingredient to providing guidance is the role of legal counsel in pay equity. In our final installment of this blog series next week, we will discuss legal counsel with respect to expert and robot approaches in pay-equity analyses.
