David Cohen, President of DCI Consulting Group, and Eric Dunleavy, Ph.D., Senior Consultant at DCI, published an article in the March 2009 Quarterly of the Personnel Testing Council of Metropolitan Washington DC (PTC/MW). The article, entitled Calling all Federal Contractors and Subcontractors: Understanding Adverse Impact analyses in OFCCP Enforcement, summarized a PTC/MW luncheon presentation from 2008.
“It is important to understand the statistical analyses that the OFCCP use in their enforcement, particularly if federal contractors want to strategically mirror OFCCP analyses and be prepared for an OFCCP audit. For example, it is useful to know that OFCCP continues to focus on systemic discrimination in hiring, and uses statistical significance tests like the Z test and the Fisher Exact Test to demonstrate systemic disparity” said David Cohen. Additionally, “OFCCP doesn’t seem to be using the 80% rule often in present day enforcement, although practical significance can be as important an issue in adverse impact litigation as statistical significance” mentioned Dr. Dunleavy.
The article, which is available through PTC/MW, also describes two statistical techniques that are useful when adverse impact analyses should be conducted across different strata (e.g., location, time period, job, etc.). In some cases it may be more difficult to mirror the reality of personnel practices than initially realized. Data aggregation decisions could have substantial implications when interpreting the results of analyses. Simply ignoring strata may produce misleading results in a single analysis. This is a particularly important issue in instances where the OFCCP aggregates multiple locations, years, and/or jobs.
In some situations it may be reasonable to aggregate data when small sample sizes limit the power of a statistical test across similarly situated groups. In other situations where sample sizes are very large, statistical significance tests may be trivial given high statistical power, and should be combined with practical significance tests. Given these issues, it is important that adverse impact analyses appropriately balance mirroring reality and statistical significance concerns. Toward this end, the presentation demonstrated examples of Simpson’s paradox. In some cases no adverse impact exists when analyses are conducted separately by year, but when the years are aggregated results are statistically significant. In other cases, significant adverse impact may exist in different years, but may be masked when aggregated into a single analysis.
Statistical techniques can also help the analyst make aggregation decisions that ensure reality is mirrored. For example, the Breslow-Day statistic can be used to determine if adverse impact is similar in magnitude across job, year, location, etc. and whether data can be reasonably combined into one analysis. If the Breslow-Day statistic is statistically significant, aggregation may be inappropriate because there are differences in adverse impact across strata and a strata-by-strata approach may best mirror reality and ensure that all meaningful results are identified. If this statistic is not significant, the magnitude of impact is likely similar and aggregation is probably reasonable from a statistical perspective. If aggregation is a reasonable decision, the Mantel-Haenszel statistic can be used to determine the aggregate significance of the impact. This statistic computes an overall “weighted” disparity across a set of 2 by 2 tables, and produces an appropriate summary probability estimate and standard deviation to assess significance. EEO analysts would likely benefit from understanding the statistical analyses that are conducted to determine EEO compliance.
