In this multi-part blog series, we will cover various topics relevant to the Federal Contractor community as it relates to statistical analyses. We begin our series with a topic that is often misunderstood by both practitioners and enforcement agencies.
This past year we heard some crazy interpretations regarding the meaning of a statistically significant standard deviation test. Our favorite was:
“A significant standard deviation means that there is a 95% chance that the difference is due to discrimination.”
As a result of such statements, we thought this might be a good time to briefly remind everyone about the meaning of the term, “statistically significant.” The idea behind statistical significance testing is that anytime you gather a sample of data (e.g., hiring rates for a janitorial position, average salary for accountants) and compare two groups (e.g., men and women), due to chance alone, it is extremely unlikely that the difference between the two groups is going to be exactly zero. For example, for a particular hiring requisition we hired 76% of White applicants but “only” 70% of Hispanic applicants. Is this 6% difference the result of chance or something more sinister? What would the numbers look like across ten requisitions?
When a difference between two groups is statistically significant (e.g., the difference in selection rates is greater than two standard deviations), it simply means that we don’t think the observed difference is due to chance. The greater the number of standard deviations, the less likely we are to believe the difference is due to chance. Some things to keep in mind:
Stay tuned for additional blogs. Also, please read our white paper to learn more about the different statistical significance tests used to analyze data for the purpose of identifying disparate impact (adverse impact) which is different from disparate treatment.
By Mike Aamodt, Principal Consultant and Yesenia Avila, Associate Consultant at DCI Consulting Group