by Art Gutman Ph.D., Professor, Florida Institute of Technology
In his recent invited address to the International Personnel Assessment Council (IPAC) (July 2010), SIOP Fellow Dr. Michael A. McDaniel gave seven specific reasons (or flaws) for abolishing of the Uniform Guidelines. Mike has made available his slides for his talk.
Briefly, the flaws are as follows:
Flaw #1: Too Much Emphasis on local validation studies
The arguments here is that the Uniform Guidelines were published before the situational specificity hypothesis was refuted, and that newer publications such as the APA Standards and the SIOP Principles recognize the futility of conducting local validation studies without large sample sizes.
Flaw #2: Ignoring Meta-Analytic Research
The argument here is that meta-analytic studies have demonstrated validity generalization of employment tests, and that the Uniform Guidelines have not been updated to reflect this research, whereas the Standards and Principles have.
Flaw #3: “Job Analysis – My Way or the Highway”
The argument here is that the Uniform Guidelines endorse a specific approach to job analysis for demonstrating content validity, whereas the Standards and Principles endorse a variety of approaches for demonstrating validity.
Flaw #4: The “Assassination” of Construct Validity
The argument here is that the Uniform Guidelines define construct validity in a way such that nobody will want to use it.
Flaw #5: A 1950’s Perspective of Separate Validity Methods
The argument here is that the Uniform Guidelines maintain an outdated “Trinitarian” view of methods of demonstrating validity (content, criterion & construct validity), with separate rules for each method. On the other hand, the Standards and Principals maintain a “Unitarian” approach in which different sources of evidence contribute to inferences drawn from selection procedures.
Flaw #6: The Diversity-Validity Dilemma
The argument here is that tests that best predict job performance tend to be the ones with the most race-based adverse impact, implying that diversity goals and merit selection are typically in conflict.
Flaw #7: The Uniform Guidelines offer recommendations on differential validity and differential prediction that are at odds with scientific knowledge and professional practice as codified in the Standards and Principles.
I’d like to thank Mike for sharing his thoughts and slides with us. I would add that there is no serious debate at this point that the Uniform Guidelines are outdated. Indeed, it is arguable that they were outdated in 1978, the year they were published. Several people (present company included) have cited many specific guidelines that have been struck down or reinterpreted in the courts. Furthermore, there is little doubt that the Standards and the Principles represent the most current thinking on test validity. Finally, Mike’s purpose in sharing with us is to elicit comments and criticisms. Please feel free to do so, both on the comment section at the end of this column and directly to Mike.
Flaw #1: Too Much Emphasis on local validation studies
The arguments here is that the Uniform Guidelines were published before the situational specificity hypothesis was refuted, and that newer publications such as the APA Standards and the SIOP Principles recognize the futility of conducting local validation studies without large sample sizes.
Flaw #2: Ignoring Meta-Analytic Research
The argument here is that meta-analytic studies have demonstrated validity generalization of employment tests, and that the Uniform Guidelines have not been updated to reflect this research, whereas the Standards and Principles have.
Flaw #3: “Job Analysis – My Way or the Highway”
The argument here is that the Uniform Guidelines endorse a specific approach to job analysis for demonstrating content validity, whereas the Standards and Principles endorse a variety of approaches for demonstrating validity.
Flaw #4: The “Assassination” of Construct Validity
The argument here is that the Uniform Guidelines define construct validity in a way such that nobody will want to use it.
Flaw #5: A 1950’s Perspective of Separate Validity Methods
The argument here is that the Uniform Guidelines maintain an outdated “Trinitarian” view of methods of demonstrating validity (content, criterion & construct validity), with separate rules for each method. On the other hand, the Standards and Principals maintain a “Unitarian” approach in which different sources of evidence contribute to inferences drawn from selection procedures.
Flaw #6: The Diversity-Validity Dilemma
The argument here is that tests that best predict job performance tend to be the ones with the most race-based adverse impact, implying that diversity goals and merit selection are typically in conflict.
Flaw #7: The Uniform Guidelines offer recommendations on differential validity and differential prediction that are at odds with scientific knowledge and professional practice as codified in the Standards and Principles.
I’d like to thank Mike for sharing his thoughts and slides with us. I would add that there is no serious debate at this point that the Uniform Guidelines are outdated. Indeed, it is arguable that they were outdated in 1978, the year they were published. Several people (present company included) have cited many specific guidelines that have been struck down or reinterpreted in the courts. Furthermore, there is little doubt that the Standards and the Principles represent the most current thinking on test validity. Finally, Mike’s purpose in sharing with us is to elicit comments and criticisms. Please feel free to do so, both on the comment section at the end of this column and directly to Mike.
