by Art Gutman Ph.D., Professor, Florida Institute of Technology
The case is Bazile v. City of Houston, decided by District Court in the Southern District of Texas decided on 2/6/12. (see 2012 U.S. Dist. LEXIS 14712). This is a very complex case that deals with some of the issues raised (but not resolved) in the Ricci ruling related to written exams versus other types of tests, most notably situational judgment tests and assessment centers. It’s a long 100-page (or so) ruling that would interest the purists among us. It involves several well-known experts. For purposes of exposition, I will overview the ruling (by Judge Lee H. Rosenthal) in this alert, and write a separate follow-up on what the experts said and how those views were evaluated by Judge Rosenthal.
The promotions in question were to captain and senior captain. The time-honored practice in Houston was to promote based on years of service and scores on a multiple choice knowledge exam, a format established by Texas Local Government Code (TLGC) and in a collective bargaining agreement (CBA) between the city and the Houston Professional Fire Fighters Association (HPFFA). The city was sued by seven black applicants in 2010 with whom it reached a settlement. More importantly, the city agreed to consent decree for changes in the promotion process to be done in two phases. Phase 1 involved a relatively minor change --- adding situational judgment multiple choice items to the knowledge items traditionally used. This exam was administered on captain exams in November 2010 with few objections. However, major changes agreed to in Phase 2 by the city for the May 2011 promotional process were opposed by the HPFFA, which argued it was in violation of the TLGC.
What the city proposed was reducing the knowledge test to pass/fail and incorporating additional situational judgment and assessment center components, followed by rank order listing of based on a computer based situational judgment test for eligibility for the assessment center, and a sliding band selection based on the assessment center (without race as a factor in the ultimate selections). Reducing the knowledge test to pass/fail would alone reduce adverse impact by expanding the pool for the situational judgment and assessment center since the passing score (70%) is far below the effective score for promotion on past knowledge exams. That is, knowledge test performers who would pass at 70% but would be well below the effective score for promotion would now be on equal footing with all others scoring above 70% entering the situational judgment test.
To make a long story short, Judge Rosenthal accepted adding the situational judgment test and assessment center, as well as minor changes to seniority points (half a point for each year of service with a maximum of 10 points allowed). However, he rejected reducing the job knowledge test to pass/fail, and eliminating discretionary decisions by the test designer (to chose the cutoff score for passing and, more importantly, choosing not to use the knowledge test at all). Judge Rosenthal also rejected rank ordering from situational judgment to assessment center and final rank ordering from the assessment center. Also rejected were sliding bands based on judgments of a consultant, documented reasons for the fire chief to make selections from within bands, and the so-called “rule of three” (selecting any of the next three top scorers).
That’s a lot to chew. I have much more to discuss with relation to determining adverse impact and issues related to validity as expressed by the various experts. Stay tuned for Part 2.