by Amanda Shapiro, Kayo Sady, Ph.D, & Eric Dunleavy, Ph.D., DCI Consulting Group
The 27th Annual Conference for the Society for Industrial and Organizational Psychology (SIOP) was held April 26-28, 2012 in San Diego, CA. Attendees and presenters from a variety of industries and backgrounds took part in the event, including but not limited to Industrial/Organizational Psychology. As is typically the case for the Annual Conference, many sessions covered topics related to current EEO-related issues, as well as important themes concerning the workplace, such as assessment and selection, job analyses, adverse impact measurement, etc.
Notably, EEOC Chair Jacqueline Berrien provided the keynote address for the conference’s theme-track. A “theme track” is essentially a conference within a conference that took place on the first day of the conference. The track included back-to-back sessions that were all focused on one topic, which was the “Science and Practice Perspectives on Contemporary Workplace Discrimination”, a particularly relevant topic for the federal contractor community. Summaries for many of the theme track sessions, as well as other relevant sessions, are provided below. We’ve grouped sessions into three sections: (1) Compliance, (2) Diversity and Inclusion, and (3) Discrimination. DCI Consulting Group staff members were involved in a number of those presentations.
Compliance
SIOP Pre-Conference Workshop
Eric Dunleavy, Principal Consultant at DCI, and Cyrus Mehri, Co-founder of Mehri and Skalet, presented a preconference workshop where participants could receive continuing education credits. This legal update focused on plaintiff, employer, and agency perspectives for identifying, minimizing, and remedying discrimination in personnel selection outcomes. The workshop focused on a variety of topics including the Supreme Court ruling in Dukes versus Wal-Mart and related cases, as well as recent EEOC and OFCCP enforcement initiatives. One of the highlights of the preconference workshop was a mock deposition where Cyrus, a well-known plaintiff’s attorney, deposed Dr. Dunleavy in a hypothetical systemic compensation case.
During the Wal-Mart section of the training, Cyrus reiterated that the ruling was narrow and that a number of courts have taken a broader approach to class certification in spite of the Supreme Court ruling. EEO agency initiatives that were reviewed included the legality of criminal background checks, credit checks and high school diplomas for use in employee selection. This section also focused on recent OFCCP enforcement with particular emphasis on pay equity enforcement and a number of high profile settlements related to hiring systems. The session ended with a review of some high profile adverse impact cases, including a set related to the Supreme Court ruling in Ricci v. Destefano.
Keynote Address – SIOP and EEOC: Finding Common Ground
Kicking off the theme track was a keynote address from the Chair of the Equal Employment Opportunity Commission (EEOC), Jacqueline A. Berrien. Chair Berrien began the discussion by focusing on the mission of the EEOC, as well as their new strategic and enforcement plans to help define the vision of the agency. Rather than reducing discrimination, the charge of the agency is to eliminate discrimination. This seemingly small change is a result of the way in which discrimination has changed since the creation of the agency; in the past discrimination was much more overt. The reduction of discrimination is a testament to the effectiveness of the efforts of the agency, however discrimination does still exist and thus Chair Berrien feels that the agency should focus on the elimination of it. Throughout the address Chair Berrien strongly emphasized a desire for an open dialogue between EEOC and SIOP. She noted that although the paths of the two groups are not identical, they are parallel. SIOP’s focus on fair and effective assessment is parallel with EEOC’s goal of equal opportunity in the workplace.
Chair Berrien was also joined by Patrick Patterson, Senior Counsel for EEOC. Together they provided a few updates, including the number of charges filed this year, reaching 100,000, which is the most in the history of EEOC. Chair Berrien estimated that of those charges, she expects 20-25% to be “cause-finding”. Other updates included the EEOC meeting on criminal checks that occurred the previous day (April 25, 2012). You can find a summary of that meeting and guidance in a
previous blog. Chair Berrien mentioned other important topics that the agency is (or has) reviewed include, credit checks, employee status, barriers to Veterans and Individuals with Disabilities, trafficking, and pregnancy/caregiver discrimination.
Job Analysis in the Legal Environment
Job analysis is vital to the effectiveness and defensibility of any selection system component. This panel session focused on the unique issues associated with conducting job analyses in a high-stakes legal environment, specifically discrimination or wage and hour litigation. The panel consisted of several experts with different areas of specialization, including Cristina Banks (Lamorinda Consulting, LLC), Mark Wilson (NC State University), Rick Bergstrom (Jones Day), Gavin Appleby (Littler Mendleson, P.C.), and Rick Jacobs (Pennsylvania State University). The panelists fielded a series of discussion questions and provided recommendations and advice based on professional experience. The panel provided a series of “best methods”:
- No one job analysis methodology is best for every situation, and even good job analysis methodologies will be scrutinized by judges or colleagues. Sometimes, a method must be “invented” in order to effectively to meet the objectives of the organization and the requirements of the courts.
- Consultants/psychologists should perform the job of interest to get a unique perspective that allows for better questions and research.
- Involve a physician whenever there is an issue of ADA compliance. It’s always important to get information from “the right people” and in the case of ADA issues; the right people are doctors and not incumbents (who are traditionally the subject matter experts).
- Judges and jurors typically like information from direct observation and structured surveys. Subject matter expert ratings and confirmation of results tend to “push a judge over the edge”
- The courts respond positively to the use of “rater confidence” metrics provided by SMEs.
- It’s a difficult balance to get the right level of detail; it’s a fight between trivializing the job with generalities and penalizing with minutia. There is also a balance between a good, thorough job analysis and the cost to clients.
- Documentation of the job analysis process and findings is vital to legal defensibility.
- An organization’s job structure should not be automatically trusted to provide accurate representations of the relationships between jobs. Many organizations have poor classifications.
- If the organization is unionized, union reps should be involved as early as possible to avoid conflict with, or sabotage by, union members.
- Cumulative knowledge (i.e., O*NET) is a great starting point for job analysis but should not be relied on exclusively.
Community of Interest on EEO Issues
Another interesting session was a community of interest on EEO issues. Community of interest sessions are informal and allow participants to come and go as they please, and ask any questions that they would like. Dr. Dunleavy facilitated this community of interest to a full room of participants. A number of interesting questions raised focused on the Americans with Disabilities Act, content oriented validation strategies, recent EEOC and OFCCP enforcement patterns, and regulatory change related to EEO/AA for veterans and persons with disabilities. Notably, the session was attended by EEOC staff and they contributed substantially to the conversation. The session ended with a preview of the Supreme Court’s decision in Fisher versus the University of Texas, which is the latest challenge to affirmative action and the use of race in college admission decisions.
Distinguished Early Career Contributions Award (Practice): Navigating the Maze of Early Career Practice: Scientist–Practitioner Reflections
In 2011, Eric Dunleavy was the first recipient of the “Distinguished Early Career Contributions Award for Practice”. In the past the award was only awarded for scientific contributions and thus this was the first year a separate practitioner award was presented. One of the honors of the award is the opportunity to present at the following year’s annual conference on a topic of the recipients choosing. Dr. Dunleavy’s presentation focused on three main topics: (1) the importance of a practitioner SIOP early career award, (2) advice to aspiring, young practitioners, and (3) a “speed-round” primer on legal risk in personnel selection (the main focus of Dr. Dunleavy’s work).
The valuable take-home information for this blog is the primer on HR Risk in Employee Selection, a topic that Dr. Dunleavy routinely consuls on with clients, governmental agencies, and other I/O psychologists. Dr. Dunleavy first provided an overview of the traditional chronology that occurs when there is a finding of adverse impact and what evidence determines if the employer or plaintiff wins the case. He followed with a discussion of the important roles of sample size and statistical power in these analyses and advised that both statistical and practical significance be considered in the evaluation of selection ratios. He provided a framework to interpret the meaning of significant or non-significant statistical analyses with large or small practical differences. Dr. Dunleavy then focused on the importance of validity evidence in conjunction with adverse impact analyses, and the possible challenges that may be brought. To combat these challenges, Dr. Dunleavy reviewed the important professionally acceptable and legal research, with a major focus on the Uniform Guidelines on Employee Selection that included some of the major criticisms of these regulatory guidelines. To conclude the presentation, Dr. Dunleavy provided a few tips for organizations and consultants to “stay out of trouble”, which include:
- Information is power: Stay on top of current research and best practices
- Use structured selection tools
- Measure adverse impact in strategic and reasonable ways that mirror reality
- Consider reasonable alternatives and document
- Think strategically about what validation strategies make sense in a given context
Diversity and Inclusion
Setting Standards for Diversity and Inclusion: Working with SHRM
This symposium included members of Society for Human Resource Management’s (SHRM) project team responsible for developing a set of standards for diversity and inclusion (D and I). More specifically, this group is tasked with establishing a national standard for defining and measuring D and I and guide the development of an accreditation process for organizations. Development of the standards started four years ago, and once they are finalized, the Standards will be available free of charge. SHRM’s hope is that organizations will use and embrace these voluntary standards. The main interest of the Standards committee was to define diversity, establish meaningful metrics, and lobby for adoption of the definition. The committee includes all “interested groups,” including academics and consultants. During the development of the standards, the committee explored the following questions: (1) What is a Diversity Program? At a minimum, what should you be doing?; (2) What is a diversity leader? What is their competence?; and (3) How do we measure the effectiveness of a diversity program (i.e., what metrics should we use)? Presenters included Lee Webster (Director HR Standards at SHRM), Lauren Ramsay (San Jose State University), Marc Bendick, Jr. (Bendick and Egan Economic Consultants, Inc.), and Rosemary Hays-Thomas (University of West Florida).
Affirmative Action in the New Millennium: Perspectives, Advancements, and Limitations
In a panel discussion about the history and future of affirmative action programs, Eugene Stone-Romero (University of Texas at San Antonio), Dianna Stone (University of Texas at San Antonio), and Gwendolyn Combs (University of Nebraska) provided an overview of affirmative action programs (AAPs) and their thoughts on the need for AAPs in the future. The panel’s assessment was that while overt discrimination may no longer be as prevalent as it once was, AAPs are still needed to protect against existing pockets of overt discrimination and widespread covert discrimination in which stereotypes, biases, and societal forces may act to limit opportunities in the absence of affirmative action plans.
After a review of relevant case law affecting affirmative programs, much of the discussion turned to the upcoming Supreme Court decision on Fisher v. University of Texas at Austin. Topics discussed included the implications of Justice Kagan’s recusal from the case and the likely SCOTUS decision. The primary question raised was whether the decision would look more like the Grutter v. Bollinger holding or the Gratz v. Bollinger holding. In Grutter v. Bollinger, the court held that race could be one of many factors considered as long as the selection policy was narrowly tailored to allow individual discretion in the consideration of race as a “plus” factor and did not result, effectively, in race quotas. In Gratz v. Bollinger, the court held that policies are unlawful to the extent that the consideration of race are too rigid and do not allow individualized consideration of applicants and result, effectively, in race quotas.
Discrimination
Racial Differences in Personnel Selection: Complex Findings and Ongoing Research
Another interesting presentation at the conference was a symposium entitled “Racial Differences in Personnel Selection: Complex Findings and Ongoing Research”. The session was chaired by Dr. Leaetta Hough and included a balanced set of paper authors from academia and practice. The first presentation from Oppler et al. focused on whether the extent to which test taker performance is over- or under-predicted or could be a factor in identifying reasonable alternatives as is required by the uniform guidelines. The second presentation by Berry et al. focused on recent research on differential validity, and suggested that variance differences were not an explanation for why differential validity may exist. Next, Scherbaum et al. summarized some recent research on the Sienna Reasoning Test (SRT) which is a relatively new measure of cognitive ability that apparently has lower subgroup differences and similar prediction as compared with traditional measures of cognitive ability. McDaniel et al. presented an alternative explanation for SRT results, which is called Spearman's hypothesis. This approach suggests that cognitive ability tests with lower subgroup differences are in fact measuring less cognitive ability and more of other constructs. The session then became heated as a debate began over the extent to which the developers of the SRT have made all background and research data available to the SIOP community. The session ended with an interesting set of discussants. Dr. Jim Outtz focused on some of the positive aspects of SRT research, while Dr. Deniz Ones conducted a series of analyses on the data made available to the panel and suggested that more research is needed to understand the SRT.
Disability in the Workplace: Contemporary Challenges Beyond ADA Compliance
The symposium focused on research of contemporary issues for disability in the workplace, specifically focusing on issues that may result in differential opportunities for individuals with disabilities but that are not currently addressed by legislation. Highlights included:
- Presenters felt that the ADA has not made a large enough impact. They noted that after Title VII was put into effect, there was a quick, and major, shift in the employment rates of women and minorities. This change hasn’t been seen for individuals with disabilities and it has been 20 years since ADA.
- When asked about the proposed changes to Section 503 by the OFCCP, the presenters felt that they would be difficult for organizations to implement. It was noted that organizations typically have a difficult job in obtaining more than a 2% population of individuals with a disability, so the 7% goal would be very difficult to achieve. Self-identification pre-offer was also met with criticism.
- Often a perception of employers is that individuals with disabilities will result in high costs to the organization because of expensive accommodations and healthcare increases.
- Individuals with disabilities are detached from the workforce.
- The differences between subtle and overt mistreatment, as individuals with disabilities tend to experience the former more often.
- Psychological and neurological disabilities are more often discriminated against than physical because organizations are worried performance will be less stable and more difficult to predict.
Test Fairness: Legal and Psychometric Issues and Controversies
John Weiner (PSI), Keith Pyburn (Fisher & Phillips, LLP), and James Sharf (Employment Risk Advisors, Inc.) conducted a session concerning legal and psychometric considerations surrounding test fairness. The panel provided overviews of test fairness from the social, legal, and statistical perspectives and highlighted how the definitions of test fairness from the three perspectives may be at odds. Audience discussions focused on leading-edge research and practice with respect to test fairness, and active exchange between panel and audience provided a lively and informative session.
Settling Workplace Discrimination Cases: The Dos, Don'ts, Costs, and Benefits
As a session part of the SIOP Theme Track, a number of national experts in employment law provided background information on the state of the science and practice with respect to unlawful employment discrimination and EEO settlements. A mock settlement negotiation conducted by defense and plaintiff attorneys followed the review. Jim Outtz provided opening and closing remarks, followed by Eric Dunleavy from DCI and Greg Mitchell from the University of Virginia School of Law who covered recent history and trends in EEOC and OFCCP activity. Dr. Dunleavy and Mr. Mitchell provided a number of interesting facts concerning EEO activity; including:
- EEOC settled 3,671 cases over the past decade with the most settlements arising from the Philadelphia district.
- Sex- and race-based cases comprised the large majority, at approximately 59% of total cases and 23% of total cases, respectively.
- Alarming from an industrial-organizational psychology perspective was the fact that very few cases required the implementation of structured processes as part of the remediation plan.
- An interesting trend in the OFCCP space concerns the increase in compensation-related settlements. As noted by the panel, in 2011, approximately 1/3 of all OFCCP settlements were compensation related, although the settlements tended to be small in pure dollar amounts.
The mock settlement negation in the latter half of the session involved a fictitious case in which the employer uses an off-the-shelf paper-and-pencil cognitive test (that has documented adverse impact against minority test-takers) as part of the process for hiring entry-level manufacturing jobs. The employer conducted a job analysis seven years prior to the case and has not conducted a local validation study, although the test vendor has conducted 10 criterion-related validation studies on entry-level jobs in separate companies, and the validity coefficients in all 10 studies were statistically significant. The primary focus of the defense strategy articulated by David Copus (Ogletree Deakins) and Geoff Weirich (Paul Hastings) was to highlight problems with the Uniform Guidelines on Employee Selection Procedures (UGESP), underscore the importance of considering validity generalization evidence, and point to the validation results conducted by the vendor as the basis for a transport validity argument. In contrast, Brad Seligman (The Impact Fund) argued the plaintiff’s case that a lack of updated job analysis data and a local validation study undermine support for the use of a cognitive test with established adverse impact against minority candidates, that a vendor’s research on its own measure is suspect as the vendor is not an impartial party, and that reasonable alternatives exist that minimize the adverse impact against minority applicants.
The closing remarks highlighted the importance of experts such as I-O psychologists in the settlement negotiation process, as they help the different sides determine the important factors to consider when settling, but all parties agreed that professional best practices in testing and hiring should not be overlooked prior to charges being brought.
Antidiscrimination Law: Past Successes, Current Concerns, and Future Directions
In a Friday afternoon session, presenters highlighted a series of independent research programs on antidiscrimination legislation. Enrica Ruggs (Rice University) presented survey research examining perceptions of the impact of anti-discrimination legislation in Science, Technology, Engineering, and Math (STEM) fields. The results of the research indicated some interesting patterns of perceptions of the Civil Rights Act (CRA) among individuals in STEM fields. For example, females reported benefiting more from the CRA than males, and White participants reported benefiting more form the CRA than Non-White participants. Interestingly, Non-White male participants reported benefiting the least from the CRA.
Cody Cox (University of Texas at Brownsville) presented findings from experimental studies evaluating how the Gross v. FBL Financial Services (2009) holding may affect employers’ treatment of older workers. Their results suggested that individuals were more likely to favor retraining over terminating older employees if they were told that it was unlawful to consider age when making employment decisions (contrary to the Gross Standard). In contrast, participants were more likely to favor terminating over retraining older employees if they were told that age was one factor, but not the only factor, they could consider when taking an employment action.
Charlie Law (Penn State Schuylkill) presented research conducted at the Defense Equal Opportunity Management Institute that suggested that perceptions of ageism, ableism, and religious discrimination negatively affect job satisfaction, and consequently, organizational commitment.
Last, Mark Roehling (Michigan State University) presented a thoughtful piece on Family Responsibilities Discrimination (FRD), which is defined as discrimination against employees based on their responsibilities to care for family members. Mark noted that FRD is not explicitly covered by Federal law, but outlined the patchwork of different state and federal angles under which FRD cases have been brought.
Dukes v. Wal-Mart
Another interesting session focused on the Supreme Court ruling and Dukes versus Wal-Mart. Dr. Eric Dunleavy provided a short summary of the Supreme Court ruling, as well as the earlier district and appeals court rulings. Readers of this blog know that the Supreme Court ruled in favor of Wal-Mart, decertifying the class because the plaintiffs did not have “the glue” that would connect the employment decisions and potential victims together. Next, Dr. William Bielby, a sociologist who was the plaintiff expert in the Wal-Mart case presented the latest research on social framework analysis. Recall that Justice Scalia was not a fan of the social framework analysis, and suggested in the written decision that it did not inform on whether there was any causal mechanism to potential disparities against women. Perhaps the point that is most interesting from Justice Scalia’s treatment of social framework analysis is the fact that it was unclear whether discrimination was the explanation for 5% of the decisions or 95% of the decisions. Dr. Bielby suggested that the social framework analysis was misunderstood by the Supreme Court, and that it may be a more useful paradigm for defense attorneys.
Dr. Jerry Barrett then presented a detailed criticism of social framework analysis, with particular emphasis on laboratory research. Dr. Barrett reviewed research concluding that there was an inherent bias that was unconscious and emphasized that none of this social psychology research conducted in the lab has been replicated in organizational settings. Dr. Barrett suggested that there is no evidence that generalizes to the workplace. The session concluded with a presentation by Dr. Mike Campion, who discussed what the Wal-Mart ruling may have changed in terms of best practices in I/O psychology, and what best practices are still best practices. One of these points focused on the fact that unstructured and subjective decision-making at Wal-Mart was decentralized and extremely subjective, and this was one reason why the class could not be certified. However, structured scientific approaches to selection may still offer more defensibility. The session concluded with a reminder that the Wal-Mart saga is not over, and that smaller classes of females are taking their cases to individual courts. Stay tuned.
As you can see by the above summaries, once again the SIOP conference included a number of informative and cutting edge EEO presentations. We expect the same next year at the SIOP conference in Houston, as I/O Psychologists continue to perform important work in the EEO context.