In late 2021, New York City (NYC) passed a law addressing the use of Automated Employment Decision Tools (AEDT) in selection processes. The law defines an AEDT as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”
The law requires tools meeting these criteria to undergo a bias audit by an independent contractor at least one year prior to being used to select or promote any resident of New York City. The use of artificial intelligence tools in employment decision-making has been a recent topic of interest for regulatory agencies, including the Equal Employment Opportunity Commission. The new NYC law appears to additionally encompass more traditional assessments used by employers, such as automated resume screens, scored applications, personality assessments, cognitive oriented tests that use data analytics or statistical modeling.
Importantly, an independent auditor must conduct the bias audit, suggesting an in-house audit may fail to comply with the bill. The law requires that the bias audit include a disparate (adverse) impact analysis for Component 1 categories including gender and race. In addition, the law requires greater transparency around the use of AEDTs. Employers must inform job candidates of the use of an AEDT, as well as disclose the job qualifications and characteristics being evaluated. The employer must also provide an alternative selection device, if requested by job candidates. There are many nuances to this law where additional guidance will hopefully be forthcoming. The law goes into effect January 1, 2023.
So what should employers do? As a starting point, we suggest employers talk to legal counsel about this law. Based on those discussions, many employers may decide to retain the services of an independent auditor with expertise in the development, evaluation, and validation of AEDTs, as well as expertise in the analysis of AEDTs for adverse impact. While the law does not provide specifics on what the bias audit should look like, based on our extensive expertise in performing independent, third-party expert reviews over the last two decades, DCI suggests the following may be worth considering in an NYC bias audit:
- Identify all AEDTs used by your organization that are covered by the bill
- Identify how each AEDT is used in your organization’s selection process(es)
- For each AEDT:
- Evaluate the purpose of the AEDT and how it was developed, considering professional and legal guidelines with a special emphasis on examining potential bias in the development process.
- Identify the job characteristics and/or qualifications being measured by the AEDT, as well as the data inputs (features) being used by the AEDT.
- Evaluate job-relatedness (validation) evidence of the AEDT considering professional and legal guidelines with a special emphasis on examining potential bias in the validation process.
- Comprehensively evaluate adverse impact (or the potential for adverse impact) based on race, ethnicity, and gender for the AEDT with consideration to how it is being used (or how it is intended to be used operationally).
- Research potential alternative assessment processes to be provided, if requested by job candidates.
DCI expert staff are available to discuss these complex and evolving issues with you. Please reach out to a DCI expert for more information.
By: Eric Dunleavy, Vice President of Employment & Litigation Services; Lisa Harpe, Director of Selection & Assessment; Dave Schmidt, Principal Consultant
