NY State Audits Bias Audit Law Enforcement, NYC Promises Improvements

By Bre Timko & Dave Schmidt

BLOG OVERVIEW: A recent audit by the New York State Comptroller's Office revealed significant shortcomings in New York City's enforcement of Local Law 144 (LL144), the landmark legislation requiring bias audits of automated employment decision tools (AEDTs) used in hiring. Key findings include inconsistent complaint intake processes, incomplete documentation, limited proactive enforcement, and inadequate bias audit reviews by the NYC Department of Consumer and Worker Protection (DCWP). With DCWP committing to adopt most of the audit's recommendations, employers using AEDTs should prepare for stronger, more structured enforcement of NYC's AI hiring law going forward.

In response to the growing reliance on automated employment decision tools (AEDTs), New York City (NYC) enacted Local Law 144 (LL144) to promote transparency, fairness, and accountability in hiring and employment practices. The law requires employers to annually conduct and publicly post independently conducted bias audits of AEDTs, notify candidates about the use of AEDTs, and comply with enforcement mechanisms that include civil penalties for violations.

Between July 2023 and June 2025, the Office of the New York State Comptroller’s Division of State Government Accountability¹ evaluated whether the NYC Department of Consumer and Worker Protection (DCWP) designed and implemented an effective system to enforce compliance with LL144 and published their findings last month in a comprehensive audit report.

This audit found major shortcomings in NYC’s enforcement, including inconsistent complaint intake processes, incomplete documentation, unresolved complaints, error-prone audits, and a lack of resources and outreach for stakeholders. As a result, DCWP has committed to strengthening enforcement, and covered employers should be prepared.

Audit Methodology

The Comptroller Office’s review included a combination of interviews, document review, and practical testing. Key steps in the audit included:

Stakeholder interviews: Engaged with personnel from DCWP, the Office of Technology and Innovation (OTI), and the City Commission on Human Rights (CCHR) to understand enforcement processes and challenges.
Policy and legal review: Examined LL144, related publications, and DCWP policies and procedures relevant to complaint intake, investigation, and enforcement.
Test calls and inquiries: Conducted test calls² and reviewed the online complaint submission process to evaluate how complaints were received and routed; reviewed call center inquiry data to evaluate complaint responsiveness.
Compliance assessment: Reviewed a DCWP evaluation of 32 companies with published bias audit results³; examined recent Equal Employment Opportunity Commission (EEOC) cases and Federal Trade Commission (FTC) complaints to identify AEDT vendors or users potentially subject to LL144 in an effort to proactively identify cases of non-compliance.

This methodology provided insight into the effectiveness of DCWP’s complaint handling, investigation, and enforcement processes, as well as broader patterns of compliance with the law. The key findings and recommendations from the audit, organized by the complaint process and the investigation and enforcement process, are summarized below.

Complaint Process Findings and Recommendations

The Comptroller’s Office reported several key findings on the complaint process, including:

Online complaint instructions are unclear, with no explicit guidance for filing AEDT-related complaints.
The complaint intake process for LL144 is inconsistent and ineffective. Most of the test complaint calls from the Comptroller’s Office to the 311 system failed to reach DCWP, and they were often misdirected to other agencies. In one circumstance, a test caller complainant was instructed to submit the complaint to the employer that was allegedly in violation of LL144.
Complaint tracking and documentation are incomplete, with information missing on the nature of the complaint and any resolution achieved. Most of the test calls were not recorded in the system as AEDT complaints.
DCWP has conducted minimal outreach or stakeholder education regarding LL144 and has not evaluated why complaint volume is extremely low.
There are structural gaps such as delayed implementation of electronic complaint records via the phone complaint process that limit DCWP’s ability to identify and address non-compliance, particularly when employers do not proactively disclose bias audits.

The Comptroller’s office made recommendations to DCWP, including:

Implementing clear, standardized procedures to ensure AEDT complaints are properly received, documented, and routed for review.
Publishing clear guidance for submitting AEDT complaints and providing cross-divisional training on complaint handling.
Establishing centralized tracking of AEDT complaints and conducting outreach to reduce barriers to reporting.

Investigation and Enforcement Findings and Recommendations

The Comptroller’s Office reported several key findings on investigation of complaints and enforcement, including:

DCWP’s enforcement efforts are limited and inconsistent, with formal investigative procedures and available OTI resources largely unused.
DCWP identified minimal non-compliance among a review of 32 companies identified in publications from Cornell and the American Civil Liberties Union (ACLU), despite the Comptroller’s Office identifying 17 potential LL144 requirement violations related to bias audit quality, auditor independence, data/methodology, and public posting.
Complaint-driven enforcement dominates, but many complaints are unresolved or inadequately reviewed.
Proactive enforcement is allowed under LL144, but is rarely used.
DCWP does not routinely monitor compliance with annual bias audit requirements, identify audit dates, or pursue proactive enforcement leads (e.g., EEOC cases or FTC complaints).
Enforcement actions have been rare and insufficiently detailed, and DCWP has not followed up to ensure remediation.

The Comptroller’s office made recommendations to DCWP, including:

Strengthening enforcement by partnering with OTI more frequently to utilize formal investigative procedures, technical expertise, and processes for structured bias audit reviews.
Adopting proactive strategies to identify potential non-compliance beyond complaints, including research-based and inter-agency leads.
Using a broader range of investigative techniques – such as interviews, system demonstrations, and documentation reviews – to support consistent and effective enforcement.

In summary, DCWP’s current approach relies heavily on a complaint process that is difficult to access and inconsistently applied, while underutilizing available tools for investigation and enforcement. Without clearer intake procedures, stronger tracking, and more robust investigative practices, DCWP’s ability to identify and address non-compliance with LL144 remains limited.

Response from DCWP

In response to the audit, DCWP stated that it does not believe there is evidence of widespread non-compliance and cited resource constraints as the basis for limiting enforcement efforts to a primarily complaint-driven enforcement approach, while also acknowledging that the Comptroller’s Office provided useful insights. DCWP agreed to fully adopt 10 (and partially adopt one) of the 13 recommendations from the audit, committing to clarifying filing instructions, cross-training staff, and formalizing complaint tracking from intake through enforcement. They also committed to consulting with OTI for technical expertise, strengthening bias audit reviews to address potential non-compliance, monitoring annual bias audit requirements, and using a broader range of investigative techniques, such as interviews, AEDT demonstrations, and reviews of system documentation.

DCWP declined to implement recommendations that involved conducting research into the low volume of complaints, creating processes to proactively identify potential non-compliance beyond complaints, or fully investigating potential violations identified in the Comptroller’s audit – though they may decide to review select instances of potential non-compliance.

Implications

The audit highlighted meaningful gaps and raised questions about the effectiveness of LL144 enforcement and oversight during the audit period. DCWP’s commitment to implementing many of the audit’s recommendations signals the potential for more structured and consistent enforcement going forward. If these changes are fully implemented, there may be clearer expectations for employers, improved monitoring of bias audit requirements by DCWP, and an increased potential for compliance failures to be identified, whether through complaints or proactive review. As enforcement practices evolve, organizations using AEDTs should closely monitor developments and ensure their compliance programs align with legal requirements.

Stay tuned to DCI's blog for continued updates on the enforcement of AI laws, and contact DCI to discuss your NYC Local Law 144 or other AI audit needs in more detail.

[1] The goal of this office is to help state and local government agencies manage resources efficiently and ensure accountability for taxpayer dollars.

[2] These test calls were made to the NYC 311 system which is available for the public to request information and submit complaints.

[3] These companies were identified as having published bias audits responsive to NYC LL144 in publications from Cornell University and via the American Civil Liberties Union (ACLU).