Which laws require bias audits for AI hiring tools in 2026?
It depends on the jurisdiction: NYC requires an annual independent bias audit for covered AI hiring tools, while federal law, Colorado, and the EU focus on broader defensibility, impact assessment, and governance records.
As of April 20, 2026, there is still no single HR-AI audit standard. New York City is the only U.S. regime now in force that expressly requires a formal bias audit for employment AI, annually, by an independent auditor, with public posting. Federal employment law still determines whether the tool is defensible when challenged, and that inquiry is broader: protected-group impact, sample adequacy, job relatedness, business necessity, and recordkeeping. Colorado and the EU AI Act point in a different direction. They require impact assessment, risk management, logging, notice, annual review, and monitoring rather than NYC's exact public-ratio model, and neither clearly copies NYC's outside-auditor structure. So bias-audited is now an overloaded claim. It may mean NYC-compliant, statistically screened, or governance-documented. Those are different answers.
Sources for this answer
Primary law
A.1 New York City Department of Consumer and Worker Protection, Automated Employm...Supports the cited proposition. (New York City Department of Consumer and Worker Protection, Automated Employm...)
has been subject to a bias audit within one year of the use
See New York City Department of Consumer and Worker Protection, Automated Employment Decision Tools (AEDT).
Primary law
A.2 29 C.F.R. § 1607.4(B)Employers are required to maintain employment selection records categorized by sex and specific race and ethnic groups to ensure compliance with equal employment opportunity guidelines and to monitor for adverse impact.
The records called for by this section are to be maintained by sex, and the following races and ethnic groups: Blacks (Negroes), American Indians (including Alaskan Natives), Asians (including Pacific Islanders), Hispanic
See 29 C.F.R. § 1607.4(B).
Primary law
A.3 Colorado General Assembly, SB25B-004 Increase Transparency for Algorithmic Sy...Colorado Senate Bill 25B-004 extends the effective date for the algorithmic transparency and consumer protection requirements originally established in Senate Bill 24-205 to June 30, 2026.
The act extends the effective date of the requirements of Senate Bill 24-205 to June 30, 2026.
See Colorado General Assembly, SB25B-004 Increase Transparency for Algorithmic Systems.
Primary law
A.4 Regulation (EU) 2024/1689Supports the cited proposition. (Regulation (EU) 2024/1689)
prior defined metrics and probabilistic thresholds
See Regulation (EU) 2024/1689.
Does a bias audit make an AI hiring tool legally defensible?
No, an audit alone does not answer the Title VII question; employers still need records showing impact, validity, job relatedness, business necessity, and monitoring.
The federal baseline is still Title VII plus the Uniform Guidelines on Employee Selection Procedures. Title VII asks whether an employer used a particular employment practice that causes a disparate impact and, if so, whether that practice is job related for the position in question and consistent with business necessity. UGESP makes the mechanics more concrete. It requires impact records by group, allows sampling only when it is appropriate and adequate in size, and treats the four-fifths (4/5ths) or eighty percent rule as a screening device rather than a full defense. That is still the core reason a defensible audit is bigger than a single fairness ratio. No direct authority surfaced in the source set requiring one universal p-value, confidence interval, bootstrap method, or minimum cell size across HR AI tools.
Once the statutes run out, the source set moves to standards rather than more law. NIST's AI RMF and SP 1270, ISO/IEC 42001, ISO/IEC 23894, ISO/IEC TR 24027, and the academic auditing literature all push toward lifecycle governance, representative data analysis, and post-deployment monitoring rather than one-number certification. They help explain why a vendor can honestly say audited without proving employment-law defensibility. They do not create a Title VII safe harbor.
The employer bar is not especially split on the floor. Seyfarth says the four-fifths rule is a general rule of thumb, not a substitute for formal statistical analysis in every case. Mayer Brown lands in the same place from the Title VII side: a passing ratio is not dispositive, and vendor assurances do not settle the employer's liability question. That is a quiet but important consensus. The firms are not saying audits are useless. They are saying the legal question is still broader than the certificate.
Sources for this answer
Primary law
B.1 42 U.S.C. § 2000e-2(k)(1)(A)(i)Under Title VII, an unlawful employment practice is established when a plaintiff demonstrates that a protected characteristic was a motivating factor for an employment decision, regardless of whether other factors also motivated the practice.
an unlawful employment practice is established when the complaining party demonstrates that race, color, religion, sex, or national origin was a motivating factor for any employment practice, even though other factors also motivated the practice.
See 42 U.S.C. § 2000e-2(k)(1)(A)(i).
Primary law
B.2 29 C.F.R. § 1607.4(B)Employers are required to maintain employment selection records categorized by sex and specific race and ethnic groups to ensure compliance with equal employment opportunity guidelines and to monitor for adverse impact.
The records called for by this section are to be maintained by sex, and the following races and ethnic groups: Blacks (Negroes), American Indians (including Alaskan Natives), Asians (including Pacific Islanders), Hispanic
See 29 C.F.R. § 1607.4(B).
Primary law
B.3 29 C.F.R. § 1607.15Employers who use selection procedures that result in an adverse impact must maintain and provide documentation of the validity of those procedures in accordance with federal guidelines.
Users of selection procedures other than those users complying with section 15A(1) below should maintain and have available for each job information on adverse impact of the selection process for that job and, where it is determined a selection process has an adverse impact, evidence of validity as set forth below.
See 29 C.F.R. § 1607.15.
Primary law
B.4 NIST AI Risk Management FrameworkThe NIST AI Risk Management Framework provides a voluntary, consensus-based structure designed to help organizations manage risks and incorporate trustworthiness into the lifecycle of artificial intelligence systems.
The NIST AI Risk Management Framework (AI RMF) is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.
See NIST AI Risk Management Framework.
Primary law
B.5 NIST SP 1270, Towards a Standard for Identifying and Managing Bias in Artific...PDFNIST Special Publication 1270 provides a socio-technical framework for identifying and managing bias in artificial intelligence systems, emphasizing that while such guidance is voluntary and does not establish legal standards, it is essential for addressing the systemic, statistical, and human factors that contribute to harmful AI outcomes.
Systemic biases result from procedures and practices of particular institutions that operate in ways which result in certain social groups being advantaged or favored and others being disadvantaged or devalued.
See NIST SP 1270, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.
Commentary
B.6 ISO/IEC 42001:2023ISO/IEC 42001:2023 provides a standardized framework for organizations to implement an Artificial Intelligence Management System (AIMS) to ensure the responsible governance, risk management, and ethical development of AI systems.
ISO/IEC 42001 is an international standard that specifies requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System (AIMS) within organizations.
See ISO/IEC 42001:2023.
Commentary
B.7 ISO/IEC 23894:2023ISO/IEC 23894:2023 provides a standardized framework for organizations to integrate and manage risks associated with the development, deployment, and use of artificial intelligence systems.
This document provides guidance on how organizations that develop, produce, deploy or use products, systems and services that utilize artificial intelligence (AI) can manage risk specifically related to AI.
See ISO/IEC 23894:2023.
Commentary
B.8 ISO/IEC TR 24027:2021ISO/IEC TR 24027:2021 provides a framework for identifying, measuring, and mitigating bias across the entire lifecycle of an AI system.
This document addresses bias in relation to AI systems, especially with regards to AI-aided decision-making.
See ISO/IEC TR 24027:2021.
Commentary
B.9 Manish Raghavan & Pauline T. Kim, Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring FairnessPDFThe 'four-fifths rule' is a non-binding rule of thumb that is insufficient as a standalone legal test for disparate impact and is a poor metric for ensuring algorithmic fairness.
The four-fifths ratio was never intended to be a rule of law, but rather a “rule of thumb.”
See Manish Raghavan & Pauline T. Kim, Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring Fairness.
Commentary
B.10 Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic AuditingInternal algorithmic auditing frameworks provide a structured, end-to-end process for organizations to identify and mitigate potential harms throughout the artificial intelligence development lifecycle.
In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development lifecycle.
See Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.
Law-firm commentary
B.11 Seyfarth Shaw commentaryThe EEOC's technical guidance clarifies that existing Title VII principles and the Uniform Guidelines on Employee Selection Procedures apply to the use of artificial intelligence and other automated tools in employment decision-making, and that employers remain liable for discriminatory outcomes even when using third-party vendor tools.
The EEOC did not unveil new policies in the TA but reiterated that its long existing policies and practices continue to apply to the technologies (such as artificial intelligence and machine learning tools) that are grabbing the public’s attention today.
See Seyfarth Shaw, EEOC Issues Technical Assistance Guidance On The Use Of Advanced Technology Tools, Including Artificial Intelligence, To Make Employment Decisions.
Law-firm commentary
B.12 Mayer Brown commentaryThe EEOC considers employer-utilized algorithmic decision-making tools to be selection procedures subject to Title VII, meaning employers remain liable for disparate impact discrimination even when relying on third-party software vendors.
The EEOC’s AI Disparate Impact Guidance makes clear that the EEOC treats employer use of algorithmic decision-making tools as an employment “selection procedure” under Title VII.
See Mayer Brown, EEOC Issues Title VII Guidance on Employer Use of AI and Other Algorithmic Decisionmaking Tools.
Does NYC require an independent bias audit for AI hiring tools?
Yes, NYC is the clearest current U.S. rule because covered automated employment decision tools need a recent independent audit, public results, and notice before use.
New York City's AEDT law is the clearest positive-law answer because it actually requires something called a bias audit. DCWP states that an AEDT cannot be used unless it “has been subject to a bias audit within one year of the use”, the summary results are publicly available, and notice is given before use. But the regime is narrower than its reputation. It is built around race, ethnicity, and sex impact ratios plus public disclosure. The source set did not surface a statutory power rule, a confidence-interval requirement, or an ADA or ADEA analogue inside Local Law 144 itself. A tool can therefore be NYC-compliant and still leave important employment-law questions open.
DLA Piper adds a different concern. Its January 30, 2026 note on a critical public audit of NYC enforcement reads less like a dispute about the statute and more like a warning that audit quality is becoming visible. Once public audit summaries, watchdog reports, and regulator scrutiny enter the picture, a thin annual deliverable can start to look worse than no grand claims at all. Perhaps that is the most practical recent development in this area: the market is no longer debating only whether an audit exists, but whether the audit says anything useful.
- How weak can a technically compliant audit be? The source set points to a real possibility that a NYC audit can satisfy the statute while staying silent on age, disability, proxy variables, language status, or model drift, because the statute is narrower than the larger employment-law question.
Sources for this answer
Primary law
C.1 New York City Department of Consumer and Worker Protection, Automated Employm...Supports the cited proposition. (New York City Department of Consumer and Worker Protection, Automated Employm...)
has been subject to a bias audit within one year of the use
See New York City Department of Consumer and Worker Protection, Automated Employment Decision Tools (AEDT).
Commentary
C.2 Rules of the City of New York, Automated Employment Decision Tools ruleThe proposed rules from the Department of Consumer and Worker Protection establish specific requirements for the use, bias auditing, and candidate notification processes for automated employment decision tools in New York City.
The proposed rules would clarify the requirements for the use of automated employment decision tools within New York City, the notices to employees and candidates for employment regarding the use of the tool, the bias audit for the tool, and the required published results of the bias audit.
See Rules of the City of New York, Automated Employment Decision Tools rule.
Law-firm commentary
C.3 DLA Piper commentaryA recent audit of the New York City Department of Consumer and Worker Protection indicates that enforcement of Local Law 144 regarding automated employment decision tools has been ineffective, signaling a shift toward more stringent regulatory scrutiny and potential penalties for employers.
The New York State Comptroller’s December 2025 audit evaluated the New York City Department of Consumer and Worker Protection’s (DCWP) enforcement of Local Law 144, which regulates the use of automated employment decision tools (AEDTs) in hiring and promotion.
See DLA Piper, Critical audit of NYC's AI hiring law signals increased risk for employers.
Law-firm commentary
C.4 DLA Piper, US: New York City set to enforce AI lawNew York City's Local Law 144 of 2021 mandates that employers using automated employment decision tools for hiring or promotion must conduct independent bias audits, publish the results, and provide notice to applicants and employees to avoid civil penalties.
Local Law 144 of 2021, which took effect on 1 January 2023, regulates employers’ use of automated employment decision tools (AEDTs) in making hiring and promotion decisions.
See DLA Piper, US: New York City set to enforce AI law.
Primary law
C.5 42 U.S.C. § 2000e-2(k)(1)(A)(i)Under Title VII, an unlawful employment practice is established when a plaintiff demonstrates that a protected characteristic was a motivating factor for an employment decision, regardless of whether other factors also motivated the practice.
an unlawful employment practice is established when the complaining party demonstrates that race, color, religion, sex, or national origin was a motivating factor for any employment practice, even though other factors also motivated the practice.
See 42 U.S.C. § 2000e-2(k)(1)(A)(i).
Do Colorado and the EU require AI hiring bias audits?
No, not in the same way as NYC; Colorado and the EU emphasize risk management, impact assessment, documentation, notices, monitoring, and human review.
Colorado's AI Act is broader, but structurally different. The law now takes effect on June 30, 2026 after a 2025 delay. It asks developers and deployers of high-risk AI systems to use reasonable care to protect consumers from known or reasonably foreseeable risks of algorithmic discrimination, and for deployers it builds a package of risk-management policy, impact assessment, annual review, notice, appeal with human review if technically feasible, and a public website statement summarizing deployed high-risk systems and how risks are managed. That is not the same thing as a NYC-style third-party statistical bias audit. In the source set, no Colorado authority required a particular fairness metric, minimum sample size, or outside auditor for that impact-assessment layer.
The EU AI Act is more prescriptive on governance and less prescriptive on audit form. Employment and worker-management systems sit in Annex III high-risk territory. The Act requires risk management, data governance using representative and statistically appropriate data, testing against “prior defined metrics and probabilistic thresholds”, technical documentation, logging, human oversight, and post-market monitoring. For most Annex III employment systems, conformity assessment runs through internal control under Article 43(2), not a mandatory outside auditor. The high-risk obligations relevant here are enacted, but the general application date for that layer is August 2, 2026. So the EU model looks less like NYC's annual public audit and more like a documented management system with lifecycle evidence.
Littler's Colorado framing is also useful because it resists calling the statute a hiring bias-audit law. Its point is that Colorado creates a broader deployer-side documentation and review burden for consequential decision systems. Fisher Phillips makes a parallel point from another angle: the first serious question is still methodology. Was the tool tested by four-fifths ratio, statistical significance, or something else? In other words, the firms keep asking what math and what documentation sit behind the audit label, not just whether the label exists.
Sources for this answer
Primary law
D.1 Colorado General Assembly, SB24-205 Consumer Protections for Artificial Intel...Colorado SB24-205 establishes consumer protection requirements for developers and deployers of high-risk artificial intelligence systems, including mandates for algorithmic risk management and transparency, and designates violations as deceptive trade practices under the Colorado Consumer Protection Act.
On and after February 1, 2026, the act requires a developer of a high-risk artificial intelligence system (high-risk system) to use reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination
See Colorado General Assembly, SB24-205 Consumer Protections for Artificial Intelligence.
Primary law
D.2 Colorado General Assembly, SB25B-004 Increase Transparency for Algorithmic Sy...Colorado Senate Bill 25B-004 extends the effective date for the algorithmic transparency and consumer protection requirements originally established in Senate Bill 24-205 to June 30, 2026.
The act extends the effective date of the requirements of Senate Bill 24-205 to June 30, 2026.
See Colorado General Assembly, SB25B-004 Increase Transparency for Algorithmic Systems.
Primary law
D.3 Colorado Session Laws, Chapter 4078Colorado law establishes specific duties for developers and deployers of high-risk artificial intelligence systems to prevent algorithmic discrimination, grants the attorney general exclusive enforcement authority, and clarifies that these provisions do not create a private right of action.
ON AND AFTER FEBRUARY 1, 2026, A DEVELOPER OF A HIGH-RISK ARTIFICIAL INTELLIGENCE SYSTEM SHALL USE REASONABLE CARE TO PROTECT CONSUMERS FROM ANY KNOWN OR REASONABLY FORESEEABLE RISKS OF ALGORITHMIC DISCRIMINATION
See Colorado Session Laws, Chapter 4078.
Primary law
D.4 Regulation (EU) 2024/1689Supports the cited proposition. (Regulation (EU) 2024/1689)
prior defined metrics and probabilistic thresholds
See Regulation (EU) 2024/1689.
Law-firm commentary
D.5 Littler commentaryColorado Senate Bill 24-205 establishes a comprehensive regulatory framework for AI in employment, imposing statutory tort liability for algorithmic discrimination and requiring employers to implement rigorous compliance, transparency, and appeal procedures.
Colorado Senate Bill 24-205 (“SB205”), landmark legislation that expressly creates statutory tort liability for AI algorithmic discrimination in the employment context, has passed both houses of the Colorado General Assembly
See Littler, Colorado's Landmark AI Legislation Would Create Significant Compliance Burden for Employers Using AI Tools.
Law-firm commentary
D.6 Littler, What Does the 2025 Artificial Intelligence Legislative and Regulatory Landscape Look LikeDue to the lack of comprehensive federal regulation, employers must navigate a growing patchwork of state and local laws that impose duties of reasonable care, transparency, and bias auditing regarding the use of AI in employment decisions.
In the absence of federal regulation, several states have either passed or are considering legislation aimed at mitigating the risk of an employer’s use of an AI system resulting in algorithmic discrimination.
See Littler, What Does the 2025 Artificial Intelligence Legislative and Regulatory Landscape Look Like.
Law-firm commentary
D.7 Fisher Phillips commentaryThe EEOC maintains that employers remain liable for Title VII violations resulting from the use of AI-driven employment tools, even when those tools are developed or administered by third-party vendors.
an improper application of AI could violate Title VII, the federal anti-discrimination law, when used for recruitment, hiring, retention, promotion, transfer, performance monitoring, demotion, or dismissal.
See Fisher Phillips, EEOC's Latest AI Guidance Sends Warning to Employers.
What should employers ask AI hiring vendors about bias audits?
Employers should ask what the audit actually measured, who performed it, whether results were public or internal, and what monitoring happens after deployment.
The non-obvious consequence is that bias audit is now an overloaded procurement term. In the current market it can mean an NYC public-ratio audit, an industrial-organizational validation project, or an AI-governance documentation workflow. HireVue's use of DCI Consulting Group illustrates the first lineage. FairNow, Holistic AI, and Warden illustrate the second. The public materials in the source set are clearer on cadence, independence, publication, and workflow than on cell sizes, missing demographic labels, or drift handling. That does not make them empty. It means the market has optimized around legally legible artifacts.
This follows directly from the law. New York City created the first recurring public deliverable, so vendors built products around annual audits and posting. Colorado and the EU reward impact-assessment records, public summaries, technical documentation, and monitoring. Federal employment law still asks the older question: what was measured, on whom, with what impact, and why keep using it after adverse impact appears. Companies buying an audit are therefore often buying one layer of the answer, not the whole answer.
Third-party independence is likewise regime-specific. NYC clearly requires an independent auditor. Colorado and the EU do not copy that rule, and internal teams may actually see more of the model, the data pipeline, and the post-deployment drift than an outside reviewer can. That leaves the market in an understandable but awkward place: outside review carries credibility, while internal review often carries access. A defensible record increasingly looks like both rather than either.
- When is outside review legally required versus merely market-preferred? New York City supplies one clear answer for one regime. Colorado and the EU do not. The harder long-term question may be whether credibility pressure and procurement norms make independent review functionally standard even where the statute does not require it.
Sources for this answer
Commentary
E.1 HireVue, HireVue leads industry in fair and ethical hiring practice, engaging external auditor DCI Consulting Group for external bias audit of algorithmsHireVue proactively engages independent third-party auditors to evaluate its AI-based hiring algorithms for bias and compliance with emerging regulatory standards such as New York City Local Law 144.
Competency-based and game-based algorithms will be audited for bias with respect to race, gender and the intersectional combination of race and gender across multiple job levels and use cases.
See HireVue, HireVue leads industry in fair and ethical hiring practice, engaging external auditor DCI Consulting Group for external bias audit of algorithms.
Commentary
E.2 FairNow, NYC Local Law 144: AI Hiring Compliance GuideOrganizations can manage evolving AI regulatory requirements and internal risk frameworks by centralizing model inventories, compliance tracking, and control monitoring.
We provide a unified platform to centralize ISO 42001, NIST AI RMF, and EU AI Act requirements, reducing the administrative burden of staying compliant.
See FairNow, NYC Local Law 144: AI Hiring Compliance Guide.
Commentary
E.3 Holistic AI, NYC Bias Audit SolutionHolistic AI provides a governance platform designed to assist organizations in meeting the independent audit, documentation, and compliance requirements mandated by NYC Local Law 144 for Automated Employment Decision Tools.
Local Law 144 requires the audit to be independent. Many organizations engage a third-party auditor to demonstrate independence.
See Holistic AI, NYC Bias Audit Solution.
Commentary
E.4 Warden, Navigating the NYC Bias Audit Law for HR Tech PlatformsNew York City's Local Law 144 imposes mandatory annual independent bias audit requirements on automated employment decision tools used in recruitment and promotion, with non-compliance subject to financial penalties.
NYC LL144 creates mandatory compliance obligations for HR tech vendors and their enterprise clients alike.
See Warden, Navigating the NYC Bias Audit Law for HR Tech Platforms.
Commentary
E.5 DCI Consulting, NYC Local Law 144: Choose Your Auditor WiselyWhile NYC Local Law 144 mandates specific independence criteria for auditors conducting bias audits of automated employment decision tools, it does not prescribe specific professional expertise requirements for those auditors.
Per NYC LL-144, independent auditors may not have: 1) been involved developing the AEDT, 2) been employed by the organization or vendor, or 3) a financial interest in the organization or vendor.
See DCI Consulting, NYC Local Law 144: Choose Your Auditor Wisely.
Primary law
E.6 New York City Department of Consumer and Worker Protection, Automated Employm...Supports the cited proposition. (New York City Department of Consumer and Worker Protection, Automated Employm...)
has been subject to a bias audit within one year of the use
See New York City Department of Consumer and Worker Protection, Automated Employment Decision Tools (AEDT).
Primary law
E.7 Colorado General Assembly, SB24-205 Consumer Protections for Artificial Intel...Colorado SB24-205 establishes consumer protection requirements for developers and deployers of high-risk artificial intelligence systems, including mandates for algorithmic risk management and transparency, and designates violations as deceptive trade practices under the Colorado Consumer Protection Act.
On and after February 1, 2026, the act requires a developer of a high-risk artificial intelligence system (high-risk system) to use reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination
See Colorado General Assembly, SB24-205 Consumer Protections for Artificial Intelligence.
Primary law
E.8 Regulation (EU) 2024/1689Supports the cited proposition. (Regulation (EU) 2024/1689)
prior defined metrics and probabilistic thresholds
See Regulation (EU) 2024/1689.
Primary law
E.9 29 C.F.R. § 1607.4(B)Employers are required to maintain employment selection records categorized by sex and specific race and ethnic groups to ensure compliance with equal employment opportunity guidelines and to monitor for adverse impact.
The records called for by this section are to be maintained by sex, and the following races and ethnic groups: Blacks (Negroes), American Indians (including Alaskan Natives), Asians (including Pacific Islanders), Hispanic
See 29 C.F.R. § 1607.4(B).
Commentary
E.10 Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic AuditingInternal algorithmic auditing frameworks provide a structured, end-to-end process for organizations to identify and mitigate potential harms throughout the artificial intelligence development lifecycle.
In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development lifecycle.
See Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.
What statistical methods matter for small AI hiring bias audits?
Unclear, because no source in this set imposes one universal p-value, confidence interval, bootstrap protocol, or minimum cell size for HR AI audits.
Small and segmented hiring funnels are where this gap is sharpest. UGESP insists on measurement and says sampling must be adequate, but the statutes and standards in the source set do not supply one universal power threshold or minimum cell size for HR AI audits. So thin applicant volumes can produce results that are formally reportable but substantively weak. Perhaps the practical divide is not compliant versus non-compliant. It is legible versus persuasive.
What statistical method counts as enough? No direct authority surfaced in the source set for a universally required p-value, confidence interval, Bayesian threshold, or bootstrap protocol. That leaves room for justified tailoring, and also room for weak bespoke science marketed as sophistication.
How should audits treat self-selection and historical-data bias? The methodology sources are fairly clear that hiring data often reflect earlier recruiting filters and social skews. A clean output ratio on that data may therefore understate how much the model is reproducing older exclusions.
How transparent can methodology become before trade-secret concerns stop it? The EU AI Act expressly preserves intellectual property, confidential business information, and trade secrets. Perhaps the likely outcome is that some of the most important audit evidence stays nonpublic unless discovery or regulator access pulls it into view.
Sources for this answer
Primary law
F.1 29 C.F.R. § 1607.4(B)Employers are required to maintain employment selection records categorized by sex and specific race and ethnic groups to ensure compliance with equal employment opportunity guidelines and to monitor for adverse impact.
The records called for by this section are to be maintained by sex, and the following races and ethnic groups: Blacks (Negroes), American Indians (including Alaskan Natives), Asians (including Pacific Islanders), Hispanic
See 29 C.F.R. § 1607.4(B).
Commentary
F.2 Manish Raghavan & Pauline T. Kim, Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring FairnessPDFThe 'four-fifths rule' is a non-binding rule of thumb that is insufficient as a standalone legal test for disparate impact and is a poor metric for ensuring algorithmic fairness.
The four-fifths ratio was never intended to be a rule of law, but rather a “rule of thumb.”
See Manish Raghavan & Pauline T. Kim, Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring Fairness.
Primary law
F.3 NIST SP 1270, Towards a Standard for Identifying and Managing Bias in Artific...PDFNIST Special Publication 1270 provides a socio-technical framework for identifying and managing bias in artificial intelligence systems, emphasizing that while such guidance is voluntary and does not establish legal standards, it is essential for addressing the systemic, statistical, and human factors that contribute to harmful AI outcomes.
Systemic biases result from procedures and practices of particular institutions that operate in ways which result in certain social groups being advantaged or favored and others being disadvantaged or devalued.
See NIST SP 1270, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.
Primary law
F.4 Regulation (EU) 2024/1689Supports the cited proposition. (Regulation (EU) 2024/1689)
prior defined metrics and probabilistic thresholds
See Regulation (EU) 2024/1689.
Primary law
F.5 NIST AI Risk Management FrameworkThe NIST AI Risk Management Framework provides a voluntary, consensus-based structure designed to help organizations manage risks and incorporate trustworthiness into the lifecycle of artificial intelligence systems.
The NIST AI Risk Management Framework (AI RMF) is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.
See NIST AI Risk Management Framework.
Commentary
F.6 Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic AuditingInternal algorithmic auditing frameworks provide a structured, end-to-end process for organizations to identify and mitigate potential harms throughout the artificial intelligence development lifecycle.
In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development lifecycle.
See Inioluwa Deborah Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.