Return-Path: Delivered-To: alvarochf@traceseis.com Received: from cs510.bluehost.com by cs510.bluehost.com with LMTP id eCDAJDDvQ2oWxwQAWiBQxw (envelope-from ) for ; Tue, 30 Jun 2026 10:30:40 -0600 Return-path: Envelope-to: alvarochf@traceseis.com Delivery-date: Tue, 30 Jun 2026 10:30:40 -0600 Received: from mail.comptrollernyc.com ([184.106.28.38]:39542) by cs510.bluehost.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.99.2) (envelope-from ) id 1webM8-00000001hwu-0qXN for alvarochf@traceseis.com; Tue, 30 Jun 2026 10:30:40 -0600 Received: by mail.comptrollernyc.com (Postfix, from userid 48) id 14F5E7F087; Tue, 30 Jun 2026 11:30:47 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.11.0 mail.comptrollernyc.com 14F5E7F087 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comptroller.nyc.gov; s=comptroller-web; t=1782837047; bh=JZKIo88azOrcawjuqtcEyGa8F/eIjsZCTisLCqaV8Wk=; h=To:Subject:Date:From:Reply-To:From; b=c3/YJdTlr7QfgXWDqL+8ZiTYb16XR/6kPRJMwhZ1iMuJ6decn9UTfPkB4t7If8fOO MBkkzJ+/C96i0FKGuS/qRdbtdBFf+4WBLERjemWuf0r1q6zYZMEKGaYzgFotEwSTAj kj3TYgfZ1UqQI8qhAB8EP2mxcbvzAQxVx01ePiaE= To: alvarochf@traceseis.com Subject: Suggest an Audit Date: Tue, 30 Jun 2026 16:30:47 +0000 From: Office of the New York City Comptroller Reply-To: no-reply@comptroller.nyc.gov Message-ID: X-Mailer: PHPMailer 7.0.0 (https://github.com/PHPMailer/PHPMailer) MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=3.8 X-Spam-Score: 38 X-Spam-Bar: +++ X-Spam-Flag: NO

Thank you for reaching out to the NYC Office of the Comptroller’s Bureau of Audit. This email confirms that we received your email.

We welcome your response to this email with any additional information you may have to share with us regarding the topic. Please note that we cannot audit or investigate all the ideas we receive, but your submission will receive due consideration.

We appreciate your civic engagement and encourage you to share the Suggest an Audit Form with anyone you know with a good idea for an audit or investigation.

Entry ID | 2070 |
Date Submitted | June 30, 2026 |
First Name | Alvaro |
Last Name | Chaveste |
Organization/Affiliation | CONFIRM/TraceSeis, Inc. |
Email | alvarochf@traceseis.com |
Phone | +1 713-890-9249 |
Your Suggestion |

I am suggesting an audit of the NYC Department of Consumer and Worker Protection's implementation and enforcement of Local Law 144 of 2021, which requires bias audits of automated employment decision tools used in hiring.
Two specific problems in the public record warrant a Comptroller audit.

Problem One: DCWP rules permit synthetic and randomly generated data as an audit basis.
Two audits in the ACLU/NYCLU's public LL144 tracker were conducted on fabricated data rather than real applicant data — and both were filed as compliant.

The first, published June 27, 2023 by ConductorAI for Alidoro's use of the Landed hiring tool, disclosed verbatim: "Each applicant was randomly generated a score between 0 and 100 using python's random.uniform function." The system was never evaluated. 35,000 fake applicants replaced real ones.
The second, published June 3, 2026 by Warden AI for Beamery's AI Talent Match, disclosed: "The system was evaluated using a purpose-built test dataset, as sufficient historical data was not available." The auditor's own disclaimer states the report "should not be interpreted as a formal certification or compliance determination." Both audits passed LL144 compliance under current DCWP rules.

Every real applicant scored by those systems in a NYC hiring context is invisible in those audits. Their outcomes carry no weight in determining whether the AI reading their resume treated them fairly.

Problem Two: DCWP rules do not require statistical significance testing.
Every public LL144 audit applies only the four-fifths rule — a ratio threshold with no test of whether observed differences are statistically meaningful. A finding in a group of 63 applicants is treated identically to a finding in a group of 400,000. Standard tools — chi-square independence testing and Cramér's V effect size — directly answer this question. None of the public LL144 audits report them.

Recommended audit scope:
Whether DCWP's current rules produce bias assessments that meaningfully protect NYC job applicants. Whether minimum data quality standards should be codified in DCWP rules. Whether statistical significance and effect size reporting requirements should be added to LL144 audit standards. How many filed audits relied on synthetic or test data, and what proportion of NYC applicants were excluded from evaluation as a result.

A detailed public analysis of the documented cases is available at: https://www.linkedin.com/pulse/2-nyc-ai-bias-audits-ran-fake-data-both-passed-alvaro-chaveste-jr--udsfe

 |
If you are suggesting an investigation, what misconduct do you suspect? | |
How do you think this audit or investigation will help the City? |

LL144 was enacted to protect NYC job applicants from discriminatory AI. If the rules permit audits that never evaluate real applicants, the law's core purpose is not being achieved. A Comptroller audit of DCWP's LL144 implementation would identify specific rule changes that close the gap between technical compliance and meaningful protection. NYC's LL144 has become a national model for AI hiring regulation. Strengthening its implementation strengthens NYC's credibility as a leader in AI accountability.

 |