@ModusAudit

AI Native Accounting firm. Backed by @lightspeedvp

 NYC
modusalliance.com
Joined February 2026
  • Tweets16
  • Following3
  • Followers283
3Media
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
We are excited to see how FinancialAuditBench can help improve audit agents and benchmark construction in privacy-sensitive domains! 🔗 Paper: arxiv.org/abs/2609.32835 🔗 Homepage: financialauditbench.com 🔗 Dataset: huggingface.co/datasets/modu… 🔗 Code: github.com/modus-audit/finan…

FinancialAuditBench: Benchmark Construction under Differential...

As AI agents are becoming widely adopted in the financial services industry, careful measurement is essential to understand where they can be reliably deployed and where oversight and professional...

arxiv.org
1
75
Finding #4: More reasoning improves audit performance, but increases both time and cost. For GPT-6 Astra, low → max reasoning raises strict pass from 23.61% to 71.11%, while mean runtime increases from 3.08 to 24.94 minutes and model cost from $1.32 to $8.27 per attempt.
1
1
82
Finding #3: When tasked with reviewing completed workpapers, agents are able to identify and correct some mistakes. All models we benchmarked improve average scores, with the strongest reviewer raising strict pass from 28.89% to 62.22%. But some workpapers get worse after review.
1
1
79
Finding #2: Agents sometimes use the wrong evidence or miss it entirely. e.g. They check client reports instead of bank statements, overlook shipping logs, and use current-year records when a procedure requires next-year data, rather than flagging the missing information.
1
1
93
Finding #1: Frontier models complete much of the work, but struggle with reliability. The best model we evaluated, Opus 5.5, scores 95.84% across our rubric criteria, yet passes all checks in only 69.17% of runs. Furthermore, only 44.44% of tasks pass all checks across 8 attempts.
1
1
105
We developed the benchmark with 1,100+ hours of auditor input, and evaluated 11 frontier models across: 90 tasks: workpaper completion and review 6 synthetic audit engagements: avg. of 179 files each 2 industries: manufacturing and staffing services
1
2
167
FinancialAuditBench is built from differentially private statistics from real audits to generate synthetic companies and their documents. A computational graph combines these priors with accounting rules and auditor expertise, keeping financial records consistent across files.
1
2
369
AI agents are being adopted into financial audit workflows, but how reliably can they complete the work? In FinancialAuditBench, we evaluate agents on realistic audit tasks, and reveal where professional review remains necessary. 🔗 Paper: arxiv.org/abs/2609.32835 Thread 👇
3
2
2
23
5,765
6/ Agents can already do a lot of staff-level audit work. They still need professional review. FAB is how we measure where that line is, and how it moves as models improve. Built by @PranavAPillai @actuallyjerryh @AlexWang1025. Paper, tasks, and code: financialauditbench.com/

FinancialAuditBench · Modus Labs

A benchmark for AI agents on financial statement audits. Compare models and explore their work.

financialauditbench.com
1
59
5/ Results vary a lot by area. Journal-entry completeness averaged a 94.5% pass rate across models. Accounts receivable averaged 4.2%. One failure we saw: testing receivables against the client's own cash-receipts report instead of the bank statement.
1
1
53
4/ Grading is strict. Every rubric check has to pass or the workpaper fails, since a single miss means a reviewer has to recheck the work. The best model, @claudeai Opus 5.5, fully passed 69% of tasks on one attempt. Required to pass all 8 attempts, it dropped to 44%.
1
1
46
3/ The binders are generated from differentially private statistics of historical audits. Ground truth and rubrics were built and checked by CPAs over 1,100+ hours.
1
1
42
2/ Each task gives an agent a full synthetic engagement binder, about 179 files on average, plus a workpaper to complete or review. Cash, receivables, payables, debt, fixed assets, revenue, and journal entries.
1
1
191
1/ Frontier models keep getting better. We wanted to know what that actually means for real audit work, so we built Financial Audit Bench (FAB). 90 audit tasks. 11 frontier models. Graded against rubrics written by licensed CPAs. Here's what we found:
1
8
1
17
973
Every model update shifts the spectrum of what is possible
Arush Jain
@Arushj01Sep 2
Replying to @AnthropicAI
@AnthropicAI released its latest frontier model Fable 5.1 this afternoon. It's the clearest example of a frontier model doing large chunks of Audit work accurately, which was not possible even two months ago.
2
697
Three college best friends, @Arushj01, @PranavAPillai, and @vinaykasat_, set out to rebuild systems that felt stuck. Years of collaboration turned into conviction, and from that conviction came @ModusAudit. We’re excited to lead Modus’ $85M Seed & Series A. Modus is rebuilding audit from the inside out. They invest in and partner with audit firms, deploy AI that improves audit quality, create a high-quality customer experience, and translate automation into new business. By rethinking audit at its core, they’re redefining what scale and quality look like in one of the most trust-critical industries. We’re proud to continue backing them on this journey. CC: Lightspeed partners Isaac Kim, Amish Desai, Alexa Boghosian, @jmover
6
7
5
120
47,863
Load more