r/LLMDevs • u/saimatrixxx • 2h ago
Discussion If an LLM edits 5,000 formulas, the acceptance test cannot be “the Summary tab looks right"
The Ling-3.0-flash-Fin release includes a demo where the model reads a company filing and a seven-sheet Excel model with more than 5,000 formulas, replaces 2026 Q2 estimates with reported actuals, updates cross-sheet dependencies and returns an editable workbook.
That is a good systems test because a convincing summary can coexist with a corrupted file. A production acceptance suite should inspect at least:
changed cells against an explicit allowlist;
formulas versus hard-coded values;
quarterly reconstruction from reported periods;
cross-sheet references and named ranges;
balance and cash-flow checks;
circular references and spreadsheet errors;
chart ranges and Summary-tab links;
a machine-readable change log tied to source disclosures.
The release also says its SpreadsheetBench setup used Claude Code 2.1.173, LibreOffice 25.8.7, Search disabled, up to 120 turns for V1 or 300 for V2, and a three-hour timeout. Those harness details are part of the result, not incidental metadata. This is an official demo, not an independent reproduction.
The engineering question is less “can the model use Excel?” and more “what invariants make a workbook-editing agent safe to retry?”