2026-08-28

A script for catching fabricated quotations found the model had been inventing lines and attributing them to the manuscript, and drafting moved to a scene-first structure.

David had Claude build check_citations.py, a small script to mechanically verify that any quoted phrase in the canon and planning files actually appeared in the chapter it claimed to come from. The first run found 206 mismatches. Of those, 110 were outright fabricated quotations that were never written, 38 were altered from the real text, and 58 were attributed to the wrong chapter. Those were fabricated or misattributed lines, caught by the script rather than by rereading. Fixing them took three repair passes through the day to bring the mismatch count down.

Separately, a long-running set of open story questions got closed out. A sweep forced forty-two undecided items into answered, ruled, or explicitly deferred status. Several sealed answer files got written, then rebuilt against adversarial critiques that checked them for evidence gaps and internal contradictions. Sealed files are written by a subagent and never shown to David, so what they contain does not appear in the transcripts either.

The day’s other major change was structural. Chapters got reorganised around 600 to 900 word scenes as the basic unit of production, assembled afterwards into chapters, and all of the project’s prose-habit budgets (how often a given device or word could appear) moved from a per-chapter count to a rate per thousand words, so they’d still make sense at the new granularity. David had Claude build a measurement tool, census.py, to check spans of prose against those rates automatically.

Voice work also reached a limit. After several more rounds of blind testing, Claude found that the scoring method itself was too dependent on who was doing the scoring to support the cross-round comparisons that had already been reported, and closed that whole line of experimentation, recording the finding. A handful of narrower recommendations survived: keep tracking what each character wants in a scene, and add an explicit check for reused rhetorical constructions to the editing pass.

Session transcript: 27-30 August