An overnight build of 27,713 hadith, and the rule that keeps models out of the rulings

Context#

As-Sunnah Search is an offline hadith reader. Each collection is a pack: the Arabic text, an Urdu translation where a translator the owner has accepted exists, the chapter structure, and, for each hadith, the ruling of a named scholar on the hadith or on its chain. By November 2026 the catalogue holds 105,530 hadith across 26 collections.

Musnad Ahmad is the largest single book in that catalogue, and the hardest to build. The source is a 50-volume critical edition whose rulings sit in footnotes, several to a page, each tied to a mark in the text. A footnote attached to the wrong hadith gives a reader a scholar's ruling on a different report. That is the failure the whole build is designed around.

The owner asked for the book on the evening of 25 October 2026 and went to sleep. This is what happened before morning, and what it took to let that happen safely.

Constraints#

• Never fabricate a scholarly ruling. A ruling on a hadith or on its chain is a judgement. The repository's working agreement forbids filling one by inference, heuristic, or model. Rulings come only from a named edition's own text, matched unambiguously; an unknown label fails the build; a label that looks wrong is withheld and listed, never corrected.
• A chain ruling is never shown as a grade. The editor's "its chain is sound" is stored and displayed as a statement about the chain, not as a verdict on the hadith.
• Nothing reaches a reader without the owner. Overnight work may build and hold a book. It may not push, release, list the book in the catalogue, or write to the clearance record. The owner's phone is not touched.
• Every agent was a cheaper model. Sonnet and Haiku did the building; the frontier models were reserved for review.

Method#

1. A plan, reviewed by a second vendor before work began. Codex read the plan for the footnote join and confirmed three ways text could leak into the wrong hadith. Each became a required test in the build.
2. Five parts, five branches, disjoint files. The entry reader, the ruling grammar, the shared builder, the app side, and the editor register were each built by its own agent on its own worktree with its own tests, then merged into one branch. The manifest said which files each agent owned; none edited another's.
3. Fail closed on the join. Every page whose footnote marks do not pair one for one with their footnotes is held, so no hadith takes a neighbour's ruling. Rulings carrying a condition, a contrast ("but"), or a status word are held with the reason recorded.
4. Three independent checkers. After the build, three separate agents re-read the result against the owner's own copy of the source. One re-parsed all 27,713 entries with code it wrote itself and compared.
5. Fixes red first. Each finding got a failing test before the fix, and the verifier and the full test suite were re-run by the orchestrating session, not by the agent that wrote the fix.
6. A morning report written by a script. The report the owner reads first is generated by a tool that calls no model and runs only allow-listed read-only commands. It cannot push, release, clear, or install, and a test proves a whole run stays inside that list.

Results#

Built and held by about 1:30 in the morning. The one real slip was a labelling rule: split hadith were not numbered N.1, N.2 as the owner had chosen. The checkers also named two places where the join should be stricter. All three were fixed with a failing test first, and the orchestrating session re-ran the verifier and all 1,127 tests itself before handing over.

Two days later the Urdu translation joined the book: 26,508 of the 27,713 hadith carry it. An independent checker re-derived every join from the raw pages without the joiner's code and found no Urdu on a wrong hadith and none doubled. The owner cleared the book for the catalogue on 29 October 2026.

What the review found#

• The plan review earned its cost before a line was built. The three leak paths Codex named were real, and each was cheap to test for and expensive to discover in a reader's hands.
• Independent re-parsing is the check that matters. Two checkers sampled; the third rebuilt the parse from scratch and compared. The slip it found was invisible to sampling because it was consistent.
• The safety net was too wide, and that was the right default. Any ruling whose opening clauses contain a contrast or a status word is held, about 1,500 of them, even where the extra words only reinforce the ruling. The owner can narrow the rule later; a reader can never be shown a ruling the pipeline was unsure of.
• The morning report is part of the harness. Because it is generated by a script and not written by a model, the first thing the owner reads after an unattended night is true by construction. Anything an agent wants to claim has to be visible in the repository the report reads.

What we would change#

• Hold less by writing finer rules, not by trusting the model more. The wide net holds rulings that are plainly sound. The fix is a narrower grammar with its own tests, decided by the owner.
• Measure the checkers. Three checkers found one slip and two tightenings. The next build should record how many of the planted and known faults each checker catches, so "three independent checkers" becomes a number rather than a count.

Why this is the hardest test#

A wrong ruling on a hadith misattributes a scholar's judgement about the Prophet's words. There is no acceptable error rate, which is why the rule is not "verify the model's rulings" but "the model never produces one". The rulings come from the printed page or they are withheld. Everything the agents did overnight was in service of copying a scholar's words to the right place and refusing to guess when the page was unclear. That constraint, written down and enforced by the pipeline, is what made it safe to let cheaper agents build the book while the owner slept.