I'll check what's mounted and where the epub lives.

scripts/update-kindle.py — run it with the Kindle mounted; already run, so the device now has the current book.

• Converts book-1.epub to MOBI via calibre's ebook-convert into a temp dir, then overwrites /Volumes/Kindle/documents/Autoroad/Wrong Man, The - Autoroad.mobi.
• Same filename deliberately, so the sibling .sdr folder (reading position, bookmarks) stays attached.
• --epub and --kindle flags override the defaults; it exits with a clear message if the Kindle isn't mounted or calibre isn't installed.

Note there's also an empty Autoroad (draft) folder on the device holding only a stale .sdr — I left it alone.

Found the actual cause. Let me rewrite the script.

The device folder is now just the one AZW3 file, nothing else. Two things were wrong, and the second was the real culprit:

Stale reader state. The .sdr sidecar held Wrong Man, The - Autoroad.apnx — the page index — dated 10:34, from before my 14:30 overwrite. Same filename, different content, old index. The reader paginates against an index that no longer matches the text, and hangs. There was also a .mbp1.bad_file sitting in there, which is the device's own record that it choked on that state. The script now deletes the .sdr on every update, so the Kindle rebuilds it.

MOBI6. Calibre's .mobi output is the legacy format; your firmware is 5.19.2, which handles AZW3 (KF8) properly. Output switched to .azw3, and the superseded .mobi is removed so you don't get a duplicate entry.

Also swept the macOS ._ AppleDouble files — those show up as phantom books in the library, though they weren't causing the freeze.

The tradeoff: you lose your reading position on each update. That's unavoidable if the page index has to be rebuilt, and the position was already corrupt. --keep-position skips the .sdr deletion if you ever want it, but it reopens the freeze.

Worth confirming the freeze is actually gone before trusting the fix — I've reasoned from the evidence on disk, not tested it on the device. If it recurs on the AZW3, the next suspect is the conversion itself and I'd look at splitting the larger chapters.

Good question — worth checking against real files rather than from memory. Other books on the device have .sdr folders I can inspect.

Yes for the page index; no, meaningfully, for the position. Let me wire up the half that works.

Split answer, because .sdr is a folder with two very different things in it.

The .apnx — yes, fully. It's a documented format and I've now wired it in. Looking at a working one from another book on your device, it's a small binary header followed by two JSON blobs and then a flat array of byte offsets:

{"contentGuid":"36d615e3","asin":"a5318a33-…","cdeType":"EBOK","fileRevisionId":"1"}
{"asin":"a5318a33-…","pageMap":"(1,a,1)"}
0x000008fc 0x000011f8 0x00001af4 …   ← one offset per page

Calibre ships APNXBuilder, so rather than hand-rolling it I call that. The script now regenerates the index from the AZW3 it just wrote, meaning it's structurally incapable of being stale — which was the failure mode. You get real page numbers back, which you'd lost under the previous commit.

The .mbp1/.mbs — parseable, but regenerating them is the wrong goal. They're an undocumented Amazon key-value serialization. The readable strings show the shape (timer.model, timer.average.calculator.distribution.normal, and fpr — furthest page read — carrying 1511217). I could reverse enough to rewrite that number.

But that number is a byte offset into the book, and this is why it isn't worth doing: when you edit chapter 3, every offset after it shifts by an unknown amount. There's no honest mapping from an old offset to a new one. Restoring the position would mean confidently dropping you at the wrong paragraph — worse than the Kindle putting you at the start and letting you know it did.

The exception is append-only updates. If you're only adding chapters at the end, offsets into existing text don't move, and the position is genuinely valid — that's the case where --keep-position is correct rather than reckless, and it's now the only case where I'd use it. If you expect to update that way often, I can make the script diff the new epub against the last-shipped one and take the safe path automatically when nothing before the append point changed.

Worth saying up front: your actual position is gone — I deleted the .sdr and the old .mobi two turns ago, so there's no old offset and no old text to match against. What I can do is build and validate the mechanism so it works from the next update on. Let me test it against a book on the device that still has a live position.

Understood — the lost position isn't a worry, making it not recur is the goal. Continuing.

fpr is a byte offset into the raw MOBI markup, confirmed: 1511217 of 1517859 bytes lands exactly on that book's end-matter TOC — a finished book. So the mapping is real and the remap is feasible. Let me check the project's test setup.

No Python infra here, so I'll add a self-contained module with tests. Tests first.

uv it is — that's what I'm running tests under. One real weakness surfaced: a long anchor breaks whenever anything is inserted just before the position.

All 16 pass. Now validate against real prose rather than synthetic fixtures — git has older versions of the epub.

Ideal test case: 5b2c6ef is a dialogue pass scattering small edits across chapters 003–031. That's the hard case.

KF8 names the file differently from MOBI6.

398/399 exact on a real edit. But my "drift" number measures content shift, not error — let me replace it with an actual accuracy check, and fix the mojibake (I'm decoding byte-by-byte).

Slow because the validator re-strips both books on every one of 399 samples. Fine for production (one call), so I'll let it finish in the background.

While that runs, let me wire the remap into the update script.

394 exact, 5 fuzzy, none wild. Note the exact rows at 0.77–0.87 aren't errors: the position is right, the next line is what the dialogue pass changed. Now, which sidecar file actually holds the position, and is there a checksum I'd break?

.mbs holds fpr, .mbp1 never does — consistent across all 20+ books, and the 8-byte header is a constant magic, not a checksum. Safe to rewrite. Wiring it in:

Now an end-to-end test on the device: stage the old build plus a synthetic position, then run the real script and see whether it lands on the same sentence.

Works end-to-end. But that staged .mbs was cloned from another book's template, so I'll clear it — the position was synthetic anyway, and I don't want redacted Wizard's reader state on your book.

Yes, both parts work. The device now has the current book, a fresh page index, and no stale state.

What the offset is. fpr in the .mbs is a byte offset into the book's raw markup. I confirmed it on a book of yours that still had a live position: redacted Wizard's fpr of 1511217, against a 1517859-byte markup, lands exactly on that book's end-matter TOC — a finished book. The container is trivial: a marker byte, a three-byte big-endian length, then the offset in ASCII.

Reading the text there and re-finding it in the new version. That's scripts/kindle_position.py. It strips markup to visible prose while keeping each character's byte offset, then works down four tiers:

• exact — the prose ending at the position is still present, verbatim and unique. Anchors run 300 → 25 characters; short ones are safe because uniqueness across the whole book is the acceptance test. The short tiers matter more than I first assumed: a 300-character anchor breaks if anything is inserted anywhere in the preceding paragraph, which is what a test caught.
• fuzzy — that passage was edited. Find the enclosing chapter by heading, diff old chapter against new, carry the position through the diff.
• chapter — chapter rewritten past recognition (similarity under 0.5); return its opening.
• reset — chapter gone; back to the start rather than inventing a location.

Validated against a real edit, not fixtures: 5b2c6ef, the dialogue pass scattering changes across chapters 003–031, old epub pulled from git. Of 399 sampled positions, 394 exact, 5 fuzzy, none wild. redacted sentence: unpublished prose

Two things to be straight about. Your original position is genuinely gone — I deleted it earlier. And while the format reading is confirmed against every book on the device (position always in .mbs, never .mbp1; the leading 0x1ab126 is a constant, not a checksum I'd be breaking), whether the firmware accepts a sidecar I rewrote can only be settled by ejecting and opening the book. That caveat is written into the module.

I also drop .mbp1 rather than carry it. It holds annotations, presumably offset-keyed too, and I don't know its layout well enough to remap them honestly — so highlights won't survive an update. Same machinery would probably do it if that turns out to matter.