Producers appended lines to .brain-queue.jsonl while the drain re-read and
os.replace'd it; the in-code comment admitted a lockless append between the
re-read and the replace was lost. Locks and rename-rotation designs were both
reviewed and rejected (each retained a tail race); the shipped design is a
maildir-style spool: one FILE per record in .brain-queue.d/ (tmp + atomic
rename), the drain snapshots filenames, processes, and deletes exactly what
it snapshotted. Writer and drainer never share an inode — nothing to race.
Semantics: at-least-once (a crash between process and unlink re-drains;
downstream content-hash dedup absorbs duplicates); retained (privacy-held)
records keep their files; unparseable records are kept + warned, never
destroyed. Legacy .brain-queue.jsonl migrates atomically on the next drain
(crash-leftover .migrating files recovered too); status/drop-queue count both
surfaces; discover-new writes spool records and advances its cursor
per-record-written. The preamble's queue-depth line switches to spool count
in this wave's template block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
