diff --git a/CHANGELOG.md b/CHANGELOG.md index 11da919cf..893185809 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,10 +5,10 @@ **`gstack-context-bill` now bills a real install.** On `~/.claude/skills` it reported "2 skills, ~134 tok" always-on, because setup installs each skill as a real directory whose `SKILL.md` and `sections/` are symlinks into the gstack checkout, and the walker only counted regular files. The tool built to answer the token-cost complaint gave a falsely tiny answer on the layout users actually have. On a fresh install it now reports 58 skills, ~6.8K tokens always-on, and 3.6 MB on disk (it was 37 KB across 2 skills). Billing the checkout itself is unchanged (66 skills, ~7.2K tokens). ### Fixed -- **`lib/context-bill.ts`**: `findSkillDirs`, `walkMd` and the foreign-file check follow symlinked files and directories. Every path the bill reports, and every file `--exact` keys on, stays the lexical path under the root you passed. A link to a directory that contains the tree being billed or the skill being walked (`..`, `/`, `$HOME`) is never followed, so a link cannot climb out of the root into its parent or your home directory, even from a symlinked skill that lives elsewhere. The markdown walk expands each real directory once and counts each real file once per skill, under the first path reached in sorted order, which is how a skill loads it: loops end, links that fan out into each other cannot multiply the walk, and a tree with more than 200,000 entries is refused with an error instead of being cut short. `--exact` walks and sends only what the bill walks (the same per-skill walks and fences, so a link no skill bills is never walked), each real file once, measures frontmatter only for the `SKILL.md` files the bill prices it for, and prices every link to it with that one measurement. It sends only files inside the tree, inside the gstack checkout that the tree's `gstack/` entry links to (a git checkout with `VERSION`, `setup` and a `SKILL.md` named `gstack`; `bin/dev-setup`'s `.claude/skills/gstack -> ` counts), or inside gstack's render folder `$GSTACK_STATE_ROOT/render` (where setup renders the `SKILL.md` files that gbrain, `--claude-model` and per-install renders link to; trusted only when `render` is a real directory, not a link). Each file is judged by its realpath, so a link out of one of those folders is refused too; anything else a symlink reaches (a skill's `notes -> ~/Documents`, say) is not sent, its directory is named in the notice, the files are listed, and they are estimated offline. The notice also says how many sent files come from the checkout and how many from the render folder. A bill in which `--exact` had to estimate any item (a refused file, or one that changed after it was measured) says so: `tokenSource` starts with `mixed:` and names the count, `tokenEstimateErrorPct` keeps the estimate's 40% band, and the text output says the counts are mixed instead of measured. A measurement is reused only for a file that still has the realpath, inode, size and mtime it was measured with, so a link retargeted or a file rewritten after the measurement (during `--diff`, say) is estimated and counted as estimated instead of inheriting the old count. Billed paths are matched to their measurement one link at a time, so a second skill sharing a link chain deeper than the kernel's limit is still priced from the one measurement. Files are opened without blocking, so a file swapped for a FIFO is refused instead of hanging the run. An empty `.md` file and a `SKILL.md` without frontmatter are measured as zero tokens without a request, so they no longer make `count_tokens` reject the run (an empty file used to drop every measurement with a 400) or turn a fully measured bill into a mixed one. Each file is read through the realpath its walk recorded and only while it is still that same file, so a link retargeted or a file swapped for a link mid-run is refused rather than sent. A link named `*.md` counts only when its target is a `.md` file, so `--exact` never sends a non-markdown file through a markdown-named link. `walkMd` skips a child repo checkout (the `gstack/` link beside the installed skills), as `findSkillDirs` already did, so `--exact` does not send the whole checkout. The walk and skill discovery read each directory through its realpath, so a chain of links deeper than the kernel's 40-link limit no longer drops files or skills silently. Skill discovery has the same 200,000-entry cap as the walk. A container skill skips only nested directories whose `SKILL.md` is a file, as discovery does, so a `sub/` whose `SKILL.md` is a directory is billed to the container instead of to nobody. The root container also skips a nested skill that a directory link reaches at any depth (discovery already bills it), so `refs/alias -> ../sub` no longer counts `sub`'s files a third time in TOTAL. `--exact` on a path that does not exist now stops with `No such tree` before any request, including `--diff` with one missing tree (it used to send its one-character envelope probe first, and `--diff` sent the whole first tree), and its notice counts the same list of files it then sends. An existing tree with no skills sends nothing and writes no egress receipt, a tree whose skills have no markdown file to send says the offline estimate was used, and a tree whose every billed file was refused says so. `--help` now says which `.md` files `--exact` sends and that others are named and estimated. +- **`lib/context-bill.ts`**: `findSkillDirs`, `walkMd` and the foreign-file check follow symlinked files and directories. Every path the bill reports, and every file `--exact` keys on, stays the lexical path under the root you passed. A link to a directory that contains the tree being billed or the skill being walked (`..`, `/`, `$HOME`) is never followed, so a link cannot climb out of the root into its parent or your home directory, even from a symlinked skill that lives elsewhere. The markdown walk expands each real directory once and counts each real file once per skill, under the first path reached in sorted order, which is how a skill loads it: loops end, links that fan out into each other cannot multiply the walk, and a tree with more than 200,000 entries is refused with an error instead of being cut short. `--exact` walks and sends only what the bill walks (the same per-skill walks and fences, so a link no skill bills is never walked), each real file once, measures frontmatter only for the `SKILL.md` files the bill prices it for, and prices every link to it with that one measurement. It sends only files inside the tree, inside the gstack checkout that the tree's `gstack/` entry links to (a git checkout with `VERSION`, `setup` and a `SKILL.md` named `gstack`; `bin/dev-setup`'s `.claude/skills/gstack -> ` counts), or inside gstack's render folder `$GSTACK_STATE_ROOT/render` (where setup renders the `SKILL.md` files that gbrain, `--claude-model` and per-install renders link to; trusted only when `render` is a real directory, not a link). Each file is judged by its realpath, so a link out of one of those folders is refused too; anything else a symlink reaches (a skill's `notes -> ~/Documents`, say) is not sent, its directory is named in the notice, the files are listed, and they are estimated offline. The notice also says how many sent files come from the checkout and how many from the render folder. A bill in which `--exact` had to estimate any item (a refused file, or one that changed after it was measured) says so: `tokenSource` starts with `mixed:` and names the count, `tokenEstimateErrorPct` keeps the estimate's 40% band, and the text output says the counts are mixed instead of measured. A measurement is reused only for a file that still has the realpath, inode, size and mtime it was measured with, so a link retargeted or a file rewritten after the measurement (during `--diff`, say) is estimated and counted as estimated instead of inheriting the old count. Billed paths are matched to their measurement one link at a time, so a second skill sharing a link chain deeper than the kernel's limit is still priced from the one measurement. Files are opened without blocking, so a file swapped for a FIFO is refused instead of hanging the run. A blank `.md` file (empty or whitespace only) and a `SKILL.md` without frontmatter are measured as zero tokens without a request, so they no longer make `count_tokens` reject the run (a blank file used to drop every measurement with a 400) or turn a fully measured bill into a mixed one. When every billed file is blank, nothing is sent and no receipt is written. The sending notice is printed after the files are read, just before the receipt, and counts exactly the files sent. A state root that is missing, unreadable or not a directory just means there is no render folder to trust; the bill still runs. Each file is read through the realpath its walk recorded and only while it is still that same file, so a link retargeted or a file swapped for a link mid-run is refused rather than sent. A link named `*.md` counts only when its target is a `.md` file, so `--exact` never sends a non-markdown file through a markdown-named link. `walkMd` skips a child repo checkout (the `gstack/` link beside the installed skills), as `findSkillDirs` already did, so `--exact` does not send the whole checkout. The walk and skill discovery read each directory through its realpath, so a chain of links deeper than the kernel's 40-link limit no longer drops files or skills silently. Skill discovery has the same 200,000-entry cap as the walk. A container skill skips only nested directories whose `SKILL.md` is a file, as discovery does, so a `sub/` whose `SKILL.md` is a directory is billed to the container instead of to nobody. The root container also skips a nested skill that a directory link reaches at any depth (discovery already bills it), so `refs/alias -> ../sub` no longer counts `sub`'s files a third time in TOTAL. `--exact` on a path that does not exist now stops with `No such tree` before any request, including `--diff` with one missing tree (it used to send its one-character envelope probe first, and `--diff` sent the whole first tree), and its notice counts the same list of files it then sends. An existing tree with no skills sends nothing and writes no egress receipt, a tree whose skills have no markdown file to send says the offline estimate was used, and a tree whose every billed file was refused says so. `--help` now says which `.md` files `--exact` sends and that others are named and estimated. ### Tests -- `test/context-bill.test.ts`: an installed-layout fixture (a checkout outside the skills root, real skill directories with symlinked `SKILL.md` and `sections/`, and a `gstack/` link to the checkout) bills both skills with the target's exact frontmatter, `SKILL.md` and section bytes, and keeps every walked path under the root. Negative controls: a dangling `SKILL.md` link and a `SKILL.md` link to a directory are not skills. A second fixture holds a self link, a parent link that reaches a skill outside the root, a loop back into the skill and a link up to the temp root; only the one real skill and its own files are billed. Further cases cover a loop to a directory below the walk root, a root whose name starts with `..`, a symlinked `agents.md` (with a dangling link and a directory of a foreign-file name as controls), `--exact` on the installed layout (every billed file measured; nothing behind `gstack/` or a `.md`-named link to a non-markdown file is sent), a 12-level fan-out of links that reaches one file by 2^12 paths (walked once, where a walker that expands every link would return all 4,096), two skills linking one checkout file and directory (each real file sent once, both skills priced from it), a symlinked skill outside the root with a link up to the root's parent, the `--exact` notice's count of files from the gstack checkout, a skill whose `SKILL.md` another skill's walk reaches first under another name (its frontmatter still measured), a container skill whose nested skill links to the container's own files (both skills still bill them), `--exact` on a missing tree and `--diff --exact` with a missing second tree (named, with no request sent), a symlinked skill with a link up to its real parent (the files there are not billed, so not sent, and no billed file is keyed through that link), an existing tree with no skills (no request, no receipt), a skill whose `SKILL.md` links to a non-markdown file (no request, no receipt, and a line saying the estimate was used), a reference file with frontmatter (no frontmatter request for it), an install whose skill also links `notes` to a documents folder (the checkout files are sent; the documents file is refused, named, listed and estimated), a `gstack/` entry that is not a gstack checkout (a plain folder, a bare `.git` folder, another project's repo, or a gstack-shaped folder without `.git`: nothing behind it counts as the checkout), a `bin/dev-setup` tree inside the gstack repo (its skill files are sent), refused files in nested and sibling folders (each folder named once), every billed file refused (said so), a link retargeted, a file replaced by a link or a hard link, or a parent folder swapped during `--diff --exact` (never sent; a swapped file stops the run), a 50-deep chain of links (every file walked, billed and, under `--exact`, measured), a skill 45 directory links deep (discovered), skill discovery past its entry cap (refused), a `sub/` whose `SKILL.md` is a directory (billed to the container), a tree with no skills under `--exact` (no estimate line), a root container with a `refs/alias -> ../sub` link and a link to an outside folder holding a skill (neither nested skill billed to the container), a file swapped for a FIFO after the walk (refused within the subprocess timeout, where it used to hang), two skills sharing a 50-deep link chain (both priced from one measurement, nothing estimated), a `SKILL.md` link retargeted to a bigger file and a reference rewritten in place after measurement (both estimated and counted as such), a bill with a refused skill (`mixed:` in `--json` and in text), a file replaced by one of the same size and mtime (estimated), a skill without frontmatter plus an empty reference under a `count_tokens` stand-in that rejects empty content (still fully measured), a render folder that is itself a link (not trusted), and a render install (its `SKILL.md` and sections sent from `$GSTACK_STATE_ROOT/render`, a link from there to a documents file refused and listed). The installed-layout and escape fixtures fail on main (the first finds no skills, the second bills the outside skill), the missing-tree case fails on main (one request sent), and each later case failed on the commit before its fix, except the hard-link and parent-swap cases, which pin the identity check added with the realpath reads (they fail when that check is removed). +- `test/context-bill.test.ts`: an installed-layout fixture (a checkout outside the skills root, real skill directories with symlinked `SKILL.md` and `sections/`, and a `gstack/` link to the checkout) bills both skills with the target's exact frontmatter, `SKILL.md` and section bytes, and keeps every walked path under the root. Negative controls: a dangling `SKILL.md` link and a `SKILL.md` link to a directory are not skills. A second fixture holds a self link, a parent link that reaches a skill outside the root, a loop back into the skill and a link up to the temp root; only the one real skill and its own files are billed. Further cases cover a loop to a directory below the walk root, a root whose name starts with `..`, a symlinked `agents.md` (with a dangling link and a directory of a foreign-file name as controls), `--exact` on the installed layout (every billed file measured; nothing behind `gstack/` or a `.md`-named link to a non-markdown file is sent), a 12-level fan-out of links that reaches one file by 2^12 paths (walked once, where a walker that expands every link would return all 4,096), two skills linking one checkout file and directory (each real file sent once, both skills priced from it), a symlinked skill outside the root with a link up to the root's parent, the `--exact` notice's count of files from the gstack checkout, a skill whose `SKILL.md` another skill's walk reaches first under another name (its frontmatter still measured), a container skill whose nested skill links to the container's own files (both skills still bill them), `--exact` on a missing tree and `--diff --exact` with a missing second tree (named, with no request sent), a symlinked skill with a link up to its real parent (the files there are not billed, so not sent, and no billed file is keyed through that link), an existing tree with no skills (no request, no receipt), a skill whose `SKILL.md` links to a non-markdown file (no request, no receipt, and a line saying the estimate was used), a reference file with frontmatter (no frontmatter request for it), an install whose skill also links `notes` to a documents folder (the checkout files are sent; the documents file is refused, named, listed and estimated), a `gstack/` entry that is not a gstack checkout (a plain folder, a bare `.git` folder, another project's repo, or a gstack-shaped folder without `.git`: nothing behind it counts as the checkout), a `bin/dev-setup` tree inside the gstack repo (its skill files are sent), refused files in nested and sibling folders (each folder named once), every billed file refused (said so), a link retargeted, a file replaced by a link or a hard link, or a parent folder swapped during `--diff --exact` (never sent; a swapped file stops the run), a 50-deep chain of links (every file walked, billed and, under `--exact`, measured), a skill 45 directory links deep (discovered), skill discovery past its entry cap (refused), a `sub/` whose `SKILL.md` is a directory (billed to the container), a tree with no skills under `--exact` (no estimate line), a root container with a `refs/alias -> ../sub` link and a link to an outside folder holding a skill (neither nested skill billed to the container), a file swapped for a FIFO after the walk (refused within the subprocess timeout, where it used to hang), two skills sharing a 50-deep link chain (both priced from one measurement, nothing estimated), a `SKILL.md` link retargeted to a bigger file and a reference rewritten in place after measurement (both estimated and counted as such), a bill with a refused skill (`mixed:` in `--json` and in text), a file replaced by one of the same size and mtime (estimated), a skill without frontmatter plus an empty reference under a `count_tokens` stand-in that rejects empty content (still fully measured), a render folder that is itself a link (not trusted), a tree whose only file is blank (no request, no receipt), a state root that is a file (the bill still runs, measured), a git checkout inside a skill or linked from one (not billed; this case passes on main and fails when the walk's `.git` check is removed), and a render install (its `SKILL.md` and sections sent from `$GSTACK_STATE_ROOT/render`, a link from there to a documents file refused and listed). The installed-layout and escape fixtures fail on main (the first finds no skills, the second bills the outside skill), the missing-tree case fails on main (one request sent), and each later case failed on the commit before its fix, except the hard-link and parent-swap cases, which pin the identity check added with the realpath reads (they fail when that check is removed). ### Gate integrity - No threshold, budget, timeout, trial count, case kind or assertion changed. The bill of this repo's own tree, which the context-budget ratchet reads, has the same skills, bytes and rounded token figures as on main. Four raw `totalMdTokens` values in `--json` differ in the last floating-point digit, because each skill's files are now summed in sorted order. diff --git a/lib/context-bill.ts b/lib/context-bill.ts index 3dc5fb8e6..34041ba5e 100644 --- a/lib/context-bill.ts +++ b/lib/context-bill.ts @@ -875,12 +875,21 @@ export interface ExactSendSet { refused: WalkedFile[]; /** Realpath of the gstack checkout the root's gstack/ entry links to, when there is one. */ checkout?: string; - /** Realpath of gstack's render folder ($GSTACK_STATE_ROOT/render), when it exists. */ + /** Realpath of gstack's render folder ($GSTACK_STATE_ROOT/render) when it is a real directory; a render link is not trusted. */ render?: string; /** Realpaths of the SKILL.md files whose frontmatter the bill prices. */ frontmatter: Set; } +/** The realpath of `dir` when it is a real directory, not a link; otherwise (missing, unreadable, a file) undefined. */ +function realDirectory(dir: string): string | undefined { + try { + return fs.lstatSync(dir).isDirectory() ? fs.realpathSync(dir) : undefined; + } catch { + return undefined; + } +} + /** A git checkout with VERSION, setup and a SKILL.md named gstack. */ function isGstackCheckout(dir: string): boolean { if (!fs.existsSync(path.join(dir, ".git"))) return false; @@ -896,7 +905,7 @@ function isGstackCheckout(dir: string): boolean { * (the root's gstack/ entry, when it is a git checkout whose SKILL.md is named * gstack and which has VERSION and setup) or inside gstack's render folder * ($GSTACK_STATE_ROOT/render, where setup renders the SKILL.md files some - * installs link to) are sent; anything else a link reaches is refused and + * installs link to; trusted only as a real directory, not a link) are sent; anything else a link reaches is refused and * estimated offline. Each file is judged by its realpath, so a link out of * one of those folders is refused too. * @@ -914,8 +923,7 @@ export function exactSendSet(root: string, stateRoot: string = resolveStateRoot( const rootReal = fs.realpathSync(resolved); const gstackEntry = path.join(resolved, "gstack"); const checkout = isGstackCheckout(gstackEntry) ? fs.realpathSync(gstackEntry) : undefined; - const renderDir = path.join(stateRoot, "render"); - const render = fs.lstatSync(renderDir, { throwIfNoEntry: false })?.isDirectory() ? fs.realpathSync(renderDir) : undefined; + const render = realDirectory(path.join(stateRoot, "render")); const approved = [rootReal, checkout, render].filter((d): d is string => d !== undefined); for (const dir of findSkillDirs(resolved)) { frontmatter.add(fs.realpathSync(path.join(dir, "SKILL.md"))); @@ -997,13 +1005,15 @@ export interface ExactMeasurement { */ export async function measureExactTokens( root: string, - { sendSet, model, apiKey, fetchImpl = fetch, onProgress, egressHome }: { + { sendSet, model, apiKey, fetchImpl = fetch, onProgress, onSend, egressHome }: { /** The send set from exactSendSet(root), when the caller already computed it. */ sendSet?: ExactSendSet; model: string; apiKey: string; fetchImpl?: FetchLike; onProgress?: (done: number, total: number) => void; + /** Called with the files about to be sent, after they are read and before the receipt; not called when nothing is sent. */ + onSend?: (files: WalkedFile[]) => void; egressHome?: string; }, ): Promise { @@ -1016,8 +1026,9 @@ export async function measureExactTokens( const opts: CountTokensOptions = { model, apiKey, fetchImpl }; const texts = new Map(); - // Empty texts are not sent (count_tokens rejects empty content); they are - // measured as zero tokens. + // Blank texts are not sent (count_tokens rejects content with no + // non-whitespace text); they are measured as zero tokens. When nothing is + // left to send, no receipt is written and no request is made. const counts = new Map(); // The send set holds each real file once, so each is sent once. The bill keys // the same file by every lexical path it is billed under; measuredAs maps a @@ -1031,7 +1042,7 @@ export async function measureExactTokens( const text = readWalkedFile(walked, root); measuredAs.set(fileIdentity(real, walked), file); realOf.set(file, real); - if (text) texts.set(file, text); + if (/\S/.test(text)) texts.set(file, text); else counts.set(file, 0); if (!frontmatter.has(real)) continue; const fm = parseFrontmatter(text); @@ -1039,6 +1050,45 @@ export async function measureExactTokens( else counts.set(frontmatterKey(file), 0); } + // A key the send set does not hold (a non-.md foreign-host file, or a file + // refused as outside the root, the checkout and the render folder) falls back to the + // estimate rather than billing zero. So does a key whose file is no longer + // the one measured (a link retargeted, or the file rewritten, since the walk). + // Misses are counted, not swallowed: a bill that is part-measured and + // part-estimated must not present itself as measured. + const missed = new Set(); + const fmSuffix = frontmatterKey(""); + const measuredKey = (key: string): string | undefined => { + const isFm = key.endsWith(fmSuffix); + let real: string; + let st: fs.Stats; + try { + real = realpathStepwise(isFm ? key.slice(0, -fmSuffix.length) : key); + st = fs.lstatSync(real); + } catch { + return undefined; + } + const lexical = measuredAs.get(fileIdentity(real, st)); + return lexical === undefined ? undefined : isFm ? frontmatterKey(lexical) : lexical; + }; + const tokensOf: TokensOf = (key, bytes) => { + const alias = measuredKey(key); + const exact = alias === undefined ? undefined : counts.get(alias); + if (exact !== undefined) return exact; + missed.add(key); + return estimateTokensOf(key, bytes); + }; + const measurement: ExactMeasurement = { + tokenSource: `count_tokens (${model})`, + tokensOf, + measuredFiles: 0, + counts, + realOf, + missedKeys: missed, + }; + if (texts.size === 0) return measurement; + onSend?.(files.filter((f) => texts.has(f.path))); + // Receipt-before-send. Content-free: file count + total bytes only. try { let totalBytes = 0; @@ -1074,43 +1124,8 @@ export async function measureExactTokens( } }; await Promise.all(Array.from({ length: Math.min(EXACT_CONCURRENCY, keys.length) }, worker)); - - // A key the send set does not hold (a non-.md foreign-host file, or a file - // refused as outside the root, the checkout and the render folder) falls back to the - // estimate rather than billing zero. So does a key whose file is no longer - // the one measured (a link retargeted, or the file rewritten, since the walk). - // Misses are counted, not swallowed: a bill that is part-measured and - // part-estimated must not present itself as measured. - const missed = new Set(); - const fmSuffix = frontmatterKey(""); - const measuredKey = (key: string): string | undefined => { - const isFm = key.endsWith(fmSuffix); - let real: string; - let st: fs.Stats; - try { - real = realpathStepwise(isFm ? key.slice(0, -fmSuffix.length) : key); - st = fs.lstatSync(real); - } catch { - return undefined; - } - const lexical = measuredAs.get(fileIdentity(real, st)); - return lexical === undefined ? undefined : isFm ? frontmatterKey(lexical) : lexical; - }; - const tokensOf: TokensOf = (key, bytes) => { - const alias = measuredKey(key); - const exact = alias === undefined ? undefined : counts.get(alias); - if (exact !== undefined) return exact; - missed.add(key); - return estimateTokensOf(key, bytes); - }; - return { - tokenSource: `count_tokens (${model})`, - tokensOf, - measuredFiles: counts.size, - counts, - realOf, - missedKeys: missed, - }; + measurement.measuredFiles = counts.size; + return measurement; } export interface Calibration { @@ -1278,27 +1293,36 @@ export async function contextBillMain(argv: string[], options: MainOptions = {}) } return {}; } - const treeReal = fs.realpathSync(tree); - const outsideTree = set.files.filter((f) => !encloses(treeReal, f.real)); - const inCheckout = outsideTree.filter((f) => set.checkout && encloses(set.checkout, f.real)).length; - const inRender = outsideTree.length - inCheckout; - const from = [ - inCheckout ? `${inCheckout} of them in the gstack checkout ${set.checkout}` : "", - inRender ? `${inRender} in gstack's render folder ${set.render}` : "", - ].filter(Boolean); - stderr.write( - `--exact: sending the content of ${set.files.length} .md file(s) under ${tree}` + - (from.length ? ` (${from.join(", ")})` : "") + - ` to api.anthropic.com for count_tokens (${flags.exactModel}). No other data leaves this machine.\n`, - ); + let announced = false; + const announce = (sent: WalkedFile[]) => { + announced = true; + const treeReal = fs.realpathSync(tree); + const outsideTree = sent.filter((f) => !encloses(treeReal, f.real)); + const inCheckout = outsideTree.filter((f) => set.checkout && encloses(set.checkout, f.real)).length; + const inRender = outsideTree.length - inCheckout; + const from = [ + inCheckout ? `${inCheckout} of them in the gstack checkout ${set.checkout}` : "", + inRender ? `${inRender} in gstack's render folder ${set.render}` : "", + ].filter(Boolean); + stderr.write( + `--exact: sending the content of ${sent.length} .md file(s) under ${tree}` + + (from.length ? ` (${from.join(", ")})` : "") + + ` to api.anthropic.com for count_tokens (${flags.exactModel}). No other data leaves this machine.\n`, + ); + }; try { const measured = await measureExactTokens(tree, { sendSet: set, model: flags.exactModel, apiKey: options.apiKey ?? process.env.ANTHROPIC_API_KEY ?? "", fetchImpl: options.fetchImpl, + onSend: announce, egressHome: options.egressHome, }); + if (!announced) { + stderr.write(`--exact: every billed .md file under ${tree} is blank; nothing was sent, so the bill uses the offline estimate.\n`); + return {}; + } return { tokensOf: measured.tokensOf, tokenSource: measured.tokenSource, diff --git a/test/context-bill.test.ts b/test/context-bill.test.ts index 728e4893c..95c4de5ac 100644 --- a/test/context-bill.test.ts +++ b/test/context-bill.test.ts @@ -195,6 +195,25 @@ describe("upstream fix a: root-as-container + walkMd exclusions", () => { fs.rmSync(tmp, { recursive: true, force: true }); }); + it("skips a git checkout inside or linked from a skill, as it skips the root's gstack/ checkout", async () => { + if (process.platform === "win32") return; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-nested-repo-")); + fs.mkdirSync(path.join(tmp, "qa", "vendor", "lib", ".git"), { recursive: true }); + fs.writeFileSync(path.join(tmp, "qa", "SKILL.md"), "---\nname: qa\ndescription: d\n---\n# qa\n"); + fs.writeFileSync(path.join(tmp, "qa", "vendor", "lib", "README.md"), "VENDORED ".repeat(500)); + const ext = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-nested-repo-ext-")); + fs.mkdirSync(path.join(ext, ".git")); + fs.writeFileSync(path.join(ext, "NOTES.md"), "LINKED-REPO"); + fs.symlinkSync(ext, path.join(tmp, "qa", "linked")); + const walked = walkMd(path.join(tmp, "qa")).map((f) => path.relative(tmp, f)); + const billed = buildBill(tmp).skills[0].totalMdBytes; + const skillBytes = fileBytes(tmp, "qa", "SKILL.md"); + fs.rmSync(tmp, { recursive: true, force: true }); + fs.rmSync(ext, { recursive: true, force: true }); + expect(walked).toEqual([path.join("qa", "SKILL.md")]); + expect(billed).toBe(skillBytes); + }); + it("walkMd skips node_modules and dot-directories", () => { const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-walk-")); fs.writeFileSync(path.join(tmp, "real.md"), "x"); @@ -1247,12 +1266,13 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.mkdirSync(path.join(tmp, "alpha")); fs.writeFileSync(path.join(tmp, "alpha", "SKILL.md"), "---\nname: alpha\ndescription: d\n---\n# alpha\n"); fs.writeFileSync(path.join(tmp, "alpha", "empty.md"), ""); + fs.writeFileSync(path.join(tmp, "alpha", "blank.md"), " \n"); fs.mkdirSync(path.join(tmp, "beta")); fs.writeFileSync(path.join(tmp, "beta", "SKILL.md"), "# beta has no frontmatter\n"); const inner = fakeFetch() as unknown as (u: string, i: { body: string }) => Promise; const strict = (async (u: string, i: { body: string }) => - JSON.parse(i.body).messages[0].content === "" - ? { ok: false, status: 400, text: async () => "messages.0: user messages must have non-empty content" } + !/\S/.test(JSON.parse(i.body).messages[0].content) + ? { ok: false, status: 400, text: async () => "messages: text content blocks must contain non-whitespace text" } : inner(u, i)) as never; const measured = await measureExactTokens(tmp, { model: "m", apiKey: "sk-test", fetchImpl: strict, egressHome }); const bill = buildBill(tmp, { tokensOf: measured.tokensOf, tokenSource: measured.tokenSource, missedKeys: measured.missedKeys }); @@ -1262,6 +1282,52 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () expect(bill.tokenSource).toBe("count_tokens (m)"); }); + it("sends nothing and writes no receipt when every billed file is blank", async () => { + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-blank-")); + fs.mkdirSync(path.join(tmp, "skills", "qa"), { recursive: true }); + fs.writeFileSync(path.join(tmp, "skills", "qa", "SKILL.md"), "\n"); + const home = path.join(tmp, "egress"); + const calls: { body: string }[] = []; + const measured = await measureExactTokens(path.join(tmp, "skills"), { model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(calls), egressHome: home }); + const stderr = capture(); + const code = await contextBillMain([path.join(tmp, "skills"), "--exact", "--json"], { + stdout: capture().stream, + stderr: stderr.stream, + apiKey: "sk-test", + fetchImpl: fakeFetch(calls), + egressHome: home, + }); + const receipts = listReceipts(home).length; + fs.rmSync(tmp, { recursive: true, force: true }); + expect(calls).toEqual([]); + expect(receipts).toBe(0); + expect([...measured.missedKeys]).toEqual([]); + expect(code).toBe(0); + expect(stderr.text()).not.toContain("sending the content"); + expect(stderr.text()).toContain("nothing was sent"); + }); + + it("still bills the tree when the state root is not a readable directory", async () => { + if (process.platform === "win32") return; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-stateroot-file-")); + const stateRoot = path.join(tmp, "statefile"); + fs.writeFileSync(stateRoot, "not a directory"); + fs.mkdirSync(path.join(tmp, "skills", "qa"), { recursive: true }); + fs.writeFileSync(path.join(tmp, "skills", "qa", "SKILL.md"), "---\nname: qa\ndescription: d\n---\n# qa\n"); + const stdout = capture(); + const code = await contextBillMain([path.join(tmp, "skills"), "--exact", "--json"], { + stdout: stdout.stream, + stderr: capture().stream, + apiKey: "sk-test", + fetchImpl: fakeFetch(), + egressHome, + stateRoot, + }); + fs.rmSync(tmp, { recursive: true, force: true }); + expect(code).toBe(0); + expect(JSON.parse(stdout.text()).tokenSource).toBe("count_tokens (claude-opus-4-5)"); + }); + it("refuses a walked file swapped for a FIFO instead of hanging", () => { if (process.platform === "win32") return; const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-fifo-"));