diff --git a/CHANGELOG.md b/CHANGELOG.md index 3105954f0..6f1225fff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,10 +5,10 @@ **`gstack-context-bill` now bills a real install.** On `~/.claude/skills` it reported "2 skills, ~134 tok" always-on, because setup installs each skill as a real directory whose `SKILL.md` and `sections/` are symlinks into the gstack checkout, and the walker only counted regular files. The tool built to answer the token-cost complaint gave a falsely tiny answer on the layout users actually have. On a fresh install it now reports 58 skills, ~6.8K tokens always-on, and 3.6 MB on disk (it was 37 KB across 2 skills). Billing the checkout itself is unchanged (66 skills, ~7.2K tokens). ### Fixed -- **`lib/context-bill.ts`**: `findSkillDirs`, `walkMd` and the foreign-file check follow symlinked files and directories. Every path the bill reports, and every file `--exact` keys on, stays the lexical path under the root you passed. A link to a directory that contains the tree being billed or the skill being walked (`..`, `/`, `$HOME`) is never followed, so a link cannot climb out of the root into its parent or your home directory, even from a symlinked skill that lives elsewhere. The markdown walk expands each real directory once and counts each real file once per skill, under the first path reached in sorted order, which is how a skill loads it: loops end, links that fan out into each other cannot multiply the walk, and a tree with more than 200,000 entries is refused with an error instead of being cut short. `--exact` walks and sends only what the bill walks (the same per-skill walks and fences, so a link no skill bills is never walked), each real file once, measures frontmatter only for the `SKILL.md` files the bill prices it for, and prices every link to it with that one measurement, and its notice says how many of the files it sends sit outside the tree behind symlinks (on an install, the checkout's skill files). A link named `*.md` counts only when its target is a `.md` file, so `--exact` never sends a non-markdown file through a markdown-named link. `walkMd` skips a child repo checkout (the `gstack/` link beside the installed skills), as `findSkillDirs` already did, so `--exact` does not send the whole checkout. `--exact` on a path that does not exist now stops with `No such tree` before any request, including `--diff` with one missing tree (it used to send its one-character envelope probe first, and `--diff` sent the whole first tree), and its notice counts the same list of files it then sends. An existing tree with no skills sends nothing and writes no egress receipt, and a tree whose skills have no markdown file to send says the offline estimate was used. `--help` now says `--exact` sends the `.md` files the bill prices, including files reached through symlinks. +- **`lib/context-bill.ts`**: `findSkillDirs`, `walkMd` and the foreign-file check follow symlinked files and directories. Every path the bill reports, and every file `--exact` keys on, stays the lexical path under the root you passed. A link to a directory that contains the tree being billed or the skill being walked (`..`, `/`, `$HOME`) is never followed, so a link cannot climb out of the root into its parent or your home directory, even from a symlinked skill that lives elsewhere. The markdown walk expands each real directory once and counts each real file once per skill, under the first path reached in sorted order, which is how a skill loads it: loops end, links that fan out into each other cannot multiply the walk, and a tree with more than 200,000 entries is refused with an error instead of being cut short. `--exact` walks and sends only what the bill walks (the same per-skill walks and fences, so a link no skill bills is never walked), each real file once, measures frontmatter only for the `SKILL.md` files the bill prices it for, and prices every link to it with that one measurement. It sends only files inside the tree or inside the gstack checkout that the tree's `gstack/` entry links to; anything else a symlink reaches (a skill's `notes -> ~/Documents`, say) is not sent, its directory is named in the notice, the files are listed, and they are estimated offline. The notice also says how many sent files come from the checkout. Each file is read through the realpath its walk recorded and only while it is still that same file, so a link retargeted or a file swapped for a link mid-run is refused rather than sent. A link named `*.md` counts only when its target is a `.md` file, so `--exact` never sends a non-markdown file through a markdown-named link. `walkMd` skips a child repo checkout (the `gstack/` link beside the installed skills), as `findSkillDirs` already did, so `--exact` does not send the whole checkout. The walk reads each directory through its realpath, so a chain of links deeper than the kernel's 40-link limit no longer drops files silently. Skill discovery has the same 200,000-entry cap as the walk. A container skill skips only nested directories whose `SKILL.md` is a file, as discovery does, so a `sub/` whose `SKILL.md` is a directory is billed to the container instead of to nobody. `--exact` on a path that does not exist now stops with `No such tree` before any request, including `--diff` with one missing tree (it used to send its one-character envelope probe first, and `--diff` sent the whole first tree), and its notice counts the same list of files it then sends. An existing tree with no skills sends nothing and writes no egress receipt, and a tree whose skills have no markdown file to send says the offline estimate was used. `--help` now says which `.md` files `--exact` sends and that others are named and estimated. ### Tests -- `test/context-bill.test.ts`: an installed-layout fixture (a checkout outside the skills root, real skill directories with symlinked `SKILL.md` and `sections/`, and a `gstack/` link to the checkout) bills both skills with the target's exact frontmatter, `SKILL.md` and section bytes, and keeps every walked path under the root. Negative controls: a dangling `SKILL.md` link and a `SKILL.md` link to a directory are not skills. A second fixture holds a self link, a parent link that reaches a skill outside the root, a loop back into the skill and a link up to the temp root; only the one real skill and its own files are billed. Further cases cover a loop to a directory below the walk root, a root whose name starts with `..`, a symlinked `agents.md` (with a dangling link and a directory of a foreign-file name as controls), `--exact` on the installed layout (every billed file measured; nothing behind `gstack/` or a `.md`-named link to a non-markdown file is sent), a 12-level fan-out of links that reaches one file by 2^12 paths (walked once, where a walker that expands every link would return all 4,096), two skills linking one checkout file and directory (each real file sent once, both skills priced from it), a symlinked skill outside the root with a link up to the root's parent, the `--exact` notice's outside-the-tree count, a skill whose `SKILL.md` another skill's walk reaches first under another name (its frontmatter still measured), a container skill whose nested skill links to the container's own files (both skills still bill them), `--exact` on a missing tree and `--diff --exact` with a missing second tree (named, with no request sent), a symlinked skill with a link up to its real parent (the files there are not billed, so not sent, and no billed file is keyed through that link), an existing tree with no skills (no request, no receipt), a skill whose `SKILL.md` links to a non-markdown file (no request, no receipt, and a line saying the estimate was used), and a reference file with frontmatter (no frontmatter request for it). The installed-layout and escape fixtures fail on main (the first finds no skills, the second bills the outside skill), the missing-tree case fails on main (one request sent), and each later case failed on the commit before its fix. +- `test/context-bill.test.ts`: an installed-layout fixture (a checkout outside the skills root, real skill directories with symlinked `SKILL.md` and `sections/`, and a `gstack/` link to the checkout) bills both skills with the target's exact frontmatter, `SKILL.md` and section bytes, and keeps every walked path under the root. Negative controls: a dangling `SKILL.md` link and a `SKILL.md` link to a directory are not skills. A second fixture holds a self link, a parent link that reaches a skill outside the root, a loop back into the skill and a link up to the temp root; only the one real skill and its own files are billed. Further cases cover a loop to a directory below the walk root, a root whose name starts with `..`, a symlinked `agents.md` (with a dangling link and a directory of a foreign-file name as controls), `--exact` on the installed layout (every billed file measured; nothing behind `gstack/` or a `.md`-named link to a non-markdown file is sent), a 12-level fan-out of links that reaches one file by 2^12 paths (walked once, where a walker that expands every link would return all 4,096), two skills linking one checkout file and directory (each real file sent once, both skills priced from it), a symlinked skill outside the root with a link up to the root's parent, the `--exact` notice's count of files from the gstack checkout, a skill whose `SKILL.md` another skill's walk reaches first under another name (its frontmatter still measured), a container skill whose nested skill links to the container's own files (both skills still bill them), `--exact` on a missing tree and `--diff --exact` with a missing second tree (named, with no request sent), a symlinked skill with a link up to its real parent (the files there are not billed, so not sent, and no billed file is keyed through that link), an existing tree with no skills (no request, no receipt), a skill whose `SKILL.md` links to a non-markdown file (no request, no receipt, and a line saying the estimate was used), a reference file with frontmatter (no frontmatter request for it), an install whose skill also links `notes` to a documents folder (the checkout files are sent; the documents file is refused, named, listed and estimated), a link or file swapped during `--diff --exact` (never sent; a swapped file stops the run), a 50-deep chain of links (every file walked and billed), skill discovery past its entry cap (refused), a `sub/` whose `SKILL.md` is a directory (billed to the container), and a tree with no skills under `--exact` (no estimate line). The installed-layout and escape fixtures fail on main (the first finds no skills, the second bills the outside skill), the missing-tree case fails on main (one request sent), and each later case failed on the commit before its fix. ### Gate integrity - No threshold, budget, timeout, trial count, case kind or assertion changed. The bill of this repo's own tree, which the context-budget ratchet reads, is byte-for-byte unchanged. diff --git a/lib/context-bill.ts b/lib/context-bill.ts index 3ba14b1cc..0a805c57d 100644 --- a/lib/context-bill.ts +++ b/lib/context-bill.ts @@ -824,13 +824,20 @@ export interface ExactFile extends WalkedFile { export interface ExactSendSet { /** Each real file once, under the first lexical path a skill's walk reaches it by. */ files: ExactFile[]; + /** Files the bill walks outside the root and the gstack checkout: never sent, estimated offline. */ + refused: WalkedFile[]; + /** Realpath of the gstack checkout the root's gstack/ entry links to, when there is one. */ + checkout?: string; /** Realpaths of the SKILL.md files whose frontmatter the bill prices. */ frontmatter: Set; } /** * What --exact sends: the same per-skill walks totalMd bills, with the same - * fences, so nothing is walked or sent that no skill bills. + * fences, so nothing is walked or sent that no skill bills. Of those, only + * files inside the root or inside the gstack checkout the skills link into + * (the root's gstack/ entry, a repo checkout) are sent; anything else a link + * reaches is refused and estimated offline. * * The root is resolved before keying. buildBill resolves its root, so a * relative root here would produce keys that never match and every lookup @@ -840,18 +847,26 @@ export function exactSendSet(root: string): ExactSendSet { const resolved = path.resolve(root); if (!fs.existsSync(resolved)) throw new Error(`No such tree: ${resolved}`); const files: ExactFile[] = []; + const refused: WalkedFile[] = []; const frontmatter = new Set(); const seen = new Set(); + const rootReal = fs.realpathSync(resolved); + const gstackEntry = path.join(resolved, "gstack"); + const checkout = fs.existsSync(path.join(gstackEntry, ".git")) ? fs.realpathSync(gstackEntry) : undefined; for (const dir of findSkillDirs(resolved)) { frontmatter.add(fs.realpathSync(path.join(dir, "SKILL.md"))); for (const file of walkMdFiles(dir, { billedRoot: resolved, skipNestedSkills: true })) { if (seen.has(file.real)) continue; seen.add(file.real); + if (!encloses(rootReal, file.real) && !(checkout && encloses(checkout, file.real))) { + refused.push(file); + continue; + } const { dev, ino } = fs.statSync(file.real); files.push({ ...file, dev, ino }); } } - return { files, frontmatter }; + return { files, refused, frontmatter, checkout }; } /** @@ -1077,8 +1092,10 @@ const USAGE = "\n" + " --exact measure tokens with Anthropic's count_tokens instead of\n" + " estimating. Off by default: it sends the content of every\n" + - " .md file the bill prices (including files reached\n" + - " through symlinks) to api.anthropic.com. Needs\n" + + " .md file the bill prices inside the tree, or inside\n" + + " the gstack checkout its gstack/ entry links to, to\n" + + " api.anthropic.com. Files that symlinks reach anywhere\n" + + " else are named, not sent, and estimated offline. Needs\n" + " ANTHROPIC_API_KEY; passing --exact is the consent. An\n" + " egress receipt is written before the send (see\n" + " gstack-egress); if it cannot be written, the run falls\n" + @@ -1135,17 +1152,28 @@ export async function contextBillMain(argv: string[], options: MainOptions = {}) onDone?: () => void; }> => { if (!set) return {}; + if (set.refused.length) { + const outsideDirs: string[] = []; + for (const d of [...new Set(set.refused.map((f) => path.dirname(f.real)))].sort()) { + if (!outsideDirs.some((kept) => encloses(kept, d))) outsideDirs.push(d); + } + stderr.write( + `--exact: not sending ${set.refused.length} .md file(s) that symlinks reach outside ${tree} and the gstack checkout ` + + `(under ${outsideDirs.join(", ")}); they are estimated offline:\n` + + set.refused.map((f) => ` ${path.relative(tree, f.path)}\n`).join(""), + ); + } if (set.files.length === 0) { - if (findSkillDirs(path.resolve(tree)).length) { + if (set.refused.length || findSkillDirs(path.resolve(tree)).length) { stderr.write(`--exact: no markdown skill file to measure under ${tree}; nothing was sent, so the bill uses the offline estimate.\n`); } return {}; } const treeReal = fs.realpathSync(tree); - const outside = set.files.filter((f) => !encloses(treeReal, f.real)).length; + const inCheckout = set.files.filter((f) => !encloses(treeReal, f.real)).length; stderr.write( `--exact: sending the content of ${set.files.length} .md file(s) under ${tree}` + - (outside ? ` (${outside} of them through symlinks to files outside it)` : "") + + (inCheckout ? ` (${inCheckout} of them in the gstack checkout ${set.checkout})` : "") + ` to api.anthropic.com for count_tokens (${flags.exactModel}). No other data leaves this machine.\n`, ); try { diff --git a/test/context-bill.test.ts b/test/context-bill.test.ts index 3e6f67241..9466c2e07 100644 --- a/test/context-bill.test.ts +++ b/test/context-bill.test.ts @@ -809,6 +809,8 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.symlinkSync(path.join(checkout, "SKILL.md"), path.join(skills, name, "SKILL.md")); fs.symlinkSync(path.join(checkout, "shared"), path.join(skills, name, "refs")); } + fs.mkdirSync(path.join(checkout, ".git")); + fs.symlinkSync(checkout, path.join(skills, "gstack")); const calls: { body: string }[] = []; const measured = await measureExactTokens(skills, { model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(calls), egressHome }); @@ -916,7 +918,7 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () expect(bill.calibration.rows.length).toBeGreaterThan(0); }); - it("says how many of the files it sends sit outside the tree behind symlinks", async () => { + it("says how many of the files it sends come from the gstack checkout", async () => { if (process.platform === "win32") return; const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-notice-")); fs.mkdirSync(path.join(tmp, "checkout", "qa", "sections"), { recursive: true }); @@ -927,6 +929,8 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.symlinkSync(path.join(tmp, "checkout", "qa", "SKILL.md"), path.join(skills, "qa", "SKILL.md")); fs.symlinkSync(path.join(tmp, "checkout", "qa", "sections"), path.join(skills, "qa", "sections")); fs.writeFileSync(path.join(skills, "qa", "notes.md"), "n"); + fs.mkdirSync(path.join(tmp, "checkout", ".git")); + fs.symlinkSync(path.join(tmp, "checkout"), path.join(skills, "gstack")); const stderr = capture(); const code = await contextBillMain([skills, "--exact", "--json"], { stdout: capture().stream, @@ -937,7 +941,8 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () }); expect(code).toBe(0); expect(stderr.text()).toContain("sending the content of 3 .md file(s)"); - expect(stderr.text()).toContain("(2 of them through symlinks to files outside it)"); + expect(stderr.text()).toContain(`(2 of them in the gstack checkout ${fs.realpathSync(path.join(tmp, "checkout"))})`); + expect(stderr.text()).not.toContain("not sending"); fs.rmSync(tmp, { recursive: true, force: true }); }); @@ -953,6 +958,8 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.writeFileSync(path.join(ext, "private", "notes.md"), "PRIVATE-NOTES"); fs.symlinkSync(ext, path.join(ext, "cc", "up")); fs.symlinkSync(path.join(ext, "cc"), path.join(skills, "cc")); + fs.mkdirSync(path.join(ext, ".git")); + fs.symlinkSync(ext, path.join(skills, "gstack")); const calls: { body: string }[] = []; const stderr = capture(); const code = await contextBillMain([skills, "--exact", "--json"], { @@ -985,6 +992,8 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.mkdirSync(skills); fs.symlinkSync(path.join(dev, "m"), path.join(skills, "m")); fs.symlinkSync(path.join(dev, "zz"), path.join(skills, "zz")); + fs.mkdirSync(path.join(dev, ".git")); + fs.symlinkSync(dev, path.join(skills, "gstack")); const calls: { body: string }[] = []; const measured = await measureExactTokens(skills, { model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(calls), egressHome }); const keys = [...measured.counts.keys()].map((k) => path.relative(skills, k)).sort(); @@ -1067,7 +1076,7 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () fs.mkdirSync(path.join(tmp, tree, "qa"), { recursive: true }); fs.writeFileSync(path.join(tmp, tree, "qa", "SKILL.md"), "---\nname: qa\ndescription: d\n---\n# qa\n"); } - const store = path.join(tmp, "store"); + const store = path.join(tmp, "b", "store"); fs.mkdirSync(store); fs.writeFileSync(path.join(store, "safe.md"), "safe"); fs.writeFileSync(path.join(store, "secret.env"), "SYNTHETIC_PRIVATE_NONMARKDOWN"); @@ -1110,6 +1119,50 @@ describe("--exact (opt-in measurement; offline here via an injected fetch)", () expect(replaced.err).toContain("changed after it was walked"); }); + it("sends files in the root and the linked gstack checkout, and refuses and names anything else a link reaches", async () => { + if (process.platform === "win32") return; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-egress-")); + const checkout = path.join(tmp, "checkout"); + fs.mkdirSync(path.join(checkout, ".git"), { recursive: true }); + fs.mkdirSync(path.join(checkout, "qa", "sections"), { recursive: true }); + fs.writeFileSync(path.join(checkout, "qa", "SKILL.md"), "---\nname: qa\ndescription: d\n---\n# CHECKOUT-QA\n"); + fs.writeFileSync(path.join(checkout, "qa", "sections", "flow.md"), "CHECKOUT-FLOW"); + const documents = path.join(tmp, "Documents"); + fs.mkdirSync(path.join(documents, "vault"), { recursive: true }); + fs.writeFileSync(path.join(documents, "vault", "diary.md"), "PRIVATE-DIARY"); + const skills = path.join(tmp, "skills"); + fs.mkdirSync(path.join(skills, "qa"), { recursive: true }); + fs.symlinkSync(checkout, path.join(skills, "gstack")); + fs.symlinkSync(path.join(checkout, "qa", "SKILL.md"), path.join(skills, "qa", "SKILL.md")); + fs.symlinkSync(path.join(checkout, "qa", "sections"), path.join(skills, "qa", "sections")); + fs.symlinkSync(documents, path.join(skills, "qa", "notes")); + + const calls: { body: string }[] = []; + const stdout = capture(); + const stderr = capture(); + const code = await contextBillMain([skills, "--exact", "--json"], { + stdout: stdout.stream, + stderr: stderr.stream, + apiKey: "sk-test", + fetchImpl: fakeFetch(calls), + egressHome, + }); + const err = stderr.text(); + const documentsReal = fs.realpathSync(path.join(documents, "vault")); + fs.rmSync(tmp, { recursive: true, force: true }); + expect(code).toBe(0); + expect(calls.some((c) => c.body.includes("CHECKOUT-QA"))).toBe(true); + expect(calls.some((c) => c.body.includes("CHECKOUT-FLOW"))).toBe(true); + expect(calls.some((c) => c.body.includes("PRIVATE-DIARY"))).toBe(false); + expect(err).toContain(`not sending 1 .md file(s)`); + expect(err).toContain(documentsReal); + expect(err).toContain(path.join("qa", "notes", "vault", "diary.md")); + expect(err).toContain("estimated"); + const bill = JSON.parse(stdout.text()); + const skillMd = "---\nname: qa\ndescription: d\n---\n# CHECKOUT-QA\n"; + expect(bill.skills[0].totalMdBytes).toBe(skillMd.length + "CHECKOUT-FLOW".length + "PRIVATE-DIARY".length); + }); + it("names a missing tree, without a raw ENOENT or any request", async () => { const parent = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-missing-")); const out = capture();