mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-03 07:18:03 +01:00
changelog: shadow mode for the CEO-only decider (GST-537 PR 3)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
aec80b10a9
commit
a30cbb153c
1 file changed
+12
@@ -0,0 +1,12 @@
|
||||
**Shadow mode for the CEO-only decider: in a driver run that turns it on, every approval gate gets one hidden model prediction per question, recorded before the owner answers (GST-537).** Before the CEO-only mode is allowed to decide for the owner, we need to know how often it would pick what he picks. The only decider before this was a fixed rule that always keeps the plan's direction, which agrees with the reviewers on every past item, so it measured nothing. Now a model (Opus 5.5 by default, about $0.07 per gate on your own API key) predicts each answer, and taste-replay round 2 in gstack-evals compares the predictions with the owner's real answers. Nothing routes to the owner differently.
|
||||
|
||||
### Added
|
||||
- **`gstack-config set autoplan_shadow on`** (off by default; `GSTACK_AUTOPLAN_SHADOW=1|0` overrides it for one run) turns shadow mode on at a run's first `next`, stored in `run.json` so a resumed run cannot flip it. `autoplan_shadow_model` picks the model (default `claude-opus-5-5`). It stays off with `SHADOW: off (<reason>)` when there is no API key for the model, the model has no price, the run is `--no-outside` or CEO-only.
|
||||
- **Every driver command that prints a gate** (`next`, `submit`, `close`, `resume`, `revise`, `run`, `check`) runs the shadow first and prints `SHADOW: recorded model=<m> usd=<x>` or `SHADOW: failed <code>`, then the gate, then the checkpoint. The prediction goes to the private `<run>/shadow/ceo-only.r<rev>.jsonl`: action, pick, confidence, cites and reason per question, beside the keep-direction rule's pick, the frozen question, the prompt hash and the cost, plus a 10% audit mark for a later live switch. The run lock is free during the call; another writer pays nothing while a call is in flight, `answer` waits for it, and after a crash `answer` records that gate first, marked retrospective. A shadow failure never blocks a gate or an answer.
|
||||
- **Shadow calls count against `--spend-cap`** under their own ledger ids labeled `shadow`, show as `shadow=<usd>` on `GATE_SPEND`, and never change the per-voice price on `GATE_REC_COST`. A call that was sent and then lost to a crash settles `unknown`, never released as undispatched.
|
||||
- **`gstack-autoplan shadow --out <dir> | --runs <file> [--json]`**, a read-only export for gstack-evals: one row per item of each answered gate, its shadow status (recorded, malformed, failed, missing, retrospective, off) and the owner's answer with how he gave it. An unanswered or interrogated gate refuses `SHADOW_BEFORE_ANSWER`.
|
||||
- **Answered decision rows record how the owner answered**, parsed from his own words: `owner_mode` (`rec`, `rec_except`, `tokens`, `as_is`, `unparsed`), `owner_named`, the `rec_hash` answered against and `answered_role`.
|
||||
- `gstack-artifact validate <run.json>` checks each shadow record against its journaled sha256.
|
||||
|
||||
### Changed
|
||||
- The shared model-call transport can write a caller's own egress receipt, returns the provider's stop reason (a reply cut at the output limit is retried at 8,000 tokens, not treated as malformed), and sends the output limit to OpenAI models as `max_completion_tokens` when a caller sets one. Existing callers send the same bytes as before.
|
||||
Reference in new issue
Block a user