Reword ETHOS sentences, scope the blinding rule, name the second candidate-visible root

Review pass 1 (frozen scope, five INFORMATIONAL findings, all in-diff, none
blocking): two ETHOS.md sentences in principles 6 and 7 were too close to the
credited source and are reworded; the Credits line names both pstack sources
(principle-* leaves and poteto-mode); eval-plan Step 1c rule 1 is scoped to
what the preregistration controls and exempts the skill text an arm loads;
the observable-questions rule in plan-tune and docs/proactive.md requires the
observation to be local, read-only and free and points at the one-way-door
rule; docs/TESTING_INTERNALS.md names skill-e2e-benchmark-* (test/skill-e2e-
deploy.test.ts:293) as a second candidate-visible root and states the audit's
matching method. SKILL.md regenerated for both unpinned skills.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-26 21:34:46 +00:00
1 parent 92002dc228
commit 13a40c41b8
8 files changed
+52 -30

No files matched your search

+15 -9
View File
@@ -1511,9 +1511,11 @@ fixtures.
(its working directory, file names, the task prompt, commit messages, the
environment) says `eval`, `test`, `judge`, `experiment`, `rubric`, `score`,
`compare`, `benchmark`, `candidate` or `arena`. The project's own `test/`
directory is exempt when the task is about those tests. The hermetic home a
case builds (`test/helpers/hermetic-env.ts`) is the boundary: what is copied
in is visible, what stays in the runner is not.
directory is exempt when the task is about those tests, and so is the skill
text an arm loads as its own instructions (a gstack SKILL.md and the ETHOS.md it
points at use these words freely); the rule governs what the case builds. The
hermetic home a case builds (`test/helpers/hermetic-env.ts`) is the boundary:
what is copied in is visible, what stays in the runner is not.
- **Fixture roots are project-shaped.** A fixture the candidate works in is
named like the project it imitates, with the files that project would hold.
Runner-side fixtures (captured transcripts, judge calibration sets, census
@@ -1528,12 +1530,16 @@ word list: `test/fixtures/arm-benchmark/` and its four children, `browse-judge/`
`cso-eval/`, `eval-history/`, `judge-calibration/` and its children, two shard
names under `paid-report-census/` and the `test/` subdirectories of
`state-matrix-6792/`. All of these are runner-side: no candidate's working
directory is one of them. The one candidate-visible gap is the other rule:
`test/helpers/arm-benchmark-harness.ts` copies each fixture into
`os.tmpdir()/arm-<fixture>-<arm>-*`, so the candidate's working directory is named
after the arm rather than after a project. The e2e hermetic roots
(`skill-e2e-<case>-*`) carry no listed word. Renaming is a separate change with
its own touchfile updates; the list is here so that change can be scoped.
directory is one of them. Two candidate-visible gaps are in the temporary roots
the harnesses create: `test/helpers/arm-benchmark-harness.ts` copies each fixture
into `os.tmpdir()/arm-<fixture>-<arm>-*`, so the candidate's working directory is
named after the arm rather than after a project, and `test/skill-e2e-deploy.test.ts`
runs the /benchmark case in `skill-e2e-benchmark-*`, a root that carries a listed
word; the other `skill-e2e-<case>-*` roots do not. The audit matched whole
lower-case words in path components under `test/fixtures/`; a substring match
(`factory-scorecard/`, `ios-qa/FixtureApp/Tests/`) was not counted. Renaming is a
separate change with its own touchfile updates; the list is here so that change
can be scoped.
## Running evals as an agent: detach
+7 -5
View File
@@ -64,9 +64,11 @@ governs what a running skill may ask you unasked: a question whose answer the
skill could observe is not yours to answer. Whether a change is faster, whether
two renders differ, whether an endpoint accepts a payload, whether a test still
fails: each of those is settled by running the thing (time it, render both,
call it, run the test) and reporting the result, not by a prompt. The only
questions that come to you are product and preference calls (what the product
should do, what you want, what you are willing to spend or risk), and they
arrive with the observation already in hand so the choice is between measured
options. `/plan-tune` applies the same rule when it audits the question log: an
call it, run the test) and reporting the result, not by a prompt. That holds
for observations that are local, read-only and free; a call that changes state,
reaches a live or third-party system or spends money is still yours to approve,
as the one-way-door rule in `/plan-tune` already says. The only questions that
come to you are product and preference calls (what the product should do, what
you want, what you are willing to spend or risk), and they arrive with the
observation already in hand so the choice is between measured options. `/plan-tune` applies the same rule when it audits the question log: an
observable question is a defect in the asking skill, not a preference to tune.