mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-05 00:03:15 +01:00
cso: ignore the compiled helpers' leading #! line in preparationSha256, on both sides of the release contract
S1 (#3312) rewrote only line 1 of lib/cso/verifier.ts and
lib/cso/preparation-container.ts. That shebang is an unused string in the
compiled helper binaries, but it moved preparationSha256, so qualify run
39535948607 refused the run-5 statements staged at 133cb36 with
STALE_RUNTIME_QUALIFICATION. Garry chose this narrow rule (option 2).
The release contract re-measures a recorded preparationSha256 that differs
from the checkout's at the staged commit (git history), accepts a record
that matches it under the current or the legacy rule, and compares and
records the current-rule value. Both release workflows fetch full history.
This commit is contained in:
1 parent
6b2b5f350d
commit
478b4ebdfb
7 files changed
+142
-10
No files matched your search
@@ -39,6 +39,7 @@ jobs:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
||||
with:
|
||||
persist-credentials: true
|
||||
fetch-depth: 0
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
|
||||
with:
|
||||
bun-version: 1.4.2
|
||||
|
||||
@@ -65,6 +65,7 @@ jobs:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
|
||||
with:
|
||||
bun-version: 1.4.2
|
||||
|
||||
@@ -1,5 +1,17 @@
|
||||
# Changelog
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
**A qualified /cso runtime set can be released again after the S1 shebang change.** The release contract fingerprints every source byte compiled into a runtime image (`preparationSha256`). S1 (#3312) changed only line 1 of the two compiled helpers, `lib/cso/verifier.ts` and `lib/cso/preparation-container.ts`, from `#!/usr/bin/env bun` to the isolated shebang. That line is embedded in the compiled binary as an unused string and never runs in the image, but it moved the fingerprint. Qualify run 39535948607 therefore refused the run-5 statements, staged at 133cb36, with `STALE_RUNTIME_QUALIFICATION: preparationSha256`. Garry chose this narrow rule change over a re-stage and a fresh paid qualification (2026-10-21, option 2).
|
||||
|
||||
### Changed
|
||||
- **`scripts/cso-requalification.ts`**: `preparationSha256` hashes a compiled helper entrypoint's leading `#!` line as absent. Every other byte of the helpers, and every byte of every other image source including its shebang lines, still counts. 133cb36 and main both measure `c721b585…`.
|
||||
- **`scripts/cso-runtime-promotion.ts`** (`releaseCandidate`): when the staging record's `preparationSha256` differs from the checkout's, the release contract re-measures it at the staged commit from git history. The record must match that commit under the current rule or the earlier one, which also hashed the shebangs; otherwise it fails with `REQUALIFICATION_TRIGGERS_MISMATCH`. The current-rule value is compared and recorded, so both sides use the same rule.
|
||||
- **`cso-runtime-qualification.yml`, `cso-runtime-promote.yml`**: check out full history so the staged commit is readable.
|
||||
|
||||
### Gate integrity
|
||||
- Gate edit, a narrowing with Garry's approval: the release contract's `preparationSha256` comparison ignores the leading `#!` line of the two compiled helpers on both sides. Nothing else gets a pass. `test/cso-runtime-promotion.test.ts` runs the contract on the real 133cb36 record (`b069f7ef…`) against a main tree. These still fail with `STALE_RUNTIME_QUALIFICATION`: a line-2 edit to either helper, an image Dockerfile edit, and a shebang edit to the image `entrypoint`. A record that matches neither rule at the staged commit fails with `REQUALIFICATION_TRIGGERS_MISMATCH`.
|
||||
|
||||
## [1.91.174.0] - 2026-10-21
|
||||
|
||||
**Two paid eval flakes from the 2026-10-20 trains are repaired at their cause, with no threshold, trial count or case kind changed.** ship-local-hook-preservation failed in train gate 39440559183 even though the agent told the user "you'll need to set it up by hand": the manual-integration detector knew "set up" and "wire in" only with the particle right after the verb. ship-docsync-current lost a focused trial to an unquoted `grep … --include=*`, which the shared read-only grammar refuses on purpose because the shell expands the glob first; the grammar's declared interface said "unquoted glob matches included" and never said that an option's glob has to be quoted.
|
||||
|
||||
@@ -280,12 +280,19 @@ policy (`ISOLATION_POLICY_HASH`, which covers the `cso-isolation-v1` policy
|
||||
version and every limit), `preparationSha256` (a digest of every source byte
|
||||
compiled or copied into a runtime image: the import closure of the image's
|
||||
verifier and preparation helpers plus the image build context), and the
|
||||
`build-inputs.json` revision. The staging run's `reviewed-inputs` job records
|
||||
`build-inputs.json` revision. Each helper entrypoint's leading `#!` line is
|
||||
hashed as absent: it is compiled into the binary as an unused string and never
|
||||
runs inside the image. Every other byte of the helpers counts, and so does
|
||||
every byte of every other file, shebang lines included. The staging run's `reviewed-inputs` job records
|
||||
these values for its source commit as the `cso-requalification-triggers`
|
||||
artifact. The ingress downloads that artifact from the verified staging run
|
||||
into the evidence, and the promotion generator writes it into
|
||||
`promotion.requalification`. Generation refuses triggers that differ from the
|
||||
promoting checkout, so a trigger that changed between staging and promotion
|
||||
`promotion.requalification`. When a recorded `preparationSha256` differs from
|
||||
the promoting checkout's, generation re-measures it at the staged commit from
|
||||
the checkout's git history. The record must match that commit under either the
|
||||
current rule or the earlier rule that also hashed the helpers' `#!` lines, and
|
||||
the current-rule value is what gets compared and recorded. Generation refuses
|
||||
triggers that differ from the promoting checkout, so a trigger that changed between staging and promotion
|
||||
requires a new staging run. The free suite's
|
||||
[`cso-requalification.test.ts`](../test/cso-requalification.test.ts) compares
|
||||
the committed catalog's recorded triggers with the current source and fails
|
||||
|
||||
@@ -5,7 +5,9 @@
|
||||
* differ from the current source is older than its inputs and must be
|
||||
* requalified (or its runtimes withdrawn) before it ships.
|
||||
*/
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { canonical, sha256 } from '../lib/cso/contracts';
|
||||
import { ISOLATION_POLICY_HASH } from '../lib/cso/docker';
|
||||
@@ -31,6 +33,37 @@ export async function imageSourceFiles(root = ROOT): Promise<string[]> {
|
||||
return [...new Set([...closure, ...context])].sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Digest of every image source byte. A compiled helper's leading `#!` line never
|
||||
* runs inside the image, so it is hashed as absent; every other byte counts.
|
||||
* `legacy` hashes the shebang too, the rule staging runs recorded before this normalization (through run 39325583697).
|
||||
*/
|
||||
function preparationSha256(root: string, files: string[], legacy = false): string {
|
||||
const helpers = new Set<string>(IMAGE_HELPER_ENTRYPOINTS);
|
||||
return sha256(canonical(files.map(file => {
|
||||
const bytes = fs.readFileSync(path.join(root, file));
|
||||
if (legacy || !helpers.has(file) || bytes[0] !== 0x23 || bytes[1] !== 0x21) return [file, sha256(bytes)];
|
||||
const newline = bytes.indexOf(0x0a);
|
||||
return [file, sha256(newline === -1 ? Buffer.alloc(0) : bytes.subarray(newline + 1))];
|
||||
})));
|
||||
}
|
||||
|
||||
/** preparationSha256 of a committed tree under the current rule and the legacy rule. */
|
||||
export async function committedPreparation(commit: string, root = ROOT): Promise<{ current: string; legacy: string }> {
|
||||
if (!/^[a-f0-9]{40}$/.test(commit)) throw new Error('INVALID_STAGED_COMMIT');
|
||||
const archive = spawnSync('git', ['-C', root, 'archive', '--format=tar', commit], { timeout: 60_000, maxBuffer: 512 * 1024 * 1024 });
|
||||
if (archive.status !== 0) throw new Error(`STAGED_COMMIT_UNAVAILABLE: ${commit} is not in this checkout; fetch full history`);
|
||||
const tree = fs.mkdtempSync(path.join(os.tmpdir(), 'cso-staged-source-'));
|
||||
try {
|
||||
const extract = spawnSync('tar', ['-x', '-C', tree], { input: archive.stdout, timeout: 60_000 });
|
||||
if (extract.status !== 0) throw new Error('STAGED_COMMIT_UNAVAILABLE: archive extraction failed');
|
||||
const files = await imageSourceFiles(tree);
|
||||
return { current: preparationSha256(tree, files), legacy: preparationSha256(tree, files, true) };
|
||||
} finally {
|
||||
fs.rmSync(tree, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function requalificationTriggers(root = ROOT): Promise<RequalificationTriggers> {
|
||||
const files = await imageSourceFiles(root);
|
||||
const buildInputs = JSON.parse(fs.readFileSync(path.join(root, IMAGE_CONTEXT, 'build-inputs.json'), 'utf8'));
|
||||
@@ -38,7 +71,7 @@ export async function requalificationTriggers(root = ROOT): Promise<Requalificat
|
||||
return {
|
||||
helperAbi: CSO_HELPER_ABI,
|
||||
isolationPolicyHash: ISOLATION_POLICY_HASH,
|
||||
preparationSha256: sha256(canonical(files.map(file => [file, sha256(fs.readFileSync(path.join(root, file)))]))),
|
||||
preparationSha256: preparationSha256(root, files),
|
||||
buildInputsRevision: buildInputs.revision,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -18,7 +18,7 @@ import {
|
||||
} from '../lib/cso/runtime-catalog';
|
||||
import { ISOLATION_POLICY_HASH } from '../lib/cso/docker';
|
||||
import { imageBuildMatrix, type ImageBuildRow } from './cso-image-matrix';
|
||||
import { requalificationTriggers, staleRequalificationTriggers } from './cso-requalification';
|
||||
import { committedPreparation, requalificationTriggers, staleRequalificationTriggers } from './cso-requalification';
|
||||
|
||||
const DIGEST = /^sha256:[a-f0-9]{64}$/;
|
||||
const WORKFLOW = /^https:\/\/github\.com\/[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+\/actions\/runs\/[0-9]+$/;
|
||||
@@ -273,6 +273,30 @@ export function catalogPromotionCandidate(
|
||||
return candidate;
|
||||
}
|
||||
|
||||
/**
|
||||
* The release contract: the promotion candidate, refused when any trigger moved since the staged commit.
|
||||
* A recorded preparationSha256 that differs from the current one is re-measured at the staged commit in
|
||||
* `source`, so a staging record made under the legacy rule compares under the current rule.
|
||||
*/
|
||||
export async function releaseCandidate(
|
||||
evidenceValues: unknown[],
|
||||
recorded: RequalificationTriggers,
|
||||
current: RequalificationTriggers,
|
||||
source?: string,
|
||||
): Promise<RuntimeCatalog> {
|
||||
let candidate = catalogPromotionCandidate(committedCatalog, buildInputs, evidenceValues, recorded);
|
||||
if (recorded.preparationSha256 !== current.preparationSha256) {
|
||||
const staged = await committedPreparation(candidate.promotion!.sourceCommit, source);
|
||||
if (recorded.preparationSha256 !== staged.current && recorded.preparationSha256 !== staged.legacy) {
|
||||
throw new Error('REQUALIFICATION_TRIGGERS_MISMATCH: preparationSha256 was not measured at the staged commit');
|
||||
}
|
||||
candidate = catalogPromotionCandidate(committedCatalog, buildInputs, evidenceValues, { ...recorded, preparationSha256: staged.current });
|
||||
}
|
||||
const stale = staleRequalificationTriggers(candidate, current);
|
||||
if (stale.length) throw new Error(`STALE_RUNTIME_QUALIFICATION: ${stale.join(', ')} changed since the staged commit; re-stage and requalify`);
|
||||
return candidate;
|
||||
}
|
||||
|
||||
/** A promotion may only replace the exact catalog revision it was generated from. */
|
||||
export function validateRuntimeCatalogTransition(currentValue: unknown, proposedValue: unknown): void {
|
||||
validateRuntimeCatalog(currentValue);
|
||||
@@ -356,9 +380,7 @@ if (import.meta.main) {
|
||||
}
|
||||
const triggers = collectEvidence(args[1], 'requalification.json');
|
||||
if (triggers.length !== 1) throw new Error('MISSING_REQUALIFICATION_TRIGGERS');
|
||||
const candidate = catalogPromotionCandidate(committedCatalog, buildInputs, collectEvidence(args[1]), triggers[0] as RequalificationTriggers);
|
||||
const stale = staleRequalificationTriggers(candidate, await requalificationTriggers());
|
||||
if (stale.length) throw new Error(`STALE_RUNTIME_QUALIFICATION: ${stale.join(', ')} changed since the staged commit; re-stage and requalify`);
|
||||
const candidate = await releaseCandidate(collectEvidence(args[1]), triggers[0] as RequalificationTriggers, await requalificationTriggers());
|
||||
const output = path.resolve(args[3]);
|
||||
fs.writeFileSync(output, JSON.stringify(candidate, null, 2) + '\n', { flag: 'wx', mode: 0o600 });
|
||||
process.stdout.write(`${output}\n`);
|
||||
|
||||
@@ -8,9 +8,9 @@ import committedCatalog from '../lib/cso/runtime-catalog.json';
|
||||
import buildInputs from '../lib/cso/images/build-inputs.json';
|
||||
import { isEvaluationRuntimeCatalog, validateRuntimeCatalog } from '../lib/cso/runtime-catalog';
|
||||
import { imageBuildMatrix, type ImageBuildRow } from '../scripts/cso-image-matrix';
|
||||
import { catalogPromotionCandidate, evaluationCandidateCatalog, validateEvaluationCandidate, validateRuntimeCatalogTransition, type RuntimeQualificationStatement } from '../scripts/cso-runtime-promotion';
|
||||
import { catalogPromotionCandidate, evaluationCandidateCatalog, releaseCandidate, validateEvaluationCandidate, validateRuntimeCatalogTransition, type RuntimeQualificationStatement } from '../scripts/cso-runtime-promotion';
|
||||
import { assertVersionOutput, probesForBuildRow } from '../scripts/cso-verify-runtime-base';
|
||||
import { requalificationTriggers } from '../scripts/cso-requalification';
|
||||
import { committedPreparation, requalificationTriggers } from '../scripts/cso-requalification';
|
||||
|
||||
const TRIGGERS = await requalificationTriggers();
|
||||
const digest = (value: string) => `sha256:${createHash('sha256').update(value).digest('hex')}`;
|
||||
@@ -220,3 +220,59 @@ describe('CSO evaluation-only runtime catalogs', () => {
|
||||
expect(isEvaluationRuntimeCatalog(committedCatalog as any)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
/** Staged by run 39325583697; its cso-requalification-triggers artifact recorded STAGED_LEGACY. */
|
||||
const STAGED = '133cb360842d0a7cf88fc1d10dcb624f2b7b841f';
|
||||
const STAGED_LEGACY = 'b069f7ef9b19dba9a770cac02a5b6b4b5d60fa68ac058321864b614de14d59f6';
|
||||
/** main after S1 (#3312) rewrote both helpers' shebang lines. */
|
||||
const AFTER_S1 = '6b2b5f350de0c5c044aa2ecb64e426cbeb45e1a6';
|
||||
|
||||
function committedLib(commit: string): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'cso-release-contract-'));
|
||||
const archive = spawnSync('git', ['archive', '--format=tar', commit, 'lib'], { cwd: join(import.meta.dir, '..'), timeout: 60_000, maxBuffer: 256 * 1024 * 1024 });
|
||||
expect(archive.status, `${commit} must be in this checkout's history`).toBe(0);
|
||||
expect(spawnSync('tar', ['-x', '-C', dir], { input: archive.stdout, timeout: 60_000 }).status).toBe(0);
|
||||
return dir;
|
||||
}
|
||||
|
||||
describe.skipIf(process.platform === 'win32')('CSO release contract preparation fingerprint', () => {
|
||||
const staged = () => statements().map(statement => ({ ...statement, sourceCommit: STAGED }));
|
||||
|
||||
test('the staged run-5 commit and main after S1 measure the same fingerprint, and only the legacy rule tells them apart', async () => {
|
||||
const [before, after] = await Promise.all([committedPreparation(STAGED), committedPreparation(AFTER_S1)]);
|
||||
expect(before.legacy).toBe(STAGED_LEGACY);
|
||||
expect(after.legacy).not.toBe(before.legacy);
|
||||
expect(after.current).toBe(before.current);
|
||||
}, 120_000);
|
||||
|
||||
test('the release contract re-measures a legacy staging record and ignores only a compiled helper\'s leading #! line', async () => {
|
||||
const recordedAt = (current: Awaited<ReturnType<typeof requalificationTriggers>>, preparationSha256 = STAGED_LEGACY) => ({ ...current, preparationSha256 });
|
||||
const variant = async (edit?: (root: string) => void) => {
|
||||
const root = committedLib(AFTER_S1);
|
||||
try { edit?.(root); return await requalificationTriggers(root); } finally { rmSync(root, { recursive: true, force: true }); }
|
||||
};
|
||||
const rewrite = (root: string, file: string, change: (lines: string[]) => void) => {
|
||||
const lines = readFileSync(join(root, file), 'utf8').split('\n'); change(lines); writeFileSync(join(root, file), lines.join('\n'));
|
||||
};
|
||||
|
||||
const main = await variant();
|
||||
const released = await releaseCandidate(staged(), recordedAt(main), main);
|
||||
expect(released.promotion!.requalification.preparationSha256).toBe(main.preparationSha256);
|
||||
expect((await committedPreparation(STAGED)).current).toBe(main.preparationSha256);
|
||||
|
||||
for (const helper of ['lib/cso/verifier.ts', 'lib/cso/preparation-container.ts']) {
|
||||
const reshebanged = await variant(root => rewrite(root, helper, lines => { lines[0] = '#!/usr/bin/env node'; }));
|
||||
expect(reshebanged.preparationSha256).toBe(main.preparationSha256);
|
||||
const drifted = await variant(root => rewrite(root, helper, lines => { lines.splice(1, 0, '// drift'); }));
|
||||
await expect(releaseCandidate(staged(), recordedAt(drifted), drifted)).rejects.toThrow('STALE_RUNTIME_QUALIFICATION: preparationSha256');
|
||||
}
|
||||
const dockerfile = await variant(root => writeFileSync(join(root, 'lib/cso/images/rails.Dockerfile'), '\n', { flag: 'a' }));
|
||||
await expect(releaseCandidate(staged(), recordedAt(dockerfile), dockerfile)).rejects.toThrow('STALE_RUNTIME_QUALIFICATION: preparationSha256');
|
||||
const entrypoint = await variant(root => rewrite(root, 'lib/cso/images/entrypoint', lines => { lines[0] = '#!/bin/bash'; }));
|
||||
await expect(releaseCandidate(staged(), recordedAt(entrypoint), entrypoint)).rejects.toThrow('STALE_RUNTIME_QUALIFICATION: preparationSha256');
|
||||
|
||||
await expect(releaseCandidate(staged(), recordedAt(main, 'd'.repeat(64)), main)).rejects.toThrow('REQUALIFICATION_TRIGGERS_MISMATCH');
|
||||
const fresh = await releaseCandidate(staged(), recordedAt(main, main.preparationSha256), main, join(tmpdir(), 'no-such-checkout'));
|
||||
expect(fresh.promotion!.requalification).toEqual(main);
|
||||
}, 180_000);
|
||||
});
|
||||
Reference in new issue
Block a user