v1.87.0.0 feat: add verified CSO audits and replayable repair bundles (#2852)

* feat(cso): add verified audits and replayable repair bundles

* fix(cso): harden qualification and setup boundaries

* fix(cso): assemble security canaries at runtime

* fix(cso): bound release proof and maintenance work

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): require complete evaluation reports

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): replay expired snapshots from supplied source

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* test(cso): synchronize DNS cancellation assertion

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore(ship): exempt repository owner from liveness proof

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* test(cso): make recheck retention overlap deterministic

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: bump version and changelog (v1.85.0.0)

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): pass native release gates

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: move release to v1.86.0.0

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): resolve rechecks by finding

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: move release to v1.87.0.0

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): pass macOS and Windows release gates

Normalize BSD wc output, compare Windows paths by filesystem identity, preserve portable snapshot race coverage, and narrow POSIX-only Windows fixtures.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): harden native verification gates

* fix(cso): refine Windows native diagnostics

* test(cso): isolate Windows Git startup failure

* test(cso): stabilize Windows native diagnostics

* fix(cso): support hardened Git on Windows

* fix(cso): close final verification gaps

* test(cso): bound cold Docker fixture setup

* fix(cso): restore cross-platform free-suite gates

---------

Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
2026-09-14 15:14:58 -07:00
committed by GitHub
co-authored by OpenAI Codex
parent 9f81911136
commit 4a3c6a8a3c
160 changed files with 24697 additions and 2288 deletions
+6 -20
View File
@@ -2,9 +2,9 @@
* AUQ behavioral matrix — drive each AUQ-heavy skill to its first
* AskUserQuestion and grade it to plan-ceo's bar (periodic, paid, SDK capture).
*
* Layer 0 (auq-format-always-loaded.test.ts) deterministically guarantees every
* skill SHIPS the format spec in its always-loaded skeleton. This test proves
* each skill's model OBEYS it: that the first real AUQ each skill fires is a
* Layer 0 (auq-format-always-loaded.test.ts) deterministically guarantees each
* listed skill SHIPS the format spec in its always-loaded skeleton. This test
* proves each skill's model OBEYS it: that the first real AUQ it fires is a
* compliant decision brief (all 7 format elements) with a substantive
* recommendation (>= 4). One parametrized case per skill so a single weak skill
* is an isolated failure, not a blocker for the rest.
@@ -20,7 +20,9 @@
* choices) are intentionally OUT of this matrix; Layer 0 covers their format
* spec, and a fixture can't fairly trigger their AUQ.
*
* Run a subset in the foreground with AUQ_MATRIX_ONLY="plan-eng-review,cso".
* CSO is intentionally absent: its private startup omits the shared AUQ block,
* and its dedicated E2E grades evidence, reporting, and proof behavior.
* Run a subset in the foreground with AUQ_MATRIX_ONLY="plan-eng-review,spec".
*/
import { test } from 'bun:test';
import { CAPTURE_MS } from './helpers/eval-budgets';
@@ -53,17 +55,6 @@ pricing page, a Postgres entitlements table, and a Redis cache — no tests
mentioned, no rollout plan, no auth check on the upgrade endpoint.
`;
const VULN_CODE = `export function login(req, res) {
// builds SQL by string concat; sets a session cookie with no flags
const user = db.query("SELECT * FROM users WHERE name = '" + req.body.name + "'");
if (user && user.password === req.body.password) {
res.cookie('session', user.id); // no HttpOnly, Secure, SameSite, or expiry
return res.json({ ok: true });
}
return res.status(401).json({ ok: false });
}
`;
interface MatrixSkill {
skill: string;
fixtures: Record<string, string>;
@@ -93,11 +84,6 @@ const MATRIX: MatrixSkill[] = [
fixtures: {},
scenario: 'The founder says: "I am building an AI tool that auto-writes unit tests for any repo. I think it is a great idea but I have zero users. Should I build it, and how do I get my first users?" Run the office-hours diagnostic until the first AskUserQuestion.',
},
{
skill: 'cso',
fixtures: { 'server/auth.js': VULN_CODE },
scenario: 'Audit the code in this repo (server/auth.js) for security issues. Walk the audit until the first AskUserQuestion (scope/stack confirmation or first finding).',
},
{
skill: 'spec',
fixtures: {},