v1.87.0.0 feat: add verified CSO audits and replayable repair bundles (#2852)

* feat(cso): add verified audits and replayable repair bundles

* fix(cso): harden qualification and setup boundaries

* fix(cso): assemble security canaries at runtime

* fix(cso): bound release proof and maintenance work

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): require complete evaluation reports

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): replay expired snapshots from supplied source

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* test(cso): synchronize DNS cancellation assertion

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore(ship): exempt repository owner from liveness proof

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* test(cso): make recheck retention overlap deterministic

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: bump version and changelog (v1.85.0.0)

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): pass native release gates

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: move release to v1.86.0.0

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): resolve rechecks by finding

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* chore: move release to v1.87.0.0

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): pass macOS and Windows release gates

Normalize BSD wc output, compare Windows paths by filesystem identity, preserve portable snapshot race coverage, and narrow POSIX-only Windows fixtures.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix(cso): harden native verification gates

* fix(cso): refine Windows native diagnostics

* test(cso): isolate Windows Git startup failure

* test(cso): stabilize Windows native diagnostics

* fix(cso): support hardened Git on Windows

* fix(cso): close final verification gaps

* test(cso): bound cold Docker fixture setup

* fix(cso): restore cross-platform free-suite gates

---------

Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
2026-09-14 15:14:58 -07:00
committed by GitHub
co-authored by OpenAI Codex
parent 9f81911136
commit 4a3c6a8a3c
160 changed files with 24697 additions and 2288 deletions
+43 -43
View File
@@ -1,65 +1,65 @@
{
"_comment": "Context-budget ratchet ceilings (~tokens). Regenerate: bun test/helpers/capture-context-budget.ts. Headroom: alwaysOnTotal x1.05, eagerPerInvocation x1.1. Graded by test/context-budget-ratchet.test.ts via lib/context-bill.ts checkBudget.",
"alwaysOnTotal": 6372,
"alwaysOnTotal": 6397,
"eagerPerInvocation": {
"autoplan": 17697,
"benchmark": 7403,
"autoplan": 18022,
"benchmark": 7657,
"benchmark-models": 3829,
"browse": 8003,
"browser-skills/hackernews-frontpage": 371,
"canary": 13522,
"canary": 13914,
"careful": 919,
"codex": 15490,
"context-restore": 9618,
"context-save": 10234,
"cso": 16110,
"design-consultation": 18392,
"design-html": 13767,
"design-review": 34483,
"design-shotgun": 13828,
"devex-review": 20302,
"codex": 15479,
"context-restore": 9607,
"context-save": 10224,
"cso": 4700,
"design-consultation": 18595,
"design-html": 14429,
"design-review": 34471,
"design-shotgun": 13856,
"devex-review": 20292,
"diagram": 4279,
"document-generate": 12362,
"document-release": 10602,
"document-generate": 12352,
"document-release": 10683,
"freeze": 990,
"gstack": 3976,
"gstack-upgrade": 4201,
"gstack-upgrade": 4586,
"guard": 889,
"health": 10816,
"investigate": 12286,
"ios-clean": 8721,
"ios-design-review": 8902,
"ios-fix": 8674,
"ios-qa": 11414,
"ios-sync": 8845,
"land-and-deploy": 18971,
"landing-report": 9527,
"learn": 9197,
"health": 10805,
"investigate": 12276,
"ios-clean": 8710,
"ios-design-review": 8891,
"ios-fix": 8664,
"ios-qa": 11403,
"ios-sync": 8834,
"land-and-deploy": 19043,
"landing-report": 9517,
"learn": 9187,
"make-pdf": 5314,
"office-hours": 22420,
"office-hours": 22374,
"open-gstack-browser": 4510,
"openclaw/skills/gstack-openclaw-ceo-review": 2764,
"openclaw/skills/gstack-openclaw-investigate": 1429,
"openclaw/skills/gstack-openclaw-office-hours": 4433,
"openclaw/skills/gstack-openclaw-retro": 2542,
"pair-agent": 11622,
"plan-ceo-review": 20550,
"plan-design-review": 20315,
"plan-devex-review": 17799,
"plan-eng-review": 14750,
"plan-tune": 14771,
"qa": 15857,
"qa-only": 16910,
"retro": 18974,
"review": 16021,
"pair-agent": 11612,
"plan-ceo-review": 20824,
"plan-design-review": 20648,
"plan-devex-review": 17884,
"plan-eng-review": 14930,
"plan-tune": 14761,
"qa": 15847,
"qa-only": 17218,
"retro": 19122,
"review": 16010,
"scrape": 6904,
"setup-browser-cookies": 3194,
"setup-deploy": 11272,
"setup-gbrain": 15565,
"ship": 20518,
"skillify": 12206,
"spec": 15011,
"sync-gbrain": 13985,
"setup-deploy": 11557,
"setup-gbrain": 15554,
"ship": 20347,
"skillify": 12196,
"spec": 14993,
"sync-gbrain": 13975,
"unfreeze": 393
}
}
+254
View File
@@ -0,0 +1,254 @@
# CSO vulnerable/fixed evaluation corpus
This corpus contains **40 immutable source pairs**: ten each for Node, Bun,
Python, and Rails. `manifest.json` pins both source hashes, expected root cause,
location, severity, and evaluation eligibility. `materialize.ts` deterministically
creates either member of each pair. Changing source requires an explicit corpus
manifest update; the loader rejects silent drift.
The ten families are SQL injection, command injection, path traversal, SSRF,
object authorization, tenant isolation, HTML injection, open redirects, mass
assignment, and resource exhaustion. Every app exposes `/action` and `/health` on
loopback port 8000. The SSRF fixtures additionally create a disposable loopback
status service on port 8001. Exhaustion assertions request 250 small records to
prove missing admission; they do not attempt to exhaust the evaluator.
Node uses standard modules, including `node:sqlite`; Bun uses `bun:sqlite`; Python
uses its standard library. Rails fixtures contain an actual Rails application
and controller, with native SQLite and Puma dependencies. Their common lock was
generated by Ruby 3.2.8 / Bundler 2.6.7 from public registry metadata only. The
exact command, lock hash, and 68 public archive hashes are recorded in
`rails-lock-provenance.json`. No fixture application was run to generate this
lock. The recorded archive hashes must be checked by runtime qualification; a
successful resolver run does not establish cold-start support.
Authentication is an explicit fixture precondition: the test adapter establishes
member-1 in tenant-a. The assessment scope is the selected endpoint's behavior.
This prevents an intentionally constant test identity from being mistaken for a
production authentication design. Application README files state legitimate
behavior; they never include attack payloads or expected finding labels.
## Preparing matched evaluations
From the repository root, export the v2 generated skill tree and build a
canonical portable payload for each version before pinning an exact model ID:
```sh
mkdir -p .context/cso-v2-source
git archive origin/main cso/SKILL.md cso/sections | \
tar -x -C .context/cso-v2-source --strip-components=1
bun scripts/cso-eval.ts payload --version v2 \
--skill-dir .context/cso-v2-source --output .context/cso-v2.payload.md
bun scripts/cso-eval.ts payload --version v3 \
--skill-dir cso --output .context/cso-v3.payload.md
bun scripts/cso-eval.ts matrix \
--model EXACT_MODEL_ID --host codex \
--v2-payload .context/cso-v2.payload.md \
--v3-payload .context/cso-v3.payload.md \
--output .context/cso-eval-matrix.json
bun scripts/cso-eval.ts materialize node-sql-injection vulnerable .context/cso-eval-app
```
The full matrix contains 960 cells: 40 pairs × two source variants × two modes ×
two skill versions × three repetitions. Each matched pair uses identical source,
model, host, and per-mode wall-clock budget. Daily runs have 600 seconds;
comprehensive runs have 1800 seconds. Each portable payload embeds the exact
generated `SKILL.md`, `sections/manifest.json`, and every generated section
listed by that manifest. The payload builder rejects missing or unlisted
generated sections and a mismatched major version. The matrix hashes the whole
payload, so a change to any section byte changes the pinned skill identity. The
commands above make no model calls and execute no fixture application.
Prepare consumable one-cell jobs and compile the generic producer runner:
```sh
bun build --compile \
--no-compile-autoload-dotenv \
--no-compile-autoload-bunfig \
--no-compile-autoload-tsconfig \
--no-compile-autoload-package-json \
scripts/cso-eval-producer.ts --outfile .context/cso-eval-producer
bun scripts/cso-eval.ts prepare .context/cso-eval-matrix.json \
--v2-payload .context/cso-v2.payload.md \
--v3-payload .context/cso-v3.payload.md \
--output .context/cso-eval-jobs
```
### Trusted five-artifact producer unit
The `.context/cso-eval-producer` command above creates a preparation artifact;
ordinary `bun run build` does not distribute the private producer. For a paid
run, start from one clean checkout and build a five-artifact staging unit in one
session:
```sh
stage="$(mktemp -d)"
bun run build:cso
bun build --compile \
--no-compile-autoload-dotenv \
--no-compile-autoload-bunfig \
--no-compile-autoload-tsconfig \
--no-compile-autoload-package-json \
scripts/cso-eval-producer.ts --outfile "$stage/cso-eval-producer"
install -m 0555 bin/gstack-cso-launcher bin/gstack-cso-core \
bin/gstack-cso-watchdog "$stage/"
install -m 0444 bin/.gstack-cso-generation "$stage/.gstack-cso-generation"
(cd "$stage" && (sha256sum cso-eval-producer gstack-cso-launcher \
gstack-cso-core gstack-cso-watchdog .gstack-cso-generation 2>/dev/null || \
shasum -a 256 cso-eval-producer gstack-cso-launcher gstack-cso-core \
gstack-cso-watchdog .gstack-cso-generation))
sudo install -d -o root -g root -m 0755 /opt/gstack-cso-producer
sudo install -o root -g root -m 0555 "$stage/cso-eval-producer" \
"$stage/gstack-cso-launcher" "$stage/gstack-cso-core" \
"$stage/gstack-cso-watchdog" /opt/gstack-cso-producer/
sudo install -o root -g root -m 0444 "$stage/.gstack-cso-generation" \
/opt/gstack-cso-producer/.gstack-cso-generation
```
Run `/opt/gstack-cso-producer/cso-eval-producer` as an unprivileged account.
Its startup check rejects missing, linked, writable, or non-root-owned members
and a writable or linked ancestor directory. Every receipt records the SHA-256
and byte length of all five artifacts in one installation identity; collection
rejects a batch if those identities differ. Preserve the printed hashes with
the release evidence and move or replace the complete unit together.
`prepare` creates a trusted `schedule.json` and 960 independent directories.
It initializes each generated application as a one-commit Git repository. Each
job has exactly one source variant and one versioned, complete instruction
payload. Root-skill pointers resolve only to sections embedded in that payload;
the producer must never load CSO instructions from `~/.claude`, another host
install, this source checkout, or the other version's payload. The generic runner
is compiled separately and contains no corpus generator, fixed alternative,
case oracle, or schedule. These preparation commands still make zero model calls.
Run each job on a fresh producer filesystem that contains only that job, the
compiled runner, the authenticated provider CLI, and the reviewed executable
gstack helper/runtime files. Do not install or copy any CSO `SKILL.md`, carved
section, generated skill tree, schedule, this repository checkout,
sibling jobs, fixed alternatives, or evaluator code to that filesystem. The
producer runner reads its opaque control file into memory and deletes it before
the agent process starts, so the case identifier and vulnerable/fixed label are
not agent inputs. Copy the selected hash-named directory as the literal path
`/producer/job`; `/producer` must contain only `job`. Put the runner in a system
tool directory and use a separate receipt mount. The runner rejects the full
prepared batch layout, extra files beside `job`, a reused job with prior state,
and receipt paths under `/producer`. The dedicated VM/container must ensure
other host paths do not contain evaluator inputs; a local directory layout does
not constrain a tool-using agent's absolute filesystem access.
The receipt must be written outside the isolated producer root:
```sh
mkdir -m 700 /receipts
CSO_EVAL_PAID=1 /opt/gstack-cso-producer/cso-eval-producer run \
/producer/job/producer-input.json /receipts/CELL_ID.json --execute-paid
```
Both the environment variable and flag are required because this is the only
command in the workflow that makes a paid model call. It reuses the repository's
Claude, Codex, and Gemini provider adapters and starts a fresh CLI process for
each cell. The requested model, normalized effective model, identity source,
timeout, output, token counts, pricing-table cost estimate, tool-call count, and
latency are bound into a hashed receipt. `provider_reported` means the CLI
resolved a different concrete ID; `requested_pin` means the adapter returned
the exact requested ID or had to fall back to that exact CLI pin. Use a concrete
model ID rather than an alias. The current adapters return only completed runs,
so first-useful-result latency is explicitly unmeasured rather than copied from
total latency. If a job is interrupted after its input is consumed, restore it
from the trusted prepared copy; never synthesize a receipt.
After returning receipts to the trusted evaluator, collect them against the
matrix and schedule:
```sh
bun scripts/cso-eval.ts collect .context/cso-eval-matrix.json \
.context/cso-eval-jobs/schedule.json .context/cso-eval-receipts \
.context/cso-eval-producer-batch.json
```
Collection makes no model calls. It rejects unknown, duplicate, changed, or
wrong-input receipts and reports scheduled/submitted/missing denominators for
every v2/v3 and daily/comprehensive group. It rejects an effective-model mismatch
between matched v2/v3 cells, leaving the source receipts available for diagnosis
and a clean rerun. The collected batch stores a compact receipt index and hashes;
raw transcripts remain in the private receipt directory for adjudication. Cost
in this batch is clearly labeled a
pricing-table estimate; only a host-billed amount may populate `EvalResult`'s
host-cost field.
The producer host needs Git, the chosen authenticated CLI, and reviewed gstack
executables and catalogs without their skill source tree. Its CSO instruction
bytes come only from the cell payload, never the installed skill or installed
carved sections. Comprehensive cells additionally need a local Docker daemon,
the exact qualified helper/runtime catalog pair, public dependency inputs or
verified cache hits, and the authenticated out-of-process assertion witness. The runtime
image workflow currently emits native staging evidence with private accuracy,
held-out repair, canary, and watchdog gates still pending. Such evidence can
measure setup failures, but it cannot support a release-qualifying tested repair
until those gates and catalog promotion are complete.
## Private oracle boundary
`test/helpers/cso-eval-oracles.ts` belongs to the **trusted evaluator**. It contains
the legitimate controls, initial security assertions, alternate held-out
assertions, and non-vacuous repair judge. It is never copied into materialized
source. The fixed alternative, expected manifest, and this evaluator module must
remain unavailable to producing agents throughout their sessions.
A separate directory alone does not restrict a host agent's file access. The
evaluation host must enforce this boundary with scoped tools or an isolated
producer filesystem and test it with the `held-out-oracle-visibility` gate. If it
cannot do that, report the gate as `not_run` and do not claim held-out
qualification. Product execution containment does not automatically isolate the
hosted producing agent.
Only trusted runtime observations enter `judgeRepair`. Boot and legitimate
controls must pass on the original; the exact security assertion must fail for
the intended reason. The patched app must pass that assertion, all held-out
assertions, controls, and existing tests with unchanged verification inputs.
Independent root-cause/feature review is mandatory; boundary-replacing mocks or
disabled functionality fail certification. A producer's `tested` or
`runtime_tested` label is never an oracle result.
`collect` preserves producer claims and performance receipts; it does not turn
them into `EvalResult`. The independent evaluator must inspect the retained CSO
report/bundle, execute the private assertions, and write finding judgments and
workflow outcomes. This separation is deliberate: automatically copying agent
claims into the trusted result contract would fabricate verification.
## Scoring and release gates
```sh
bun scripts/cso-eval.ts score .context/cso-eval-matrix.json \
.context/cso-eval-producer-batch.json .context/trusted-cso-results.json \
.context/cso-qualification.json
```
The result contract is `EvalResult` in `scripts/cso-eval.ts`. The independent
evaluator supplies finding judgments, the matching `producerReceiptHash`, and
hashes of private evidence. The release scoring command requires a complete
producer batch, checks its integrity and effective-model parity, and rejects any
trusted result that is not bound to that cell's receipt. Recheck
success additionally requires a distinct current-source observation and its
source hash. Correct alternative patches need not match the reference fix's
text. Keep actual transcripts and reports in private evaluation state; do not
commit them beside the fixtures.
Each metric includes its numerator and denominator. Duplicate findings cannot
inflate true positives. Fixed variants contribute false positives. Missing and
setup-blocked supported scenarios remain misses in recall and workflow rates.
Hypotheses do not become supported discoveries; v2 legacy review evidence cannot
become v3 reproduced, runtime-tested, or tested evidence. Unknown usage remains
unknown, and wall-clock budgets are not represented as model-spend caps.
The scorer checks 95% daily precision, 80% comprehensive high/critical recall,
no aggregate high/critical recall regression against v2, all core cold starts,
zero falsely certified runtime-tested repairs, and a correct held-out repair in
every application stack. It also requires complete matched results, mandatory
reports, containment/canary receipts, and enforced oracle separation. Missing
evaluations return
`unmeasured` or `partial`, never passing release gates.
The free accounting tests use clearly labeled synthetic observations to test the
scorer. They are **not measured agent performance**. No paid comparison, fixture
cold-start qualification, or containment qualification is claimed by this corpus.
+686
View File
@@ -0,0 +1,686 @@
{
"schemaVersion": 1,
"version": "cso-v3-pairs-3",
"cases": [
{
"id": "node-sql-injection",
"stack": "node",
"family": "sql-injection",
"severity": "high",
"rootCause": "sql-injection",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "87267be77a89123b31f36c86c9a52e6eafbd266a313f3aca788a16fc47be7c65",
"fixed": "069fb74d1b471879b4151ee60de6dcae5abe6b86935a811b3160afc888091ee5"
}
},
{
"id": "node-command-injection",
"stack": "node",
"family": "command-injection",
"severity": "high",
"rootCause": "command-injection",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "4336fcd76b10ee6c36455288e14411c0ac2c62407e7ef22ae83cf957f77d17ed",
"fixed": "bcf6a044ca201dba861c4764696e800230d9b215644465cd7ed31479d218c12d"
}
},
{
"id": "node-path-traversal",
"stack": "node",
"family": "path-traversal",
"severity": "high",
"rootCause": "path-traversal",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "d69e959678cc3076202dbe9963ecde251cfece5744ee6718f4e82387efd5c994",
"fixed": "48ba7360137d0732b139dc6061640768cbfa1e31f896d5feb93258bb341a4e34"
}
},
{
"id": "node-ssrf",
"stack": "node",
"family": "ssrf",
"severity": "high",
"rootCause": "ssrf",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "dddf678fa3c900b2dd75953df953308c8e649eba1b061018075338163dba9d41",
"fixed": "c6b0a7d485fb0becae52f41e072d51096610d9c31c0cf790225350f40acb01b1"
}
},
{
"id": "node-object-authorization",
"stack": "node",
"family": "object-authorization",
"severity": "high",
"rootCause": "object-authorization",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "de4a888619db2d1992b2d46671f0b5fcde48903d6909cb23321a4019a574674a",
"fixed": "32c3ddaa88abe9286e1905f68ee3261aaf9a4a209964d7e014f622d25f05dc2c"
}
},
{
"id": "node-tenant-isolation",
"stack": "node",
"family": "tenant-isolation",
"severity": "high",
"rootCause": "tenant-isolation",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "fe9a035c0362f3513c7fb54d48acb5a835c84c424a9148be8a453f67dd1823fb",
"fixed": "6df2ee7243dcb8e624c988320d8df23814912ce0d3c2fb9282c50070f3f5b159"
}
},
{
"id": "node-html-injection",
"stack": "node",
"family": "html-injection",
"severity": "medium",
"rootCause": "html-injection",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "a373a32baef89b2d6ae97aac6c0a8aa56aa2d43c2f4f1d034e24249b272cf1da",
"fixed": "c3c114c86bd25b1c1f1b2766ad39cbe100f1c42edaa961314a20a7c64d5cf8a3"
}
},
{
"id": "node-open-redirect",
"stack": "node",
"family": "open-redirect",
"severity": "medium",
"rootCause": "open-redirect",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "12f0a273a9dc9960b7be8345794482274f7207f98b416c5211c0eb2e73633ba9",
"fixed": "1547a8c5cce7a47b054e8b0d9ce5a065703d0a686c2fea8be3d70016290cbe4d"
}
},
{
"id": "node-mass-assignment",
"stack": "node",
"family": "mass-assignment",
"severity": "high",
"rootCause": "mass-assignment",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "f0c6f6e3465069d9eba208a6ca899b1a6438d679937b89ebcd56c768d9cbd85b",
"fixed": "cb08b01645e2186adfae2bdb17bd3d41240cb389f9ddd65223e286d91ac72d05"
}
},
{
"id": "node-resource-exhaustion",
"stack": "node",
"family": "resource-exhaustion",
"severity": "high",
"rootCause": "resource-exhaustion",
"location": {
"path": "app.mjs",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "1671cb1dd6639c19aa687ec02f70190c39bbc1b7ccf436783560a173f9bb8d57",
"fixed": "a78998de27bd98b4a90600352f2e431f34c0cc7a8a4c9f5de4fc8ec1877bb9ba"
}
},
{
"id": "bun-sql-injection",
"stack": "bun",
"family": "sql-injection",
"severity": "high",
"rootCause": "sql-injection",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "6e55eb6286eb0cd7a550bc0e2057b2344af21d34e0e73ff950e12446a5e306bc",
"fixed": "b7e9df66ff955ebf4ed538798fbd8187e5929c1e4d3bd82c655e7513cb024269"
}
},
{
"id": "bun-command-injection",
"stack": "bun",
"family": "command-injection",
"severity": "high",
"rootCause": "command-injection",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "d4b222144463b15d360de50c675bbd4e01cd91b6fe97c090124e11406970bb0e",
"fixed": "0522b906f1460ebb514ec45c7d4261575bf8bccb67c3add14ec83bc5afce4e3c"
}
},
{
"id": "bun-path-traversal",
"stack": "bun",
"family": "path-traversal",
"severity": "high",
"rootCause": "path-traversal",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "8d9225578ad93d0bf243d23e23db719ddf77f486b25065ff8797c093e4d8529b",
"fixed": "380d546d31f4563edcecd8eeaed7c02aadb169a662776a9a4b026f7ea514468f"
}
},
{
"id": "bun-ssrf",
"stack": "bun",
"family": "ssrf",
"severity": "high",
"rootCause": "ssrf",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "80728866a613c4a3c1c5ffe468a098152f6e3632afa049905feb0ed286664f99",
"fixed": "05992952d8922605ac472d1a39e1aed9b058453eb95296b469d022c62b6d1114"
}
},
{
"id": "bun-object-authorization",
"stack": "bun",
"family": "object-authorization",
"severity": "high",
"rootCause": "object-authorization",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "2d72c60a1aa2b560f00d3f066bdc0947bc52978755de7323402ee2f75498f9a6",
"fixed": "b839b3f4f8354407eb6577f423d1651b40f7c3a68b079b04624b0b0d6de0294b"
}
},
{
"id": "bun-tenant-isolation",
"stack": "bun",
"family": "tenant-isolation",
"severity": "high",
"rootCause": "tenant-isolation",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "9d880c06e01294c8e785471165642a899a555c15e9409346946802f0189db53a",
"fixed": "dd1b4c7fb3cce974bfe998a5df337f6270f7675a97ce91ca9647067d29bd4368"
}
},
{
"id": "bun-html-injection",
"stack": "bun",
"family": "html-injection",
"severity": "medium",
"rootCause": "html-injection",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "280970e0c738f3f98f72388b2094b30a81ae8de24631aab9759c5c467eb04dce",
"fixed": "01d9ca12aeac526627c851609d034a35e57326b9f27e798853ad4338c8d6f0fe"
}
},
{
"id": "bun-open-redirect",
"stack": "bun",
"family": "open-redirect",
"severity": "medium",
"rootCause": "open-redirect",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "92e0f59a1a5cc9346dee8941ef218a82f739b5fb1ca06595a38ae4e3be3e5e50",
"fixed": "93da544141e869c13e5136b834f4ae59f0469d17b55bc5455e69c9b2efe0e144"
}
},
{
"id": "bun-mass-assignment",
"stack": "bun",
"family": "mass-assignment",
"severity": "high",
"rootCause": "mass-assignment",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "046ae2d46f8bc9a185a7384a1dac8902cd0d4b8f23447564b77a27354e2587eb",
"fixed": "79bb3ced148331d009b3e680c6c2040f6885e62873daedad0667fd2fd6173e1b"
}
},
{
"id": "bun-resource-exhaustion",
"stack": "bun",
"family": "resource-exhaustion",
"severity": "high",
"rootCause": "resource-exhaustion",
"location": {
"path": "app.ts",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "785c96f9a0ee7b2e8a07135dd46c43fdf28d06caa2e5380d8c6d2a469397cc2c",
"fixed": "d8ed22d72e5eb14c5e745b5d702377147b53a071de634597e4239435fd3a1c42"
}
},
{
"id": "python-sql-injection",
"stack": "python",
"family": "sql-injection",
"severity": "high",
"rootCause": "sql-injection",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "fc2cb6a773be0e191372c33014deed8837738fa48843fa99eb8ff63367f5675d",
"fixed": "07518c636f923497ed4d474b68e471216af0efb77bbd0751a3fdbc222055c71e"
}
},
{
"id": "python-command-injection",
"stack": "python",
"family": "command-injection",
"severity": "high",
"rootCause": "command-injection",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "4137304be1b38300a68202f943e41680a0897a75c89a8fc222781e39c22f22a6",
"fixed": "99bf0f2010d595321018e25428e76d408cf5efc6c96284dd7144c5802c259091"
}
},
{
"id": "python-path-traversal",
"stack": "python",
"family": "path-traversal",
"severity": "high",
"rootCause": "path-traversal",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "c261d74c7a569025259984448056928741ccb07383684f9fda5e51b9280a4d5c",
"fixed": "0fa6edad6b84f90efda50d1e49c584c8ca51ddb8ef9998034ad86b2bb7d81b76"
}
},
{
"id": "python-ssrf",
"stack": "python",
"family": "ssrf",
"severity": "high",
"rootCause": "ssrf",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "5b06615f38d6d43ea4900551155eabd5b8ba9f856d15f78e79fde70856177792",
"fixed": "814315a11b6b600c5e338822fc42fc9c4d6f66b88fa86de664ab0ccdeb91166e"
}
},
{
"id": "python-object-authorization",
"stack": "python",
"family": "object-authorization",
"severity": "high",
"rootCause": "object-authorization",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "a151cda5d96ba2e7e9b940ceb936e28070b680d572079ba5364ab56a993599da",
"fixed": "a8083c530ddb9fa4d0c0b0f6de4487ce0df7b811d2fb79c8cdbcdae0d8e3e2b4"
}
},
{
"id": "python-tenant-isolation",
"stack": "python",
"family": "tenant-isolation",
"severity": "high",
"rootCause": "tenant-isolation",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "d8cad33e13f914d34c97eb365f5d7fed4f63c0cb64546abf182f28b8050c03e0",
"fixed": "7ac649e25edcf30030f0ed3c22aa3bfacb38516ab435db4dee240c5d8e8c45a4"
}
},
{
"id": "python-html-injection",
"stack": "python",
"family": "html-injection",
"severity": "medium",
"rootCause": "html-injection",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "aa9ca878ed3c100845728ee70fb1e4254d1ee5ef76d655b3c17d1269c17b190b",
"fixed": "5a6cd758d4e007291eb01d922ef7994cc2f36371a7e96c99fcb4fabe643e3ce8"
}
},
{
"id": "python-open-redirect",
"stack": "python",
"family": "open-redirect",
"severity": "medium",
"rootCause": "open-redirect",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "0c052ebc5fba7f165ba4e173ca477911f741ed512bd152e2b43a1b526f815267",
"fixed": "2a3e70475e00d98fe6e2d6505d0ff71b9a9985f701e6a9fd081801d0162dbbd1"
}
},
{
"id": "python-mass-assignment",
"stack": "python",
"family": "mass-assignment",
"severity": "high",
"rootCause": "mass-assignment",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "74ffa12e965869af108f8cfbde9903a66d3a96fc76478ece6739a36712f97f69",
"fixed": "f4311765ac908b668ecfa3665d8023cf910e6afb1aff799802036a9f212c75ff"
}
},
{
"id": "python-resource-exhaustion",
"stack": "python",
"family": "resource-exhaustion",
"severity": "high",
"rootCause": "resource-exhaustion",
"location": {
"path": "app.py",
"symbol": "action"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "50f4a5c5ad4ac5fd9fc0a935a37a0ede2d6735dbf31f7ed9fae9f63029d98181",
"fixed": "a25653768c0d551373d594a1505510325a01e06536fd1158c7b8617084fdb2dc"
}
},
{
"id": "rails-sql-injection",
"stack": "rails",
"family": "sql-injection",
"severity": "high",
"rootCause": "sql-injection",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "ebcfded5f3a00b76f803986cf1e82ca725df8ecd5268758ffe5c7b1ffed602e6",
"fixed": "3905e9acf8f564fb3ead4e53c823fca10a2bc6e28212b90cad95bf4cff515c7d"
}
},
{
"id": "rails-command-injection",
"stack": "rails",
"family": "command-injection",
"severity": "high",
"rootCause": "command-injection",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "7938f3f18bbd5effddbd28029476f305a443148351707142267411d56d5fb0b8",
"fixed": "a88dcb4e5d0bc19d79577791bbe4eccc946f22f7ad66a0471af005d83106bd5d"
}
},
{
"id": "rails-path-traversal",
"stack": "rails",
"family": "path-traversal",
"severity": "high",
"rootCause": "path-traversal",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "b49d633543845557b4dea97674b04542df2309d53c46c61dad6532b561b4179e",
"fixed": "a77ce178798cb01abaa55a815da3f8f16ab95add4103d540b27775735d857d9e"
}
},
{
"id": "rails-ssrf",
"stack": "rails",
"family": "ssrf",
"severity": "high",
"rootCause": "ssrf",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "e8fbb44ae9e01e44615dc991920c655ab97ad16c3ef8d425f9d94a3d215beedd",
"fixed": "0c2d952137fa644f4eba215b43988ad0092a4ce902863196309e65b30438661d"
}
},
{
"id": "rails-object-authorization",
"stack": "rails",
"family": "object-authorization",
"severity": "high",
"rootCause": "object-authorization",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "5378c421c287d6a79f1d7c4836e472fe0851a7436aaee431e23b317a7fc259ca",
"fixed": "976c0a27c7c647ddbd5e31d5a36668deb6e1ca5851702bbb280a2d1f81a3d84e"
}
},
{
"id": "rails-tenant-isolation",
"stack": "rails",
"family": "tenant-isolation",
"severity": "high",
"rootCause": "tenant-isolation",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "0a082cf0146c15cb4ca585b50a024e4596642dbb6462c3370b5452a8c25eb550",
"fixed": "5814fae616945cb1ead1d7046c89281c0f7215ccf4cb298a4cc517297adc0ba8"
}
},
{
"id": "rails-html-injection",
"stack": "rails",
"family": "html-injection",
"severity": "medium",
"rootCause": "html-injection",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "118599300873d0a10f3e920157707f71703cd58cae9d597e5c98cf046b410507",
"fixed": "12a2d8d6db8c6ed60faa1658aa63df414d60c6b85dcd7e417d229e7017a05949"
}
},
{
"id": "rails-open-redirect",
"stack": "rails",
"family": "open-redirect",
"severity": "medium",
"rootCause": "open-redirect",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "77cce00579bc9b10bfc0be2755b5b6446034585456799e4e3771740a0d7f59ab",
"fixed": "113b5e5c346fd4aebab9b4e3242d14784ee70e71c6e938c289e46dfcb968c4a8"
}
},
{
"id": "rails-mass-assignment",
"stack": "rails",
"family": "mass-assignment",
"severity": "high",
"rootCause": "mass-assignment",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "17fce4f956eaa654b1b90f6fe78ccc604accc820f9496517296800ca541bc436",
"fixed": "58b5132f336f6e115ff38b3c7b28bc0a00fca2444669008e3b30c3f6037a915e"
}
},
{
"id": "rails-resource-exhaustion",
"stack": "rails",
"family": "resource-exhaustion",
"severity": "high",
"rootCause": "resource-exhaustion",
"location": {
"path": "app/controllers/cases_controller.rb",
"symbol": "CasesController#show"
},
"coreColdStart": true,
"heldOut": true,
"filesHash": {
"vulnerable": "08e656bee75b724ba5b958c1887589df9931babec2c742e3b644236a677e007c",
"fixed": "fb7f5a5011d05e1f9d27d19c70aca5f64c11bd2cbc68d3396053e7169d8e94ec"
}
}
]
}
+323
View File
@@ -0,0 +1,323 @@
/** Immutable benchmark source generator. Expected outcomes live outside producer inputs. */
import { createHash } from 'node:crypto';
import { existsSync, lstatSync, mkdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
export const CORPUS_VERSION = 'cso-v3-pairs-3';
export const STACKS = ['node', 'bun', 'python', 'rails'] as const;
export const FAMILIES = ['sql-injection', 'command-injection', 'path-traversal', 'ssrf', 'object-authorization', 'tenant-isolation', 'html-injection', 'open-redirect', 'mass-assignment', 'resource-exhaustion'] as const;
export type EvalStack = typeof STACKS[number];
export type EvalFamily = typeof FAMILIES[number];
export type EvalVariant = 'vulnerable' | 'fixed';
export interface CorpusCase {
id: string; stack: EvalStack; family: EvalFamily; severity: 'critical' | 'high' | 'medium';
rootCause: string; location: { path: string; symbol: string };
coreColdStart: true; heldOut: true;
filesHash: { vulnerable: string; fixed: string };
}
export interface CorpusManifest { schemaVersion: 1; version: string; cases: CorpusCase[] }
const hash = (input: string) => createHash('sha256').update(input).digest('hex');
export function sourceHash(files: Record<string, string>): string {
return hash(JSON.stringify(Object.keys(files).sort().map(path => [path, hash(files[path])])));
}
function stackCase(id: string): { stack: EvalStack; family: EvalFamily } {
for (const stack of STACKS) for (const family of FAMILIES) if (id === `${stack}-${family}`) return { stack, family };
throw new Error('UNKNOWN_CORPUS_CASE');
}
const description: Record<EvalFamily, string> = {
'sql-injection': 'Look up the note belonging to an exact customer name.',
'command-injection': 'Format a display name using the system formatting utility. Display names contain letters, digits, underscores, and hyphens.',
'path-traversal': 'Read a file from the public documents directory.',
ssrf: 'Load the public status document from the internal status service.',
'object-authorization': 'Read an invoice belonging to the authenticated member.',
'tenant-isolation': 'List documents in the authenticated member\'s tenant.',
'html-injection': 'Render a welcome page containing a visitor-supplied display name as text.',
'open-redirect': 'Redirect a signed-in member to an application-relative path.',
'mass-assignment': 'Update the authenticated member\'s display name. Their role is managed by an administrator.',
'resource-exhaustion': 'Generate up to 100 preview records per request.',
};
interface PublicControl {
method: 'GET' | 'POST';
path: string;
body?: Record<string, string>;
status: number;
bodyIncludes: string;
location?: string;
}
/**
* Producer-visible regression controls exercise the feature's intended use.
* Attack payloads and alternate assertions stay in cso-eval-oracles.ts.
*/
function publicControl(family: EvalFamily): PublicControl {
const controls: Record<EvalFamily, PublicControl> = {
'sql-injection': { method: 'GET', path: '/action?name=alice', status: 200, bodyIncludes: 'HELLO_ALICE' },
'command-injection': { method: 'GET', path: '/action?name=Ada', status: 200, bodyIncludes: 'Ada' },
'path-traversal': { method: 'GET', path: '/action?file=hello.txt', status: 200, bodyIncludes: 'PUBLIC_HELLO' },
ssrf: { method: 'GET', path: '/action?url=http%3A%2F%2F127.0.0.1%3A8001%2Fpublic', status: 200, bodyIncludes: 'PUBLIC_STATUS' },
'object-authorization': { method: 'GET', path: '/action?id=invoice-1', status: 200, bodyIncludes: 'member-1' },
'tenant-isolation': { method: 'GET', path: '/action?tenant=tenant-a', status: 200, bodyIncludes: 'OWN_TENANT_DOCUMENT' },
'html-injection': { method: 'GET', path: '/action?name=Alice', status: 200, bodyIncludes: '<main>Welcome Alice</main>' },
'open-redirect': { method: 'GET', path: '/action?next=%2Faccount', status: 302, bodyIncludes: '/account', location: '/account' },
'mass-assignment': { method: 'POST', path: '/action', body: { display_name: 'Ada' }, status: 200, bodyIncludes: 'Ada' },
'resource-exhaustion': { method: 'GET', path: '/action?count=5', status: 200, bodyIncludes: 'count' },
};
return controls[family];
}
function javascriptControlTest(stack: 'node' | 'bun', family: EvalFamily): string {
const control = publicControl(family);
const imports = stack === 'node'
? 'import test from "node:test";\nimport assert from "node:assert/strict";\nimport { spawn } from "node:child_process";'
: 'import { test, expect } from "bun:test";';
const start = stack === 'node'
? 'const child = spawn(process.execPath, ["app.mjs"], {stdio: "ignore"});'
: 'const child = Bun.spawn({cmd: [process.execPath, "--no-install", "app.ts"], stdout: "ignore", stderr: "ignore"});';
const stop = stack === 'node'
? 'child.kill("SIGTERM");\n if (child.exitCode === null) await new Promise(resolve => child.once("exit", resolve));'
: 'child.kill();\n await child.exited;';
const equal = stack === 'node'
? `assert.equal(response.status, ${control.status});\n assert.ok(body.includes(${JSON.stringify(control.bodyIncludes)}));${control.location ? `\n assert.equal(response.headers.get("location"), ${JSON.stringify(control.location)});` : ''}`
: `expect(response.status).toBe(${control.status});\n expect(body).toContain(${JSON.stringify(control.bodyIncludes)});${control.location ? `\n expect(response.headers.get("location")).toBe(${JSON.stringify(control.location)});` : ''}`;
const options = control.body
? `{method: "POST", redirect: "manual", headers: {"content-type": "application/json"}, body: ${JSON.stringify(JSON.stringify(control.body))}}`
: '{redirect: "manual"}';
return `${imports}
async function waitUntilReady() {
for (let attempt = 0; attempt < 100; attempt++) {
try { if ((await fetch("http://127.0.0.1:8000/health")).status === 200) return; } catch {}
await new Promise(resolve => setTimeout(resolve, 20));
}
throw new Error("application did not become ready");
}
test("the intended member workflow remains available", async () => {
${start}
try {
await waitUntilReady();
const response = await fetch(${JSON.stringify(`http://127.0.0.1:8000${control.path}`)}, ${options});
const body = await response.text();
${equal}
} finally { ${stop} }
});
`;
}
function pythonControlTest(family: EvalFamily): string {
const control = publicControl(family);
const body = control.body ? JSON.stringify(control.body) : '';
return `import http.client
import json
import subprocess
import sys
import time
import unittest
class IntendedWorkflowTest(unittest.TestCase):
def test_intended_member_workflow_remains_available(self):
child = subprocess.Popen([sys.executable, "-I", "app.py"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
try:
for _ in range(100):
try:
connection = http.client.HTTPConnection("127.0.0.1", 8000, timeout=1)
connection.request("GET", "/health")
if connection.getresponse().status == 200:
break
except OSError:
time.sleep(0.02)
else:
self.fail("application did not become ready")
connection = http.client.HTTPConnection("127.0.0.1", 8000, timeout=2)
payload = ${JSON.stringify(body)}
headers = {"content-type": "application/json"} if payload else {}
connection.request(${JSON.stringify(control.method)}, ${JSON.stringify(control.path)}, body=payload or None, headers=headers)
response = connection.getresponse()
response_body = response.read().decode()
self.assertEqual(response.status, ${control.status})
self.assertIn(${JSON.stringify(control.bodyIncludes)}, response_body)
${control.location ? ` self.assertEqual(response.getheader("location"), ${JSON.stringify(control.location)})\n` : ''} finally:
child.terminate()
try:
child.wait(timeout=2)
except subprocess.TimeoutExpired:
child.kill()
if __name__ == "__main__":
unittest.main()
`;
}
function railsControlTest(family: EvalFamily): string {
const control = publicControl(family);
const verb = control.method.toLowerCase();
const request = control.body
? `${verb} ${JSON.stringify(control.path)}, params: ${JSON.stringify(control.body)}`
: `${verb} ${JSON.stringify(control.path)}`;
return `require "test_helper"
class IntendedWorkflowTest < ActionDispatch::IntegrationTest
test "the intended member workflow remains available" do
${request}
assert_response ${control.status}
assert_includes response.body, ${JSON.stringify(control.bodyIncludes)}
${control.location ? ` assert_equal ${JSON.stringify(control.location)}, response.headers["Location"]\n` : ''} end
end
`;
}
function jsSource(stack: 'node' | 'bun', family: EvalFamily, fixed: boolean): string {
const imports = ['import { createServer } from "node:http";'];
let setup = '';
if (family === 'sql-injection') {
imports.push(stack === 'node' ? 'import { DatabaseSync } from "node:sqlite";' : 'import { Database } from "bun:sqlite";');
setup += `const db = new ${stack === 'node' ? 'DatabaseSync' : 'Database'}(":memory:");\ndb.exec("CREATE TABLE notes (name TEXT, note TEXT); INSERT INTO notes VALUES ('alice', 'HELLO_ALICE'), ('admin', 'SECRET_ADMIN_NOTE')");\nconst query = (sql, ...args) => db.${stack === 'node' ? 'prepare' : 'query'}(sql).all(...args);\n`;
}
if (family === 'command-injection') imports.push('import { execSync, execFileSync } from "node:child_process";');
if (family === 'path-traversal') imports.push('import { readFileSync } from "node:fs";', 'import { resolve, sep } from "node:path";');
if (family === 'ssrf') setup += `createServer((request, response) => { response.end(request.url === "/public" ? "PUBLIC_STATUS" : "INTERNAL_SERVICE_SECRET"); }).listen(8001, "127.0.0.1");\n`;
if (family === 'object-authorization') setup += 'const invoices = {"invoice-1": {owner: "member-1", total: 12}, "invoice-2": {owner: "member-2", total: 9800}};\n';
if (family === 'tenant-isolation') setup += 'const documents = [{tenant: "tenant-a", body: "OWN_TENANT_DOCUMENT"}, {tenant: "tenant-b", body: "OTHER_TENANT_SECRET"}];\n';
if (family === 'mass-assignment') setup += 'const account = {id: "member-1", display_name: "Alice", role: "member"};\n';
const actions: Record<EvalFamily, string> = {
'sql-injection': fixed
? 'return send(200, query("SELECT note FROM notes WHERE name = ?", params.get("name") ?? ""));'
: 'return send(200, query("SELECT note FROM notes WHERE name = \'" + (params.get("name") ?? "") + "\'"));',
'command-injection': fixed
? 'const name = params.get("name") ?? "";\n if (!/^[A-Za-z0-9_-]{1,60}$/.test(name)) return send(400, {error: "invalid display name"});\n return send(200, {output: execFileSync("/usr/bin/printf", ["%s", name], {encoding: "utf8"})});'
: 'const name = params.get("name") ?? "";\n return send(200, {output: execSync(\'printf "%s" "\' + name + \'"\', {encoding: "utf8"})});',
'path-traversal': fixed
? 'const base = resolve("public");\n const file = resolve(base, params.get("file") ?? "hello.txt");\n if (!file.startsWith(base + sep)) return send(403, {error: "outside public documents"});\n return send(200, {document: readFileSync(file, "utf8")});'
: 'const file = resolve("public", params.get("file") ?? "hello.txt");\n return send(200, {document: readFileSync(file, "utf8")});',
ssrf: `${fixed ? 'const target = new URL(params.get("url") ?? "http://127.0.0.1:8001/public");\n if (target.origin !== "http://127.0.0.1:8001" || target.pathname !== "/public" || target.username || target.password || target.search || target.hash) return send(403, {error: "unsupported status source"});\n const result = await fetch(target, {redirect: "error"});' : 'const result = await fetch(params.get("url") ?? "http://127.0.0.1:8001/public");'}\n return send(200, {status: await result.text()});`,
'object-authorization': `const invoice = invoices[params.get("id") ?? "invoice-1"];\n if (!invoice) return send(404, {error: "not found"});\n ${fixed ? 'if (invoice.owner !== actor.id) return send(403, {error: "forbidden"});\n ' : ''}return send(200, invoice);`,
'tenant-isolation': `return send(200, documents.filter(document => document.tenant === ${fixed ? 'actor.tenant' : '(params.get("tenant") ?? actor.tenant)'}));`,
'html-injection': `const name = params.get("name") ?? "Alice";\n ${fixed ? 'const safe = name.replace(/[&<>"\']/g, char => ({"&": "&amp;", "<": "&lt;", ">": "&gt;", \'"\': "&quot;", "\'": "&#39;"}[char]));' : 'const safe = name;'}\n return send(200, "<main>Welcome " + safe + "</main>", "text/html");`,
'open-redirect': `const target = params.get("next") ?? "/account";\n ${fixed ? 'if (!/^\\/(?!\\/)/.test(target) || /[\\\\\\r\\n]/.test(target)) return send(400, {error: "invalid redirect"});\n ' : ''}response.setHeader("Location", target);\n return send(302, {next: target});`,
'mass-assignment': `${fixed ? 'if (typeof body.display_name === "string") account.display_name = body.display_name;' : 'Object.assign(account, body);'}\n return send(200, account);`,
'resource-exhaustion': `const count = Number(params.get("count") ?? "5");\n ${fixed ? 'if (!Number.isInteger(count) || count < 1 || count > 100) return send(400, {error: "preview count must be 1-100"});\n ' : ''}const previews = Array.from({length: count}, (_, index) => "preview-" + index);\n return send(200, {count: previews.length});`,
};
return `${imports.join('\n')}\n${setup}// Authentication middleware has established this member before the endpoint.\nconst actor = Object.freeze({id: "member-1", tenant: "tenant-a", role: "member"});\nasync function action(params, body, response, send) {\n ${actions[family]}\n}\ncreateServer(async (request, response) => {\n const send = (status, value, type = "application/json") => {response.writeHead(status, {"content-type": type}); response.end(type === "application/json" ? JSON.stringify(value) : value);};\n const url = new URL(request.url, "http://127.0.0.1:8000");\n if (url.pathname === "/health") return send(200, {ready: true});\n if (url.pathname !== "/action") return send(404, {error: "not found"});\n try {\n let raw = "";\n for await (const part of request) {raw += part; if (raw.length > 16384) return send(413, {error: "body too large"});}\n await action(url.searchParams, raw ? JSON.parse(raw) : {}, response, send);\n } catch {send(400, {error: "invalid request"});}\n}).listen(8000, "127.0.0.1");\n`;
}
function pythonSource(family: EvalFamily, fixed: boolean): string {
const setup: Partial<Record<EvalFamily, string>> = {
'sql-injection': `db = sqlite3.connect(":memory:", check_same_thread=False)\ndb.executescript("CREATE TABLE notes (name TEXT, note TEXT); INSERT INTO notes VALUES ('alice', 'HELLO_ALICE'), ('admin', 'SECRET_ADMIN_NOTE')")`,
'object-authorization': 'invoices = {"invoice-1": {"owner": "member-1", "total": 12}, "invoice-2": {"owner": "member-2", "total": 9800}}',
'tenant-isolation': 'documents = [{"tenant": "tenant-a", "body": "OWN_TENANT_DOCUMENT"}, {"tenant": "tenant-b", "body": "OTHER_TENANT_SECRET"}]',
'mass-assignment': 'account = {"id": "member-1", "display_name": "Alice", "role": "member"}',
ssrf: `class StatusService(BaseHTTPRequestHandler):\n def do_GET(self):\n self.send_response(200)\n self.end_headers()\n self.wfile.write(b"PUBLIC_STATUS" if self.path == "/public" else b"INTERNAL_SERVICE_SECRET")\n def log_message(self, *args):\n pass\nthreading.Thread(target=ThreadingHTTPServer(("127.0.0.1", 8001), StatusService).serve_forever, daemon=True).start()`,
};
const actions: Record<EvalFamily, string> = {
'sql-injection': `rows = db.execute(${fixed ? '"SELECT note FROM notes WHERE name = ?", (params.get("name", ""),)' : '"SELECT note FROM notes WHERE name = \'" + params.get("name", "") + "\'"'}).fetchall()\n return 200, [{"note": row[0]} for row in rows]`,
'command-injection': `name = params.get("name", "")\n ${fixed ? 'if not re.fullmatch(r"[A-Za-z0-9_-]{1,60}", name):\n return 400, {"error": "invalid display name"}\n output = subprocess.check_output(["/usr/bin/printf", "%s", name], text=True)' : 'output = subprocess.check_output(\'printf "%s" "\' + name + \'"\', shell=True, text=True)'}\n return 200, {"output": output}`,
'path-traversal': `base = pathlib.Path("public").resolve()\n path = (base / params.get("file", "hello.txt")).resolve()\n ${fixed ? 'if base not in path.parents:\n return 403, {"error": "outside public documents"}\n ' : ''}return 200, {"document": path.read_text()}`,
ssrf: `target = params.get("url", "http://127.0.0.1:8001/public")\n ${fixed ? 'parsed = urllib.parse.urlsplit(target)\n if parsed.scheme != "http" or parsed.netloc != "127.0.0.1:8001" or parsed.path != "/public" or parsed.query or parsed.fragment:\n return 403, {"error": "unsupported status source"}\n ' : ''}with urllib.request.urlopen(target, timeout=2) as result:\n return 200, {"status": result.read(16384).decode()}`,
'object-authorization': `invoice = invoices.get(params.get("id", "invoice-1"))\n if invoice is None:\n return 404, {"error": "not found"}\n ${fixed ? 'if invoice["owner"] != actor["id"]:\n return 403, {"error": "forbidden"}\n ' : ''}return 200, invoice`,
'tenant-isolation': `return 200, [document for document in documents if document["tenant"] == ${fixed ? 'actor["tenant"]' : 'params.get("tenant", actor["tenant"])'}]`,
'html-injection': `name = params.get("name", "Alice")\n return 200, "<main>Welcome " + ${fixed ? 'html.escape(name, quote=True)' : 'name'} + "</main>"`,
'open-redirect': `target = params.get("next", "/account")\n ${fixed ? 'if not target.startswith("/") or target.startswith("//") or any(char in target for char in "\\\\\\r\\n"):\n return 400, {"error": "invalid redirect"}\n ' : ''}return 302, {"next": target}`,
'mass-assignment': `${fixed ? 'if isinstance(body.get("display_name"), str):\n account["display_name"] = body["display_name"]' : 'account.update(body)'}\n return 200, account`,
'resource-exhaustion': `count = int(params.get("count", "5"))\n ${fixed ? 'if count < 1 or count > 100:\n return 400, {"error": "preview count must be 1-100"}\n ' : ''}previews = ["preview-" + str(index) for index in range(count)]\n return 200, {"count": len(previews)}`,
};
return `import html\nimport json\nimport pathlib\nimport re\nimport sqlite3\nimport subprocess\nimport threading\nimport urllib.parse\nimport urllib.request\nfrom http.server import BaseHTTPRequestHandler, ThreadingHTTPServer\n\n${setup[family] ?? ''}\n# Authentication middleware has established this member before the endpoint.\nactor = {"id": "member-1", "tenant": "tenant-a", "role": "member"}\n\ndef action(params, body):\n ${actions[family]}\n\nclass Application(BaseHTTPRequestHandler):\n def do_GET(self):\n self.dispatch()\n def do_POST(self):\n self.dispatch()\n def dispatch(self):\n url = urllib.parse.urlsplit(self.path)\n params = dict(urllib.parse.parse_qsl(url.query))\n try:\n size = int(self.headers.get("Content-Length", "0"))\n if size < 0 or size > 16384:\n status, value = 413, {"error": "body too large"}\n elif url.path == "/health":\n status, value = 200, {"ready": True}\n elif url.path != "/action":\n status, value = 404, {"error": "not found"}\n else:\n body = json.loads(self.rfile.read(size)) if size else {}\n status, value = action(params, body)\n except Exception:\n status, value = 400, {"error": "invalid request"}\n self.send_response(status)\n if status == 302:\n self.send_header("Location", value["next"])\n self.send_header("Content-Type", "text/html" if isinstance(value, str) else "application/json")\n self.end_headers()\n self.wfile.write((value if isinstance(value, str) else json.dumps(value)).encode())\n def log_message(self, *args):\n pass\n\nThreadingHTTPServer(("127.0.0.1", 8000), Application).serve_forever()\n`;
}
function railsSources(family: EvalFamily, fixed: boolean): Record<string, string> {
const setup: Partial<Record<EvalFamily, string>> = {
'sql-injection': '$database = SQLite3::Database.new(":memory:")\n$database.results_as_hash = true\n$database.execute_batch("CREATE TABLE notes (name TEXT, note TEXT); INSERT INTO notes VALUES (\'alice\', \'HELLO_ALICE\'), (\'admin\', \'SECRET_ADMIN_NOTE\')")',
'object-authorization': '$invoices = {"invoice-1" => {owner: "member-1", total: 12}, "invoice-2" => {owner: "member-2", total: 9800}}',
'tenant-isolation': '$documents = [{tenant: "tenant-a", body: "OWN_TENANT_DOCUMENT"}, {tenant: "tenant-b", body: "OTHER_TENANT_SECRET"}]',
'mass-assignment': '$account = {"id" => "member-1", "display_name" => "Alice", "role" => "member"}',
ssrf: `Thread.new do\n server = TCPServer.new("127.0.0.1", 8001)\n loop do\n client = server.accept\n request = client.gets.to_s\n while (line = client.gets) && line != "\\r\\n"; end\n body = request.start_with?("GET /public ") ? "PUBLIC_STATUS" : "INTERNAL_SERVICE_SECRET"\n client.write("HTTP/1.1 200 OK\\r\\nContent-Length: #{body.bytesize}\\r\\nConnection: close\\r\\n\\r\\n#{body}")\n client.close\n end\nend`,
};
const actions: Record<EvalFamily, string> = {
'sql-injection': `rows = $database.execute(${fixed ? '"SELECT note FROM notes WHERE name = ?", [params[:name].to_s]' : '"SELECT note FROM notes WHERE name = \'#{params[:name]}\'"'})\n render json: rows`,
'command-injection': `name = params[:name].to_s\n ${fixed ? 'return render(json: {error: "invalid display name"}, status: 400) unless /\\A[A-Za-z0-9_-]{1,60}\\z/.match?(name)\n output, = Open3.capture2("/usr/bin/printf", "%s", name)' : 'output, = Open3.capture2("sh", "-c", \'printf "%s" "\' + name + \'"\')'}\n render json: {output: output}`,
'path-traversal': `base = Rails.root.join("public").to_s\n path = File.expand_path(params[:file] || "hello.txt", base)\n ${fixed ? 'return render(json: {error: "outside public documents"}, status: 403) unless path.start_with?(base + File::SEPARATOR)\n ' : ''}render json: {document: File.read(path)}`,
ssrf: `target = URI.parse(params[:url] || "http://127.0.0.1:8001/public")\n ${fixed ? 'return render(json: {error: "unsupported status source"}, status: 403) unless target.scheme == "http" && target.host == "127.0.0.1" && target.port == 8001 && target.path == "/public" && !target.userinfo && !target.query && !target.fragment\n ' : ''}render json: {status: Net::HTTP.get(target)}`,
'object-authorization': `invoice = $invoices[params[:id] || "invoice-1"]\n return render(json: {error: "not found"}, status: 404) unless invoice\n ${fixed ? 'return render(json: {error: "forbidden"}, status: 403) unless invoice[:owner] == actor[:id]\n ' : ''}render json: invoice`,
'tenant-isolation': `render json: $documents.select { |document| document[:tenant] == ${fixed ? 'actor[:tenant]' : '(params[:tenant] || actor[:tenant])'} }`,
'html-injection': `name = params[:name] || "Alice"\n render html: ("<main>Welcome " + ${fixed ? 'ERB::Util.html_escape(name)' : 'name'} + "</main>").html_safe`,
'open-redirect': `target = params[:next] || "/account"\n ${fixed ? 'return render(json: {error: "invalid redirect"}, status: 400) unless target.start_with?("/") && !target.start_with?("//") && !/[\\\\\\r\\n]/.match?(target)\n ' : ''}response.set_header("Location", target)\n render json: {next: target}, status: 302`,
'mass-assignment': `$account.merge!(${fixed ? 'params.permit(:display_name).to_h' : 'params.permit!.to_h.except("controller", "action")'})\n render json: $account`,
'resource-exhaustion': `count = Integer(params[:count] || "5")\n ${fixed ? 'return render(json: {error: "preview count must be 1-100"}, status: 400) unless (1..100).cover?(count)\n ' : ''}previews = Array.new(count) { |index| "preview-#{index}" }\n render json: {count: previews.length}`,
};
return {
'Gemfile': 'source "https://rubygems.org"\ngem "rails", "= 8.1.2"\ngem "puma", "= 7.2.0"\ngem "sqlite3", "= 2.9.0"\n',
'Gemfile.lock': readFileSync(new URL('./rails.Gemfile.lock', import.meta.url), 'utf8'),
'config.ru': 'require_relative "config/environment"\nrun Rails.application\n',
'config/boot.rb': 'ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)\nrequire "bundler/setup"\n',
'config/application.rb': `require_relative "boot"\nrequire "rails"\nrequire "action_controller/railtie"\nrequire "sqlite3"\nrequire "open3"\nrequire "net/http"\nrequire "socket"\nrequire "erb"\nmodule MemberPortal\n class Application < Rails::Application\n config.load_defaults 8.1\n config.eager_load = false\n config.secret_key_base = "cso-synthetic-test-key-not-a-production-credential"\n config.hosts = ["127.0.0.1", "localhost"]\n config.action_controller.allow_forgery_protection = false if Rails.env.test?\n end\nend\n${setup[family] ?? ''}\n`,
'config/environment.rb': 'require_relative "application"\nRails.application.initialize!\n',
'config/routes.rb': 'Rails.application.routes.draw do\n match "/action", to: "cases#show", via: [:get, :post]\n get "/health", to: proc { [200, {"content-type" => "application/json"}, [\'{"ready":true}\']] }\nend\n',
'app/controllers/cases_controller.rb': `class CasesController < ActionController::Base\n def show\n # Authentication middleware has established this member before the endpoint.\n actor = {id: "member-1", tenant: "tenant-a", role: "member"}\n ${actions[family]}\n end\nend\n`,
'bin/rails': '#!/usr/bin/env ruby\nAPP_PATH = File.expand_path("../config/application", __dir__)\nrequire_relative "../config/boot"\nrequire "rails/commands"\n',
};
}
/** Only application files are returned. Neither oracle inputs nor expected labels are included. */
export function sourceFiles(id: string, variant: EvalVariant): Record<string, string> {
const { stack, family } = stackCase(id);
if (variant !== 'vulnerable' && variant !== 'fixed') throw new Error('INVALID_CORPUS_VARIANT');
const fixed = variant === 'fixed';
let files: Record<string, string>;
if (stack === 'node' || stack === 'bun') {
const node = stack === 'node';
files = {
[node ? 'app.mjs' : 'app.ts']: jsSource(stack, family, fixed),
[node ? 'test/control.test.mjs' : 'test/control.test.ts']: javascriptControlTest(stack, family),
'package.json': JSON.stringify({
name: 'member-portal', version: '1.0.0', private: true, type: 'module',
scripts: { start: node ? 'node app.mjs' : 'bun app.ts', test: node ? 'node --test' : 'bun test' },
}, null, 2) + '\n',
};
files[stack === 'node' ? 'package-lock.json' : 'bun.lock'] = JSON.stringify(stack === 'node'
? { name: 'member-portal', version: '1.0.0', lockfileVersion: 3, packages: { '': { name: 'member-portal', version: '1.0.0' } } }
: { lockfileVersion: 1, configVersion: 1, workspaces: { '': { name: 'member-portal' } }, packages: {} }, null, 2) + '\n';
} else if (stack === 'python') files = {
'app.py': pythonSource(family, fixed),
'test_control.py': pythonControlTest(family),
'requirements.txt': '# Uses only the pinned Python runtime standard library.\n',
};
else files = {
...railsSources(family, fixed),
'test/test_helper.rb': 'ENV["RAILS_ENV"] ||= "test"\nrequire_relative "../config/environment"\nrequire "rails/test_help"\n',
'test/integration/intended_workflow_test.rb': railsControlTest(family),
};
files['README.md'] = `# Member portal\n\n${description[family]}\n\nThe endpoint is /action. A test authentication adapter establishes member-1 in\ntenant-a with the member role before application logic. Authentication itself\nis outside this fixture's assessment scope. /health is a readiness endpoint.\n\nRun only through the CSO execution helper. This application is a security\nbenchmark and must not be exposed to a host network or deployed.\n`;
if (family === 'path-traversal') { files['public/hello.txt'] = 'PUBLIC_HELLO'; files['private.txt'] = 'PRIVATE_DOCUMENT_SECRET'; }
return files;
}
export function createManifest(): CorpusManifest {
return { schemaVersion: 1, version: CORPUS_VERSION, cases: STACKS.flatMap(stack => FAMILIES.map(family => {
const id = `${stack}-${family}`;
return { id, stack, family, severity: ['html-injection', 'open-redirect'].includes(family) ? 'medium' as const : 'high' as const,
rootCause: family, location: { path: stack === 'rails' ? 'app/controllers/cases_controller.rb' : stack === 'node' ? 'app.mjs' : stack === 'bun' ? 'app.ts' : 'app.py', symbol: stack === 'rails' ? 'CasesController#show' : 'action' },
coreColdStart: true as const, heldOut: true as const, filesHash: { vulnerable: sourceHash(sourceFiles(id, 'vulnerable')), fixed: sourceHash(sourceFiles(id, 'fixed')) } };
})) };
}
export function loadCorpusManifest(): CorpusManifest {
const manifest = JSON.parse(readFileSync(new URL('./manifest.json', import.meta.url), 'utf8')) as CorpusManifest;
if (JSON.stringify(manifest) !== JSON.stringify(createManifest())) throw new Error('CORPUS_INTEGRITY_MISMATCH');
return manifest;
}
export function materializeCase(id: string, variant: EvalVariant, destination: string): { path: string; sourceHash: string } {
const manifest = loadCorpusManifest();
const spec = manifest.cases.find(item => item.id === id);
if (!spec) throw new Error('UNKNOWN_CORPUS_CASE');
const path = resolve(destination);
if (existsSync(path)) throw new Error('CORPUS_DESTINATION_EXISTS');
const parent = dirname(path);
if (realpathSync(parent) !== parent || !lstatSync(parent).isDirectory()) throw new Error('UNSAFE_CORPUS_DESTINATION');
const files = sourceFiles(id, variant);
if (sourceHash(files) !== spec.filesHash[variant]) throw new Error('CORPUS_INTEGRITY_MISMATCH');
mkdirSync(path, { mode: 0o700 });
for (const [file, contents] of Object.entries(files)) {
const output = join(path, file); mkdirSync(dirname(output), { recursive: true, mode: 0o700 });
writeFileSync(output, contents, { flag: 'wx', mode: 0o600 });
}
return { path, sourceHash: spec.filesHash[variant] };
}
+488
View File
@@ -0,0 +1,488 @@
{
"schemaVersion": 1,
"generatedAt": "2026-09-11",
"ruby": "3.2.8",
"bundler": "2.6.7",
"command": "BUNDLE_FORCE_RUBY_PLATFORM=true BUNDLE_IGNORE_CONFIG=true bundle _2.6.7_ update --bundler=2.6.7",
"gemfileSha256": "f09fb9778d39ae776be3c559f7c49402cfa82b06853939e558bda81dde0dfc6c",
"lockSha256": "ab7c4cbb2f6f81a04cd400467bfd78cf7e7aa75fc600530d2646986f7ffaf139",
"archives": [
{
"name": "action_text-trix",
"version": "2.1.19",
"sha256": "7012f59421009cf284aa651294896414d653a61a2417c9b8714c8476d2f74009",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actioncable",
"version": "8.1.2",
"sha256": "dc31efc34cca9cdefc5c691ddb8b4b214c0ea5cd1372108cbc1377767fb91969",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actionmailbox",
"version": "8.1.2",
"sha256": "058b2fb1980e5d5a894f675475fcfa45c62631103d5a2596d9610ec81581889b",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actionmailer",
"version": "8.1.2",
"sha256": "f4c1d2060f653bfe908aa7fdc5a61c0e5279670de992146582f2e36f8b9175e9",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actionpack",
"version": "8.1.2",
"sha256": "ced74147a1f0daafaa4bab7f677513fd4d3add574c7839958f7b4f1de44f8423",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actiontext",
"version": "8.1.2",
"sha256": "0bf57da22a9c19d970779c3ce24a56be31b51c7640f2763ec64aa72e358d2d2d",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "actionview",
"version": "8.1.2",
"sha256": "80455b2588911c9b72cec22d240edacb7c150e800ef2234821269b2b2c3e2e5b",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "activejob",
"version": "8.1.2",
"sha256": "908dab3713b101859536375819f4156b07bdf4c232cc645e7538adb9e302f825",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "activemodel",
"version": "8.1.2",
"sha256": "e21358c11ce68aed3f9838b7e464977bc007b4446c6e4059781e1d5c03bcf33e",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "activerecord",
"version": "8.1.2",
"sha256": "acfbe0cadfcc50fa208011fe6f4eb01cae682ebae0ef57145ba45380c74bcc44",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "activestorage",
"version": "8.1.2",
"sha256": "8a63a48c3999caeee26a59441f813f94681fc35cc41aba7ce1f836add04fba76",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "activesupport",
"version": "8.1.2",
"sha256": "88842578ccd0d40f658289b0e8c842acfe9af751afee2e0744a7873f50b6fdae",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "base64",
"version": "0.3.0",
"sha256": "27337aeabad6ffae05c265c450490628ef3ebd4b67be58257393227588f5a97b",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "bigdecimal",
"version": "4.1.2",
"sha256": "ccc836eab720a525529f70ed0de26a206fdbc9a9e8ac67b3b4ac7318b03e114d",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "builder",
"version": "3.3.0",
"sha256": "497918d2f9dca528fdca4b88d84e4ef4387256d984b8154e9d5d3fe5a9c8835f",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "concurrent-ruby",
"version": "1.3.8",
"sha256": "b2f1be836e968ccc78ccfce277ea79c72a88633f22306782c16ff23fb415d1e1",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "connection_pool",
"version": "3.0.2",
"sha256": "33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "crass",
"version": "1.0.7",
"sha256": "94868719948664c89ddcaf0a37c65048413dfcb1c869470a5f7a7ceb5390b295",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "date",
"version": "3.5.1",
"sha256": "12e09477dc932afe45bf768cd362bf73026804e0db1e6c314186d6cd0bee3344",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "drb",
"version": "2.2.3",
"sha256": "0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "erb",
"version": "6.0.7",
"sha256": "ef8339f928aa33be9205534be19e9c0daf310c4cc4eb85fd409141c094c57d61",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "erubi",
"version": "1.13.1",
"sha256": "a082103b0885dbc5ecf1172fede897f9ebdb745a4b97a5e8dc63953db1ee4ad9",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "globalid",
"version": "1.4.0",
"sha256": "037f12fbf1d9d7a014d501c2d5c77356fd4ddd96d7a7991d6700bba96706f427",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "i18n",
"version": "1.15.2",
"sha256": "00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "io-console",
"version": "0.9.2",
"sha256": "efa74f891dd03c0939a931dfc6e74c2813d904763d456ea9762b0525e748db08",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "irb",
"version": "1.18.0",
"sha256": "de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "json",
"version": "3.0.2",
"sha256": "2afafb9c82faabfb60ed2f37707973fdcb766dba2c99f0b9ed85f073d1d4c3d0",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "logger",
"version": "1.7.0",
"sha256": "196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "loofah",
"version": "2.25.2",
"sha256": "2007f746959ac65552456e04b433e83deb22759ab38c838b4445c70e43425918",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "mail",
"version": "2.9.1",
"sha256": "06574eca475253d6c18145dd70af80d0eb970182d55053497c5f4d797ea160e8",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "marcel",
"version": "1.2.1",
"sha256": "1678e9360e32f9eafa917c80029e2f6d10b2715c66a4b87b6d0da9b9cd1f859f",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "mini_mime",
"version": "1.1.5",
"sha256": "8681b7e2e4215f2a159f9400b5816d85e9d8c6c6b491e96a12797e798f8bccef",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "mini_portile2",
"version": "2.8.9",
"sha256": "0cd7c7f824e010c072e33f68bc02d85a00aeb6fce05bb4819c03dfd3c140c289",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "minitest",
"version": "6.0.6",
"sha256": "153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "net-imap",
"version": "0.6.6",
"sha256": "96aa4ee50df3060203e649efc341f53480b791d49e150f2fdebf68beb141a8df",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "net-pop",
"version": "0.1.2",
"sha256": "848b4e982013c15b2f0382792268763b748cce91c9e91e36b0f27ed26420dff3",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "net-protocol",
"version": "0.3.0",
"sha256": "ba310c3d4f1cad46bb1ab20336b06669b1ff8f7c568d9cb9342b32a718547472",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "net-smtp",
"version": "0.5.1",
"sha256": "ed96a0af63c524fceb4b29b0d352195c30d82dd916a42f03c62a3a70e5b70736",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "nio4r",
"version": "2.7.5",
"sha256": "d14779d2a9b012ec0148a53344fbb2ed2a3c4d90c5dd923bf281135ab983b2c9",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "nokogiri",
"version": "1.19.4",
"sha256": "50c951611c92bca05c51411aef45f1cbc50f2821c4802758c5c6d34696533ab5",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "pp",
"version": "0.6.4",
"sha256": "dfcb0fce700c41456265922884f9fe195d7fbb0674a3578e6c0f69588e82b570",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "prettyprint",
"version": "0.2.0",
"sha256": "2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "prism",
"version": "1.9.0",
"sha256": "7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "puma",
"version": "7.2.0",
"sha256": "5ef97cc64c0579e6a507cded86286869b6387d58d28abe347c1dd1d7decdf6d0",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "racc",
"version": "1.8.1",
"sha256": "54f2e6d1e1b91c154013277d986f52a90e5ececbe91465d29172e49342732b98",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "rack",
"version": "3.2.7",
"sha256": "93e13e1c24f93556671d85d2d79fa228c3485815c50d7e2f265b5330c6528fb7",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rack-session",
"version": "2.1.2",
"sha256": "595434f8c0c3473ae7d7ac56ecda6cc6dfd9d37c0b2b5255330aa1576967ffe8",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rack-test",
"version": "2.2.0",
"sha256": "005a36692c306ac0b4a9350355ee080fd09ddef1148a5f8b2ac636c720f5c463",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rackup",
"version": "2.3.1",
"sha256": "6c79c26753778e90983761d677a48937ee3192b3ffef6bc963c0950f94688868",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rails",
"version": "8.1.2",
"sha256": "5069061b23dfa8706b9f0159ae8b9d35727359103178a26962b868a680ba7d95",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rails-dom-testing",
"version": "2.3.0",
"sha256": "8acc7953a7b911ca44588bf08737bc16719f431a1cc3091a292bca7317925c1d",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rails-html-sanitizer",
"version": "1.7.1",
"sha256": "e797a7c9b01e567307e317c576b49ab4168017e63eea4dba9ce3cb587e2f22c2",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "railties",
"version": "8.1.2",
"sha256": "1289ece76b4f7668fc46d07e55cc992b5b8751f2ad85548b7da351b8c59f8055",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rake",
"version": "13.4.2",
"sha256": "cb825b2bd5f1f8e91ca37bddb4b9aaf345551b4731da62949be002fa89283701",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "rbs",
"version": "4.1.3",
"sha256": "193582897752597ee7cd2b9d6bf0a7014acd05490d86eb47769cdfd9f2d00ff5",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "rdoc",
"version": "8.0.0",
"sha256": "03bf8c08a9639658855a0cfd77c0abca8325c227693f7f33f82957811348c469",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "reline",
"version": "0.7.0",
"sha256": "5b012d8e55dbf9d450f12bde2cf7d15ff546ae80b3f8f3b30e570d431815583d",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "securerandom",
"version": "0.4.1",
"sha256": "cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "sqlite3",
"version": "2.9.0",
"sha256": "ece9c00b32ec5f550d3a4a35c41ea8d738563589f090b9dfd0d510b7ae5f296c",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "thor",
"version": "1.5.0",
"sha256": "e3a9e55fe857e44859ce104a84675ab6e8cd59c650a49106a05f55f136425e73",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "timeout",
"version": "0.6.1",
"sha256": "78f57368a7e7bbadec56971f78a3f5ecbcfb59b7fcbb0a3ed6ddc08a5094accb",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "tsort",
"version": "0.2.0",
"sha256": "9650a793f6859a43b6641671278f79cfead60ac714148aabe4e3f0060480089f",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "tzinfo",
"version": "2.0.6",
"sha256": "8daf828cc77bcf7d63b0e3bdb6caa47e2272dcfaf4fbfe46f8c3a9df087a829b",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "uri",
"version": "1.1.1",
"sha256": "379fa58d27ffb1387eaada68c749d1426738bd0f654d812fcc07e7568f5c57c6",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "useragent",
"version": "0.16.11",
"sha256": "700e6413ad4bb954bb63547fa098dddf7b0ebe75b40cc6f93b8d54255b173844",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "websocket-driver",
"version": "0.8.2",
"sha256": "f60120d1377cccf24100bcd1ebc25ceea6f3d5a013fc8e9d16b721016200946e",
"source": "RubyGems version API",
"platform": "java"
},
{
"name": "websocket-extensions",
"version": "0.1.5",
"sha256": "1c6ba63092cda343eb53fc657110c71c754c56484aad42578495227d717a8241",
"source": "RubyGems version API",
"platform": "ruby"
},
{
"name": "zeitwerk",
"version": "2.8.3",
"sha256": "2c85125a8467ce069e20123d1e709a08955c9d29c118c25b46b7b7fafdbb92e5",
"source": "RubyGems version API",
"platform": "ruby"
}
],
"qualification": "not-run; metadata resolution only, no fixture application executed"
}
+210
View File
@@ -0,0 +1,210 @@
GEM
remote: https://rubygems.org/
specs:
action_text-trix (2.1.19)
railties
actioncable (8.1.2)
actionpack (= 8.1.2)
activesupport (= 8.1.2)
nio4r (~> 2.0)
websocket-driver (>= 0.6.1)
zeitwerk (~> 2.6)
actionmailbox (8.1.2)
actionpack (= 8.1.2)
activejob (= 8.1.2)
activerecord (= 8.1.2)
activestorage (= 8.1.2)
activesupport (= 8.1.2)
mail (>= 2.8.0)
actionmailer (8.1.2)
actionpack (= 8.1.2)
actionview (= 8.1.2)
activejob (= 8.1.2)
activesupport (= 8.1.2)
mail (>= 2.8.0)
rails-dom-testing (~> 2.2)
actionpack (8.1.2)
actionview (= 8.1.2)
activesupport (= 8.1.2)
nokogiri (>= 1.8.5)
rack (>= 2.2.4)
rack-session (>= 1.0.1)
rack-test (>= 0.6.3)
rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.6)
useragent (~> 0.16)
actiontext (8.1.2)
action_text-trix (~> 2.1.15)
actionpack (= 8.1.2)
activerecord (= 8.1.2)
activestorage (= 8.1.2)
activesupport (= 8.1.2)
globalid (>= 0.6.0)
nokogiri (>= 1.8.5)
actionview (8.1.2)
activesupport (= 8.1.2)
builder (~> 3.1)
erubi (~> 1.11)
rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.6)
activejob (8.1.2)
activesupport (= 8.1.2)
globalid (>= 0.3.6)
activemodel (8.1.2)
activesupport (= 8.1.2)
activerecord (8.1.2)
activemodel (= 8.1.2)
activesupport (= 8.1.2)
timeout (>= 0.4.0)
activestorage (8.1.2)
actionpack (= 8.1.2)
activejob (= 8.1.2)
activerecord (= 8.1.2)
activesupport (= 8.1.2)
marcel (~> 1.0)
activesupport (8.1.2)
base64
bigdecimal
concurrent-ruby (~> 1.0, >= 1.3.1)
connection_pool (>= 2.2.5)
drb
i18n (>= 1.6, < 2)
json
logger (>= 1.4.2)
minitest (>= 5.1)
securerandom (>= 0.3)
tzinfo (~> 2.0, >= 2.0.5)
uri (>= 0.13.1)
base64 (0.3.0)
bigdecimal (4.1.2)
builder (3.3.0)
concurrent-ruby (1.3.8)
connection_pool (3.0.2)
crass (1.0.7)
date (3.5.1)
drb (2.2.3)
erb (6.0.7)
erubi (1.13.1)
globalid (1.4.0)
activesupport (>= 6.1)
i18n (1.15.2)
concurrent-ruby (~> 1.0)
io-console (0.9.2)
irb (1.18.0)
pp (>= 0.6.0)
prism (>= 1.3.0)
rdoc (>= 4.0.0)
reline (>= 0.4.2)
json (3.0.2)
logger (1.7.0)
loofah (2.25.2)
crass (~> 1.0.2)
nokogiri (>= 1.12.0)
mail (2.9.1)
logger
mini_mime (>= 0.1.1)
net-imap
net-pop
net-smtp
marcel (1.2.1)
mini_mime (1.1.5)
mini_portile2 (2.8.9)
minitest (6.0.6)
drb (~> 2.0)
prism (~> 1.5)
net-imap (0.6.6)
date
net-protocol
net-pop (0.1.2)
net-protocol
net-protocol (0.3.0)
timeout
net-smtp (0.5.1)
net-protocol
nio4r (2.7.5)
nokogiri (1.19.4)
mini_portile2 (~> 2.8.2)
racc (~> 1.4)
pp (0.6.4)
prettyprint
prettyprint (0.2.0)
prism (1.9.0)
puma (7.2.0)
nio4r (~> 2.0)
racc (1.8.1)
rack (3.2.7)
rack-session (2.1.2)
base64 (>= 0.1.0)
rack (>= 3.0.0)
rack-test (2.2.0)
rack (>= 1.3)
rackup (2.3.1)
rack (>= 3)
rails (8.1.2)
actioncable (= 8.1.2)
actionmailbox (= 8.1.2)
actionmailer (= 8.1.2)
actionpack (= 8.1.2)
actiontext (= 8.1.2)
actionview (= 8.1.2)
activejob (= 8.1.2)
activemodel (= 8.1.2)
activerecord (= 8.1.2)
activestorage (= 8.1.2)
activesupport (= 8.1.2)
bundler (>= 1.15.0)
railties (= 8.1.2)
rails-dom-testing (2.3.0)
activesupport (>= 5.0.0)
minitest
nokogiri (>= 1.6)
rails-html-sanitizer (1.7.1)
loofah (~> 2.25, >= 2.25.2)
nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
railties (8.1.2)
actionpack (= 8.1.2)
activesupport (= 8.1.2)
irb (~> 1.13)
rackup (>= 1.0.0)
rake (>= 12.2)
thor (~> 1.0, >= 1.2.2)
tsort (>= 0.2)
zeitwerk (~> 2.6)
rake (13.4.2)
rbs (4.1.3)
logger
prism (>= 1.6.0)
tsort
rdoc (8.0.0)
erb
prism (>= 1.6.0)
rbs (>= 4.0.0)
tsort
reline (0.7.0)
io-console (~> 0.5)
securerandom (0.4.1)
sqlite3 (2.9.0)
mini_portile2 (~> 2.8.0)
thor (1.5.0)
timeout (0.6.1)
tsort (0.2.0)
tzinfo (2.0.6)
concurrent-ruby (~> 1.0)
uri (1.1.1)
useragent (0.16.11)
websocket-driver (0.8.2)
base64
websocket-extensions (>= 0.1.0)
websocket-extensions (0.1.5)
zeitwerk (2.8.3)
PLATFORMS
ruby
DEPENDENCIES
puma (= 7.2.0)
rails (= 8.1.2)
sqlite3 (= 2.9.0)
BUNDLED WITH
2.6.7
+41
View File
@@ -0,0 +1,41 @@
/* Trusted, dependency-free loopback service for runtime/verifier smoke tests.
* This is infrastructure evidence, not a vulnerable/fixed evaluation pair. */
#include <arpa/inet.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
int main(void) {
signal(SIGPIPE, SIG_IGN);
int server = socket(AF_INET, SOCK_STREAM, 0), reuse = 1;
struct sockaddr_in address = {.sin_family = AF_INET, .sin_port = htons(34568),
.sin_addr.s_addr = htonl(INADDR_LOOPBACK)};
if (server < 0 || setsockopt(server, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)) ||
bind(server, (void *)&address, sizeof(address)) || listen(server, 8)) return 2;
for (;;) {
int client = accept(server, NULL, NULL);
if (client < 0) continue;
char request[2048] = {0}, response[256];
ssize_t received = read(client, request, sizeof(request) - 1);
const char *status = "404 Not Found", *body = "MISSING";
if (received > 0 && strncmp(request, "GET /control ", 13) == 0) {
status = "200 OK"; body = "CONTROL_OK";
} else if (received > 0 && strncmp(request, "GET /security ", 14) == 0) {
status = "403 Forbidden"; body = "DENIED";
}
int length = snprintf(response, sizeof(response),
"HTTP/1.1 %s\r\nContent-Length: %zu\r\nConnection: close\r\n\r\n%s",
status, strlen(body), body);
if (length > 0 && (size_t)length < sizeof(response)) {
size_t sent = 0;
while (sent < (size_t)length) {
ssize_t count = write(client, response + sent, (size_t)length - sent);
if (count <= 0) break;
sent += (size_t)count;
}
}
close(client);
}
}