diff --git a/CHANGELOG.md b/CHANGELOG.md index 1baf18f1e..07f6a13fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,30 @@ # Changelog +## [1.91.22.0] - 2026-10-05 + +**`/cso` scanner execution has reviewed, signed scanner images to build from.** + +`lib/cso/scanner-images/build-inputs.json` now pins one attested base image per scanner (gitleaks, OSV-Scanner, Semgrep, zizmor, Trivy, Schemathesis) for linux/amd64 and linux/arm64, plus the SBOM generator. None of the upstream images publish GitHub artifact attestations, so the bases come from [`garrytan/gstack-cso-scanner-bases`](https://github.com/garrytan/gstack-cso-scanner-bases) release `bases-2026.10.04.3`: each re-publishes a pinned upstream digest with the offline Semgrep rules or OSV/Trivy databases added, and its workflow signs SLSA provenance and an SPDX SBOM. Every identity, statement-set digest and asset hash was re-verified independently before the state changed to `reviewed`. Dispatching `cso-scanner-images.yml` on main now builds, qualifies and proposes the scanner catalog; until that catalog merges, scans still report `not assessed`. + +### The numbers that matter + +| Check | Result | +|---|---| +| Attested images (6 scanners x 2 platforms + generator) verified with gstack's own `gh attestation verify` flags | 13/13, both predicates | +| Statement-set digests recomputed with `scripts/cso-attestation-evidence.ts` | 26/26 equal | +| Offline asset hashes recomputed from both platforms with `hash-asset` | 6/6 equal | +| Scanners completing a real network-none scan through the production adapter on the published amd64 bases | 6/6, no coverage gaps | + +### What changes for you + +- **OSV and Trivy reports survive redaction.** Advisory links such as `ntap-20210312-0006` read as phone numbers and four-part versions read as IP addresses, which used to discard the whole report. Scanner output now has each located span replaced in place; output is still withheld when a span cannot be located or crosses stdout and stderr. +- **Schemathesis scans finish.** Schemathesis 4.26+ fuzzes until `--max-time` is spent; the adapter now budgets 75% of the scanner deadline and accepts a `max_time` stop once every selected operation ran. +- **The scanner qualification test runs.** Its fixture now lives where the executor mounts sources and declares a real dependency, so OSV and Trivy exercise their offline databases. + +### For contributors + +- Refresh advisory databases by pushing a new `bases-*` tag in `gstack-cso-scanner-bases`, then review the release's `build-inputs.json` candidate here. + ## [1.91.19.0] - 2026-10-03 **A check that did not run now says so, and memory stops losing transcripts.** diff --git a/VERSION b/VERSION index c68c06199..c55ab8cf7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.19.0 +1.91.22.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 152ce8864..556b33e73 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.19.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.22.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/cso-scanners.md b/docs/cso-scanners.md index 5212278dd..342ef23ee 100644 --- a/docs/cso-scanners.md +++ b/docs/cso-scanners.md @@ -53,8 +53,11 @@ Each execution probes the scanner version before scanning and validates the retained source both before and after collection. Unknown output, failed diagnostics, a timeout, changed source, or incomplete cleanup cannot become an empty clean result. The helper retains valid partial candidates when the -adapter can distinguish them from failed coverage. Output that cannot be -redacted is withheld. +adapter can distinguish them from failed coverage. Scanner output is redacted +in place: each located sensitive span becomes a `` marker, so an +advisory URL or version string that resembles a phone number or IP address does +not discard the whole report. Output that cannot be redacted, including a span +split across stdout and stderr, is withheld. Scanner and SARIF collection hold the run's mutation lock and reject finished or interrupted reports. Every outcome is written exclusively under a unique @@ -67,18 +70,23 @@ qualification test runs the production adapter, image, watchdog, and isolation policy. It emits the version-output hash only after a representative scan completes without a coverage gap. -The release inputs in -[`build-inputs.json`](../lib/cso/scanner-images/build-inputs.json) remain -`pending` until reviewers provide all six scanners on both native platforms. -The inputs require immutable base and SBOM-generator image digests, exact source +The reviewed release inputs in +[`build-inputs.json`](../lib/cso/scanner-images/build-inputs.json) cover all six +scanners on both native platforms. The inputs require immutable base and SBOM-generator image digests, exact source commits and versions, signer workflow identities and digests, and reviewed canonical SLSA/SPDX statement-set digests. The workflow cryptographically re-verifies each image with those repository, source, signer, and predicate constraints before use. Semgrep additionally requires a reviewed local rules bundle. OSV and Trivy require complete offline database bundles with recorded content hashes, -freshness, and ecosystem coverage. Publishing those asset-bearing base images -and reviewing their evidence are external prerequisites; the release workflow -does not invent or silently replace them. +freshness, and ecosystem coverage. The release workflow does not invent or +silently replace them. None of the upstream scanner images publish GitHub +artifact attestations, so the reviewed bases come from +[`garrytan/gstack-cso-scanner-bases`](https://github.com/garrytan/gstack-cso-scanner-bases). +That repository re-publishes each pinned upstream digest with the offline assets +added and signs provenance and an SPDX SBOM with its own workflow. Each tagged +release attaches a `pending` `build-inputs.json` candidate; a reviewer +independently re-verifies the identities and hashes before the state here +changes to `reviewed`. [`cso-scanner-images.yml`](../.github/workflows/cso-scanner-images.yml) lets a dispatched branch run read-only input and contract validation. Image publishing, @@ -99,5 +107,9 @@ Scanner flags follow the primary documentation linked in the exact chosen version: OSV's maintainers documented a cache-location regression in 2.5.0, which demonstrates why a version string alone is insufficient. [OSV issue #2983](https://github.com/google/osv-scanner/issues/2983). -Trivy uses an explicit memory scan cache alongside read-only baked databases. +Trivy uses an explicit memory scan cache alongside read-only baked databases +and a baked misconfiguration checks bundle. Schemathesis 4.26 and later repeat +fuzzing until `--max-time` is spent, so the adapter sets that budget to three +quarters of the scanner deadline and treats a `max_time` stop as complete only +when every selected operation was exercised. [Trivy filesystem reference](https://trivy.dev/docs/v0.68/guide/references/configuration/cli/trivy_filesystem/). diff --git a/lib/cso/docker.ts b/lib/cso/docker.ts index 89fa079fb..48f5adde3 100644 --- a/lib/cso/docker.ts +++ b/lib/cso/docker.ts @@ -842,7 +842,7 @@ export class DockerGroup { async execCapture( id: string, command: string[], - options: { workdir?: string; env?: Record } = {}, + options: { workdir?: string; env?: Record; redaction?: 'withhold' | 'splice' } = {}, ): Promise<{ code: number; stdout: string; stderr: string }> { if (!command.length || !command[0].startsWith('/')) throw new CsoError('INVALID_SCHEMA', 'Exec needs an absolute executable'); @@ -864,6 +864,7 @@ export class DockerGroup { env: dockerEnvironment(this.endpoint, this.config), timeoutMs: remaining, maxBytes: this.remainingOutput, + redaction: options.redaction, }); this.remainingOutput = Math.max(0, this.remainingOutput - r.capturedBytes); if (r.timedOut) throw new CsoError('DEADLINE', 'Target command exceeded the reproduction deadline'); diff --git a/lib/cso/process.ts b/lib/cso/process.ts index 6807eab5d..d450ca539 100644 --- a/lib/cso/process.ts +++ b/lib/cso/process.ts @@ -74,6 +74,47 @@ export function childEnvironment(home: string): Record { GIT_ATTR_NOSYSTEM: '1', }; } +/** + * Redact each same-stream run of captured output in place. The result is + * released only when every view a reader could assemble from it (each channel, + * both concatenation orders, and the chronological interleaving) is free of + * findings; a span crossing a channel boundary or one that cannot be located + * therefore still withholds both channels. + */ +function spliceRedaction( + ordered: Buffer[], + streams: Array, + out: Buffer[], +): { stdout: string; stderr: string } | null { + const runs: Array<{ stdout: boolean; text: string }> = []; + for (let i = 0; i < ordered.length;) { + let j = i; + while (j < ordered.length && streams[j] === streams[i]) j++; + const bytes = Buffer.concat(ordered.slice(i, j)), + text = bytes.toString('utf8'); + if (!Buffer.from(text, 'utf8').equals(bytes)) return null; + runs.push({ stdout: streams[i] === out, text }); + i = j; + } + const released: Array<{ stdout: boolean; text: string }> = []; + for (const run of runs) { + const text = redactFindingSpans(run.text, { maxBytes: MAX_OUTPUT }); + if (text === null) return null; + released.push({ stdout: run.stdout, text }); + } + const stdout = released + .filter((run) => run.stdout) + .map((run) => run.text) + .join(''), + stderr = released + .filter((run) => !run.stdout) + .map((run) => run.text) + .join(''), + chronological = released.map((run) => run.text).join(''); + const views = [stdout, stderr, stdout + stderr, stderr + stdout, chronological]; + if (views.some((view) => redactFindingSpans(view, { maxBytes: 4 * MAX_OUTPUT }) !== view)) return null; + return { stdout, stderr }; +} export function redact(value: string): string { // Scan the complete bounded stream, including across write/chunk boundaries. const output = redactFindingSpans(value, { maxBytes: MAX_OUTPUT }); @@ -455,6 +496,8 @@ export async function runProcess( maxBytes?: number; input?: string; raw?: boolean; // Only for inert Git framing or private helper/Docker control JSON that is validated before use. Never print or persist raw results. + /** `splice` replaces each located sensitive span with a marker instead of withholding both channels. */ + redaction?: 'withhold' | 'splice'; }, ): Promise { if (!isAbsolute(file) || !isAbsolute(opts.cwd) || !existsSync(opts.cwd)) @@ -476,7 +519,8 @@ export async function runProcess( }); const out: Buffer[] = [], err: Buffer[] = [], - ordered: Buffer[] = []; + ordered: Buffer[] = [], + streams: Array = []; let bytes = 0, timedOut = false, truncated = false; @@ -502,6 +546,7 @@ export async function runProcess( } target.push(chunk); ordered.push(chunk); + streams.push(target); }; child.stdout.on('data', capture(out)); child.stderr.on('data', capture(err)); @@ -526,6 +571,12 @@ export async function runProcess( const forward = stdout + stderr, reverse = stderr + stdout, chronological = Buffer.concat(ordered).toString('utf8'); + const spliced = + opts.redaction === 'splice' && !truncated ? spliceRedaction(ordered, streams, out) : null; + if (spliced) { + resolve({ code: code ?? -1, ...spliced, timedOut, truncated, capturedBytes: bytes }); + return; + } if ([stdout, stderr, forward, reverse, chronological].some((value) => redact(value) !== value)) { resolve({ code: code ?? -1, diff --git a/lib/cso/scanner-executor.ts b/lib/cso/scanner-executor.ts index ed60978ec..05ba4135f 100644 --- a/lib/cso/scanner-executor.ts +++ b/lib/cso/scanner-executor.ts @@ -728,7 +728,11 @@ export async function createDockerScannerRunner(context: ScannerRunnerContext): await group.start(scanner); const capture = async (command: string[]): Promise => { if (!scanner) throw new CsoError('ISOLATION_FAILED', 'Scanner container is unavailable'); - const result = await group!.execCapture(scanner, command, { workdir: '/work', env: plan.env }); + const result = await group!.execCapture(scanner, command, { + workdir: '/work', + env: plan.env, + redaction: 'splice', + }); return { stdout: result.stdout, stderr: result.stderr, exitCode: result.code }; }; return { diff --git a/lib/cso/scanner-images/build-inputs.json b/lib/cso/scanner-images/build-inputs.json index 4a3664225..e54955685 100644 --- a/lib/cso/scanner-images/build-inputs.json +++ b/lib/cso/scanner-images/build-inputs.json @@ -1,8 +1,235 @@ { "schemaVersion": 1, "helperAbi": 3, - "state": "pending", - "sbomGenerator": null, - "profiles": [], + "state": "reviewed", + "sbomGenerator": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:cc9e54f7e2f35ad35c932e2eda93e3e4f3d593d60937bac576ec3e4cbcb725f7", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:861e51b3cef80d09e058bec8cdbdc1c352ece4b9b146336d90c3f7a3827ec534", + "sbomStatementDigest": "sha256:f22c40474da437e92e8e08eff70686317af74b0365aebbaef5c60d8730686fc0" + }, + "profiles": [ + { + "scanner": "gitleaks", + "version": "8.30.1", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:19e4968269d3c0539f99d1ed41640a6d35bfb99973e4d614f13df8a3c05aefc2", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:cfd20937dcbb90b65699b1fec63ce688a00c389c1ead339997cb27449b4a072a", + "sbomStatementDigest": "sha256:968bdbde3d716d33c3d234bc723d7592b5fd32e94c7319cb71e29056688972c6" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:3e2a466e5f5aa549476d2f9c06a8c4a1dcd9e5811ae0428626108386793e2fb7", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:7e7bef7b910a3da111d1cdfa809b67498c26b908e681bc3dfd1bc9f8f0a53823", + "sbomStatementDigest": "sha256:02f83d779cc0dddad32efb6cbe09835e8307d7446b8c589cde68807e78901422" + } + }, + "executable": "/usr/bin/gitleaks" + }, + { + "scanner": "osv", + "version": "2.6.0", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:41aac7a4f1a757aac39ac8305815ae70ec8811304b19c34cad88b6e876e50580", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:03c45f8ff2ffe1a7d36ed6a7c6bdb5de41b8b6fb781e62536672d46bb420fb54", + "sbomStatementDigest": "sha256:0585e75ecd0cef1bc6b6abf34e8a55cbf7afed23b59ad9552f40794297b61c73" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:239118d886444b07f693a20ce4e95bc34f1f6bfbb2d814b3cc7d73df961fb084", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:331656efade5f777be4810031a45b92fedf3974a46e49dd340296e090da35041", + "sbomStatementDigest": "sha256:b559793bf5d9660baede7576e9596e34363794f4a827d0b4e858a94e6b68a6ea" + } + }, + "executable": "/osv-scanner", + "assets": { + "advisoryDatabase": { + "path": "/opt/cso/scanner-data/osv", + "contentSha256": "d9721a347e3ccf3c7ee7a4f6c679ba73cb56a7abd48d13e2bd03d0992ccd784c", + "updatedAt": "2026-10-04T18:32:04.000Z", + "ecosystems": [ + "Go", + "Hex", + "Maven", + "NuGet", + "Packagist", + "Pub", + "PyPI", + "RubyGems", + "SwiftURL", + "crates.io", + "npm" + ] + } + } + }, + { + "scanner": "semgrep", + "version": "1.179.0", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:13f53c51469a3e4e52d24d449b200f38df55a9e75bbfc97e69accd167ed88970", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:dfb008d7eff78a49ee2bfbf4e33aaa0d3ba63b497b91a6903821417ca56e78e7", + "sbomStatementDigest": "sha256:668fe85d7b46d456f12a0ce9ffe264382b287f7a0ebbcfa8ee3c484b8fa3d4ce" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:feaeb83adc333ed42ce43494d0040e7a801df2e50af4ef940b6fd98737a327a1", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:856ce948d2b79409bbce39b93cb5b12dd9a162ba54532b343c48b30dae90b559", + "sbomStatementDigest": "sha256:029a16a441818234e95ca8066e8113ac72e8425092fb98c1b19c75aabdcbc875" + } + }, + "executable": "/usr/bin/semgrep", + "assets": { + "semgrepRules": { + "path": "/policy/catalog/semgrep", + "sha256": "2b00d88982338e907f66dd8c1d60c791dff64a5a8ea51e9dca8c5958c51b0502" + } + } + }, + { + "scanner": "zizmor", + "version": "1.30.1", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:e277bc5a9755bf48f28553ed382187018b14e1ce419de0e53537978f9687223e", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:52a332ca5db5d05c7852f0c6ac1bce1fa7c07c8530a4fae909cc30fbf769b204", + "sbomStatementDigest": "sha256:bdd205208e8f1c3f3026e893a541ed62e05da25ef4a391ac297abc9c6e451d43" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:a4ab6c8c06794011583850d4556f4aca556b1d1841f5386d79fa34975bc8dfe9", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:439604b15e8243d2b5723c5642f30de97fbb7701b232d4f0d9da95f687303df3", + "sbomStatementDigest": "sha256:5ae76c860909197e7e5cc98967979272f0fb78ddd9c22e9320881b6e203f65a8" + } + }, + "executable": "/usr/bin/zizmor" + }, + { + "scanner": "trivy", + "version": "0.75.0", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:6578b21f64b66219f1efeac84a556faec01f997c44411824789367c5996c8762", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:e09d43cd66ffafd1f411f796f23c92093d6ce5084247da6f1fb6601818890029", + "sbomStatementDigest": "sha256:aae3165cf84764c5cef1a33718c70822a8cc83e32a5d8b07cc2b5b6263c3f464" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:8e1bedf13c12823d81aa5dc8e617cca62011d62a557d62933e7010bbce84d6b6", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:fc3fea7e89938ccc4430ab15fd967ca59907d5fda5fe9340476c971274212903", + "sbomStatementDigest": "sha256:83944514345b397ab4630bbc2839f8ba8b481d350c455ca85efdac0803ef6c24" + } + }, + "executable": "/usr/local/bin/trivy", + "assets": { + "advisoryDatabase": { + "path": "/opt/cso/scanner-data/trivy", + "contentSha256": "e292173b9cf49eb5414745c87af4e9355f1a76cc93aa3f68f12edbe8e18b8eee", + "updatedAt": "2026-10-04T14:28:15.000Z", + "ecosystems": [ + "alpine", + "bitnami", + "cargo", + "cocoapods", + "composer", + "conan", + "debian", + "go", + "hex", + "k8s", + "maven", + "npm", + "nuget", + "pip", + "pub", + "redhat", + "rubygems", + "swift", + "ubuntu" + ] + } + } + }, + { + "scanner": "schemathesis", + "version": "4.29.1", + "baseImages": { + "linux/amd64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:e330773e46b1219acb75b18401778e816f3c2a5b95a511a8c35006211804fbc0", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:254e952b20a27495243bd6b1d92ca225fb03eec194443db080feb75e7604606f", + "sbomStatementDigest": "sha256:1374e2f065236acf72908497ce4025e48e3b710be3e2e457caacbffd2f20a077" + }, + "linux/arm64": { + "image": "ghcr.io/garrytan/gstack-cso-scanner-bases@sha256:a0615aa9322b90fa5cf81c3002f0abd80a8baa3d4b3ea18b34b9598969371cc5", + "repository": "https://github.com/garrytan/gstack-cso-scanner-bases", + "sourceCommit": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "release": "bases-2026.10.04.3", + "signerWorkflow": "github.com/garrytan/gstack-cso-scanner-bases/.github/workflows/publish.yml", + "signerDigest": "f708b2021d2bf5e00813a263e5641fc3b2ba5e1f", + "provenanceStatementDigest": "sha256:a7c21101bbe4e82c0f62ad330976ef9a79830a5925e357d157567311f6d3b45e", + "sbomStatementDigest": "sha256:706a3b72e4771871046130e26954da85a86a9844f49f16eb87cc91a0ed8e6dd7" + } + }, + "executable": "/opt/cso-base/bin/schemathesis", + "applicationExecutable": "/usr/local/bin/python3" + } + ], "instructions": "Add one reviewed, digest-pinned SBOM generator and exactly one profile for each scanner. Every platform image and the generator must declare its GitHub repository, source commit, release, signer workflow and signer digest, plus canonical SLSA and SPDX statement-set digests that the workflow re-verifies cryptographically before use. Semgrep bases must contain the reviewed rules bundle. OSV and Trivy bases must contain the declared offline databases. Change state to reviewed only after all identities and content hashes have been independently checked." } diff --git a/lib/cso/scanners.ts b/lib/cso/scanners.ts index 823e5e696..836751841 100644 --- a/lib/cso/scanners.ts +++ b/lib/cso/scanners.ts @@ -451,7 +451,7 @@ export function scannerPlans(opts: ScannerOptions): ScannerPlan[] { String(examples), '--max-failures=10', '--max-time', - String(timeout), + String(Math.max(1, Math.floor(timeout * 0.75))), '--seed', String(seed), '--request-timeout=5', @@ -921,7 +921,9 @@ function parseResults( case 'schemathesis-json': { str(doc.schemathesis_version); const operations = doc.operations === null ? null : obj(doc.operations); - if (doc.complete !== true || doc.stop_reason !== 'completed') + // Schemathesis 4.26+ repeats fuzzing until --max-time is spent, so the + // declared time budget is a normal completion once every operation ran. + if (doc.complete !== true || (doc.stop_reason !== 'completed' && doc.stop_reason !== 'max_time')) gap('SKIPPED_INPUT', 'Schemathesis did not finish its declared operation assessment.'); if (!operations || typeof operations.tested !== 'number' || operations.tested === 0) gap('SKIPPED_INPUT', 'Schemathesis exercised no operations.'); diff --git a/package.json b/package.json index 9ab478887..21e8ad127 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.19", + "version": "1.91.22", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/cso-scanner-docker-integration.test.ts b/test/cso-scanner-docker-integration.test.ts index f215d34f1..94bedb69f 100644 --- a/test/cso-scanner-docker-integration.test.ts +++ b/test/cso-scanner-docker-integration.test.ts @@ -44,8 +44,8 @@ function write(name: string, body: string): void { } function fixture(): void { write('app.js', 'export const scannerQualification = true;\n'); - write('package.json', '{"name":"cso-scanner-qualification","version":"1.0.0"}\n'); - write('package-lock.json', '{"name":"cso-scanner-qualification","version":"1.0.0","lockfileVersion":3,"packages":{"":{"name":"cso-scanner-qualification","version":"1.0.0"}}}\n'); + write('package.json', '{"name":"cso-scanner-qualification","version":"1.0.0","dependencies":{"lodash":"4.17.20"}}\n'); + write('package-lock.json', '{"name":"cso-scanner-qualification","version":"1.0.0","lockfileVersion":3,"packages":{"":{"name":"cso-scanner-qualification","version":"1.0.0","dependencies":{"lodash":"4.17.20"}},"node_modules/lodash":{"version":"4.17.20","resolved":"https://registry.npmjs.org/lodash/-/lodash-4.17.20.tgz","integrity":"sha512-PlhdFcillOINfeV7Ni6oF1TAEayyZBoZ8bcshTHqOYJYlrqzRK5hagpagky5o4HfCzzd1TRkXPMFq6cKk9rGmA=="}}}\n'); write('.github/workflows/qualification.yml', 'name: qualification\non: [push]\npermissions: {}\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - run: echo qualification\n'); write('server.py', [ 'from http.server import BaseHTTPRequestHandler, HTTPServer', @@ -80,7 +80,7 @@ beforeAll(async () => { if (staged.platform !== platform) throw new Error(`Scanner Docker qualification requires native ${platform}`); if (staged.scanner === 'schemathesis' && (typeof staged.applicationExecutable !== 'string' || !staged.applicationExecutable.startsWith('/'))) throw new Error('Schemathesis qualification requires its reviewed fixture runtime executable'); root = fs.mkdtempSync(path.join(fs.realpathSync(os.tmpdir()), 'cso-scanner-docker-')); - source = secureDirectory(path.join(root, 'source')); fixture(); + source = secureDirectory(path.join(root, 'snapshot')); fixture(); process.env.GSTACK_HOME = path.join(root, 'state'); watchdog = path.resolve(import.meta.dir, '../bin/gstack-cso-watchdog'); if (!fs.existsSync(watchdog)) throw new Error('Scanner Docker qualification requires the compiled watchdog'); diff --git a/test/cso-scanner-release.test.ts b/test/cso-scanner-release.test.ts index 41b4abc43..9bee9e1e8 100644 --- a/test/cso-scanner-release.test.ts +++ b/test/cso-scanner-release.test.ts @@ -129,9 +129,10 @@ describe('CSO scanner qualification workflow', () => { expect(raw.slice(publicPromotion, sourcePromotion)).toContain('--remove-after'); for (const job of Object.values(workflow.jobs) as any[]) for (const step of job.steps) if (step.uses) expect(step.uses).toMatch(/@[a-f0-9]{40}$/); }); - test('commits no invented release input and normalizes scanner images to the constrained helper contract', () => { + test('commits either the empty placeholder or a complete reviewed matrix, and normalizes scanner images to the constrained helper contract', () => { const inputs = JSON.parse(fs.readFileSync(path.join(ROOT, 'lib/cso/scanner-images/build-inputs.json'), 'utf8')); - expect(inputs).toMatchObject({ state: 'pending', profiles: [], sbomGenerator: null }); + if (inputs.state === 'pending') expect(inputs).toMatchObject({ profiles: [], sbomGenerator: null }); + else expect(scannerBuildMatrix(inputs).include).toHaveLength(12); const dockerfile = fs.readFileSync(path.join(ROOT, 'lib/cso/scanner-images/scanner.Dockerfile'), 'utf8'); for (const value of ['/opt/cso/entrypoint', '/opt/cso/bin/scanner', 'USER 10001:10001', 'test -x /bin/sleep', 'test -x /bin/cat']) expect(dockerfile).toContain(value); expect(fs.readFileSync(path.join(ROOT, 'lib/cso/scanner-executor.ts'), 'utf8')).toContain("capture(['/bin/cat', plan.outputPath])"); diff --git a/test/cso-scanners.test.ts b/test/cso-scanners.test.ts index 8adbb1ebb..5e352f570 100644 --- a/test/cso-scanners.test.ts +++ b/test/cso-scanners.test.ts @@ -168,6 +168,21 @@ describe('CSO scanner candidate normalization', () => { expect(result.candidates[0]).not.toHaveProperty('reproduced'); }); + test('Schemathesis time-budget completion counts only when every selected operation ran', () => { + const complete = parse('schemathesis', { schemathesis_version: '4.29.1', complete: true, stop_reason: 'max_time', operations: { selected: 1, tested: 1, errored: 0, skipped: 0 }, errors: [], failures: [] }, 0); + expect(complete.status).toBe('complete'); + expect(complete.gaps).toEqual([]); + const untested = parse('schemathesis', { schemathesis_version: '4.29.1', complete: true, stop_reason: 'max_time', operations: { selected: 2, tested: 1, errored: 0, skipped: 0 }, errors: [], failures: [] }, 0); + expect(untested.status).toBe('partial'); + const interrupted = parse('schemathesis', { schemathesis_version: '4.29.1', complete: false, stop_reason: 'max_time', operations: { selected: 1, tested: 1, errored: 0, skipped: 0 }, errors: [], failures: [] }, 0); + expect(interrupted.status).toBe('partial'); + }); + + test('Schemathesis time budget leaves room inside the execution deadline', () => { + const plan = scannerPlans({ snapshotRoot: '/source', offline: true, selected: ['schemathesis'], deadlineSeconds: 120, schemaPath: '/policy/openapi.json', baseUrl: 'http://127.0.0.1:3000/', operationIds: ['listItems'] })[0]; + expect(plan.args[plan.args.indexOf('--max-time') + 1]).toBe('90'); + }); + test('startup failure and zero exercised operations remain not covered', () => { const result = parse('schemathesis', { schemathesis_version: '4.0.0', complete: false, stop_reason: 'interrupted', operations: null, errors: [{ title: 'Schema load failed' }], failures: [] }, 1); expect(result.status).toBe('partial'); diff --git a/test/cso-snapshot-state.test.ts b/test/cso-snapshot-state.test.ts index da57fa234..20d866061 100644 --- a/test/cso-snapshot-state.test.ts +++ b/test/cso-snapshot-state.test.ts @@ -63,6 +63,9 @@ describe('private state and process output',()=>{ test('private state rejects a non-sticky group-writable ancestor',()=>{if(process.platform==='win32')return;const base=tmp();fs.chmodSync(base,0o770);expect(()=>secureDirectory(path.join(base,'private'))).toThrow('group- or world-writable ancestor');}); test('split process output is joined and redacted before return',async()=>{const dir=tmp(),r=await runProcess('/bin/sh',['-c',"printf 'ghp_abcdefghijklmnop'; printf 'qrstuvwxyz1234567890' >&2"],{cwd:dir,env:{PATH:'/usr/bin:/bin'}});expect(r.stdout+r.stderr).not.toContain('ghp_');expect(r.stdout.toLowerCase()).toContain('redacted');}); test('alternating stdout and stderr chunks cannot split a secret marker',async()=>{const dir=tmp(),r=await runProcess('/bin/sh',['-c',"printf g; sleep .02; printf hp_abcdefghijklmnop >&2; sleep .02; printf qrstuvwxyz1234567890"],{cwd:dir,env:{PATH:'/usr/bin:/bin'}});expect(r.stdout+r.stderr).not.toContain('abcdefghijklmnop');expect(r.stdout.toLowerCase()).toContain('redacted');}); + test('splice redaction keeps structured scanner output and marks only located spans',async()=>{const dir=tmp(),r=await runProcess('/bin/sh',['-c',"printf '{\"url\":\"https://security.netapp.com/advisory/ntap-20210312-0006\",\"ok\":true}'; printf 'log line' >&2"],{cwd:dir,env:{PATH:'/usr/bin:/bin'},redaction:'splice'});expect(r.stdout).not.toContain('20210312-0006');expect(JSON.parse(r.stdout).ok).toBe(true);expect(r.stdout).toContain('{const dir=tmp();for(const script of ["printf 'ghp_abcdefghijklmnop'; printf 'qrstuvwxyz1234567890' >&2","printf g; sleep .02; printf hp_abcdefghijklmnop >&2; sleep .02; printf qrstuvwxyz1234567890"]){const r=await runProcess('/bin/sh',['-c',script],{cwd:dir,env:{PATH:'/usr/bin:/bin'},redaction:'splice'});expect(r.stdout+r.stderr).not.toContain('abcdefghijklmnop');expect(r.stdout).toBe('[sensitive process output redacted]');}}); + test('splice redaction replaces a whole-channel secret instead of leaking it',async()=>{const dir=tmp(),token=['ghp','abcdefghijklmnopqrstuvwxyz1234567890'].join('_'),r=await runProcess('/bin/sh',['-c',`printf 'token=${token} done'`],{cwd:dir,env:{PATH:'/usr/bin:/bin'},redaction:'splice'});expect(r.stdout).not.toContain('abcdefghijklmnop');expect(r.stdout).toContain('done');}); test('marker-only private keys withhold the whole payload',async()=>{const dir=tmp();expect(runProcess('/bin/sh',['-c',"printf '%s' \"$1\"",'sh',['-----BEGIN ','PRIVATE KEY-----\nraw\n-----END ','PRIVATE KEY-----'].join('')],{cwd:dir,env:{PATH:'/usr/bin:/bin'}})).rejects.toMatchObject({code:'REDACTION_FAILED'});}); test('untrusted hash-like keys do not bypass structured redaction',()=>{const dir=tmp(),file=path.join(dir,'value.json');writeJson(file,{payloadHash:['ghp_','abcdefghijklmnopqrstuvwxyz1234567890'].join('')});expect(fs.readFileSync(file,'utf8')).not.toContain('ghp_');}); test('only schema-bound helper metadata bypasses content redaction',()=>{const token='1abcdefabcdefabcdefabcdefabcdefa',hash='a'.repeat(64),untrusted=sanitizeForJson({id:token,findingId:token,sourceHash:hash}) as any;expect(untrusted.id).not.toBe(token);expect(untrusted.findingId).not.toBe(token);const trusted=sanitizeHelperForJson({id:token,findingId:token,artifactId:token,reviewArtifactId:token,bundleId:token,sourceHash:hash,preparedManifestHash:hash}) as any;expect(trusted).toEqual({id:token,findingId:token,artifactId:token,reviewArtifactId:token,bundleId:token,sourceHash:hash,preparedManifestHash:hash});const dir=tmp(),file=path.join(dir,'claim.json');writeHelperJson(file,{findingId:token,outcome:'open'});expect(readJson(file)).toEqual({findingId:token,outcome:'open'});});