diff --git a/CHANGELOG.md b/CHANGELOG.md index ecf2db2b9..9da7ab543 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## [1.91.29.0] - 2026-10-05 + +**Native runtime qualification can pass for Node, Bun, Python and Rails.** + +Main run 37345275258 of `cso-runtime-images.yml` staged every image, but eight `qualify-native` rows failed. Two causes showed up in the logs, and more were hiding behind them. Every cause below is fixed, and each was reproduced on a local amd64 Docker daemon against freshly built images. + +- **Promotion evidence is recomputed for injected runtimes.** The cold-start test put the staged runtimes into its test catalog without re-binding `promotion.evidenceDigest`, so Bun, Python and Rails failed with `RUNTIME_PROMOTION_EVIDENCE_MISMATCH`. A shared `installStagedRuntime` helper re-binds the digest, and the test now validates the catalog it builds. +- **npm dependency acquisition works.** npm 11 refuses to load the same file as both its user and global config. The preparation commands passed `/opt/cso/empty-config` for both, so every Node acquisition failed. The global config now points at its own empty file, `/opt/cso/empty-globalconfig`, which the Node image creates. +- **Prepared output can leave its container.** `docker cp` cannot read tmpfs mounts. The acquisition `/archives` and offline `/work` directories are now Docker local volumes backed by tmpfs, with the same size, owner, mode and `nosuid`/`nodev` options as before (`/archives` stays `noexec`). The runner refuses to start a container when Docker did not keep those options. +- **The offline seed stays up.** The in-image `seed` command exited while the offline commands were still to run, because it was waiting on a promise that nothing kept alive. It now holds a timer until the container stops. +- **Prepared trees can execute.** Docker's tmpfs default is `noexec`, so virtualenv Python and binstubs under `/work` exited with status 126. `/work` now mounts `exec` explicitly. +- **The registry broker releases its connections.** Under Bun, `pipe()` never closed the upstream socket. A Rails acquisition opens about 70 tunnels, which filled the broker's 64-connection limit and timed out. The broker and the in-container forwarder now forward end and close explicitly, and a free test opens 70 tunnels. +- **Gem acquisition is batched.** Each `gem fetch` process spends about 11 seconds loading the RubyGems index, so acquisition now runs one `gem fetch` per platform for all exact gems. +- **Rails commands wait for the source copy.** `db:prepare` ran while `run-app` was still copying the prepared tree, and failed with "Could not locate Gemfile". Rails application and test containers now start through `startHeldApplication`, which waits until the copy has finished. +- **Test fixtures.** The Python cold fixture called a method that `unittest.TestCase` does not have, and the Rails cold fixture had no `bin/rails`, so `rails server` printed the `rails new` help instead of booting. +- **Dependency installs get a bounded longer ceiling.** The Rails fixture's lockfile lists only the `ruby` platform, so `bundle install --local` compiles nokogiri, sqlite3 and pg. At the app container's 0.85 CPU share that takes about 385 seconds, past the 300-second limit on one command. Dependency fetch and install commands in the preparation phases now get `PREPARATION_COMMAND_TIMEOUT_MS` (900 seconds). Every other command keeps 300 seconds, both stay inside the journey deadline, and a free test pins which calls can ask for the longer ceiling. The CPU share and `ISOLATION_POLICY_HASH` are unchanged. +- **Export refusals name their reason.** A prepared-tree export the helper refuses now fails with `PREPARED_EXPORT_REJECTED`, the helper's exit status and its fixed reason or filesystem errno. Target paths are never included, and nothing retries. One unexplained refusal was seen locally in 20 exports, so a recurrence on CI will now say why. + +- **The `/cso --diff` eval accepts every correct name for the webhook bug.** `cso-diff-mode` matched only `signature|authenticat|forg` in the root cause and impact, so a correct finding titled "no authenticity verification" failed it. It now also matches `authentic`, `spoof` and `hmac`, and checks the finding title. A solo panel on main measured the case at 10/10 with and without the qualified scanner images available. + +The images change (`/opt/cso/preparation` and the Node recipe), so qualification needs a fresh staging run on main. + ## [1.91.27.0] - 2026-10-05 **The first protected-main runtime staging run can finish, and a qualified scanner catalog can ship.** diff --git a/VERSION b/VERSION index 816ffb756..ad61f5dd7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.27.0 +1.91.29.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 551c1eec7..09b43a5fd 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.27.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.29.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/cso-release-qualification.md b/docs/cso-release-qualification.md index 239e8dadb..5af039395 100644 --- a/docs/cso-release-qualification.md +++ b/docs/cso-release-qualification.md @@ -94,6 +94,38 @@ and once with two databases in a fresh staged PostgreSQL sidecar. The Rails fixture builds the source-platform `sqlite3` and `pg` native gems only in the offline execution phase. +Preparation output leaves its containers through `docker cp`, which cannot read +a tmpfs mount. The acquisition `/archives` directory and the offline `/work` +directory are therefore Docker local volumes backed by bounded tmpfs, created +per container with private mode, the runtime uid/gid, `nosuid`, and `nodev` +(`/archives` is also `noexec`; `/work` must stay executable for virtualenv and +binstub launchers). The runner inspects each container and refuses to start it +if Docker did not preserve those options. npm's user and global configs point +at two different empty files, because npm refuses to load one file in both +roles. Rails application and test containers hold on `sleep` after `run-app` +copies the source; `db:prepare` starts only once that copy has finished. +Changes to `/opt/cso/preparation` or an image recipe need a new staging run +before native qualification can pass. + +Each supervised command has a 300-second ceiling (`COMMAND_TIMEOUT_MS` in +`lib/cso/process.ts`). Only the dependency fetch and install commands of the +acquisition and offline preparation phases get `PREPARATION_COMMAND_TIMEOUT_MS`, +900 seconds. A Rails lockfile that lists only the `ruby` platform compiles +nokogiri, sqlite3 and pg offline, which takes about 385 seconds at the app +role's 0.85 CPU share. Both ceilings are cut to the time left before the +journey's aggregate deadline, so the longer ceiling never extends a journey. +Helper probes, exports, application starts, `db:prepare` and project tests keep +the 300-second ceiling. A free test in `test/cso-docker-mounts.test.ts` pins +which calls may request the longer ceiling. The CPU share is unchanged, so +`ISOLATION_POLICY_HASH` and the profiles bound to it are unaffected. + +When the in-image helper refuses to export a prepared tree, preparation fails +with error code `PREPARED_EXPORT_REJECTED` instead of the generic +`TOOL_FAILED`. The message carries the helper's exit status and its fixed +reason, such as `prepared tree contains a hard-linked file`, or the filesystem +errno and syscall. It never includes target paths. Nothing retries. Treat a +recurrence as evidence for a defect, not as a flake to rerun. + For hashed Python requirements resolved by pip, the broker records the requested index URL, allowed contacted hosts, and verified wheel hash. Pip's internal `files.pythonhosted.org` response URL is opaque to that TLS boundary, so diff --git a/lib/cso/contracts.ts b/lib/cso/contracts.ts index 743537345..485b2918e 100644 --- a/lib/cso/contracts.ts +++ b/lib/cso/contracts.ts @@ -24,7 +24,8 @@ export type ErrorCode = | 'CANCELLED' | 'PREREQUISITE' | 'INCOMPATIBLE_INPUT' - | 'ASSERTION_FAILED'; + | 'ASSERTION_FAILED' + | 'PREPARED_EXPORT_REJECTED'; export class CsoError extends Error { constructor( public code: ErrorCode, diff --git a/lib/cso/docker.ts b/lib/cso/docker.ts index 48f5adde3..51ce2566b 100644 --- a/lib/cso/docker.ts +++ b/lib/cso/docker.ts @@ -4,7 +4,7 @@ import { canonical, CsoError, MAX_OUTPUT, sha256 } from './contracts'; import { spawn } from 'node:child_process'; import { dirname } from 'node:path'; import { GROUP_LIMITS, Role, ROLE_LIMITS, Lease, admit, markSupervised, release, total } from './admission'; -import { childEnvironment, executable, runProcess } from './process'; +import { childEnvironment, commandTimeoutMs, executable, runProcess } from './process'; import { secureDirectory } from './state'; export const CONTAINER_SHM_BYTES = 8 * 1024 * 1024; export const ISOLATION_POLICY_HASH = sha256( @@ -319,6 +319,12 @@ export interface ContainerSpec { workTmpfsBytes?: number; temporaryTmpfsBytes?: number; readonlyArchiveDirectory?: string; + /** + * The prepared-tree export is read back with docker cp, which cannot see a + * tmpfs mount. This keeps /work in memory as a size-capped tmpfs-backed local + * volume instead; removal with --volumes discards it like the container. + */ + exportableWork?: true; registrySocket?: string; /** Non-secret, fixed-user-readable PostgreSQL database-name policy. */ postgresDatabasePolicy?: string; @@ -397,6 +403,26 @@ export function validateSingleContainerProcessOutput(output: string): void { 'Offline lifecycle left a background process; prepared output was withheld', ); } +/** + * Names why the in-image export helper refused a prepared tree. Only the + * helper's fixed "prepared ..." messages and filesystem errno/syscall + * names pass through, so target-controlled paths never reach the error. + */ +export function preparedExportRejection(stderr: string): string { + const reason = stderr.match(/^error: (prepared [A-Za-z ,-]+)$/m)?.[1]; + if (reason) return reason; + const errno = stderr.match(/^\s*code: "(E[A-Z]+)",?$/m)?.[1], + syscall = stderr.match(/^\s*syscall: "([a-z]+)",?$/m)?.[1]; + if (errno) return `filesystem error ${errno}${syscall ? ` during ${syscall}` : ''}`; + return 'unrecognized helper failure'; +} +export function heldApplicationReady(output: string): boolean { + const lines = output + .split('\n') + .map((line) => line.trim()) + .filter(Boolean); + return lines.length === 2 && /^PID\s+COMMAND$/.test(lines[0]) && /^\d+\s+sleep$/.test(lines[1]); +} export class DockerGroup { private ids: { role: Role; id: string }[] = []; private roles: Role[] = ['anchor']; @@ -609,6 +635,16 @@ export class DockerGroup { ); const uid = spec.role === 'postgres' ? 10001 : (hostUid as number), gid = spec.role === 'postgres' ? 10001 : (hostGid as number); + // docker cp reads volumes but not tmpfs mounts, so export sources use + // tmpfs-backed local volumes with the same size, owner and flag policy. + const expectedVolumes = new Map(); + const memoryVolume = (destination: string, options: string): string[] => { + expectedVolumes.set(destination, options); + return [ + '--mount', + `type=volume,dst=${destination},volume-driver=local,volume-opt=type=tmpfs,volume-opt=device=tmpfs,"volume-opt=o=${options}"`, + ]; + }; const args = [ 'create', '--pull=never', @@ -638,14 +674,16 @@ export class DockerGroup { `${CONTAINER_SHM_BYTES}b`, '--tmpfs', `/tmp:rw,noexec,nosuid,nodev,size=${tmpBytes},mode=1777`, - '--tmpfs', - `/work:rw,nosuid,nodev,size=${workBytes},mode=700,uid=${uid},gid=${gid}`, + ...(spec.exportableWork + ? memoryVolume('/work', `size=${workBytes},mode=700,uid=${uid},gid=${gid},nosuid,nodev`) + : // Docker's tmpfs default is noexec; prepared venvs and native extensions execute from /work. + ['--tmpfs', `/work:rw,exec,nosuid,nodev,size=${workBytes},mode=700,uid=${uid},gid=${gid}`]), '--network', joinAnchor ? `container:${this.anchor}` : 'none', '--platform', process.arch === 'arm64' ? 'linux/arm64' : 'linux/amd64', ]; - const expectedTmpfs = new Set(['/tmp', '/work']), + const expectedTmpfs = new Set(spec.exportableWork ? ['/tmp'] : ['/tmp', '/work']), expectedMounts = new Set(); for (const [k, v] of Object.entries(spec.env ?? {})) { if (!/^[A-Z][A-Z0-9_]{0,63}$/.test(k) || v.includes('\0')) @@ -749,10 +787,11 @@ export class DockerGroup { 'Preparation archive tmpfs exceeds the 2 GiB group storage policy', ); args.push( - '--tmpfs', - `/archives:rw,noexec,nosuid,nodev,size=${spec.archiveTmpfsBytes},mode=700,uid=${uid},gid=${gid}`, + ...memoryVolume( + '/archives', + `size=${spec.archiveTmpfsBytes},mode=700,uid=${uid},gid=${gid},noexec,nosuid,nodev`, + ), ); - expectedTmpfs.add('/archives'); } if (spec.readonlyArchiveDirectory) directoryMount(spec.readonlyArchiveDirectory, '/archives', true); if (spec.registrySocket) { @@ -784,7 +823,25 @@ export class DockerGroup { throw new CsoError('ISOLATION_FAILED', 'Docker did not return valid admitted-container configuration'); } const hostConfig = inspectedContainer?.HostConfig, - mounts = inspectedContainer?.Mounts; + mounts = inspectedContainer?.Mounts, + volumes = (Array.isArray(hostConfig?.Mounts) ? hostConfig.Mounts : []).filter( + (mount: any) => mount?.Type === 'volume', + ); + if ( + volumes.length !== expectedVolumes.size || + volumes.some((mount: any) => { + const driver = mount.VolumeOptions?.DriverConfig; + return ( + !expectedVolumes.has(mount.Target) || + mount.Source !== undefined || + mount.ReadOnly === true || + driver?.Name !== 'local' || + canonical(driver?.Options ?? null) !== + canonical({ device: 'tmpfs', o: expectedVolumes.get(mount.Target), type: 'tmpfs' }) + ); + }) + ) + throw new CsoError('ISOLATION_FAILED', 'Docker did not preserve the bounded in-memory export volumes'); if ( hostConfig?.ReadonlyRootfs !== true || hostConfig?.ShmSize !== CONTAINER_SHM_BYTES || @@ -796,10 +853,12 @@ export class DockerGroup { !mount || !( (mount.Type === 'bind' && expectedMounts.has(mount.Destination)) || - (mount.Type === 'tmpfs' && expectedTmpfs.has(mount.Destination)) + (mount.Type === 'tmpfs' && expectedTmpfs.has(mount.Destination)) || + (mount.Type === 'volume' && mount.Driver === 'local' && expectedVolumes.has(mount.Destination)) ), ) || - mounts.filter((mount: any) => mount?.Type === 'bind').length !== expectedMounts.size + mounts.filter((mount: any) => mount?.Type === 'bind').length !== expectedMounts.size || + mounts.filter((mount: any) => mount?.Type === 'volume').length !== expectedVolumes.size ) throw new CsoError('ISOLATION_FAILED', 'Docker did not preserve the bounded writable-storage policy'); // Durable journal publication precedes in-memory admission. If this write @@ -842,7 +901,12 @@ export class DockerGroup { async execCapture( id: string, command: string[], - options: { workdir?: string; env?: Record; redaction?: 'withhold' | 'splice' } = {}, + options: { + workdir?: string; + env?: Record; + redaction?: 'withhold' | 'splice'; + preparationCommand?: true; + } = {}, ): Promise<{ code: number; stdout: string; stderr: string }> { if (!command.length || !command[0].startsWith('/')) throw new CsoError('INVALID_SCHEMA', 'Exec needs an absolute executable'); @@ -850,7 +914,7 @@ export class DockerGroup { throw new CsoError('INVALID_SCHEMA', 'Exec working directory is outside the preparation contract'); if (this.remainingOutput <= 0) throw new CsoError('REDACTION_FAILED', 'Reproduction-group output budget is exhausted'); - const remaining = Math.max(1, Math.min(300_000, this.deadline - Date.now())); + const remaining = commandTimeoutMs(this.deadline, options.preparationCommand ? 'preparation' : 'command'); const args = ['exec']; if (options.workdir) args.push('--workdir', options.workdir); for (const [key, value] of Object.entries(options.env ?? {})) { @@ -865,6 +929,7 @@ export class DockerGroup { timeoutMs: remaining, maxBytes: this.remainingOutput, redaction: options.redaction, + preparationCommand: options.preparationCommand, }); this.remainingOutput = Math.max(0, this.remainingOutput - r.capturedBytes); if (r.timedOut) throw new CsoError('DEADLINE', 'Target command exceeded the reproduction deadline'); @@ -897,6 +962,18 @@ export class DockerGroup { throw new CsoError('REDACTION_FAILED', 'Aggregate reproduction output exceeded 1 MiB and was withheld'); return { code: r.code, output: r.stdout + r.stderr }; } + async startHeldApplication(id: string): Promise { + await this.start(id); + for (;;) { + if (heldApplicationReady(await this.docker(['top', id, '-eo', 'pid,comm'], 64 * 1024))) return; + if (Date.now() >= this.deadline) + throw new CsoError( + 'DEADLINE', + 'Application source copy did not finish within the reproduction deadline', + ); + await new Promise((resolve) => setTimeout(resolve, 100)); + } + } async assertOnlyInitProcess(id: string): Promise { if (!this.ids.some((item) => item.id === id)) throw new CsoError( diff --git a/lib/cso/images/node.Dockerfile b/lib/cso/images/node.Dockerfile index a345eecdd..cfa7378c6 100644 --- a/lib/cso/images/node.Dockerfile +++ b/lib/cso/images/node.Dockerfile @@ -1,7 +1,7 @@ # BASE_IMAGE must be a reviewed, provenance-verified node image@sha256 digest. ARG BASE_IMAGE FROM ${BASE_IMAGE} -RUN mkdir -p /opt/cso /work /metadata /archives /source /policy /fixtures && touch /opt/cso/empty-config \ +RUN mkdir -p /opt/cso /work /metadata /archives /source /policy /fixtures && touch /opt/cso/empty-config /opt/cso/empty-globalconfig \ && chown 10001:10001 /work /metadata /archives COPY --chmod=0555 entrypoint /opt/cso/entrypoint COPY --chmod=0555 run-app /opt/cso/run-app diff --git a/lib/cso/preparation-container.ts b/lib/cso/preparation-container.ts index 1dbdcdbcf..41bb67fa4 100644 --- a/lib/cso/preparation-container.ts +++ b/lib/cso/preparation-container.ts @@ -681,8 +681,14 @@ async function forwarder(socketPath: string, listen: string): Promise { client.destroy(); upstream.destroy(); }); - client.pipe(upstream); - upstream.pipe(client); + // Bun's stream.pipe does not end the destination socket; forward each + // side's end and close so finished registry tunnels release the broker. + client.once('end', () => upstream.end()); + upstream.once('end', () => client.end()); + client.once('close', () => upstream.destroy()); + upstream.once('close', () => client.destroy()); + client.pipe(upstream, { end: false }); + upstream.pipe(client, { end: false }); }); server.listen(18443, '127.0.0.1'); await once(server, 'listening'); @@ -1054,7 +1060,7 @@ async function seed(policyPath: string): Promise { '--userconfig', '/opt/cso/empty-config', '--globalconfig', - '/opt/cso/empty-config', + '/opt/cso/empty-globalconfig', ], { cwd: '/work', @@ -1078,7 +1084,11 @@ async function seed(policyPath: string): Promise { mode: 0o400, flag: 'wx', }); - await new Promise(() => {}); + // A pending promise alone lets the process exit and the container stop before + // the offline commands run; a timer holds it until the group removes it. + await new Promise(() => { + setInterval(() => {}, 2 ** 31 - 1); + }); } function ready(): void { diff --git a/lib/cso/preparation-docker.ts b/lib/cso/preparation-docker.ts index f8d7a4280..e36e0a34d 100644 --- a/lib/cso/preparation-docker.ts +++ b/lib/cso/preparation-docker.ts @@ -7,7 +7,7 @@ import { spawn } from 'node:child_process'; import { dirname, isAbsolute, join, resolve, sep } from 'node:path'; import { atomicWriteSync } from '../fs-atomic'; import { canonical, CsoError, sha256 } from './contracts'; -import { DockerGroup, type DockerEndpoint } from './docker'; +import { DockerGroup, preparedExportRejection, type DockerEndpoint } from './docker'; import { secureDirectory } from './state'; import { admittedPreparationRuntime, @@ -640,6 +640,13 @@ export class RegistryEgressBroker { this.sockets.add(upstream); upstream.setTimeout(Math.max(1, Math.min(30_000, this.deadline - Date.now()))); upstream.once('close', () => this.sockets.delete(upstream)); + // Bun's stream.pipe does not end the destination socket, so a finished + // tunnel would otherwise keep its upstream open and fill the broker's + // connection limit. Each side's end and close are forwarded explicitly. + socket.once('end', () => upstream.end()); + upstream.once('end', () => socket.end()); + socket.once('close', () => upstream.destroy()); + upstream.once('close', () => socket.destroy()); upstream.once('error', () => { if (!this.closing) this.violation ??= 'Registry connection failed after DNS pinning'; socket.destroy(); @@ -663,8 +670,8 @@ export class RegistryEgressBroker { }; socket.on('data', count); upstream.on('data', count); - socket.pipe(upstream); - upstream.pipe(socket); + socket.pipe(upstream, { end: false }); + upstream.pipe(socket, { end: false }); this.contactedHosts.add(host); socket.write('HTTP/1.1 200 Connection Established\r\n\r\n'); if (remainder.length && addBytes(remainder.length)) upstream.write(remainder); @@ -1228,7 +1235,12 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner group: DockerGroup, id: string, command: string[], - options: { workdir?: string; env?: Record } = {}, + options: { + workdir?: string; + env?: Record; + preparationCommand?: true; + redaction?: 'splice'; + } = {}, ) { const forbidden = new Set([ 'BUN_OPTIONS', @@ -1255,7 +1267,11 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner .map(([key, value]) => `${key}=${value}`), ...command, ]; - return group.execCapture(id, argv, { workdir: options.workdir }); + return group.execCapture(id, argv, { + workdir: options.workdir, + preparationCommand: options.preparationCommand, + redaction: options.redaction, + }); } async acquire(request: PreparationAcquireRequest): Promise { @@ -1357,6 +1373,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner result = await this.execClean(group, container, [command.executable, ...command.args], { workdir: command.cwd, env: { ...command.env, ...proxy }, + preparationCommand: true, }); broker.assertClean(); commandResults.push(commandReceipt(command, index, result.code)); @@ -1557,6 +1574,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner readonlyFiles: [{ host: offlinePolicy, container: '/policy/offline.json' }], readonlyMetadata: metadata, readonlyArchiveDirectory: archives, + exportableWork: true, command: ['/opt/cso/run-app', '/opt/cso/preparation', 'seed', '/policy/offline.json'], }); await group.start(app); @@ -1572,6 +1590,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner result = await this.execClean(group, app, [command.executable, ...command.args], { workdir: command.cwd, env: command.env, + preparationCommand: true, }); commands.push(commandReceipt(command, index, result.code)); if (result.code !== 0) @@ -1581,15 +1600,23 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner if (finalized.code !== 0) fail('TOOL_FAILED', 'Offline preparation scratch cleanup failed'); await group.assertOnlyInitProcess(app); const containerExport = `/work/.gstack-cso-export-${randomBytes(12).toString('hex')}`; - const exported = await this.execClean(group, app, [ - '/opt/cso/preparation', - 'export-prepared', - '/work', - containerExport, - String(request.limits.writableBytes), - ]); + const exported = await this.execClean( + group, + app, + [ + '/opt/cso/preparation', + 'export-prepared', + '/work', + containerExport, + String(request.limits.writableBytes), + ], + { redaction: 'splice' }, + ); if (exported.code !== 0) - fail('TOOL_FAILED', 'Qualified prepared-tree export rejected offline application output'); + fail( + 'PREPARED_EXPORT_REJECTED', + `Qualified prepared-tree export rejected offline application output (exit ${exported.code}: ${preparedExportRejection(exported.stderr)})`, + ); await group.assertOnlyInitProcess(app); await group.pause(app); const inertExport = secureDirectory(join(executionCopies, 'prepared-export')), diff --git a/lib/cso/preparation.ts b/lib/cso/preparation.ts index f822e74ad..2a5563819 100644 --- a/lib/cso/preparation.ts +++ b/lib/cso/preparation.ts @@ -336,7 +336,7 @@ function inspectNode(root: string, plan: PreparationPlan) { '--userconfig', '/opt/cso/empty-config', '--globalconfig', - '/opt/cso/empty-config', + '/opt/cso/empty-globalconfig', ]; const offline = [ 'ci', @@ -348,7 +348,7 @@ function inspectNode(root: string, plan: PreparationPlan) { '--userconfig', '/opt/cso/empty-config', '--globalconfig', - '/opt/cso/empty-config', + '/opt/cso/empty-globalconfig', ]; plan.acquisition.push( command('/usr/local/bin/npm', [...acquisition, '--registry', 'https://registry.npmjs.org'], '/metadata', { @@ -369,7 +369,7 @@ function inspectNode(root: string, plan: PreparationPlan) { '--userconfig', '/opt/cso/empty-config', '--globalconfig', - '/opt/cso/empty-config', + '/opt/cso/empty-globalconfig', ], '/work', ), @@ -988,17 +988,22 @@ function inspectRails(root: string, plan: PreparationPlan) { input.integrity = hash; input.integritySource = 'lock'; } - // gem fetch downloads without evaluating a Gemfile/gemspec or building extensions. + } + // gem fetch downloads without evaluating a Gemfile/gemspec or building extensions. + // Each process loads the full RubyGems index (~11 s at the app role's CPU share), + // so all exact NAME:VERSION gems of one platform are fetched in one process. + const platforms = [...new Set(plan.inputs.map((input) => input.platform!))]; + for (const platform of platforms) { plan.acquisition.push( command( '/usr/local/bin/gem', [ 'fetch', - input.name, - '--version', - input.version, + ...plan.inputs + .filter((input) => input.platform === platform) + .map((input) => `${input.name}:${input.version}`), '--platform', - input.platform!, + platform, '--clear-sources', '--source', 'https://rubygems.org', diff --git a/lib/cso/process.ts b/lib/cso/process.ts index 6f3b7a77f..15d4d40f3 100644 --- a/lib/cso/process.ts +++ b/lib/cso/process.ts @@ -496,6 +496,26 @@ function hardenGit(file: string, args: string[]): { args: string[]; configs?: Gi ], }; } +/** Ceiling for one supervised child command. */ +export const COMMAND_TIMEOUT_MS = 300_000; +/** + * Ceiling for one dependency fetch or install command inside a preparation + * container. Rails lockfiles that pin only the `ruby` platform compile native + * gems offline, which takes about 385 s at the app role's CPU share. Every + * other command keeps COMMAND_TIMEOUT_MS, and both stay inside the caller's + * aggregate deadline. + */ +export const PREPARATION_COMMAND_TIMEOUT_MS = 900_000; +export function commandTimeoutMs( + deadline: number, + phase: 'command' | 'preparation', + now = Date.now(), +): number { + return Math.max( + 1, + Math.min(phase === 'preparation' ? PREPARATION_COMMAND_TIMEOUT_MS : COMMAND_TIMEOUT_MS, deadline - now), + ); +} export async function runProcess( file: string, args: string[], @@ -508,6 +528,8 @@ export async function runProcess( raw?: boolean; // Only for inert Git framing or private helper/Docker control JSON that is validated before use. Never print or persist raw results. /** `splice` replaces each located sensitive span with a marker instead of withholding both channels. */ redaction?: 'withhold' | 'splice'; + /** Allows PREPARATION_COMMAND_TIMEOUT_MS; only preparation dependency commands set it. */ + preparationCommand?: true; }, ): Promise { if (!isAbsolute(file) || !isAbsolute(opts.cwd) || !existsSync(opts.cwd)) @@ -545,7 +567,13 @@ export async function runProcess( timedOut = true; kill(); }, - Math.max(1, Math.min(opts.timeoutMs ?? 30_000, 300_000)), + Math.max( + 1, + Math.min( + opts.timeoutMs ?? 30_000, + opts.preparationCommand ? PREPARATION_COMMAND_TIMEOUT_MS : COMMAND_TIMEOUT_MS, + ), + ), ); const capture = (target: Buffer[]) => (chunk: Buffer) => { bytes += chunk.length; diff --git a/lib/cso/scanner-executor.ts b/lib/cso/scanner-executor.ts index 05ba4135f..db62f05f9 100644 --- a/lib/cso/scanner-executor.ts +++ b/lib/cso/scanner-executor.ts @@ -378,6 +378,7 @@ function failure(plan: ScannerPlan, error: unknown, version?: string): ScannerOu PREREQUISITE: 'PREREQUISITE', INCOMPATIBLE_INPUT: 'PREREQUISITE', ASSERTION_FAILED: 'TOOL_FAILED', + PREPARED_EXPORT_REJECTED: 'TOOL_FAILED', }; const code = codes[e.code]; return { @@ -798,7 +799,7 @@ export async function createDockerScannerRunner(context: ScannerRunnerContext): ? ['/opt/cso/run-app', '/bin/sleep', '2147483647'] : ['/opt/cso/run-app', api.start.executable, ...api.start.args], }); - await group!.start(app); + await (rails ? group!.startHeldApplication(app) : group!.start(app)); if (rails) { const clean = [ '/usr/bin/env', diff --git a/lib/cso/verification.ts b/lib/cso/verification.ts index 3ea84d1a5..d45394e7b 100644 --- a/lib/cso/verification.ts +++ b/lib/cso/verification.ts @@ -1718,7 +1718,7 @@ export class DockerVerificationExecutor implements VerificationExecutor { command: ['/opt/cso/run-app', '/bin/sleep', '2147483647'], readonlyDirectories: [{ host: fixtures, container: '/fixtures' }], }); - await group.start(app); + await group.startHeldApplication(app); await dbPrepare(app); await group.execDetached(app, [request.start.executable, ...request.start.args]); } else { @@ -1784,7 +1784,7 @@ export class DockerVerificationExecutor implements VerificationExecutor { }); let testResult: { code: number; output: string }; if (rails) { - await group.start(testId); + await group.startHeldApplication(testId); await dbPrepare(testId); const result = await group.execCapture(testId, cleanCommand([test.executable, ...test.args]), { workdir: '/work', diff --git a/package.json b/package.json index 1c8758aed..6367e04c2 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.27", + "version": "1.91.29", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/cso-docker-mounts.test.ts b/test/cso-docker-mounts.test.ts index 427ba425e..fdcad2e43 100644 --- a/test/cso-docker-mounts.test.ts +++ b/test/cso-docker-mounts.test.ts @@ -2,7 +2,9 @@ import { afterEach, describe, expect, spyOn, test } from 'bun:test'; import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; -import { CONTAINER_SHM_BYTES, DockerGroup, type ContainerSpec } from '../lib/cso/docker'; +import { CONTAINER_SHM_BYTES, DockerGroup, preparedExportRejection, type ContainerSpec } from '../lib/cso/docker'; +import * as processModule from '../lib/cso/process'; +import { COMMAND_TIMEOUT_MS, PREPARATION_COMMAND_TIMEOUT_MS, commandTimeoutMs } from '../lib/cso/process'; const roots: string[] = []; const restores: Array<() => void> = []; @@ -22,6 +24,7 @@ function fixture() { restores.push(() => listener.stop(true)); const image = `sha256:${'a'.repeat(64)}`, id = 'b'.repeat(64), calls: string[][] = []; let createArgs: string[] = []; + const inspectHooks: Array<(value: any) => void> = []; const group = new (DockerGroup as any)({}, 'mount-regression', root, Date.now() + 60_000, {}) as DockerGroup; if (process.getuid!() === 0) { const uid = spyOn(process, 'getuid').mockReturnValue(1001); @@ -43,17 +46,25 @@ function fixture() { if (args[0] === 'create') { createArgs = args; return id; } if (args[0] === 'inspect') { const tmpfs = createArgs.flatMap((arg, index) => arg === '--tmpfs' ? [createArgs[index + 1].split(':')[0]] : []); - const binds = createArgs.flatMap((arg, index) => arg === '--mount' ? [createArgs[index + 1]] : []); - return JSON.stringify({ - HostConfig: { ReadonlyRootfs: true, ShmSize: CONTAINER_SHM_BYTES, Tmpfs: Object.fromEntries(tmpfs.map(target => [target, 'rw'])) }, + const mounts = createArgs.flatMap((arg, index) => arg === '--mount' ? [createArgs[index + 1]] : []); + const binds = mounts.filter(mount => !mount.startsWith('type=volume,')); + const volumes = mounts.filter(mount => mount.startsWith('type=volume,')).map(mount => ({ + target: /dst=([^,]+)/.exec(mount)![1], o: /"volume-opt=o=([^"]+)"/.exec(mount)![1], + })); + const value = { + HostConfig: { ReadonlyRootfs: true, ShmSize: CONTAINER_SHM_BYTES, Tmpfs: Object.fromEntries(tmpfs.map(target => [target, 'rw'])), + Mounts: [...binds.map(() => ({ Type: 'bind' })), ...volumes.map(volume => ({ Type: 'volume', Target: volume.target, + VolumeOptions: { DriverConfig: { Name: 'local', Options: { device: 'tmpfs', o: volume.o, type: 'tmpfs' } } } }))] }, Mounts: [...tmpfs.map(Destination => ({ Type: 'tmpfs', Destination })), ...binds.map(mount => ({ Type: 'bind', Destination: mount.split(',').find(part => part.startsWith('dst='))!.slice(4), - }))], - }); + })), ...volumes.map(volume => ({ Type: 'volume', Destination: volume.target, Driver: 'local' }))], + }; + for (const hook of inspectHooks) hook(value); + return JSON.stringify(value); } throw new Error(`Unexpected Docker call: ${args.join(' ')}`); }; - return { root, directory, file, socket, group, image, id, calls }; + return { root, directory, file, socket, group, image, id, calls, inspectHooks }; } describe.skipIf(process.platform === 'win32')('CSO Docker nonrecursive bind mounts', () => { @@ -106,4 +117,86 @@ describe.skipIf(process.platform === 'win32')('CSO Docker nonrecursive bind moun expect(f.calls.some(call => call[0] === 'create')).toBe(false); expect(fs.existsSync(path.join(f.root, 'resources.journal'))).toBe(false); }); + + // docker cp cannot read tmpfs mounts, so both export sources are tmpfs-backed local volumes. + test.each([ + ['acquisition archives', '/archives', { archiveTmpfsBytes: 4096 }, 'noexec,nosuid,nodev'], + ['prepared-tree work', '/work', { exportableWork: true as const }, 'nosuid,nodev'], + ])('%s export from a bounded in-memory volume docker cp can read', async (_name, target, spec, flags) => { + const f = fixture(); + expect(await f.group.createContainer({ role: 'app', image: f.image, command: ['/bin/sleep', '1'], ...(spec as Partial) })).toBe(f.id); + const args = f.calls.find(call => call[0] === 'create')!; + const volume = args.flatMap((arg, index) => arg === '--mount' && args[index + 1].startsWith('type=volume,') ? [args[index + 1]] : []); + expect(volume).toHaveLength(1); + expect(volume[0]).toStartWith(`type=volume,dst=${target},volume-driver=local,volume-opt=type=tmpfs,volume-opt=device=tmpfs,"volume-opt=o=size=`); + expect(volume[0]).toEndWith(`,mode=700,uid=${process.getuid!()},gid=${process.getgid!()},${flags}"`); + expect(args.flatMap((arg, index) => arg === '--tmpfs' ? [args[index + 1].split(':')[0]] : [])).not.toContain(target); + }); + + test.each([ + ['dropped tmpfs driver options', (value: any) => { value.HostConfig.Mounts.at(-1).VolumeOptions.DriverConfig.Options.type = 'none'; }], + ['a host-backed volume source', (value: any) => { value.HostConfig.Mounts.at(-1).Source = '/var/tmp/export'; }], + ['an extra daemon volume', (value: any) => { value.Mounts.push({ Type: 'volume', Destination: '/data', Driver: 'local' }); }], + ['a missing volume mount', (value: any) => { value.HostConfig.Mounts.pop(); }], + ])('export volumes fail closed on %s', async (_name, mutate) => { + const f = fixture(); f.inspectHooks.push(mutate); + await expect(f.group.createContainer({ role: 'app', image: f.image, command: ['/bin/sleep', '1'], archiveTmpfsBytes: 4096 })).rejects.toThrow('Docker did not preserve'); + expect(fs.existsSync(path.join(f.root, 'resources.journal'))).toBe(false); + }); +}); + +describe('CSO preparation command ceiling', () => { + test('only the preparation phase gets the longer ceiling, and neither outlives the deadline', () => { + const now = 1_000_000, later = now + 2 * 60 * 60_000; + expect(PREPARATION_COMMAND_TIMEOUT_MS).toBe(900_000); + expect(COMMAND_TIMEOUT_MS).toBe(300_000); + expect(commandTimeoutMs(later, 'command', now)).toBe(COMMAND_TIMEOUT_MS); + expect(commandTimeoutMs(later, 'preparation', now)).toBe(PREPARATION_COMMAND_TIMEOUT_MS); + expect(commandTimeoutMs(now + 60_000, 'preparation', now)).toBe(60_000); + expect(commandTimeoutMs(now + 60_000, 'command', now)).toBe(60_000); + expect(commandTimeoutMs(now - 1, 'preparation', now)).toBe(1); + }); + + test('execCapture passes the preparation ceiling to the child only when asked, bounded by the group deadline', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'cso-t-')); + roots.push(root); + const seen: Array<{ timeoutMs?: number; preparationCommand?: true }> = []; + const run = spyOn(processModule, 'runProcess').mockImplementation(async (_file, _args, opts) => { + seen.push({ timeoutMs: opts.timeoutMs, preparationCommand: opts.preparationCommand }); + return { code: 0, stdout: '', stderr: '', timedOut: false, truncated: false, capturedBytes: 0 }; + }); + restores.push(() => run.mockRestore()); + const group = (deadline: number) => + new (DockerGroup as any)({ executable: '/usr/bin/docker' }, 'timeout-ceiling', root, deadline, {}) as DockerGroup; + const far = group(Date.now() + 2 * 60 * 60_000), near = group(Date.now() + 60_000); + await far.execCapture('x', ['/bin/true']); + await far.execCapture('x', ['/bin/true'], { preparationCommand: true }); + await near.execCapture('x', ['/bin/true'], { preparationCommand: true }); + expect(seen[0]).toEqual({ timeoutMs: COMMAND_TIMEOUT_MS, preparationCommand: undefined }); + expect(seen[1]).toEqual({ timeoutMs: PREPARATION_COMMAND_TIMEOUT_MS, preparationCommand: true }); + expect(seen[2].preparationCommand).toBe(true); + expect(seen[2].timeoutMs!).toBeLessThanOrEqual(60_000); + }); + + test('only the acquisition and offline dependency command loops request the preparation ceiling', () => { + const directory = path.join(import.meta.dir, '../lib/cso'); + const uses = fs.readdirSync(directory).filter(name => name.endsWith('.ts')).flatMap(name => { + const lines = fs.readFileSync(path.join(directory, name), 'utf8').split('\n'); + return lines.flatMap((line, index) => line.includes('preparationCommand: true') ? [{ name, context: lines.slice(Math.max(0, index - 4), index).join('\n') }] : []); + }); + expect(uses.map(use => use.name)).toEqual(['preparation-docker.ts', 'preparation-docker.ts']); + for (const use of uses) expect(use.context).toContain('[command.executable, ...command.args]'); + }); +}); + +describe('CSO prepared export rejection reason', () => { + test('names the helper reason or errno and never echoes paths', () => { + expect(preparedExportRejection('4 | throw new Error("prepared tree contains a hard-linked file");\n ^\nerror: prepared tree contains a hard-linked file\n at f (/$bunfs/root/preparation:4:9)\n')) + .toBe('prepared tree contains a hard-linked file'); + expect(preparedExportRejection("ENOENT: no such file or directory, lstat '/work/secret-name'\n path: \"/work/secret-name\",\n syscall: \"lstat\",\n errno: -2,\n code: \"ENOENT\"\n")) + .toBe('filesystem error ENOENT during lstat'); + expect(preparedExportRejection('error: prepared /work/secret-name\n')).toBe('unrecognized helper failure'); + expect(preparedExportRejection('error: prepared tree contains a FIFO, socket, device, or other special object\n')).toBe('prepared tree contains a FIFO, socket, device, or other special object'); + expect(preparedExportRejection('')).toBe('unrecognized helper failure'); + }); }); diff --git a/test/cso-preparation-container.test.ts b/test/cso-preparation-container.test.ts index 1a3e44cf7..eb8b42581 100644 --- a/test/cso-preparation-container.test.ts +++ b/test/cso-preparation-container.test.ts @@ -5,7 +5,7 @@ import * as path from 'node:path'; import { spawnSync } from 'node:child_process'; import { createPreparedExport, recordNpmArchiveEntry } from '../lib/cso/preparation-container'; import { materializePreparedExport } from '../lib/cso/preparation-docker'; -import { validateSingleContainerProcessOutput } from '../lib/cso/docker'; +import { heldApplicationReady, validateSingleContainerProcessOutput } from '../lib/cso/docker'; const roots: string[] = []; function temporary(): string { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'cso-prepared-export-')); roots.push(root); return root; } @@ -60,4 +60,11 @@ describe('CSO preparation container archive limits', () => { expect(() => validateSingleContainerProcessOutput('PID\n123\n456\n')).toThrow('background process'); expect(() => validateSingleContainerProcessOutput('PID COMMAND\n123 bun\n')).toThrow('background process'); }); + + test('treats a held application as ready only after run-app has finished copying its source', () => { + expect(heldApplicationReady('PID COMMAND\n273296 sleep\n')).toBe(true); + expect(heldApplicationReady('PID COMMAND\n1 sh\n2 cp\n')).toBe(false); + expect(heldApplicationReady('PID COMMAND\n1 sh\n2 sleep\n')).toBe(false); + expect(heldApplicationReady('PID COMMAND\n1 sh\n')).toBe(false); + }); }); diff --git a/test/cso-preparation-executor.test.ts b/test/cso-preparation-executor.test.ts index d5e90ff66..f6d63cb7b 100644 --- a/test/cso-preparation-executor.test.ts +++ b/test/cso-preparation-executor.test.ts @@ -366,6 +366,43 @@ describe('CSO constrained dependency preparation executor', () => { } finally { await broker.close(); } }); + // Rails acquires ~70 gems in sequence. Under Bun, stream.pipe left each finished + // tunnel's upstream open, so the broker hit its 64-connection limit mid-acquisition. + test('finished registry tunnels release both sockets across more tunnels than the connection limit', async () => { + const base = root('cso-broker-release-'), socketPath = path.join(base, 'registry.sock'); + const registry = net.createServer(upstream => { upstream.on('data', chunk => upstream.write(chunk)); upstream.on('end', () => upstream.end()); }); + await new Promise(resolve => registry.listen(0, '127.0.0.1', resolve)); + const registryPort = (registry.address() as net.AddressInfo).port, dial = net.connect; + const connect = spyOn(net, 'connect').mockImplementation(((options: any, ...rest: any[]) => + dial({ host: '127.0.0.1', port: options?.port === 443 ? registryPort : options?.port }, ...rest)) as typeof net.connect); + const broker = new RegistryEgressBroker(socketPath, ['rubygems.org'], Date.now() + 60_000, 64 * 1024 * 1024, + () => ({ promise: Promise.resolve([{ address: '151.101.1.227', family: 4 as const }]), cancel: () => {} })); + await broker.start(); + try { + for (let index = 0; index < 70; index++) { + const echoed = await new Promise((resolveEcho, reject) => { + const client = net.createConnection({ path: socketPath }); let output = ''; + client.setTimeout(5_000, () => client.destroy(new Error(`tunnel ${index} stalled`))); + client.once('connect', () => client.write('CONNECT rubygems.org:443 HTTP/1.1\r\nHost: rubygems.org:443\r\n\r\n')); + client.on('data', chunk => { + output += chunk.toString(); + if (output === 'HTTP/1.1 200 Connection Established\r\n\r\n') { client.write(`gem-${index}`); return; } + if (output.endsWith(`gem-${index}`)) client.end(); + }); + client.once('close', () => resolveEcho(output)); client.once('error', reject); + }); + expect(echoed).toEndWith(`gem-${index}`); + } + const deadline = Date.now() + 2_000; + while ((broker as any).sockets.size && Date.now() < deadline) await Bun.sleep(10); + expect((broker as any).sockets.size).toBe(0); + expect(() => broker.assertClean()).not.toThrow(); + expect([...broker.contactedHosts]).toEqual(['rubygems.org']); + } finally { + connect.mockRestore(); await broker.close(); await new Promise(resolve => registry.close(() => resolve())); + } + }); + test('registry address classification rejects mapped, translated, private, and unspecified destinations', () => { for (const address of ['127.0.0.1', '169.254.169.254', '10.2.3.4', '100.64.0.1']) expect(isBlockedRegistryAddress(address, 4)).toBe(true); diff --git a/test/cso-preparation.test.ts b/test/cso-preparation.test.ts index 1f663b970..6cc8fc1b8 100644 --- a/test/cso-preparation.test.ts +++ b/test/cso-preparation.test.ts @@ -1,5 +1,5 @@ import { afterEach, describe, expect, test } from 'bun:test'; -import { mkdtempSync, mkdirSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { mkdtempSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; import { inspectPreparation, railsTestConfiguration } from '../lib/cso/preparation'; @@ -59,6 +59,17 @@ describe('CSO inert Node preparation', () => { const plan = inspectPreparation(fixture(nodeFiles(1))); expect(plan.prerequisites[0].code).toBe('UNSUPPORTED_LOCK'); expect(plan.acquisition).toEqual([]); expect(plan.metadata).toEqual([]); }); + // npm 11 refuses one file loaded as both user and global config ("double-loading config"). + test('every npm command loads distinct empty user and global configs', () => { + const plan = inspectPreparation(fixture(nodeFiles(3))); + const npm = [...plan.acquisition, ...plan.offline].filter(command => command.executable === '/usr/local/bin/npm'); + expect(npm.length).toBeGreaterThanOrEqual(3); + for (const command of npm) { + const value = (flag: string) => command.args[command.args.indexOf(flag) + 1]; + expect([value('--userconfig'), value('--globalconfig')]).toEqual(['/opt/cso/empty-config', '/opt/cso/empty-globalconfig']); + } + expect(readFileSync(join(import.meta.dir, '../lib/cso/images/node.Dockerfile'), 'utf8')).toContain('touch /opt/cso/empty-config /opt/cso/empty-globalconfig'); + }); for (const url of [CREDENTIAL_ARCHIVE_URL, 'http://registry.npmjs.org/a.tgz', 'https://registry.npmjs.org.evil.invalid/a.tgz', 'https://127.0.0.1/a.tgz', 'git+ssh://github.com/a/b', 'https://registry.npmjs.org/a.tgz?token=secret']) test(`rejects non-public archive ${url.split('@').at(-1)}`, () => { const files = nodeFiles(); files['package-lock.json'].packages['node_modules/cookie'].resolved = url; const plan = inspectPreparation(fixture(files)); expect(plan.status).toBe('prerequisites'); expect(plan.acquisition).toEqual([]); @@ -159,6 +170,12 @@ describe('CSO inert Rails preparation', () => { const plan = inspectPreparation(fixture({ 'Gemfile.lock': gemLock, 'Gemfile': 'system("touch /tmp/CSO_UNSAFE")\nsource "https://rubygems.org"' })); expect(plan.status).toBe('ready'); expect(plan.stack).toBe('rails'); expect(plan.metadata.some(m => m.path === 'Gemfile')).toBe(false); expect(plan.acquisition).toHaveLength(2); expect(plan.acquisition.every(c => c.args[0] === 'fetch')).toBe(true); + // One gem process per platform: each process loads the full RubyGems index once. + for (const command of plan.acquisition) { + const platform = command.args[command.args.indexOf('--platform') + 1]; + expect(command.args.slice(1, command.args.indexOf('--platform'))).toEqual(plan.inputs.filter(input => input.platform === platform).map(input => `${input.name}:${input.version}`)); + expect(command.args).not.toContain('--version'); + } expect(plan.inputs[1].platform).toBe('x86_64-linux-gnu'); expect(plan.inputs[0].integritySource).toBe('registry-on-acquisition'); expect(plan.runtimeRequirements).toEqual({ ruby: '3.3.6', bundler: '2.6.9' }); expect(plan.offline[0].args).toContain('--local'); expect(plan.offline[0].env.BUNDLE_IGNORE_CONFIG).toBe('true'); diff --git a/test/cso-rails-verification.test.ts b/test/cso-rails-verification.test.ts index af0f0876c..fba3d1769 100644 --- a/test/cso-rails-verification.test.ts +++ b/test/cso-rails-verification.test.ts @@ -18,7 +18,7 @@ describe('CSO Rails database verification lifecycle',()=>{ return { anchor:`anchor-${group}`, createContainer:async(spec:any)=>{const id=`${group}-${spec.role}-${++next}`;events.push(`create:${id}:${spec.command.join(' ')}`);if(spec.postgresDatabasePolicy){policies.push(fs.readFileSync(spec.postgresDatabasePolicy,'utf8').trim().split('\n'));expect(fs.statSync(spec.postgresDatabasePolicy).mode&0o777).toBe(0o444);}return id;}, - start:async(id:string)=>{events.push(`start:${id}`);}, + start:async(id:string)=>{events.push(`start:${id}`);},startHeldApplication:async(id:string)=>{events.push(`held:${id}`);}, execCapture:async(id:string,command:string[])=>{events.push(`exec:${id}:${command.join(' ')}`);return{code:0,stdout:'',stderr:''};}, execDetached:async(id:string,command:string[])=>{events.push(`detach:${id}:${command.join(' ')}`);}, startAttach:async(id:string)=>{events.push(`attach:${id}`);return{code:0,output:JSON.stringify({booted:true,legitimate:true,security:'pass',existingTests:false,output:'ok',inputHash:''})};}, @@ -37,6 +37,8 @@ describe('CSO Rails database verification lifecycle',()=>{ expect(policies).toEqual([['cso_primary','cso_queue'],['cso_primary','cso_queue']]); expect(events.filter(event=>event.includes('/opt/cso/postgresql-ready'))).toHaveLength(2); expect(events.filter(event=>event.includes('/usr/local/bin/bundle exec rails db:prepare'))).toHaveLength(6); + for(const [index,event] of events.entries())if(event.includes('exec rails db:prepare'))expect(events.slice(0,index)).toContain(`held:${event.split(':')[1]}`); + expect(events.some(event=>/^start:\d-(app|tests)-/.test(event))).toBe(false); for(const group of [1,2]){ const removeApp=events.findIndex(event=>event===`remove:${group}-app-2`),createFirstTest=events.findIndex(event=>event.startsWith(`create:${group}-tests-4:`)); expect(removeApp).toBeGreaterThan(-1);expect(createFirstTest).toBeGreaterThan(removeApp); @@ -47,11 +49,11 @@ describe('CSO Rails database verification lifecycle',()=>{ const root=fs.mkdtempSync(path.join(os.tmpdir(),'cso-rails-sqlite-'));roots.push(root);for(const name of ['source','work','control'])fs.mkdirSync(path.join(root,name)); const commands:string[]=[];let next=0; const create=spyOn(DockerGroup,'create').mockResolvedValue({ - createContainer:async(spec:any)=>{commands.push(`create:${spec.role}`);return`${spec.role}-${++next}`;},start:async()=>{}, + createContainer:async(spec:any)=>{commands.push(`create:${spec.role}`);return`${spec.role}-${++next}`;},start:async()=>{},startHeldApplication:async(id:string)=>{commands.push(`held:${id}`);}, execCapture:async(_id:string,command:string[])=>{commands.push(command.join(' '));return{code:0,stdout:'',stderr:''};},execDetached:async()=>{}, startAttach:async()=>({code:0,output:JSON.stringify({booted:true,legitimate:true,security:'pass',existingTests:false,output:'ok',inputHash:''})}),removeContainer:async()=>{},cleanup:async()=>{}, } as any); try{const runtime:any={stack:'rails',image:`runtime@sha256:${'a'.repeat(64)}`},request:any={findingId:'d'.repeat(32),port:3456,start:{executable:'/usr/local/bin/bundle',args:['exec','rails','server']},legitimate:[],security:{},fixtures:{},existingTests:[{executable:'/usr/local/bin/bundle',args:['exec','rails','test']}]};await new DockerVerificationExecutor({} as any,'/watchdog',Date.now()+60_000).observe(path.join(root,'source'),'before',request,runtime,runtime,path.join(root,'work'),path.join(root,'control'),{environment:{},database:{adapter:'sqlite',connections:['primary']}});}finally{create.mockRestore();} - expect(commands.some(command=>command==='create:postgres')).toBe(false);expect(commands.filter(command=>command.includes('exec rails db:prepare'))).toHaveLength(2); + expect(commands.some(command=>command==='create:postgres')).toBe(false);expect(commands.filter(command=>command.includes('exec rails db:prepare'))).toHaveLength(2);expect(commands.filter(command=>command.startsWith('held:'))).toHaveLength(2); }); }); diff --git a/test/cso-stack-cold-integration.test.ts b/test/cso-stack-cold-integration.test.ts index 124937d58..89db2822d 100644 --- a/test/cso-stack-cold-integration.test.ts +++ b/test/cso-stack-cold-integration.test.ts @@ -11,8 +11,8 @@ import { type PreparedApplication, type PreparationRuntimeAdmission, type RailsDatabaseSelection, } from '../lib/cso/preparation-executor'; import { inspectPreparation, type CsoStack, type PreparationPlan } from '../lib/cso/preparation'; -import { CSO_HELPER_ABI, type QualifiedRuntime, type RuntimeCatalog } from '../lib/cso/runtime-catalog'; -import { completeRuntimeCatalogFixture } from './helpers/cso-runtime-catalog'; +import { CSO_HELPER_ABI, validateRuntimeCatalog, type QualifiedRuntime, type RuntimeCatalog } from '../lib/cso/runtime-catalog'; +import { completeRuntimeCatalogFixture, installStagedRuntime } from './helpers/cso-runtime-catalog'; import { secureDirectory } from '../lib/cso/state'; import { canonicalStartPlan, canonicalTestPlan, DockerVerificationExecutor } from '../lib/cso/verification'; @@ -82,7 +82,10 @@ import django django.setup() from django.test import Client class ColdStartTest(unittest.TestCase): - def test_dependency_and_control(self): self.assertContains(Client().get("/control"),"CONTROL_OK",status_code=200) + def test_dependency_and_control(self): + response = Client().get("/control") + self.assertEqual(response.status_code, 200) + self.assertIn(b"CONTROL_OK", response.content) `, }); } @@ -102,6 +105,7 @@ function railsSource(directory: string, bundlerVersion: string): void { 'config/database.yml': `default: &default\n pool: 3\ntest:\n primary:\n <<: *default\n adapter: sqlite3\n database: storage/test.sqlite3\n queue:\n <<: *default\n adapter: postgresql\n database: cso_queue\n`, 'app/controllers/application_controller.rb': `class ApplicationController < ActionController::Base;end\n`, 'app/controllers/probe_controller.rb': `class ProbeController < ApplicationController\n def control;render plain:"CONTROL_OK:Rails";end\n def security;render plain:"DENIED",status: :forbidden;end\nend\n`, + 'bin/rails': `#!/usr/bin/env ruby\nAPP_PATH = File.expand_path("../config/application", __dir__)\nrequire_relative "../config/boot"\nrequire "rails/commands"\n`, 'Rakefile': `require_relative "config/application"\nRails.application.load_tasks\n`, 'config.ru': `require_relative "config/environment"\nrun Rails.application\n`, 'db/schema.rb': `ActiveRecord::Schema[8.1].define(version: 1) do\n create_table :cold_records, force: true do |t|\n t.string :name\n end\nend\n`, @@ -163,21 +167,14 @@ beforeAll(async () => { if (!fs.existsSync(watchdog)) throw new Error(`${requestedStack} cold-start qualification requires the compiled CSO watchdog`); endpoint = await dockerEndpoint(secureDirectory(path.join(root, 'docker-home')), { HOME: root, DOCKER_HOST: process.env.DOCKER_HOST ?? 'unix:///var/run/docker.sock' }); catalog = completeRuntimeCatalogFixture(`${requestedStack}-staged-cold`); - const installRuntime = (value: QualifiedRuntime): QualifiedRuntime => { - const runtimeIndex = catalog.runtimes.findIndex(item => item.stack === value.stack && item.platform === value.platform); - const profile = catalog.profiles.find(item => item.stack === value.stack && item.platform === value.platform)!; - const installed = { ...value, id: profile.id }; - catalog.runtimes[runtimeIndex] = installed; - profile.versions = { ...installed.versions }; - return installed; - }; - runtime = installRuntime(qualified(requestedStack as CsoStack, image, versions)); + runtime = installStagedRuntime(catalog, qualified(requestedStack as CsoStack, image, versions)); if (requestedStack === 'rails') { const postgresImage = process.env.GSTACK_CSO_TEST_POSTGRES_IMAGE; const postgresVersion = process.env.GSTACK_CSO_TEST_POSTGRES_VERSION; if (!postgresImage || !/^\d+\.\d+(?:\.\d+)?$/.test(postgresVersion ?? '')) throw new Error('Rails cold-start qualification requires a staged PostgreSQL digest and exact version'); - installRuntime(qualified('postgresql', postgresImage, { postgresql: postgresVersion! })); + installStagedRuntime(catalog, qualified('postgresql', postgresImage, { postgresql: postgresVersion! })); } + validateRuntimeCatalog(catalog); }); afterAll(() => { if (root) fs.rmSync(root, { recursive: true, force: true }); delete process.env.GSTACK_HOME; }); diff --git a/test/helpers/cso-qualified-cli.ts b/test/helpers/cso-qualified-cli.ts index 1c80fcc1c..543c8c7be 100644 --- a/test/helpers/cso-qualified-cli.ts +++ b/test/helpers/cso-qualified-cli.ts @@ -1,9 +1,8 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { dispatchCsoCommand, type CsoCliDependencies } from '../../lib/cso/cli'; -import { canonical, sha256 } from '../../lib/cso/contracts'; import { validateRuntimeCatalog, type QualifiedRuntime, type RuntimeCatalog, type RuntimePlatform } from '../../lib/cso/runtime-catalog'; -import { completeRuntimeCatalogFixture } from './cso-runtime-catalog'; +import { completeRuntimeCatalogFixture, installStagedRuntime } from './cso-runtime-catalog'; export interface QualifiedCsoCli { readonly catalog: RuntimeCatalog; @@ -31,17 +30,9 @@ export function qualifiedNodeCli(options: { } const catalog = completeRuntimeCatalogFixture('node-lifecycle-fixture'); - const runtimeIndex = catalog.runtimes.findIndex(item => item.stack === 'node' && item.platform === options.platform); - const profile = catalog.profiles.find(item => item.stack === 'node' && item.platform === options.platform); - if (runtimeIndex < 0 || !profile || !catalog.promotion) throw new Error('Node lifecycle catalog fixture is incomplete'); - const runtime: QualifiedRuntime = { - ...catalog.runtimes[runtimeIndex], - image: options.image, - versions: { ...options.versions }, - }; - catalog.runtimes[runtimeIndex] = runtime; - profile.versions = { ...runtime.versions }; - catalog.promotion.evidenceDigest = `sha256:${sha256(canonical(catalog.runtimes))}`; + const fixture = catalog.runtimes.find(item => item.stack === 'node' && item.platform === options.platform); + if (!fixture) throw new Error('Node lifecycle catalog fixture is incomplete'); + const runtime = installStagedRuntime(catalog, { ...fixture, image: options.image, versions: options.versions }); validateRuntimeCatalog(catalog); const dependencies: CsoCliDependencies = Object.freeze({ diff --git a/test/helpers/cso-runtime-catalog.ts b/test/helpers/cso-runtime-catalog.ts index c7bdd90bf..a480399f9 100644 --- a/test/helpers/cso-runtime-catalog.ts +++ b/test/helpers/cso-runtime-catalog.ts @@ -107,3 +107,19 @@ export function completeRuntimeCatalogFixture( runtimes, }; } + +/** + * Put a staged runtime into its stack/platform slot under the reviewed profile + * id, as promotion does, and re-bind the promotion evidence digest the + * validator recomputes from the runtime matrix. + */ +export function installStagedRuntime(catalog: RuntimeCatalog, runtime: QualifiedRuntime): QualifiedRuntime { + const index = catalog.runtimes.findIndex(item => item.stack === runtime.stack && item.platform === runtime.platform); + const profile = catalog.profiles.find(item => item.stack === runtime.stack && item.platform === runtime.platform); + if (index < 0 || !profile || !catalog.promotion) throw new Error(`Runtime catalog fixture lacks ${runtime.stack} on ${runtime.platform}`); + const installed = { ...runtime, id: profile.id, versions: { ...runtime.versions } }; + catalog.runtimes[index] = installed; + profile.versions = { ...installed.versions }; + catalog.promotion.evidenceDigest = `sha256:${sha256(canonical(catalog.runtimes))}`; + return installed; +} diff --git a/test/skill-e2e-cso.test.ts b/test/skill-e2e-cso.test.ts index ad702779f..e01473e50 100644 --- a/test/skill-e2e-cso.test.ts +++ b/test/skill-e2e-cso.test.ts @@ -178,7 +178,7 @@ http.createServer((req, res) => { expect(report.policy.diff).toBe(true); expect(report.policy.base).toBe('main'); expect(report.source.baseCommit).toBe(git(f.repo, 'rev-parse', 'main').trim()); - expect(report.findings.some(item => item.location.path === 'webhook.js' && /signature|authenticat|forg/i.test(`${item.rootCause} ${item.impact}`))).toBe(true); + expect(report.findings.some(item => item.location.path === 'webhook.js' && /signature|authentic|forg|spoof|hmac/i.test(`${item.title} ${item.rootCause} ${item.impact}`))).toBe(true); expect(report.findings.every(item => item.location.path === 'webhook.js')).toBe(true); passed = result.browseErrors.length === 0; } finally {