fix(cso): unblock native runtime qualification (v1.91.28.0)

Fixes the qualify-native failures from main run 37345275258:
- re-bind promotion evidence for staged runtimes in cold-start tests
- give npm a distinct empty global config (npm 11 refuses double-loading)
- export /archives and /work through tmpfs-backed local volumes (docker cp
  cannot read tmpfs), with exec on /work
- keep the offline seed process alive
- end broker/forwarder tunnels explicitly under Bun
- batch gem fetch per platform
- wait for run-app's source copy before Rails db:prepare and tests
- fix the Python cold fixture and add bin/rails to the Rails cold fixture

CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## [1.91.28.0] - 2026-10-05
+
+**Native runtime qualification can pass for Node, Bun and Python. Rails gets past every earlier failure, but building its native gems from source still takes longer than the five-minute command limit.**
+
+Main run 37345275258 of `cso-runtime-images.yml` staged every image, but eight `qualify-native` rows failed. Two causes showed up in the logs, and six more were hiding behind them. All eight are fixed below. Each was reproduced on a local Docker daemon against freshly built images.
+
+- **Promotion evidence is recomputed for injected runtimes.** The cold-start test put the staged runtimes into its test catalog without re-binding `promotion.evidenceDigest`, so Bun, Python and Rails failed with `RUNTIME_PROMOTION_EVIDENCE_MISMATCH`. A shared `installStagedRuntime` helper re-binds the digest, and the test now validates the catalog it builds.
+- **npm dependency acquisition works.** npm 11 refuses to load the same file as both its user and global config. The preparation commands passed `/opt/cso/empty-config` for both, so every Node acquisition failed. The global config now points at its own empty file, `/opt/cso/empty-globalconfig`, which the Node image creates.
+- **Prepared output can leave its container.** `docker cp` cannot read tmpfs mounts. The acquisition `/archives` and offline `/work` directories are now Docker local volumes backed by tmpfs, with the same size, owner, mode and `nosuid`/`nodev` options as before (`/archives` stays `noexec`). The runner refuses to start a container when Docker did not keep those options.
+- **The offline seed stays up.** The in-image `seed` command exited while the offline commands were still to run, because it was waiting on a promise that nothing kept alive. It now holds a timer until the container stops.
+- **Prepared trees can execute.** Docker's tmpfs default is `noexec`, so virtualenv Python and binstubs under `/work` exited with status 126. `/work` now mounts `exec` explicitly.
+- **The registry broker releases its connections.** Under Bun, `pipe()` never closed the upstream socket. A Rails acquisition opens about 70 tunnels, which filled the broker's 64-connection limit and timed out. The broker and the in-container forwarder now forward end and close explicitly, and a free test opens 70 tunnels.
+- **Gem acquisition is batched.** Each `gem fetch` process spends about 11 seconds loading the RubyGems index, so acquisition now runs one `gem fetch` per platform for all exact gems.
+- **Rails commands wait for the source copy.** `db:prepare` ran while `run-app` was still copying the prepared tree, and failed with "Could not locate Gemfile". Rails application and test containers now start through `startHeldApplication`, which waits until the copy has finished.
+- **Test fixtures.** The Python cold fixture called a method that `unittest.TestCase` does not have, and the Rails cold fixture had no `bin/rails`, so `rails server` printed the `rails new` help instead of booting.
+
+The images change (`/opt/cso/preparation` and the Node recipe), so qualification needs a fresh staging run on main. The Rails fixture's lockfile lists only the `ruby` platform, so `bundle install --local` compiles nokogiri, sqlite3 and pg. At the app container's 0.85 CPU share that takes about 385 seconds, which is longer than the 300-second limit on a single command. Rails stays red until that limit or the CPU share changes.
+
 ## [1.91.27.0] - 2026-10-05
 
 **The first protected-main runtime staging run can finish, and a qualified scanner catalog can ship.**

VERSION

@@ -1 +1 @@
-1.91.27.0
+1.91.28.0

agents-digest/gstack-AGENTS.md

@@ -1,4 +1,4 @@
-# gstack digest v1.91.27.0 — regenerate/re-copy after upgrading gstack
+# gstack digest v1.91.28.0 — regenerate/re-copy after upgrading gstack
 
 Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
 for agent hosts without a full skill install. The full skills add workflows,

docs/cso-release-qualification.md

@@ -94,6 +94,19 @@ and once with two databases in a fresh staged PostgreSQL sidecar. The Rails
 fixture builds the source-platform `sqlite3` and `pg` native gems only in the
 offline execution phase.
 
+Preparation output leaves its containers through `docker cp`, which cannot read
+a tmpfs mount. The acquisition `/archives` directory and the offline `/work`
+directory are therefore Docker local volumes backed by bounded tmpfs, created
+per container with private mode, the runtime uid/gid, `nosuid`, and `nodev`
+(`/archives` is also `noexec`; `/work` must stay executable for virtualenv and
+binstub launchers). The runner inspects each container and refuses to start it
+if Docker did not preserve those options. npm's user and global configs point
+at two different empty files, because npm refuses to load one file in both
+roles. Rails application and test containers hold on `sleep` after `run-app`
+copies the source; `db:prepare` starts only once that copy has finished.
+Changes to `/opt/cso/preparation` or an image recipe need a new staging run
+before native qualification can pass.
+
 For hashed Python requirements resolved by pip, the broker records the requested
 index URL, allowed contacted hosts, and verified wheel hash. Pip's internal
 `files.pythonhosted.org` response URL is opaque to that TLS boundary, so

lib/cso/docker.ts

@@ -319,6 +319,12 @@ export interface ContainerSpec {
   workTmpfsBytes?: number;
   temporaryTmpfsBytes?: number;
   readonlyArchiveDirectory?: string;
+  /**
+   * The prepared-tree export is read back with docker cp, which cannot see a
+   * tmpfs mount. This keeps /work in memory as a size-capped tmpfs-backed local
+   * volume instead; removal with --volumes discards it like the container.
+   */
+  exportableWork?: true;
   registrySocket?: string;
   /** Non-secret, fixed-user-readable PostgreSQL database-name policy. */
   postgresDatabasePolicy?: string;
@@ -397,6 +403,13 @@ export function validateSingleContainerProcessOutput(output: string): void {
       'Offline lifecycle left a background process; prepared output was withheld',
     );
 }
+export function heldApplicationReady(output: string): boolean {
+  const lines = output
+    .split('\n')
+    .map((line) => line.trim())
+    .filter(Boolean);
+  return lines.length === 2 && /^PID\s+COMMAND$/.test(lines[0]) && /^\d+\s+sleep$/.test(lines[1]);
+}
 export class DockerGroup {
   private ids: { role: Role; id: string }[] = [];
   private roles: Role[] = ['anchor'];
@@ -609,6 +622,16 @@ export class DockerGroup {
       );
     const uid = spec.role === 'postgres' ? 10001 : (hostUid as number),
       gid = spec.role === 'postgres' ? 10001 : (hostGid as number);
+    // docker cp reads volumes but not tmpfs mounts, so export sources use
+    // tmpfs-backed local volumes with the same size, owner and flag policy.
+    const expectedVolumes = new Map<string, string>();
+    const memoryVolume = (destination: string, options: string): string[] => {
+      expectedVolumes.set(destination, options);
+      return [
+        '--mount',
+        `type=volume,dst=${destination},volume-driver=local,volume-opt=type=tmpfs,volume-opt=device=tmpfs,"volume-opt=o=${options}"`,
+      ];
+    };
     const args = [
       'create',
       '--pull=never',
@@ -638,14 +661,16 @@ export class DockerGroup {
       `${CONTAINER_SHM_BYTES}b`,
       '--tmpfs',
       `/tmp:rw,noexec,nosuid,nodev,size=${tmpBytes},mode=1777`,
-      '--tmpfs',
-      `/work:rw,nosuid,nodev,size=${workBytes},mode=700,uid=${uid},gid=${gid}`,
+      ...(spec.exportableWork
+        ? memoryVolume('/work', `size=${workBytes},mode=700,uid=${uid},gid=${gid},nosuid,nodev`)
+        : // Docker's tmpfs default is noexec; prepared venvs and native extensions execute from /work.
+          ['--tmpfs', `/work:rw,exec,nosuid,nodev,size=${workBytes},mode=700,uid=${uid},gid=${gid}`]),
       '--network',
       joinAnchor ? `container:${this.anchor}` : 'none',
       '--platform',
       process.arch === 'arm64' ? 'linux/arm64' : 'linux/amd64',
     ];
-    const expectedTmpfs = new Set(['/tmp', '/work']),
+    const expectedTmpfs = new Set(spec.exportableWork ? ['/tmp'] : ['/tmp', '/work']),
       expectedMounts = new Set<string>();
     for (const [k, v] of Object.entries(spec.env ?? {})) {
       if (!/^[A-Z][A-Z0-9_]{0,63}$/.test(k) || v.includes('\0'))
@@ -749,10 +774,11 @@ export class DockerGroup {
           'Preparation archive tmpfs exceeds the 2 GiB group storage policy',
         );
       args.push(
-        '--tmpfs',
-        `/archives:rw,noexec,nosuid,nodev,size=${spec.archiveTmpfsBytes},mode=700,uid=${uid},gid=${gid}`,
+        ...memoryVolume(
+          '/archives',
+          `size=${spec.archiveTmpfsBytes},mode=700,uid=${uid},gid=${gid},noexec,nosuid,nodev`,
+        ),
       );
-      expectedTmpfs.add('/archives');
     }
     if (spec.readonlyArchiveDirectory) directoryMount(spec.readonlyArchiveDirectory, '/archives', true);
     if (spec.registrySocket) {
@@ -784,7 +810,25 @@ export class DockerGroup {
       throw new CsoError('ISOLATION_FAILED', 'Docker did not return valid admitted-container configuration');
     }
     const hostConfig = inspectedContainer?.HostConfig,
-      mounts = inspectedContainer?.Mounts;
+      mounts = inspectedContainer?.Mounts,
+      volumes = (Array.isArray(hostConfig?.Mounts) ? hostConfig.Mounts : []).filter(
+        (mount: any) => mount?.Type === 'volume',
+      );
+    if (
+      volumes.length !== expectedVolumes.size ||
+      volumes.some((mount: any) => {
+        const driver = mount.VolumeOptions?.DriverConfig;
+        return (
+          !expectedVolumes.has(mount.Target) ||
+          mount.Source !== undefined ||
+          mount.ReadOnly === true ||
+          driver?.Name !== 'local' ||
+          canonical(driver?.Options ?? null) !==
+            canonical({ device: 'tmpfs', o: expectedVolumes.get(mount.Target), type: 'tmpfs' })
+        );
+      })
+    )
+      throw new CsoError('ISOLATION_FAILED', 'Docker did not preserve the bounded in-memory export volumes');
     if (
       hostConfig?.ReadonlyRootfs !== true ||
       hostConfig?.ShmSize !== CONTAINER_SHM_BYTES ||
@@ -796,10 +840,12 @@ export class DockerGroup {
           !mount ||
           !(
             (mount.Type === 'bind' && expectedMounts.has(mount.Destination)) ||
-            (mount.Type === 'tmpfs' && expectedTmpfs.has(mount.Destination))
+            (mount.Type === 'tmpfs' && expectedTmpfs.has(mount.Destination)) ||
+            (mount.Type === 'volume' && mount.Driver === 'local' && expectedVolumes.has(mount.Destination))
           ),
       ) ||
-      mounts.filter((mount: any) => mount?.Type === 'bind').length !== expectedMounts.size
+      mounts.filter((mount: any) => mount?.Type === 'bind').length !== expectedMounts.size ||
+      mounts.filter((mount: any) => mount?.Type === 'volume').length !== expectedVolumes.size
     )
       throw new CsoError('ISOLATION_FAILED', 'Docker did not preserve the bounded writable-storage policy');
     // Durable journal publication precedes in-memory admission. If this write
@@ -897,6 +943,18 @@ export class DockerGroup {
       throw new CsoError('REDACTION_FAILED', 'Aggregate reproduction output exceeded 1 MiB and was withheld');
     return { code: r.code, output: r.stdout + r.stderr };
   }
+  async startHeldApplication(id: string): Promise<void> {
+    await this.start(id);
+    for (;;) {
+      if (heldApplicationReady(await this.docker(['top', id, '-eo', 'pid,comm'], 64 * 1024))) return;
+      if (Date.now() >= this.deadline)
+        throw new CsoError(
+          'DEADLINE',
+          'Application source copy did not finish within the reproduction deadline',
+        );
+      await new Promise((resolve) => setTimeout(resolve, 100));
+    }
+  }
   async assertOnlyInitProcess(id: string): Promise<void> {
     if (!this.ids.some((item) => item.id === id))
       throw new CsoError(

lib/cso/images/node.Dockerfile

@@ -1,7 +1,7 @@
 # BASE_IMAGE must be a reviewed, provenance-verified node image@sha256 digest.
 ARG BASE_IMAGE
 FROM ${BASE_IMAGE}
-RUN mkdir -p /opt/cso /work /metadata /archives /source /policy /fixtures && touch /opt/cso/empty-config \
+RUN mkdir -p /opt/cso /work /metadata /archives /source /policy /fixtures && touch /opt/cso/empty-config /opt/cso/empty-globalconfig \
     && chown 10001:10001 /work /metadata /archives
 COPY --chmod=0555 entrypoint /opt/cso/entrypoint
 COPY --chmod=0555 run-app /opt/cso/run-app

lib/cso/preparation-container.ts

@@ -681,8 +681,14 @@ async function forwarder(socketPath: string, listen: string): Promise<void> {
       client.destroy();
       upstream.destroy();
     });
-    client.pipe(upstream);
-    upstream.pipe(client);
+    // Bun's stream.pipe does not end the destination socket; forward each
+    // side's end and close so finished registry tunnels release the broker.
+    client.once('end', () => upstream.end());
+    upstream.once('end', () => client.end());
+    client.once('close', () => upstream.destroy());
+    upstream.once('close', () => client.destroy());
+    client.pipe(upstream, { end: false });
+    upstream.pipe(client, { end: false });
   });
   server.listen(18443, '127.0.0.1');
   await once(server, 'listening');
@@ -1054,7 +1060,7 @@ async function seed(policyPath: string): Promise<void> {
           '--userconfig',
           '/opt/cso/empty-config',
           '--globalconfig',
-          '/opt/cso/empty-config',
+          '/opt/cso/empty-globalconfig',
         ],
         {
           cwd: '/work',
@@ -1078,7 +1084,11 @@ async function seed(policyPath: string): Promise<void> {
     mode: 0o400,
     flag: 'wx',
   });
-  await new Promise<void>(() => {});
+  // A pending promise alone lets the process exit and the container stop before
+  // the offline commands run; a timer holds it until the group removes it.
+  await new Promise<void>(() => {
+    setInterval(() => {}, 2 ** 31 - 1);
+  });
 }
 
 function ready(): void {

lib/cso/preparation-docker.ts

@@ -640,6 +640,13 @@ export class RegistryEgressBroker {
       this.sockets.add(upstream);
       upstream.setTimeout(Math.max(1, Math.min(30_000, this.deadline - Date.now())));
       upstream.once('close', () => this.sockets.delete(upstream));
+      // Bun's stream.pipe does not end the destination socket, so a finished
+      // tunnel would otherwise keep its upstream open and fill the broker's
+      // connection limit. Each side's end and close are forwarded explicitly.
+      socket.once('end', () => upstream.end());
+      upstream.once('end', () => socket.end());
+      socket.once('close', () => upstream.destroy());
+      upstream.once('close', () => socket.destroy());
       upstream.once('error', () => {
         if (!this.closing) this.violation ??= 'Registry connection failed after DNS pinning';
         socket.destroy();
@@ -663,8 +670,8 @@ export class RegistryEgressBroker {
         };
         socket.on('data', count);
         upstream.on('data', count);
-        socket.pipe(upstream);
-        upstream.pipe(socket);
+        socket.pipe(upstream, { end: false });
+        upstream.pipe(socket, { end: false });
         this.contactedHosts.add(host);
         socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
         if (remainder.length && addBytes(remainder.length)) upstream.write(remainder);
@@ -1557,6 +1564,7 @@ export class DockerPreparationSandboxRunner implements PreparationSandboxRunner
         readonlyFiles: [{ host: offlinePolicy, container: '/policy/offline.json' }],
         readonlyMetadata: metadata,
         readonlyArchiveDirectory: archives,
+        exportableWork: true,
         command: ['/opt/cso/run-app', '/opt/cso/preparation', 'seed', '/policy/offline.json'],
       });
       await group.start(app);

lib/cso/preparation.ts

@@ -336,7 +336,7 @@ function inspectNode(root: string, plan: PreparationPlan) {
     '--userconfig',
     '/opt/cso/empty-config',
     '--globalconfig',
-    '/opt/cso/empty-config',
+    '/opt/cso/empty-globalconfig',
   ];
   const offline = [
     'ci',
@@ -348,7 +348,7 @@ function inspectNode(root: string, plan: PreparationPlan) {
     '--userconfig',
     '/opt/cso/empty-config',
     '--globalconfig',
-    '/opt/cso/empty-config',
+    '/opt/cso/empty-globalconfig',
   ];
   plan.acquisition.push(
     command('/usr/local/bin/npm', [...acquisition, '--registry', 'https://registry.npmjs.org'], '/metadata', {
@@ -369,7 +369,7 @@ function inspectNode(root: string, plan: PreparationPlan) {
         '--userconfig',
         '/opt/cso/empty-config',
         '--globalconfig',
-        '/opt/cso/empty-config',
+        '/opt/cso/empty-globalconfig',
       ],
       '/work',
     ),
@@ -988,17 +988,22 @@ function inspectRails(root: string, plan: PreparationPlan) {
       input.integrity = hash;
       input.integritySource = 'lock';
     }
-    // gem fetch downloads without evaluating a Gemfile/gemspec or building extensions.
+  }
+  // gem fetch downloads without evaluating a Gemfile/gemspec or building extensions.
+  // Each process loads the full RubyGems index (~11 s at the app role's CPU share),
+  // so all exact NAME:VERSION gems of one platform are fetched in one process.
+  const platforms = [...new Set(plan.inputs.map((input) => input.platform!))];
+  for (const platform of platforms) {
     plan.acquisition.push(
       command(
         '/usr/local/bin/gem',
         [
           'fetch',
-          input.name,
-          '--version',
-          input.version,
+          ...plan.inputs
+            .filter((input) => input.platform === platform)
+            .map((input) => `${input.name}:${input.version}`),
           '--platform',
-          input.platform!,
+          platform,
           '--clear-sources',
           '--source',
           'https://rubygems.org',

lib/cso/scanner-executor.ts

@@ -798,7 +798,7 @@ export async function createDockerScannerRunner(context: ScannerRunnerContext):
               ? ['/opt/cso/run-app', '/bin/sleep', '2147483647']
               : ['/opt/cso/run-app', api.start.executable, ...api.start.args],
           });
-          await group!.start(app);
+          await (rails ? group!.startHeldApplication(app) : group!.start(app));
           if (rails) {
             const clean = [
               '/usr/bin/env',

lib/cso/verification.ts

@@ -1718,7 +1718,7 @@ export class DockerVerificationExecutor implements VerificationExecutor {
           command: ['/opt/cso/run-app', '/bin/sleep', '2147483647'],
           readonlyDirectories: [{ host: fixtures, container: '/fixtures' }],
         });
-        await group.start(app);
+        await group.startHeldApplication(app);
         await dbPrepare(app);
         await group.execDetached(app, [request.start.executable, ...request.start.args]);
       } else {
@@ -1784,7 +1784,7 @@ export class DockerVerificationExecutor implements VerificationExecutor {
         });
         let testResult: { code: number; output: string };
         if (rails) {
-          await group.start(testId);
+          await group.startHeldApplication(testId);
           await dbPrepare(testId);
           const result = await group.execCapture(testId, cleanCommand([test.executable, ...test.args]), {
             workdir: '/work',

package.json

@@ -1,6 +1,6 @@
 {
   "name": "gstack",
-  "version": "1.91.27",
+  "version": "1.91.28",
   "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
   "license": "MIT",
   "type": "module",

test/cso-docker-mounts.test.ts

@@ -22,6 +22,7 @@ function fixture() {
   restores.push(() => listener.stop(true));
   const image = `sha256:${'a'.repeat(64)}`, id = 'b'.repeat(64), calls: string[][] = [];
   let createArgs: string[] = [];
+  const inspectHooks: Array<(value: any) => void> = [];
   const group = new (DockerGroup as any)({}, 'mount-regression', root, Date.now() + 60_000, {}) as DockerGroup;
   if (process.getuid!() === 0) {
     const uid = spyOn(process, 'getuid').mockReturnValue(1001);
@@ -43,17 +44,25 @@ function fixture() {
     if (args[0] === 'create') { createArgs = args; return id; }
     if (args[0] === 'inspect') {
       const tmpfs = createArgs.flatMap((arg, index) => arg === '--tmpfs' ? [createArgs[index + 1].split(':')[0]] : []);
-      const binds = createArgs.flatMap((arg, index) => arg === '--mount' ? [createArgs[index + 1]] : []);
-      return JSON.stringify({
-        HostConfig: { ReadonlyRootfs: true, ShmSize: CONTAINER_SHM_BYTES, Tmpfs: Object.fromEntries(tmpfs.map(target => [target, 'rw'])) },
+      const mounts = createArgs.flatMap((arg, index) => arg === '--mount' ? [createArgs[index + 1]] : []);
+      const binds = mounts.filter(mount => !mount.startsWith('type=volume,'));
+      const volumes = mounts.filter(mount => mount.startsWith('type=volume,')).map(mount => ({
+        target: /dst=([^,]+)/.exec(mount)![1], o: /"volume-opt=o=([^"]+)"/.exec(mount)![1],
+      }));
+      const value = {
+        HostConfig: { ReadonlyRootfs: true, ShmSize: CONTAINER_SHM_BYTES, Tmpfs: Object.fromEntries(tmpfs.map(target => [target, 'rw'])),
+          Mounts: [...binds.map(() => ({ Type: 'bind' })), ...volumes.map(volume => ({ Type: 'volume', Target: volume.target,
+            VolumeOptions: { DriverConfig: { Name: 'local', Options: { device: 'tmpfs', o: volume.o, type: 'tmpfs' } } } }))] },
         Mounts: [...tmpfs.map(Destination => ({ Type: 'tmpfs', Destination })), ...binds.map(mount => ({
           Type: 'bind', Destination: mount.split(',').find(part => part.startsWith('dst='))!.slice(4),
-        }))],
-      });
+        })), ...volumes.map(volume => ({ Type: 'volume', Destination: volume.target, Driver: 'local' }))],
+      };
+      for (const hook of inspectHooks) hook(value);
+      return JSON.stringify(value);
     }
     throw new Error(`Unexpected Docker call: ${args.join(' ')}`);
   };
-  return { root, directory, file, socket, group, image, id, calls };
+  return { root, directory, file, socket, group, image, id, calls, inspectHooks };
 }
 
 describe.skipIf(process.platform === 'win32')('CSO Docker nonrecursive bind mounts', () => {
@@ -106,4 +115,30 @@ describe.skipIf(process.platform === 'win32')('CSO Docker nonrecursive bind moun
     expect(f.calls.some(call => call[0] === 'create')).toBe(false);
     expect(fs.existsSync(path.join(f.root, 'resources.journal'))).toBe(false);
   });
+
+  // docker cp cannot read tmpfs mounts, so both export sources are tmpfs-backed local volumes.
+  test.each([
+    ['acquisition archives', '/archives', { archiveTmpfsBytes: 4096 }, 'noexec,nosuid,nodev'],
+    ['prepared-tree work', '/work', { exportableWork: true as const }, 'nosuid,nodev'],
+  ])('%s export from a bounded in-memory volume docker cp can read', async (_name, target, spec, flags) => {
+    const f = fixture();
+    expect(await f.group.createContainer({ role: 'app', image: f.image, command: ['/bin/sleep', '1'], ...(spec as Partial<ContainerSpec>) })).toBe(f.id);
+    const args = f.calls.find(call => call[0] === 'create')!;
+    const volume = args.flatMap((arg, index) => arg === '--mount' && args[index + 1].startsWith('type=volume,') ? [args[index + 1]] : []);
+    expect(volume).toHaveLength(1);
+    expect(volume[0]).toStartWith(`type=volume,dst=${target},volume-driver=local,volume-opt=type=tmpfs,volume-opt=device=tmpfs,"volume-opt=o=size=`);
+    expect(volume[0]).toEndWith(`,mode=700,uid=${process.getuid!()},gid=${process.getgid!()},${flags}"`);
+    expect(args.flatMap((arg, index) => arg === '--tmpfs' ? [args[index + 1].split(':')[0]] : [])).not.toContain(target);
+  });
+
+  test.each([
+    ['dropped tmpfs driver options', (value: any) => { value.HostConfig.Mounts.at(-1).VolumeOptions.DriverConfig.Options.type = 'none'; }],
+    ['a host-backed volume source', (value: any) => { value.HostConfig.Mounts.at(-1).Source = '/var/tmp/export'; }],
+    ['an extra daemon volume', (value: any) => { value.Mounts.push({ Type: 'volume', Destination: '/data', Driver: 'local' }); }],
+    ['a missing volume mount', (value: any) => { value.HostConfig.Mounts.pop(); }],
+  ])('export volumes fail closed on %s', async (_name, mutate) => {
+    const f = fixture(); f.inspectHooks.push(mutate);
+    await expect(f.group.createContainer({ role: 'app', image: f.image, command: ['/bin/sleep', '1'], archiveTmpfsBytes: 4096 })).rejects.toThrow('Docker did not preserve');
+    expect(fs.existsSync(path.join(f.root, 'resources.journal'))).toBe(false);
+  });
 });

test/cso-preparation-container.test.ts

@@ -5,7 +5,7 @@ import * as path from 'node:path';
 import { spawnSync } from 'node:child_process';
 import { createPreparedExport, recordNpmArchiveEntry } from '../lib/cso/preparation-container';
 import { materializePreparedExport } from '../lib/cso/preparation-docker';
-import { validateSingleContainerProcessOutput } from '../lib/cso/docker';
+import { heldApplicationReady, validateSingleContainerProcessOutput } from '../lib/cso/docker';
 
 const roots: string[] = [];
 function temporary(): string { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'cso-prepared-export-')); roots.push(root); return root; }
@@ -60,4 +60,11 @@ describe('CSO preparation container archive limits', () => {
     expect(() => validateSingleContainerProcessOutput('PID\n123\n456\n')).toThrow('background process');
     expect(() => validateSingleContainerProcessOutput('PID COMMAND\n123 bun\n')).toThrow('background process');
   });
+
+  test('treats a held application as ready only after run-app has finished copying its source', () => {
+    expect(heldApplicationReady('PID                 COMMAND\n273296              sleep\n')).toBe(true);
+    expect(heldApplicationReady('PID COMMAND\n1 sh\n2 cp\n')).toBe(false);
+    expect(heldApplicationReady('PID COMMAND\n1 sh\n2 sleep\n')).toBe(false);
+    expect(heldApplicationReady('PID COMMAND\n1 sh\n')).toBe(false);
+  });
 });

test/cso-preparation-executor.test.ts

@@ -366,6 +366,43 @@ describe('CSO constrained dependency preparation executor', () => {
     } finally { await broker.close(); }
   });
 
+  // Rails acquires ~70 gems in sequence. Under Bun, stream.pipe left each finished
+  // tunnel's upstream open, so the broker hit its 64-connection limit mid-acquisition.
+  test('finished registry tunnels release both sockets across more tunnels than the connection limit', async () => {
+    const base = root('cso-broker-release-'), socketPath = path.join(base, 'registry.sock');
+    const registry = net.createServer(upstream => { upstream.on('data', chunk => upstream.write(chunk)); upstream.on('end', () => upstream.end()); });
+    await new Promise<void>(resolve => registry.listen(0, '127.0.0.1', resolve));
+    const registryPort = (registry.address() as net.AddressInfo).port, dial = net.connect;
+    const connect = spyOn(net, 'connect').mockImplementation(((options: any, ...rest: any[]) =>
+      dial({ host: '127.0.0.1', port: options?.port === 443 ? registryPort : options?.port }, ...rest)) as typeof net.connect);
+    const broker = new RegistryEgressBroker(socketPath, ['rubygems.org'], Date.now() + 60_000, 64 * 1024 * 1024,
+      () => ({ promise: Promise.resolve([{ address: '151.101.1.227', family: 4 as const }]), cancel: () => {} }));
+    await broker.start();
+    try {
+      for (let index = 0; index < 70; index++) {
+        const echoed = await new Promise<string>((resolveEcho, reject) => {
+          const client = net.createConnection({ path: socketPath }); let output = '';
+          client.setTimeout(5_000, () => client.destroy(new Error(`tunnel ${index} stalled`)));
+          client.once('connect', () => client.write('CONNECT rubygems.org:443 HTTP/1.1\r\nHost: rubygems.org:443\r\n\r\n'));
+          client.on('data', chunk => {
+            output += chunk.toString();
+            if (output === 'HTTP/1.1 200 Connection Established\r\n\r\n') { client.write(`gem-${index}`); return; }
+            if (output.endsWith(`gem-${index}`)) client.end();
+          });
+          client.once('close', () => resolveEcho(output)); client.once('error', reject);
+        });
+        expect(echoed).toEndWith(`gem-${index}`);
+      }
+      const deadline = Date.now() + 2_000;
+      while ((broker as any).sockets.size && Date.now() < deadline) await Bun.sleep(10);
+      expect((broker as any).sockets.size).toBe(0);
+      expect(() => broker.assertClean()).not.toThrow();
+      expect([...broker.contactedHosts]).toEqual(['rubygems.org']);
+    } finally {
+      connect.mockRestore(); await broker.close(); await new Promise<void>(resolve => registry.close(() => resolve()));
+    }
+  });
+
   test('registry address classification rejects mapped, translated, private, and unspecified destinations', () => {
     for (const address of ['127.0.0.1', '169.254.169.254', '10.2.3.4', '100.64.0.1'])
       expect(isBlockedRegistryAddress(address, 4)).toBe(true);

test/cso-preparation.test.ts

@@ -1,5 +1,5 @@
 import { afterEach, describe, expect, test } from 'bun:test';
-import { mkdtempSync, mkdirSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
+import { mkdtempSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
 import { join } from 'node:path';
 import { tmpdir } from 'node:os';
 import { inspectPreparation, railsTestConfiguration } from '../lib/cso/preparation';
@@ -59,6 +59,17 @@ describe('CSO inert Node preparation', () => {
     const plan = inspectPreparation(fixture(nodeFiles(1)));
     expect(plan.prerequisites[0].code).toBe('UNSUPPORTED_LOCK'); expect(plan.acquisition).toEqual([]); expect(plan.metadata).toEqual([]);
   });
+  // npm 11 refuses one file loaded as both user and global config ("double-loading config").
+  test('every npm command loads distinct empty user and global configs', () => {
+    const plan = inspectPreparation(fixture(nodeFiles(3)));
+    const npm = [...plan.acquisition, ...plan.offline].filter(command => command.executable === '/usr/local/bin/npm');
+    expect(npm.length).toBeGreaterThanOrEqual(3);
+    for (const command of npm) {
+      const value = (flag: string) => command.args[command.args.indexOf(flag) + 1];
+      expect([value('--userconfig'), value('--globalconfig')]).toEqual(['/opt/cso/empty-config', '/opt/cso/empty-globalconfig']);
+    }
+    expect(readFileSync(join(import.meta.dir, '../lib/cso/images/node.Dockerfile'), 'utf8')).toContain('touch /opt/cso/empty-config /opt/cso/empty-globalconfig');
+  });
   for (const url of [CREDENTIAL_ARCHIVE_URL, 'http://registry.npmjs.org/a.tgz', 'https://registry.npmjs.org.evil.invalid/a.tgz', 'https://127.0.0.1/a.tgz', 'git+ssh://github.com/a/b', 'https://registry.npmjs.org/a.tgz?token=secret']) test(`rejects non-public archive ${url.split('@').at(-1)}`, () => {
     const files = nodeFiles(); files['package-lock.json'].packages['node_modules/cookie'].resolved = url;
     const plan = inspectPreparation(fixture(files)); expect(plan.status).toBe('prerequisites'); expect(plan.acquisition).toEqual([]);
@@ -159,6 +170,12 @@ describe('CSO inert Rails preparation', () => {
     const plan = inspectPreparation(fixture({ 'Gemfile.lock': gemLock, 'Gemfile': 'system("touch /tmp/CSO_UNSAFE")\nsource "https://rubygems.org"' }));
     expect(plan.status).toBe('ready'); expect(plan.stack).toBe('rails'); expect(plan.metadata.some(m => m.path === 'Gemfile')).toBe(false);
     expect(plan.acquisition).toHaveLength(2); expect(plan.acquisition.every(c => c.args[0] === 'fetch')).toBe(true);
+    // One gem process per platform: each process loads the full RubyGems index once.
+    for (const command of plan.acquisition) {
+      const platform = command.args[command.args.indexOf('--platform') + 1];
+      expect(command.args.slice(1, command.args.indexOf('--platform'))).toEqual(plan.inputs.filter(input => input.platform === platform).map(input => `${input.name}:${input.version}`));
+      expect(command.args).not.toContain('--version');
+    }
     expect(plan.inputs[1].platform).toBe('x86_64-linux-gnu'); expect(plan.inputs[0].integritySource).toBe('registry-on-acquisition');
     expect(plan.runtimeRequirements).toEqual({ ruby: '3.3.6', bundler: '2.6.9' });
     expect(plan.offline[0].args).toContain('--local'); expect(plan.offline[0].env.BUNDLE_IGNORE_CONFIG).toBe('true');

test/cso-rails-verification.test.ts

@@ -18,7 +18,7 @@ describe('CSO Rails database verification lifecycle',()=>{
       return {
         anchor:`anchor-${group}`,
         createContainer:async(spec:any)=>{const id=`${group}-${spec.role}-${++next}`;events.push(`create:${id}:${spec.command.join(' ')}`);if(spec.postgresDatabasePolicy){policies.push(fs.readFileSync(spec.postgresDatabasePolicy,'utf8').trim().split('\n'));expect(fs.statSync(spec.postgresDatabasePolicy).mode&0o777).toBe(0o444);}return id;},
-        start:async(id:string)=>{events.push(`start:${id}`);},
+        start:async(id:string)=>{events.push(`start:${id}`);},startHeldApplication:async(id:string)=>{events.push(`held:${id}`);},
         execCapture:async(id:string,command:string[])=>{events.push(`exec:${id}:${command.join(' ')}`);return{code:0,stdout:'',stderr:''};},
         execDetached:async(id:string,command:string[])=>{events.push(`detach:${id}:${command.join(' ')}`);},
         startAttach:async(id:string)=>{events.push(`attach:${id}`);return{code:0,output:JSON.stringify({booted:true,legitimate:true,security:'pass',existingTests:false,output:'ok',inputHash:''})};},
@@ -37,6 +37,8 @@ describe('CSO Rails database verification lifecycle',()=>{
     expect(policies).toEqual([['cso_primary','cso_queue'],['cso_primary','cso_queue']]);
     expect(events.filter(event=>event.includes('/opt/cso/postgresql-ready'))).toHaveLength(2);
     expect(events.filter(event=>event.includes('/usr/local/bin/bundle exec rails db:prepare'))).toHaveLength(6);
+    for(const [index,event] of events.entries())if(event.includes('exec rails db:prepare'))expect(events.slice(0,index)).toContain(`held:${event.split(':')[1]}`);
+    expect(events.some(event=>/^start:\d-(app|tests)-/.test(event))).toBe(false);
     for(const group of [1,2]){
       const removeApp=events.findIndex(event=>event===`remove:${group}-app-2`),createFirstTest=events.findIndex(event=>event.startsWith(`create:${group}-tests-4:`));
       expect(removeApp).toBeGreaterThan(-1);expect(createFirstTest).toBeGreaterThan(removeApp);
@@ -47,11 +49,11 @@ describe('CSO Rails database verification lifecycle',()=>{
     const root=fs.mkdtempSync(path.join(os.tmpdir(),'cso-rails-sqlite-'));roots.push(root);for(const name of ['source','work','control'])fs.mkdirSync(path.join(root,name));
     const commands:string[]=[];let next=0;
     const create=spyOn(DockerGroup,'create').mockResolvedValue({
-      createContainer:async(spec:any)=>{commands.push(`create:${spec.role}`);return`${spec.role}-${++next}`;},start:async()=>{},
+      createContainer:async(spec:any)=>{commands.push(`create:${spec.role}`);return`${spec.role}-${++next}`;},start:async()=>{},startHeldApplication:async(id:string)=>{commands.push(`held:${id}`);},
       execCapture:async(_id:string,command:string[])=>{commands.push(command.join(' '));return{code:0,stdout:'',stderr:''};},execDetached:async()=>{},
       startAttach:async()=>({code:0,output:JSON.stringify({booted:true,legitimate:true,security:'pass',existingTests:false,output:'ok',inputHash:''})}),removeContainer:async()=>{},cleanup:async()=>{},
     } as any);
     try{const runtime:any={stack:'rails',image:`runtime@sha256:${'a'.repeat(64)}`},request:any={findingId:'d'.repeat(32),port:3456,start:{executable:'/usr/local/bin/bundle',args:['exec','rails','server']},legitimate:[],security:{},fixtures:{},existingTests:[{executable:'/usr/local/bin/bundle',args:['exec','rails','test']}]};await new DockerVerificationExecutor({} as any,'/watchdog',Date.now()+60_000).observe(path.join(root,'source'),'before',request,runtime,runtime,path.join(root,'work'),path.join(root,'control'),{environment:{},database:{adapter:'sqlite',connections:['primary']}});}finally{create.mockRestore();}
-    expect(commands.some(command=>command==='create:postgres')).toBe(false);expect(commands.filter(command=>command.includes('exec rails db:prepare'))).toHaveLength(2);
+    expect(commands.some(command=>command==='create:postgres')).toBe(false);expect(commands.filter(command=>command.includes('exec rails db:prepare'))).toHaveLength(2);expect(commands.filter(command=>command.startsWith('held:'))).toHaveLength(2);
   });
 });

test/cso-stack-cold-integration.test.ts

@@ -11,8 +11,8 @@ import {
   type PreparedApplication, type PreparationRuntimeAdmission, type RailsDatabaseSelection,
 } from '../lib/cso/preparation-executor';
 import { inspectPreparation, type CsoStack, type PreparationPlan } from '../lib/cso/preparation';
-import { CSO_HELPER_ABI, type QualifiedRuntime, type RuntimeCatalog } from '../lib/cso/runtime-catalog';
-import { completeRuntimeCatalogFixture } from './helpers/cso-runtime-catalog';
+import { CSO_HELPER_ABI, validateRuntimeCatalog, type QualifiedRuntime, type RuntimeCatalog } from '../lib/cso/runtime-catalog';
+import { completeRuntimeCatalogFixture, installStagedRuntime } from './helpers/cso-runtime-catalog';
 import { secureDirectory } from '../lib/cso/state';
 import { canonicalStartPlan, canonicalTestPlan, DockerVerificationExecutor } from '../lib/cso/verification';
 
@@ -82,7 +82,10 @@ import django
 django.setup()
 from django.test import Client
 class ColdStartTest(unittest.TestCase):
-  def test_dependency_and_control(self): self.assertContains(Client().get("/control"),"CONTROL_OK",status_code=200)
+  def test_dependency_and_control(self):
+    response = Client().get("/control")
+    self.assertEqual(response.status_code, 200)
+    self.assertIn(b"CONTROL_OK", response.content)
 `,
   });
 }
@@ -102,6 +105,7 @@ function railsSource(directory: string, bundlerVersion: string): void {
     'config/database.yml': `default: &default\n  pool: 3\ntest:\n  primary:\n    <<: *default\n    adapter: sqlite3\n    database: storage/test.sqlite3\n  queue:\n    <<: *default\n    adapter: postgresql\n    database: cso_queue\n`,
     'app/controllers/application_controller.rb': `class ApplicationController < ActionController::Base;end\n`,
     'app/controllers/probe_controller.rb': `class ProbeController < ApplicationController\n def control;render plain:"CONTROL_OK:Rails";end\n def security;render plain:"DENIED",status: :forbidden;end\nend\n`,
+    'bin/rails': `#!/usr/bin/env ruby\nAPP_PATH = File.expand_path("../config/application", __dir__)\nrequire_relative "../config/boot"\nrequire "rails/commands"\n`,
     'Rakefile': `require_relative "config/application"\nRails.application.load_tasks\n`,
     'config.ru': `require_relative "config/environment"\nrun Rails.application\n`,
     'db/schema.rb': `ActiveRecord::Schema[8.1].define(version: 1) do\n create_table :cold_records, force: true do |t|\n  t.string :name\n end\nend\n`,
@@ -163,21 +167,14 @@ beforeAll(async () => {
   if (!fs.existsSync(watchdog)) throw new Error(`${requestedStack} cold-start qualification requires the compiled CSO watchdog`);
   endpoint = await dockerEndpoint(secureDirectory(path.join(root, 'docker-home')), { HOME: root, DOCKER_HOST: process.env.DOCKER_HOST ?? 'unix:///var/run/docker.sock' });
   catalog = completeRuntimeCatalogFixture(`${requestedStack}-staged-cold`);
-  const installRuntime = (value: QualifiedRuntime): QualifiedRuntime => {
-    const runtimeIndex = catalog.runtimes.findIndex(item => item.stack === value.stack && item.platform === value.platform);
-    const profile = catalog.profiles.find(item => item.stack === value.stack && item.platform === value.platform)!;
-    const installed = { ...value, id: profile.id };
-    catalog.runtimes[runtimeIndex] = installed;
-    profile.versions = { ...installed.versions };
-    return installed;
-  };
-  runtime = installRuntime(qualified(requestedStack as CsoStack, image, versions));
+  runtime = installStagedRuntime(catalog, qualified(requestedStack as CsoStack, image, versions));
   if (requestedStack === 'rails') {
     const postgresImage = process.env.GSTACK_CSO_TEST_POSTGRES_IMAGE;
     const postgresVersion = process.env.GSTACK_CSO_TEST_POSTGRES_VERSION;
     if (!postgresImage || !/^\d+\.\d+(?:\.\d+)?$/.test(postgresVersion ?? '')) throw new Error('Rails cold-start qualification requires a staged PostgreSQL digest and exact version');
-    installRuntime(qualified('postgresql', postgresImage, { postgresql: postgresVersion! }));
+    installStagedRuntime(catalog, qualified('postgresql', postgresImage, { postgresql: postgresVersion! }));
   }
+  validateRuntimeCatalog(catalog);
 });
 afterAll(() => { if (root) fs.rmSync(root, { recursive: true, force: true }); delete process.env.GSTACK_HOME; });
 

test/helpers/cso-qualified-cli.ts

@@ -1,9 +1,8 @@
 import * as fs from 'node:fs';
 import * as path from 'node:path';
 import { dispatchCsoCommand, type CsoCliDependencies } from '../../lib/cso/cli';
-import { canonical, sha256 } from '../../lib/cso/contracts';
 import { validateRuntimeCatalog, type QualifiedRuntime, type RuntimeCatalog, type RuntimePlatform } from '../../lib/cso/runtime-catalog';
-import { completeRuntimeCatalogFixture } from './cso-runtime-catalog';
+import { completeRuntimeCatalogFixture, installStagedRuntime } from './cso-runtime-catalog';
 
 export interface QualifiedCsoCli {
   readonly catalog: RuntimeCatalog;
@@ -31,17 +30,9 @@ export function qualifiedNodeCli(options: {
   }
 
   const catalog = completeRuntimeCatalogFixture('node-lifecycle-fixture');
-  const runtimeIndex = catalog.runtimes.findIndex(item => item.stack === 'node' && item.platform === options.platform);
-  const profile = catalog.profiles.find(item => item.stack === 'node' && item.platform === options.platform);
-  if (runtimeIndex < 0 || !profile || !catalog.promotion) throw new Error('Node lifecycle catalog fixture is incomplete');
-  const runtime: QualifiedRuntime = {
-    ...catalog.runtimes[runtimeIndex],
-    image: options.image,
-    versions: { ...options.versions },
-  };
-  catalog.runtimes[runtimeIndex] = runtime;
-  profile.versions = { ...runtime.versions };
-  catalog.promotion.evidenceDigest = `sha256:${sha256(canonical(catalog.runtimes))}`;
+  const fixture = catalog.runtimes.find(item => item.stack === 'node' && item.platform === options.platform);
+  if (!fixture) throw new Error('Node lifecycle catalog fixture is incomplete');
+  const runtime = installStagedRuntime(catalog, { ...fixture, image: options.image, versions: options.versions });
   validateRuntimeCatalog(catalog);
 
   const dependencies: CsoCliDependencies = Object.freeze({

test/helpers/cso-runtime-catalog.ts

@@ -107,3 +107,19 @@ export function completeRuntimeCatalogFixture(
     runtimes,
   };
 }
+
+/**
+ * Put a staged runtime into its stack/platform slot under the reviewed profile
+ * id, as promotion does, and re-bind the promotion evidence digest the
+ * validator recomputes from the runtime matrix.
+ */
+export function installStagedRuntime(catalog: RuntimeCatalog, runtime: QualifiedRuntime): QualifiedRuntime {
+  const index = catalog.runtimes.findIndex(item => item.stack === runtime.stack && item.platform === runtime.platform);
+  const profile = catalog.profiles.find(item => item.stack === runtime.stack && item.platform === runtime.platform);
+  if (index < 0 || !profile || !catalog.promotion) throw new Error(`Runtime catalog fixture lacks ${runtime.stack} on ${runtime.platform}`);
+  const installed = { ...runtime, id: profile.id, versions: { ...runtime.versions } };
+  catalog.runtimes[index] = installed;
+  profile.versions = { ...installed.versions };
+  catalog.promotion.evidenceDigest = `sha256:${sha256(canonical(catalog.runtimes))}`;
+  return installed;
+}