mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 06:28:59 +02:00
fix(scripts): stop heredoc bodies deadlocking under Homebrew bash
`./setup --help` can hang forever on macOS, printing nothing, with no way
to tell it apart from a slow install. Eleven scripts carry the same
latent hang, `setup` itself being the one every user hits first.
bash 5.2+ delivers a heredoc body of 64KiB or less through a pipe: the
forked child writes the entire body before exec, and nothing reads the
other end until the command starts. Under macOS pipe-KVA pressure the
kernel hands a fresh pipe a 512-byte buffer instead of the usual 16-64KiB,
so any body of 512 bytes or more blocks write() permanently. The capacity
check bash would need to notice (F_GETPIPE_SZ) is Linux-only, so it never
fires here. It is pressure-dependent, which is why it reads as "worked on
my machine" — the same script runs fine all day and then wedges.
Homebrew bash is what `#!/usr/bin/env bash` resolves to on a Mac with brew
on PATH, which is most of them. Apple's /bin/bash 3.2 predates the pipe
path and is unaffected, so the bug is invisible to anyone testing with the
system shell.
The fix is `BASH_COMPAT=50` in each affected script, which restores the
pre-5.2 tempfile path:
$ bash -c 'probe() { [ -p /dev/stdin ] && echo PIPE || echo TEMPFILE; }
probe <<EOF
$(printf "x%.0s" $(seq 1 1000))
EOF'
PIPE
$ BASH_COMPAT=50 bash -c '...same...'
TEMPFILE
- Not a `#!/bin/bash` shebang swap: that pins the script to whatever bash
lives at /bin (3.2 on macOS, absent on some Linux distributions) and is
bypassed entirely by `bash script.sh` call sites. The variable survives
both.
- Not exported, so child processes keep their own compat level.
- Placed below any `--help` sed range that reads $0, so usage output is
unchanged (verified on all eleven).
- Every guarded script is bash-3.2-clean — no associative arrays, case
conversion, or mapfile — so compat level 50 costs them nothing.
test/heredoc-pipe-deadlock.test.ts scans every tracked shell script for a
heredoc body in the 512B-64KiB window and fails without the guard, and
proves the mechanism at runtime on bash 5.2+ by asserting the body moves
from PIPE to TEMPFILE. On older bash the runtime half is skipped, since
the pipe path does not exist there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Absorbed from PR #2640 with authorship preserved. Wave adaptations: the pipe-probe test skips on minimal-/dev environments without /dev/stdin (it would report OTHER for an unobservable fd), and one caveat verified during review: on bash 4.3/4.4 (e.g. Git Bash), assigning BASH_COMPAT=50 prints a non-fatal 'invalid value' warning to stderr — those bashes are already on tempfiles, so the guard is a no-op there; windows-setup-e2e exercises this empirically.
This commit is contained in:
committed by
Garry Tan
co-authored by
Claude Opus 5
parent
e01ee8fe86
commit
4543b3c66b
@@ -34,6 +34,16 @@
|
||||
# GSTACK_HOME — override ~/.gstack
|
||||
# USER — fallback for repo naming if $USER is unset
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
|
||||
@@ -25,6 +25,16 @@
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
|
||||
@@ -17,6 +17,16 @@
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack (aligns with writers).
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
|
||||
@@ -20,6 +20,16 @@
|
||||
# auditability via --status when you want it.
|
||||
# Per D6: Anthropic SDK direct call, fail-loud on missing ANTHROPIC_API_KEY.
|
||||
set -euo pipefail
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
GSTACK_HOME="${GSTACK_STATE_ROOT:-${GSTACK_HOME:-$HOME/.gstack}}"
|
||||
|
||||
@@ -24,6 +24,16 @@
|
||||
# 0 — merge succeeded, result written to <ours>
|
||||
# 1 — error; git treats as conflict and stops the merge
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
if [ "$#" -lt 3 ]; then
|
||||
|
||||
@@ -54,6 +54,16 @@
|
||||
# on disk. `rollback` is a single-step undo of the last real mutation.
|
||||
# - writes are atomic: unique tmp file + rename (a fixed tmp name would let
|
||||
# two concurrent writers rename a half-written file into place).
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ACTION="${1:-}"
|
||||
|
||||
@@ -7,6 +7,16 @@
|
||||
#
|
||||
# Run from the root of your team's repo (not from the gstack directory).
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:-}"
|
||||
|
||||
@@ -24,6 +24,16 @@
|
||||
# the brain admin to run on the brain host
|
||||
#
|
||||
# All steps are idempotent. Re-running after partial completion is safe.
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if [ -z "${HOME:-}" ]; then
|
||||
|
||||
@@ -21,6 +21,16 @@
|
||||
# on completion. Re-running this script is silent if the touchfile exists,
|
||||
# OR if local_code_index_offered=true.
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if [ -z "${HOME:-}" ]; then
|
||||
|
||||
@@ -13,6 +13,16 @@
|
||||
# ./scripts/build-app.sh # Build .app + DMG
|
||||
# ./scripts/build-app.sh --no-dmg # Build .app only
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever and the script hangs at startup with no
|
||||
# output. Compat level 50 restores the tempfile path. These scripts are
|
||||
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash script.sh` call sites that
|
||||
# bypass the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
@@ -3,6 +3,16 @@
|
||||
set -e
|
||||
umask 077 # Restrict new files to owner-only (0o600 files, 0o700 dirs)
|
||||
|
||||
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
||||
# pipe in the forked child before exec, with no reader on the other end. On
|
||||
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
||||
# body >=512B blocks write() forever — ./setup --help would hang with no
|
||||
# output. Compat level 50 restores the tempfile path. This script is
|
||||
# bash-3.2-clean, so the compat level costs it nothing. Not exported: the
|
||||
# guard is per-script, and it survives `bash setup` call sites that bypass
|
||||
# the shebang.
|
||||
BASH_COMPAT=50
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
gstack setup — install gstack skills + build browse binary
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { execSync, spawnSync } from 'child_process';
|
||||
|
||||
/**
|
||||
* bash 5.2+ delivers a heredoc body of 64KiB or less through a pipe: the
|
||||
* forked child writes the whole body before exec, and nothing reads the other
|
||||
* end until the command starts. On macOS under pipe-KVA pressure the kernel
|
||||
* hands a fresh pipe a 512-byte buffer, so any body of 512 bytes or more
|
||||
* blocks write() forever — the script hangs at startup, silently, with no
|
||||
* output and no error. The runtime capacity check bash would need
|
||||
* (F_GETPIPE_SZ) is Linux-only.
|
||||
*
|
||||
* Compat level 50 restores the pre-5.2 tempfile path. Every script that ships
|
||||
* an in-window heredoc must set it, and this scanner fails the suite when a
|
||||
* new one appears without the guard.
|
||||
*
|
||||
* The guard is deliberately not a `#!/bin/bash` shebang swap: that pins the
|
||||
* script to whatever bash lives at /bin (3.2 on macOS, absent on some Linux
|
||||
* distributions) and is bypassed entirely by `bash script.sh` call sites.
|
||||
*/
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
|
||||
// Inclusive byte window where the pipe path is taken AND a starved pipe can
|
||||
// block. Bodies over 64KiB fall back to a tempfile on their own.
|
||||
const MIN_BODY = 512;
|
||||
const MAX_BODY = 64 * 1024;
|
||||
|
||||
const GUARD_RE = /^\s*(?::\s*"\$\{)?BASH_COMPAT(?:[:=]|\}")/m;
|
||||
|
||||
function trackedShellScripts(): string[] {
|
||||
const out = execSync('git ls-files', { cwd: ROOT, encoding: 'utf-8', maxBuffer: 32 * 1024 * 1024 });
|
||||
return out
|
||||
.split('\n')
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean)
|
||||
.filter((f) => {
|
||||
const abs = path.join(ROOT, f);
|
||||
if (!fs.existsSync(abs) || !fs.statSync(abs).isFile()) return false;
|
||||
if (f.endsWith('.sh')) return true;
|
||||
const head = fs.readFileSync(abs).subarray(0, 64).toString('utf-8');
|
||||
return /^#!.*\b(bash|sh)\b/.test(head);
|
||||
});
|
||||
}
|
||||
|
||||
/** Heredocs in `content` whose body lands inside the deadlock window. */
|
||||
function inWindowHeredocs(content: string): { line: number; tag: string; bytes: number }[] {
|
||||
const lines = content.split('\n');
|
||||
const hits: { line: number; tag: string; bytes: number }[] = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const m = /<<-?\s*'?([A-Za-z_][A-Za-z0-9_]*)'?/.exec(lines[i]);
|
||||
if (!m) continue;
|
||||
const tag = m[1];
|
||||
let j = i + 1;
|
||||
const body: string[] = [];
|
||||
while (j < lines.length && lines[j].trim() !== tag) body.push(lines[j++]);
|
||||
const bytes = Buffer.byteLength(body.join('\n')) + 1;
|
||||
if (bytes >= MIN_BODY && bytes <= MAX_BODY) hits.push({ line: i + 1, tag, bytes });
|
||||
i = j;
|
||||
}
|
||||
return hits;
|
||||
}
|
||||
|
||||
describe('heredoc pipe-deadlock guard', () => {
|
||||
test('every script with an in-window heredoc sets BASH_COMPAT', () => {
|
||||
const violations: string[] = [];
|
||||
for (const rel of trackedShellScripts()) {
|
||||
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
||||
const hits = inWindowHeredocs(content);
|
||||
if (hits.length === 0) continue;
|
||||
if (GUARD_RE.test(content)) continue;
|
||||
for (const h of hits) violations.push(`${rel}:${h.line} <<${h.tag} body=${h.bytes}B`);
|
||||
}
|
||||
if (violations.length > 0) {
|
||||
throw new Error(
|
||||
`Heredoc bodies in the ${MIN_BODY}-${MAX_BODY}B pipe window without a BASH_COMPAT guard:\n ` +
|
||||
violations.join('\n ') +
|
||||
`\n\nFix: add \`BASH_COMPAT=50\` near the top of the script (below any ` +
|
||||
`\`--help\` sed range that reads $0), or shrink the body under ${MIN_BODY}B, ` +
|
||||
`or pipe it in with printf so a live reader exists.`,
|
||||
);
|
||||
}
|
||||
expect(violations).toEqual([]);
|
||||
});
|
||||
|
||||
test('the guard actually moves the body off the pipe', () => {
|
||||
const bash = spawnSync('bash', ['-c', 'echo "${BASH_VERSINFO[0]}.${BASH_VERSINFO[1]}"'], {
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
const version = (bash.stdout ?? '').trim();
|
||||
const [maj, min] = version.split('.').map((n) => parseInt(n, 10));
|
||||
// Only 5.2+ takes the pipe path at all; older bash is already on tempfiles.
|
||||
if (!(maj > 5 || (maj === 5 && min >= 2))) {
|
||||
expect(version).toBeTruthy();
|
||||
return;
|
||||
}
|
||||
|
||||
// Some sandboxes/containers ship a minimal /dev without /dev/stdin — the
|
||||
// probe medium itself is absent there, so -p/-f both report false and the
|
||||
// probe would answer OTHER for an unobservable fd. Skip rather than fail.
|
||||
const devStdin = spawnSync('bash', ['-c', '[ -e /dev/stdin ] && echo yes || echo no'], {
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
if ((devStdin.stdout ?? '').trim() !== 'yes') return;
|
||||
|
||||
const probe = (guard: string) => `#!/usr/bin/env bash
|
||||
${guard}
|
||||
body=$(printf 'x%.0s' $(seq 1 1000))
|
||||
probe() { if [ -p /dev/stdin ]; then echo PIPE; elif [ -f /dev/stdin ]; then echo TEMPFILE; else echo OTHER; fi; }
|
||||
probe <<EOF
|
||||
$body
|
||||
EOF
|
||||
`;
|
||||
const run = (guard: string) =>
|
||||
(spawnSync('bash', ['-c', probe(guard)], { encoding: 'utf-8' }).stdout ?? '').trim();
|
||||
|
||||
expect(run('')).toBe('PIPE');
|
||||
expect(run('BASH_COMPAT=50')).toBe('TEMPFILE');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user