mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 14:38:59 +02:00
Benjamin D. SmithandClaude Opus 5
4543b3c66b
`./setup --help` can hang forever on macOS, printing nothing, with no way
to tell it apart from a slow install. Eleven scripts carry the same
latent hang, `setup` itself being the one every user hits first.
bash 5.2+ delivers a heredoc body of 64KiB or less through a pipe: the
forked child writes the entire body before exec, and nothing reads the
other end until the command starts. Under macOS pipe-KVA pressure the
kernel hands a fresh pipe a 512-byte buffer instead of the usual 16-64KiB,
so any body of 512 bytes or more blocks write() permanently. The capacity
check bash would need to notice (F_GETPIPE_SZ) is Linux-only, so it never
fires here. It is pressure-dependent, which is why it reads as "worked on
my machine" — the same script runs fine all day and then wedges.
Homebrew bash is what `#!/usr/bin/env bash` resolves to on a Mac with brew
on PATH, which is most of them. Apple's /bin/bash 3.2 predates the pipe
path and is unaffected, so the bug is invisible to anyone testing with the
system shell.
The fix is `BASH_COMPAT=50` in each affected script, which restores the
pre-5.2 tempfile path:
$ bash -c 'probe() { [ -p /dev/stdin ] && echo PIPE || echo TEMPFILE; }
probe <<EOF
$(printf "x%.0s" $(seq 1 1000))
EOF'
PIPE
$ BASH_COMPAT=50 bash -c '...same...'
TEMPFILE
- Not a `#!/bin/bash` shebang swap: that pins the script to whatever bash
lives at /bin (3.2 on macOS, absent on some Linux distributions) and is
bypassed entirely by `bash script.sh` call sites. The variable survives
both.
- Not exported, so child processes keep their own compat level.
- Placed below any `--help` sed range that reads $0, so usage output is
unchanged (verified on all eleven).
- Every guarded script is bash-3.2-clean — no associative arrays, case
conversion, or mapfile — so compat level 50 costs them nothing.
test/heredoc-pipe-deadlock.test.ts scans every tracked shell script for a
heredoc body in the 512B-64KiB window and fails without the guard, and
proves the mechanism at runtime on bash 5.2+ by asserting the body moves
from PIPE to TEMPFILE. On older bash the runtime half is skipped, since
the pipe path does not exist there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Absorbed from PR #2640 with authorship preserved. Wave adaptations: the pipe-probe test skips on minimal-/dev environments without /dev/stdin (it would report OTHER for an unobservable fd), and one caveat verified during review: on bash 4.3/4.4 (e.g. Git Bash), assigning BASH_COMPAT=50 prints a non-fatal 'invalid value' warning to stderr — those bashes are already on tempfiles, so the guard is a no-op there; windows-setup-e2e exercises this empirically.
106 lines
3.8 KiB
Bash
Executable File
106 lines
3.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# gstack-jsonl-merge — git merge driver for append-only JSONL files.
|
|
#
|
|
# Usage (called by git, not by users):
|
|
# gstack-jsonl-merge <base> <ours> <theirs>
|
|
#
|
|
# Registered in local git config by bin/gstack-artifacts-init and
|
|
# bin/gstack-brain-restore:
|
|
# git config merge.jsonl-append.driver \
|
|
# "$GSTACK_BIN/gstack-jsonl-merge %O %A %B"
|
|
#
|
|
# Behavior:
|
|
# Concatenate base + ours + theirs, dedup exact-duplicate lines, sort by
|
|
# ISO "ts" field when present, fall back to SHA-256 of the line for
|
|
# deterministic order. Write result to <ours> (the %A file per the git
|
|
# merge-driver contract).
|
|
#
|
|
# Two machines appending to the same JSONL file between pushes produces
|
|
# a same-line conflict at the file tail. This driver resolves it cleanly:
|
|
# both appends survive, ordered by wall-clock timestamp where available,
|
|
# content hash otherwise.
|
|
#
|
|
# Exit codes:
|
|
# 0 — merge succeeded, result written to <ours>
|
|
# 1 — error; git treats as conflict and stops the merge
|
|
|
|
# Heredoc delivery guard. bash 5.2+ writes a heredoc body <=64KiB through a
|
|
# pipe in the forked child before exec, with no reader on the other end. On
|
|
# macOS under pipe-KVA pressure a fresh pipe gets a 512-byte buffer, so any
|
|
# body >=512B blocks write() forever and the script hangs at startup with no
|
|
# output. Compat level 50 restores the tempfile path. These scripts are
|
|
# bash-3.2-clean, so the compat level costs them nothing. Not exported: the
|
|
# guard is per-script, and it survives `bash script.sh` call sites that
|
|
# bypass the shebang.
|
|
BASH_COMPAT=50
|
|
|
|
set -uo pipefail
|
|
|
|
if [ "$#" -lt 3 ]; then
|
|
echo "gstack-jsonl-merge: expected 3 args (base ours theirs), got $#" >&2
|
|
exit 1
|
|
fi
|
|
|
|
BASE="$1"
|
|
OURS="$2"
|
|
THEIRS="$3"
|
|
|
|
TMP=$(mktemp /tmp/gstack-jsonl-merge.XXXXXX) || exit 1
|
|
trap 'rm -f "$TMP" 2>/dev/null || true' EXIT
|
|
|
|
python3 - "$BASE" "$OURS" "$THEIRS" > "$TMP" <<'PYEOF'
|
|
import sys, json, hashlib
|
|
|
|
paths = sys.argv[1:4] # base, ours, theirs
|
|
seen = {} # line content -> sort_key
|
|
|
|
for path in paths:
|
|
try:
|
|
with open(path, 'r', encoding='utf-8') as f:
|
|
for line in f:
|
|
line = line.rstrip('\n')
|
|
if not line:
|
|
continue
|
|
if line in seen:
|
|
continue
|
|
# Prefer ISO ts field for sort; fall back to SHA-256. The line
|
|
# content is the final tiebreaker so the order is total: two
|
|
# entries sharing a ts must resolve identically regardless of
|
|
# which side they arrive on. Without it, equal-ts entries fall
|
|
# back to insertion order (base, ours, theirs), and since ours
|
|
# and theirs are swapped depending on which machine runs the
|
|
# merge, the two sides produce divergent files that never
|
|
# converge.
|
|
sort_key = None
|
|
try:
|
|
obj = json.loads(line)
|
|
ts = obj.get('ts') or obj.get('timestamp')
|
|
if isinstance(ts, str):
|
|
sort_key = (0, ts, line)
|
|
except (json.JSONDecodeError, ValueError, TypeError):
|
|
pass
|
|
if sort_key is None:
|
|
h = hashlib.sha256(line.encode('utf-8')).hexdigest()
|
|
sort_key = (1, h, line)
|
|
seen[line] = sort_key
|
|
except FileNotFoundError:
|
|
# Absent base / absent ours / absent theirs are all valid.
|
|
continue
|
|
except OSError:
|
|
# Permission / IO errors are fatal — caller sees non-zero exit.
|
|
sys.exit(1)
|
|
|
|
# Timestamp-ordered entries first (group 0), then hash-ordered (group 1).
|
|
for line, _ in sorted(seen.items(), key=lambda item: item[1]):
|
|
print(line)
|
|
PYEOF
|
|
|
|
_PYEXIT=$?
|
|
if [ "$_PYEXIT" != "0" ]; then
|
|
exit 1
|
|
fi
|
|
|
|
mv "$TMP" "$OURS" || exit 1
|
|
trap - EXIT
|
|
exit 0
|