mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-03 15:27:56 +01:00
v1.92.69.0 fix: lock developer-profile writers so overlapping derives keep every session (#3437, #3435) (#3738)
* fix(plan-tune): lock developer-profile writes and give derive a unique temp file (#3437) Concurrent background --derive runs shared developer-profile.json.tmp and rewrote the whole profile unlocked, so renames failed with ENOENT and a concurrent --log-session was dropped. Every read-modify-write now holds developer-profile.json.lock and writes a mktemp file. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * chore(plan-tune): remove applySignal and newDimensionTotals (#3435) Both lost their last production caller when --derive moved onto accumulateSignals; only three unit tests referenced them. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes Use the PID in the lock token, anchor a relative state root once so do_derive's cd cannot leak the lock, and note the stale-takeover tradeoff. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (pass 2) Keep Windows drive-letter state roots absolute when anchoring a relative root, and say which writers do not take the lock yet. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: take over a lock only from a dead holder, one taker at a time Step 11 voices: a late stale-lock mv could evict a fresh owner, a failed mv spun past the timeout, and the 10 s give-up sat under the 30 s stale age. Takeover now checks the holder's pid, re-reads the owner under a takeover lock, falls through to the give-up on failure, and reports an unwritable state directory at once. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: take over an owned lock older than 5 minutes (pid reuse) A crashed holder whose pid now belongs to a live process kept its lock forever. An owned lock past 5 minutes now counts as abandoned, and cleanup removes a takeover lock this writer holds. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: changelog fragment for the profile lock (#3437) and dead-helper removal (#3435) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: say the 5-minute takeover applies whatever the pid Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: keep the give-up test inside the type ratchet Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
1a22ae9652
commit
7762c39a4e
8 files changed
+288
-84
No files matched your search
@@ -1,5 +1,20 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [1.92.69.0] - 2026-11-02
|
||||||
|
|
||||||
|
**Background `--derive` runs no longer drop /office-hours sessions or fail with ENOENT when they overlap (#3437).** `gstack-question-log` starts a background `gstack-developer-profile --derive` after every append, and a multi-question AskUserQuestion logs several rows at once. Every derive wrote the same `developer-profile.json.tmp` and rewrote the whole profile without a lock. On main at 3bfe566, 30 rounds of 8 overlapping derives plus one `--log-session` saw 9 of 270 runs fail with `ENOENT` on rename, and 2 of the 30 sessions disappeared. With this change the same run has 0 failures and keeps 30 of 30 sessions.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **`bin/gstack-developer-profile`**: `--derive`, `--log-session`, `--reconcile` and `--migrate` hold `developer-profile.json.lock` from before they read the profile until after they rename it, and each writes its own `mktemp` file. The owner file names the holder's process ID, and the lock is ownership-checked on release. A lock is taken over when its holder process is gone, or once it is 5 minutes old even if a process with that ID is running, since no writer takes that long and the ID may have been reused. Takeovers go one at a time under `developer-profile.json.lock.takeover`, so two waiters can't both take over. A writer that can't get the lock within 10 seconds writes nothing and exits 5. An unwritable state directory is reported right away instead of after the wait. A relative state root is anchored once, so the lock is released under the same path it was taken. Windows drive-letter roots stay as they are.
|
||||||
|
### Removed
|
||||||
|
- **`scripts/psychographic-signals.ts`**: `applySignal` and `newDimensionTotals`, which lost their last production caller when `--derive` moved onto `accumulateSignals` (#3435), and their three unit tests.
|
||||||
|
|
||||||
|
### Not covered yet
|
||||||
|
- `gstack-distill-apply` and /plan-tune's declared edits still write the profile without the lock.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
- `test/gstack-developer-profile.test.ts`: a directory at the old fixed temp path; a held lock blocking `--derive` and `--log-session`; takeover of an ownerless stale lock, of a dead holder's lock, and of a 5-minute-old lock whose process ID is alive; a takeover already in progress; a takeover that can't remove the lock still giving up; a relative state root; and 6 rounds of 6 derives plus a `--log-session` keeping every session. Each test failed on the code before its fix.
|
||||||
|
|
||||||
## [1.92.68.0] - 2026-11-02
|
## [1.92.68.0] - 2026-11-02
|
||||||
|
|
||||||
**`/health` now finds its tools with gstack's detector, the same one `/verify` uses, and a test proves it still finds every check it found before.** /health Step 1 used to carry its own shell block for detecting a type checker, linter, test runner, knip, shellcheck and gbrain, a second copy of the list in `lib/checks.ts`. It now runs `gstack-checks detect --for health`. Scoring, the gbrain sub-score and history are unchanged. GST-116, PR 2 of 2.
|
**`/health` now finds its tools with gstack's detector, the same one `/verify` uses, and a test proves it still finds every check it found before.** /health Step 1 used to carry its own shell block for detecting a type checker, linter, test runner, knip, shellcheck and gbrain, a second copy of the list in `lib/checks.ts`. It now runs `gstack-checks detect --for health`. Scoring, the gbrain sub-score and history are unchanged. GST-116, PR 2 of 2.
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# gstack digest v1.92.68.0 — regenerate/re-copy after upgrading gstack
|
# gstack digest v1.92.69.0 — regenerate/re-copy after upgrading gstack
|
||||||
|
|
||||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||||
for agent hosts without a full skill install. The full skills add workflows,
|
for agent hosts without a full skill install. The full skills add workflows,
|
||||||
|
|||||||
+121
-25
@@ -25,6 +25,10 @@
|
|||||||
# `completed` events in projects/*/timeline.jsonl.
|
# `completed` events in projects/*/timeline.jsonl.
|
||||||
# Idempotent; backfilled rows count for tenure only.
|
# Idempotent; backfilled rows count for tenure only.
|
||||||
#
|
#
|
||||||
|
# Writers (--derive, --log-session, --reconcile, --migrate) hold
|
||||||
|
# developer-profile.json.lock for their read-modify-write and exit 5 when
|
||||||
|
# another writer keeps it past the give-up window (#3437).
|
||||||
|
#
|
||||||
# Profile file: ~/.gstack/developer-profile.json (unified schema — see
|
# Profile file: ~/.gstack/developer-profile.json (unified schema — see
|
||||||
# docs/designs/PLAN_TUNING_V0.md). Event file: ~/.gstack/projects/{SLUG}/
|
# docs/designs/PLAN_TUNING_V0.md). Event file: ~/.gstack/projects/{SLUG}/
|
||||||
# question-events.jsonl.
|
# question-events.jsonl.
|
||||||
@@ -37,6 +41,10 @@ ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
|||||||
. "$(dirname "$0")/gstack-state-root.sh" 2>/dev/null || { echo "$0: cannot resolve the gstack state root: $(dirname "$0")/gstack-state-root.sh is missing. fix: reinstall with ./setup or /gstack-upgrade (docs/state-root.md)" >&2; exit 1; }
|
. "$(dirname "$0")/gstack-state-root.sh" 2>/dev/null || { echo "$0: cannot resolve the gstack state root: $(dirname "$0")/gstack-state-root.sh is missing. fix: reinstall with ./setup or /gstack-upgrade (docs/state-root.md)" >&2; exit 1; }
|
||||||
gstack_state_root_select
|
gstack_state_root_select
|
||||||
GSTACK_HOME="$_gstack_sr_root"
|
GSTACK_HOME="$_gstack_sr_root"
|
||||||
|
# A relative state root is anchored to the starting directory once, here:
|
||||||
|
# do_derive changes directory, and the profile lock must name the same path
|
||||||
|
# when it is taken and when it is released.
|
||||||
|
case "$GSTACK_HOME" in /*|[A-Za-z]:[\\/]*) ;; *) GSTACK_HOME="$PWD/$GSTACK_HOME" ;; esac
|
||||||
# Windows git-bash: GSTACK_HOME resolves to an MSYS path (/c/Users/...), which
|
# Windows git-bash: GSTACK_HOME resolves to an MSYS path (/c/Users/...), which
|
||||||
# Bun on Windows cannot open as a filesystem path (bites --derive). Normalize
|
# Bun on Windows cannot open as a filesystem path (bites --derive). Normalize
|
||||||
# once, here, before PROFILE_FILE/LEGACY_FILE are derived from it below —
|
# once, here, before PROFILE_FILE/LEGACY_FILE are derived from it below —
|
||||||
@@ -52,10 +60,100 @@ SLUG="${SLUG:-unknown}"
|
|||||||
CMD="${1:---read}"
|
CMD="${1:---read}"
|
||||||
shift || true
|
shift || true
|
||||||
|
|
||||||
|
# -----------------------------------------------------------------------
|
||||||
|
# Profile lock (#3437). Every subcommand that rewrites the profile reads,
|
||||||
|
# changes and renames the whole file, and gstack-question-log starts a
|
||||||
|
# background --derive after every append, so writers overlap: a shared temp
|
||||||
|
# name made renames fail and an unlocked rewrite dropped a concurrent
|
||||||
|
# --log-session. Writers hold a mkdir lock (<profile>.lock/, whose owner file
|
||||||
|
# names the holder's pid and a random token; release is ownership-checked)
|
||||||
|
# from before their read until after their rename, and write a unique temp
|
||||||
|
# file. A lock whose holder process is gone is taken over at once; one older
|
||||||
|
# than 5 minutes is taken over even when its pid is alive, since no writer
|
||||||
|
# runs that long and the pid may have been reused. Takeovers are serialized by a second mkdir lock
|
||||||
|
# (<profile>.lock.takeover/) and re-read the owner under it, so two waiters
|
||||||
|
# cannot both take over or remove a lock a third writer just took. A lock or
|
||||||
|
# takeover directory older than 30 s with no owner belongs to a writer that
|
||||||
|
# died before writing one; cleanup removes a takeover lock this writer holds. A writer that cannot get the lock within the
|
||||||
|
# give-up window writes nothing and exits 5. Writers outside this binary
|
||||||
|
# (gstack-distill-apply, /plan-tune's declared edits) do not take it yet.
|
||||||
|
# -----------------------------------------------------------------------
|
||||||
|
LOCK_DIR="$PROFILE_FILE.lock"
|
||||||
|
TAKEOVER_DIR="$LOCK_DIR.takeover"
|
||||||
|
LOCK_TOKEN=""
|
||||||
|
HOLDS_TAKEOVER=""
|
||||||
|
PROFILE_TMP=""
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
if [ -n "$PROFILE_TMP" ]; then rm -f "$PROFILE_TMP"; fi
|
||||||
|
if [ -n "$HOLDS_TAKEOVER" ]; then rmdir "$TAKEOVER_DIR" 2>/dev/null || true; fi
|
||||||
|
if [ -n "$LOCK_TOKEN" ] && [ "$(cat "$LOCK_DIR/owner" 2>/dev/null || true)" = "$LOCK_TOKEN" ]; then
|
||||||
|
rm -rf "$LOCK_DIR"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
trap 'exit 130' INT
|
||||||
|
trap 'exit 143' TERM
|
||||||
|
|
||||||
|
dir_age_s() {
|
||||||
|
local mtime
|
||||||
|
# GNU stat first: BSD-style stat -f on Linux prints a filesystem block.
|
||||||
|
mtime=$(stat -c %Y "$1" 2>/dev/null || stat -f %m "$1" 2>/dev/null || echo "")
|
||||||
|
case "$mtime" in ""|*[!0-9]*) echo 0 ;; *) echo $(( $(date +%s) - mtime )) ;; esac
|
||||||
|
}
|
||||||
|
|
||||||
|
lock_abandoned() {
|
||||||
|
if [ -n "$1" ]; then
|
||||||
|
! kill -0 "${1%%-*}" 2>/dev/null || [ "$(dir_age_s "$LOCK_DIR")" -gt 300 ]
|
||||||
|
else
|
||||||
|
[ "$(dir_age_s "$LOCK_DIR")" -gt 30 ]
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
acquire_lock() {
|
||||||
|
if [ -n "$LOCK_TOKEN" ]; then return 0; fi
|
||||||
|
mkdir -p "$GSTACK_HOME"
|
||||||
|
# GSTACK_PROFILE_LOCK_TIMEOUT_MS: test-only override for the give-up.
|
||||||
|
local give_up_ms="${GSTACK_PROFILE_LOCK_TIMEOUT_MS:-10000}" waited_ms=0 started=$SECONDS vanished=0 owner
|
||||||
|
local token="$$-$RANDOM$RANDOM"
|
||||||
|
until mkdir "$LOCK_DIR" 2>/dev/null; do
|
||||||
|
# mkdir failed with no lock to wait for: retry once in case a holder just
|
||||||
|
# released it, then report the directory instead of waiting out the window.
|
||||||
|
if [ ! -e "$LOCK_DIR" ]; then
|
||||||
|
if [ "$vanished" -eq 0 ]; then vanished=1; continue; fi
|
||||||
|
echo "gstack-developer-profile: cannot create $LOCK_DIR (is $GSTACK_HOME writable?), so nothing was written (exit 5)." >&2
|
||||||
|
exit 5
|
||||||
|
fi
|
||||||
|
vanished=0
|
||||||
|
owner=$(cat "$LOCK_DIR/owner" 2>/dev/null || true)
|
||||||
|
if [ -d "$LOCK_DIR" ] && lock_abandoned "$owner"; then
|
||||||
|
if [ -d "$TAKEOVER_DIR" ] && [ "$(dir_age_s "$TAKEOVER_DIR")" -gt 30 ]; then rmdir "$TAKEOVER_DIR" 2>/dev/null || true; fi
|
||||||
|
if mkdir "$TAKEOVER_DIR" 2>/dev/null; then
|
||||||
|
HOLDS_TAKEOVER=1
|
||||||
|
if [ "$(cat "$LOCK_DIR/owner" 2>/dev/null || true)" = "$owner" ] && lock_abandoned "$owner"; then
|
||||||
|
rm -rf "$LOCK_DIR" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
rmdir "$TAKEOVER_DIR" 2>/dev/null || true
|
||||||
|
HOLDS_TAKEOVER=""
|
||||||
|
if [ ! -e "$LOCK_DIR" ]; then continue; fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ "$waited_ms" -ge "$give_up_ms" ] || [ $(( SECONDS - started )) -gt $(( give_up_ms / 1000 )) ]; then
|
||||||
|
echo "gstack-developer-profile: another writer held $LOCK_DIR for ${give_up_ms} ms, so nothing was written (exit 5)." >&2
|
||||||
|
exit 5
|
||||||
|
fi
|
||||||
|
sleep 0.05
|
||||||
|
waited_ms=$(( waited_ms + 50 ))
|
||||||
|
done
|
||||||
|
printf '%s\n' "$token" > "$LOCK_DIR/owner"
|
||||||
|
LOCK_TOKEN="$token"
|
||||||
|
}
|
||||||
|
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
# Migration: builder-profile.jsonl → developer-profile.json
|
# Migration: builder-profile.jsonl → developer-profile.json
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
do_migrate() {
|
do_migrate() {
|
||||||
|
acquire_lock
|
||||||
if [ ! -f "$LEGACY_FILE" ]; then
|
if [ ! -f "$LEGACY_FILE" ]; then
|
||||||
echo "MIGRATE: no legacy file to migrate"
|
echo "MIGRATE: no legacy file to migrate"
|
||||||
return 0
|
return 0
|
||||||
@@ -68,9 +166,7 @@ do_migrate() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# Run migration in a temp file, then atomic rename.
|
# Run migration in a temp file, then atomic rename.
|
||||||
local TMPOUT
|
PROFILE_TMP=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
||||||
TMPOUT=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
|
||||||
trap 'rm -f "$TMPOUT"' EXIT
|
|
||||||
|
|
||||||
cat "$LEGACY_FILE" | bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
cat "$LEGACY_FILE" | bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
||||||
const lines = (await Bun.stdin.text()).trim().split('\n').filter(Boolean);
|
const lines = (await Bun.stdin.text()).trim().split('\n').filter(Boolean);
|
||||||
@@ -113,11 +209,11 @@ do_migrate() {
|
|||||||
schema_version: 1,
|
schema_version: 1,
|
||||||
};
|
};
|
||||||
console.log(JSON.stringify(profile, null, 2));
|
console.log(JSON.stringify(profile, null, 2));
|
||||||
" > "$TMPOUT"
|
" > "$PROFILE_TMP"
|
||||||
|
|
||||||
# Atomic rename.
|
# Atomic rename.
|
||||||
mv "$TMPOUT" "$PROFILE_FILE"
|
mv "$PROFILE_TMP" "$PROFILE_FILE"
|
||||||
trap - EXIT
|
PROFILE_TMP=""
|
||||||
|
|
||||||
# gbrain-sync: enqueue the migrated file for cross-machine sync (no-op if off).
|
# gbrain-sync: enqueue the migrated file for cross-machine sync (no-op if off).
|
||||||
SCRIPT_DIR_E="$(cd "$(dirname "$0")" && pwd)"
|
SCRIPT_DIR_E="$(cd "$(dirname "$0")" && pwd)"
|
||||||
@@ -205,13 +301,12 @@ do_log_session() {
|
|||||||
*) echo "gstack-developer-profile: bun printed no verdict while validating the session; nothing was logged" >&2; return 1 ;;
|
*) echo "gstack-developer-profile: bun printed no verdict while validating the session; nothing was logged" >&2; return 1 ;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
acquire_lock
|
||||||
ensure_profile
|
ensure_profile
|
||||||
|
|
||||||
local TMPOUT
|
PROFILE_TMP=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
||||||
TMPOUT=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
|
||||||
trap 'rm -f "$TMPOUT"' EXIT
|
|
||||||
|
|
||||||
PROFILE_FILE_PATH="$PROFILE_FILE" RECORD_INPUT="$INPUT" TMPOUT_PATH="$TMPOUT" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
PROFILE_FILE_PATH="$PROFILE_FILE" RECORD_INPUT="$INPUT" TMPOUT_PATH="$PROFILE_TMP" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
||||||
const fs = require('fs');
|
const fs = require('fs');
|
||||||
const entry = JSON.parse(process.env.RECORD_INPUT);
|
const entry = JSON.parse(process.env.RECORD_INPUT);
|
||||||
if (!entry.ts) entry.ts = new Date().toISOString();
|
if (!entry.ts) entry.ts = new Date().toISOString();
|
||||||
@@ -238,8 +333,8 @@ do_log_session() {
|
|||||||
fs.writeFileSync(process.env.TMPOUT_PATH, JSON.stringify(profile, null, 2));
|
fs.writeFileSync(process.env.TMPOUT_PATH, JSON.stringify(profile, null, 2));
|
||||||
"
|
"
|
||||||
|
|
||||||
mv "$TMPOUT" "$PROFILE_FILE"
|
mv "$PROFILE_TMP" "$PROFILE_FILE"
|
||||||
trap - EXIT
|
PROFILE_TMP=""
|
||||||
"$SCRIPT_DIR/gstack-brain-enqueue" "developer-profile.json" 2>/dev/null &
|
"$SCRIPT_DIR/gstack-brain-enqueue" "developer-profile.json" 2>/dev/null &
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -252,13 +347,12 @@ do_log_session() {
|
|||||||
# rewritten only when something was added.
|
# rewritten only when something was added.
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
do_reconcile() {
|
do_reconcile() {
|
||||||
|
acquire_lock
|
||||||
ensure_profile
|
ensure_profile
|
||||||
|
|
||||||
local TMPOUT
|
PROFILE_TMP=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
||||||
TMPOUT=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
|
||||||
trap 'rm -f "$TMPOUT"' EXIT
|
|
||||||
|
|
||||||
PROFILE_FILE_PATH="$PROFILE_FILE" PROJECTS_DIR="$GSTACK_HOME/projects" TMPOUT_PATH="$TMPOUT" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
PROFILE_FILE_PATH="$PROFILE_FILE" PROJECTS_DIR="$GSTACK_HOME/projects" TMPOUT_PATH="$PROFILE_TMP" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
||||||
const fs = require('fs');
|
const fs = require('fs');
|
||||||
const path = require('path');
|
const path = require('path');
|
||||||
const profile = JSON.parse(fs.readFileSync(process.env.PROFILE_FILE_PATH, 'utf-8'));
|
const profile = JSON.parse(fs.readFileSync(process.env.PROFILE_FILE_PATH, 'utf-8'));
|
||||||
@@ -313,13 +407,13 @@ do_reconcile() {
|
|||||||
console.log('RECONCILE: ' + added + ' backfilled from ' + runs.length + ' office-hours run(s)');
|
console.log('RECONCILE: ' + added + ' backfilled from ' + runs.length + ' office-hours run(s)');
|
||||||
"
|
"
|
||||||
|
|
||||||
if [ -s "$TMPOUT" ]; then
|
if [ -s "$PROFILE_TMP" ]; then
|
||||||
mv "$TMPOUT" "$PROFILE_FILE"
|
mv "$PROFILE_TMP" "$PROFILE_FILE"
|
||||||
trap - EXIT
|
PROFILE_TMP=""
|
||||||
"$SCRIPT_DIR/gstack-brain-enqueue" "developer-profile.json" 2>/dev/null &
|
"$SCRIPT_DIR/gstack-brain-enqueue" "developer-profile.json" 2>/dev/null &
|
||||||
else
|
else
|
||||||
rm -f "$TMPOUT"
|
rm -f "$PROFILE_TMP"
|
||||||
trap - EXIT
|
PROFILE_TMP=""
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -506,6 +600,7 @@ function accumulateSignals(lines, QUESTIONS, SIGNAL_MAP, dims) {
|
|||||||
# Derive: recompute inferred dimensions from question-events.jsonl
|
# Derive: recompute inferred dimensions from question-events.jsonl
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
do_derive() {
|
do_derive() {
|
||||||
|
acquire_lock
|
||||||
ensure_profile
|
ensure_profile
|
||||||
local EVENTS="$GSTACK_HOME/projects/$SLUG/question-log.jsonl"
|
local EVENTS="$GSTACK_HOME/projects/$SLUG/question-log.jsonl"
|
||||||
local REGISTRY="$ROOT_DIR/scripts/question-registry.ts"
|
local REGISTRY="$ROOT_DIR/scripts/question-registry.ts"
|
||||||
@@ -515,8 +610,9 @@ do_derive() {
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
PROFILE_TMP=$(mktemp "$GSTACK_HOME/developer-profile.json.tmp.XXXXXX")
|
||||||
cd "$ROOT_DIR"
|
cd "$ROOT_DIR"
|
||||||
PROFILE_FILE_PATH="$PROFILE_FILE" EVENTS_PATH="$EVENTS" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
PROFILE_FILE_PATH="$PROFILE_FILE" EVENTS_PATH="$EVENTS" TMPOUT_PATH="$PROFILE_TMP" bun --no-env-file --config="${GSTACK_BUN_CONFIG:-/dev/null}" -e "
|
||||||
import('./scripts/question-registry.ts').then(async (regmod) => {
|
import('./scripts/question-registry.ts').then(async (regmod) => {
|
||||||
const sigmod = await import('./scripts/psychographic-signals.ts');
|
const sigmod = await import('./scripts/psychographic-signals.ts');
|
||||||
const fs = require('fs');
|
const fs = require('fs');
|
||||||
@@ -568,12 +664,12 @@ do_derive() {
|
|||||||
profile.gap = gap;
|
profile.gap = gap;
|
||||||
profile.derived_at = new Date().toISOString();
|
profile.derived_at = new Date().toISOString();
|
||||||
|
|
||||||
const tmp = profilePath + '.tmp';
|
fs.writeFileSync(process.env.TMPOUT_PATH, JSON.stringify(profile, null, 2));
|
||||||
fs.writeFileSync(tmp, JSON.stringify(profile, null, 2));
|
|
||||||
fs.renameSync(tmp, profilePath);
|
|
||||||
console.log('DERIVE: ok — ' + count + ' events, ' + skills.size + ' skills, ' + qids.size + ' questions');
|
console.log('DERIVE: ok — ' + count + ' events, ' + skills.size + ' skills, ' + qids.size + ' questions');
|
||||||
}).catch(err => { console.error('DERIVE:', err.message); process.exit(1); });
|
}).catch(err => { console.error('DERIVE:', err.message); process.exit(1); });
|
||||||
"
|
"
|
||||||
|
mv "$PROFILE_TMP" "$PROFILE_FILE"
|
||||||
|
PROFILE_TMP=""
|
||||||
}
|
}
|
||||||
|
|
||||||
# -----------------------------------------------------------------------
|
# -----------------------------------------------------------------------
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "gstack",
|
"name": "gstack",
|
||||||
"version": "1.92.68",
|
"version": "1.92.69",
|
||||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
|||||||
@@ -306,29 +306,6 @@ export function resolveOptionKey(question_id: string, choice: string): string |
|
|||||||
return tie ? undefined : best;
|
return tie ? undefined : best;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Apply a user choice for a question to the running dimension totals.
|
|
||||||
*
|
|
||||||
* @param dims - running total of dimension nudges (mutated)
|
|
||||||
* @param signal_key - from the question registry entry
|
|
||||||
* @param user_choice - the option key the user selected
|
|
||||||
* @returns list of dimension deltas applied (empty if no mapping)
|
|
||||||
*/
|
|
||||||
export function applySignal(
|
|
||||||
dims: Record<Dimension, number>,
|
|
||||||
signal_key: string,
|
|
||||||
user_choice: string,
|
|
||||||
): DimensionDelta[] {
|
|
||||||
const subMap = SIGNAL_MAP[signal_key];
|
|
||||||
if (!subMap) return [];
|
|
||||||
const deltas = subMap[user_choice];
|
|
||||||
if (!deltas) return [];
|
|
||||||
for (const { dim, delta } of deltas) {
|
|
||||||
dims[dim] = (dims[dim] ?? 0) + delta;
|
|
||||||
}
|
|
||||||
return deltas;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Validate that every signal_key referenced in the registry has a matching
|
* Validate that every signal_key referenced in the registry has a matching
|
||||||
* entry in SIGNAL_MAP. Called by tests to catch drift.
|
* entry in SIGNAL_MAP. Called by tests to catch drift.
|
||||||
@@ -353,17 +330,6 @@ export function validateRegistrySignalKeys(): {
|
|||||||
return { missing, extra };
|
return { missing, extra };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Empty dimension totals — starting point for derivation. */
|
|
||||||
export function newDimensionTotals(): Record<Dimension, number> {
|
|
||||||
return {
|
|
||||||
scope_appetite: 0,
|
|
||||||
risk_tolerance: 0,
|
|
||||||
detail_preference: 0,
|
|
||||||
autonomy: 0,
|
|
||||||
architecture_care: 0,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Sigmoid clamp: map accumulated delta total to [0, 1]. */
|
/** Sigmoid clamp: map accumulated delta total to [0, 1]. */
|
||||||
export function normalizeToDimensionValue(total: number): number {
|
export function normalizeToDimensionValue(total: number): number {
|
||||||
// Simple sigmoid: each 1.0 of accumulated delta approaches saturation.
|
// Simple sigmoid: each 1.0 of accumulated delta approaches saturation.
|
||||||
|
|||||||
@@ -770,6 +770,155 @@ describe('gstack-developer-profile --log-session (#1671 fix)', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// Concurrent writers (#3437). gstack-question-log starts a background
|
||||||
|
// --derive after every append, and a multi-question AskUserQuestion logs
|
||||||
|
// several rows at once, so derives overlap each other and /office-hours'
|
||||||
|
// --log-session. Each subcommand rewrites the whole profile, so every
|
||||||
|
// read-modify-write holds the profile lock and writes a unique temp file.
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
async function runDevAsync(...args: string[]): Promise<{ stdout: string; stderr: string; status: number }> {
|
||||||
|
const proc = Bun.spawn([BIN_DEV, ...args], {
|
||||||
|
env: { ...process.env, GSTACK_HOME: tmpHome },
|
||||||
|
cwd: ROOT,
|
||||||
|
stdout: 'pipe',
|
||||||
|
stderr: 'pipe',
|
||||||
|
});
|
||||||
|
const timer = setTimeout(() => proc.kill(), 60_000);
|
||||||
|
const [stdout, stderr, status] = await Promise.all([
|
||||||
|
new Response(proc.stdout).text(),
|
||||||
|
new Response(proc.stderr).text(),
|
||||||
|
proc.exited,
|
||||||
|
]);
|
||||||
|
clearTimeout(timer);
|
||||||
|
return { stdout, stderr, status };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('gstack-developer-profile concurrent writers (#3437)', () => {
|
||||||
|
const lockDir = () => path.join(tmpHome, 'developer-profile.json.lock');
|
||||||
|
|
||||||
|
test('--derive does not write through a fixed temp path', () => {
|
||||||
|
fs.mkdirSync(path.join(tmpHome, 'developer-profile.json.tmp'));
|
||||||
|
const r = runDev('--derive');
|
||||||
|
expect(r.stderr).toBe('');
|
||||||
|
expect(r.status).toBe(0);
|
||||||
|
expect(typeof (readProfile() as { derived_at?: string }).derived_at).toBe('string');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a held profile lock keeps --derive and --log-session from writing', () => {
|
||||||
|
expect(runDev('--derive').status).toBe(0);
|
||||||
|
const before = fs.readFileSync(path.join(tmpHome, 'developer-profile.json'), 'utf-8');
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
fs.writeFileSync(path.join(lockDir(), 'owner'), process.pid + '-held\n');
|
||||||
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_PROFILE_LOCK_TIMEOUT_MS: '300' };
|
||||||
|
for (const args of [['--derive'], ['--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder' })]]) {
|
||||||
|
const res = spawnSync(BIN_DEV, args, { env, encoding: 'utf-8', cwd: ROOT, timeout: 30_000 });
|
||||||
|
expect(res.status).toBe(5);
|
||||||
|
expect(res.stderr).toContain('developer-profile.json.lock');
|
||||||
|
}
|
||||||
|
expect(fs.readFileSync(path.join(tmpHome, 'developer-profile.json'), 'utf-8')).toBe(before);
|
||||||
|
expect(fs.readFileSync(path.join(lockDir(), 'owner'), 'utf-8')).toBe(process.pid + '-held\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a stale lock left by a crashed writer is taken over', () => {
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
const old = new Date(Date.now() - 120_000);
|
||||||
|
fs.utimesSync(lockDir(), old, old);
|
||||||
|
const r = runDev('--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder', project_slug: 'p' }));
|
||||||
|
expect(r.status).toBe(0);
|
||||||
|
expect((readProfile() as { sessions: unknown[] }).sessions.length).toBe(1);
|
||||||
|
expect(fs.existsSync(lockDir())).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a lock whose holder process is gone is taken over at once', () => {
|
||||||
|
const gone = spawnSync('true', [], { timeout: 10_000 }).pid;
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
fs.writeFileSync(path.join(lockDir(), 'owner'), gone + '-crashed\n');
|
||||||
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_PROFILE_LOCK_TIMEOUT_MS: '300' };
|
||||||
|
const res = spawnSync(BIN_DEV, ['--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder', project_slug: 'p' })], { env, encoding: 'utf-8', cwd: ROOT, timeout: 30_000 });
|
||||||
|
expect(res.stderr).toBe('');
|
||||||
|
expect(res.status).toBe(0);
|
||||||
|
expect((readProfile() as { sessions: unknown[] }).sessions.length).toBe(1);
|
||||||
|
expect(fs.existsSync(lockDir())).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a lock older than 5 minutes is taken over even when its pid is alive (pid reuse)', () => {
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
fs.writeFileSync(path.join(lockDir(), 'owner'), process.pid + '-reused\n');
|
||||||
|
const old = new Date(Date.now() - 10 * 60_000);
|
||||||
|
fs.utimesSync(lockDir(), old, old);
|
||||||
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_PROFILE_LOCK_TIMEOUT_MS: '300' };
|
||||||
|
const res = spawnSync(BIN_DEV, ['--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder', project_slug: 'p' })], { env, encoding: 'utf-8', cwd: ROOT, timeout: 30_000 });
|
||||||
|
expect(res.stderr).toBe('');
|
||||||
|
expect(res.status).toBe(0);
|
||||||
|
expect((readProfile() as { sessions: unknown[] }).sessions.length).toBe(1);
|
||||||
|
expect(fs.existsSync(lockDir())).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a takeover already in progress is not repeated', () => {
|
||||||
|
const gone = spawnSync('true', [], { timeout: 10_000 }).pid;
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
fs.writeFileSync(path.join(lockDir(), 'owner'), gone + '-crashed\n');
|
||||||
|
fs.mkdirSync(lockDir() + '.takeover');
|
||||||
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_PROFILE_LOCK_TIMEOUT_MS: '300' };
|
||||||
|
const res = spawnSync(BIN_DEV, ['--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder' })], { env, encoding: 'utf-8', cwd: ROOT, timeout: 30_000 });
|
||||||
|
expect(res.status).toBe(5);
|
||||||
|
expect(fs.readFileSync(path.join(lockDir(), 'owner'), 'utf-8')).toBe(gone + '-crashed\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a takeover that cannot remove the lock still gives up', () => {
|
||||||
|
expect(runDev('--derive').status).toBe(0);
|
||||||
|
fs.mkdirSync(lockDir());
|
||||||
|
const old = new Date(Date.now() - 120_000);
|
||||||
|
fs.utimesSync(lockDir(), old, old);
|
||||||
|
fs.chmodSync(tmpHome, 0o555);
|
||||||
|
try {
|
||||||
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_PROFILE_LOCK_TIMEOUT_MS: '300' };
|
||||||
|
const res = spawnSync(BIN_DEV, ['--derive'], { env, encoding: 'utf-8', cwd: ROOT, timeout: 10_000 });
|
||||||
|
expect(res.signal).toBeNull();
|
||||||
|
expect([0, 5]).toContain(res.status ?? -1);
|
||||||
|
} finally {
|
||||||
|
fs.chmodSync(tmpHome, 0o755);
|
||||||
|
}
|
||||||
|
}, 20_000);
|
||||||
|
|
||||||
|
test('a relative state root does not leave the lock behind', () => {
|
||||||
|
const env = { ...process.env, GSTACK_HOME: 'state' };
|
||||||
|
for (const args of [['--derive'], ['--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder', project_slug: 'p' })]]) {
|
||||||
|
const res = spawnSync(BIN_DEV, args, { env, encoding: 'utf-8', cwd: tmpHome, timeout: 30_000 });
|
||||||
|
expect(res.stderr).toBe('');
|
||||||
|
expect(res.status).toBe(0);
|
||||||
|
}
|
||||||
|
const p = JSON.parse(fs.readFileSync(path.join(tmpHome, 'state', 'developer-profile.json'), 'utf-8'));
|
||||||
|
expect(p.sessions.length).toBe(1);
|
||||||
|
expect(typeof p.derived_at).toBe('string');
|
||||||
|
expect(fs.existsSync(path.join(tmpHome, 'state', 'developer-profile.json.lock'))).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('overlapping derives and log-sessions keep every session and leave valid JSON', async () => {
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
logQuestion({ skill: 'plan-ceo-review', question_id: 'plan-ceo-review-mode', question_summary: 'mode', user_choice: 'expand', session_id: 's' + i });
|
||||||
|
}
|
||||||
|
expect(runDev('--derive').status).toBe(0);
|
||||||
|
const rounds = 6;
|
||||||
|
for (let round = 0; round < rounds; round++) {
|
||||||
|
const runs = await Promise.all([
|
||||||
|
...Array.from({ length: 6 }, () => runDevAsync('--derive')),
|
||||||
|
runDevAsync('--log-session', JSON.stringify({ date: '2026-10-31', mode: 'builder', project_slug: 'round-' + round })),
|
||||||
|
]);
|
||||||
|
for (const r of runs) {
|
||||||
|
expect(r.stderr).toBe('');
|
||||||
|
expect(r.status).toBe(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const p = readProfile() as { sessions: Array<{ project_slug: string }>; inferred: { sample_size: number } };
|
||||||
|
expect(p.sessions.map((s) => s.project_slug).sort()).toEqual(Array.from({ length: rounds }, (_, i) => 'round-' + i).sort());
|
||||||
|
expect(p.inferred.sample_size).toBe(5);
|
||||||
|
expect(fs.readdirSync(tmpHome).filter((f) => f.startsWith('developer-profile.json.'))).toEqual([]);
|
||||||
|
}, 120_000);
|
||||||
|
});
|
||||||
|
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
// SESSION_COUNT / TIER / NUDGE_ELIGIBLE must ignore mode:resources entries.
|
// SESSION_COUNT / TIER / NUDGE_ELIGIBLE must ignore mode:resources entries.
|
||||||
//
|
//
|
||||||
|
|||||||
@@ -29,9 +29,7 @@ import {
|
|||||||
} from '../scripts/one-way-doors';
|
} from '../scripts/one-way-doors';
|
||||||
import {
|
import {
|
||||||
SIGNAL_MAP,
|
SIGNAL_MAP,
|
||||||
applySignal,
|
|
||||||
validateRegistrySignalKeys,
|
validateRegistrySignalKeys,
|
||||||
newDimensionTotals,
|
|
||||||
normalizeToDimensionValue,
|
normalizeToDimensionValue,
|
||||||
resolveOptionKey,
|
resolveOptionKey,
|
||||||
ALL_DIMENSIONS,
|
ALL_DIMENSIONS,
|
||||||
@@ -246,26 +244,6 @@ describe('psychographic signal map', () => {
|
|||||||
expect(missing).toEqual([]);
|
expect(missing).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('applySignal mutates dimension totals per mapping', () => {
|
|
||||||
const dims = newDimensionTotals();
|
|
||||||
const applied = applySignal(dims, 'scope-appetite', 'expand');
|
|
||||||
expect(applied.length).toBeGreaterThan(0);
|
|
||||||
expect(dims.scope_appetite).toBeCloseTo(0.06, 5);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('applySignal returns [] for unknown signal_key', () => {
|
|
||||||
const dims = newDimensionTotals();
|
|
||||||
const applied = applySignal(dims, 'no-such-signal', 'anything');
|
|
||||||
expect(applied).toEqual([]);
|
|
||||||
expect(dims.scope_appetite).toBe(0);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('applySignal returns [] for unknown user_choice', () => {
|
|
||||||
const dims = newDimensionTotals();
|
|
||||||
const applied = applySignal(dims, 'scope-appetite', 'definitely-not-a-real-choice');
|
|
||||||
expect(applied).toEqual([]);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('normalizeToDimensionValue maps 0 → 0.5 (neutral)', () => {
|
test('normalizeToDimensionValue maps 0 → 0.5 (neutral)', () => {
|
||||||
expect(normalizeToDimensionValue(0)).toBeCloseTo(0.5, 5);
|
expect(normalizeToDimensionValue(0)).toBeCloseTo(0.5, 5);
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in new issue
Block a user