fix(auq): harden error-fallback hook + harness per adversarial review

Codex pre-landing review found three real issues:
- The PostToolUse fallback hook shared source 'plan-tune-cathedral' with the
  question-log hook (same event+matcher); gstack-settings-hook replaces the entry,
  so it would have clobbered plan-tune capture. Give it its own 'auq-error-fallback'
  source (separate entry, both run); ALREADY_INSTALLED now requires both sources.
- isErrorResponse triggered on any string containing 'internal error'/'is_error',
  so a real answer or a {"is_error": false} payload could fire the fallback after a
  successful question. Narrow it to the missing-result sentinel + boolean is_error.
- The SDK runner mutated process.env.GSTACK_HEADLESS process-wide (leaked headless
  into later tests). Removed; GSTACK_HEADLESS=1 now lives in the eval package.json
  scripts, scoped to the invocation and inherited by the SDK child.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-06-07 19:53:01 -07:00
parent 0ddaf27aa2
commit 6a6900f15b
5 changed files with 53 additions and 32 deletions
+10 -5
View File
@@ -98,20 +98,25 @@ export function isErrorResponse(response: unknown): boolean {
if (typeof response === 'string') {
const s = response.trim();
if (s === '') return true;
return /tool result missing|internal error|\bis_error\b/i.test(s);
// Match ONLY the specific missing-result sentinel phrase, not any string that
// merely contains "error" — a real answer like "Investigate the internal error"
// must NOT trigger the fallback. (Codex review finding.)
return /tool result missing/i.test(s);
}
if (typeof response === 'object') {
const rec = response as Record<string, unknown>;
if (rec.is_error === true || rec.isError === true || rec.error) return true;
// Structured flag must be the boolean true — not the substring "is_error" inside
// a serialized success payload like '{"is_error": false}'.
if (rec.is_error === true || rec.isError === true) return true;
if (typeof rec.error === 'string' && rec.error.trim() !== '') return true;
// Some hosts wrap the payload as { content: "..." } or { content: [{text}] }.
const content = rec.content;
if (typeof content === 'string') return isErrorResponse(content);
if (typeof content === 'string') return /tool result missing/i.test(content);
if (Array.isArray(content)) {
const text = content
.map((c) => (typeof c === 'string' ? c : (c as Record<string, unknown>)?.text ?? ''))
.join(' ');
if (text.trim() === '') return false; // empty content array on success is ambiguous; don't trigger
return /tool result missing|internal error/i.test(text);
return /tool result missing/i.test(text);
}
}
return false;