mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-14 08:59:01 +02:00
Garry TanandClaude Fable 5
4c20eca33b
Ponytail's honest-benchmark method pointed at gstack itself: 3 build-shaped
tasks (native-platform over-build trap, CRUD endpoint, bug fix with planted
decoys) x 2 arms, real claude -p sessions, scored on the git diff left
behind. A research instrument, not a release gate — no assertion compares
arm scores.
Arms use the PROVEN project-scope pattern: the with-arm installs a
build-discipline skill (extracted reuse-ladder + bounded-closer content, not
whole-file copies) into the fixture's .claude/skills/ with a CLAUDE.md
routing line and an explicit invocation; a live spike confirmed claude -p
discovers and invokes project-scope skills via the Skill tool (3 turns,
exact-output probe). Fixtures are git init + local bare origin; diff capture
is three lines of git, no worktree machinery.
Failure taxonomy: zero-diff arms are VALID scored cells (deterministic
0/none, no API call), harvest failures record harvest:null, judge_error
cells are excluded from aggregates but named in the report — nothing drops
silently. armJudge: fixed sonnet judge, 0-3 unrequested-structure rubric,
must name the construct or say none, bounded retry-on-malformed; callJudge
gains optional temperature/max_tokens (defaults unchanged). recordE2E now
populates tokens_used for every E2E. Eval schema v2: harvest gains
{insertions, deletions, net}, tolerant reads keep v1 runs comparable.
Registered periodic in E2E_TIERS + touchfiles (with the auq-repetition-cut
A/B); periodic detach timeout raised to the new shard-census floor. Free
selftest (8 tests, zero API) pins fixtures, extraction, arm asymmetry, diff
capture, judge plumbing, and the retry bound.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
23 lines
668 B
JavaScript
23 lines
668 B
JavaScript
const http = require('node:http');
|
|
const { handleRequest } = require('./app');
|
|
|
|
const server = http.createServer((req, res) => {
|
|
let raw = '';
|
|
req.on('data', (chunk) => { raw += chunk; });
|
|
req.on('end', () => {
|
|
let body = null;
|
|
if (raw) {
|
|
try { body = JSON.parse(raw); } catch { body = null; }
|
|
}
|
|
const result = handleRequest(req.method, req.url, body);
|
|
res.writeHead(result.status, { 'content-type': 'application/json' });
|
|
res.end(result.body === undefined ? '' : JSON.stringify(result.body));
|
|
});
|
|
});
|
|
|
|
if (require.main === module) {
|
|
server.listen(3000, () => console.log('notes api on :3000'));
|
|
}
|
|
|
|
module.exports = { server };
|