CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-14 17:05:28 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix: default cross-model workflows to frontier models

Browse Source
This commit is contained in:
Garry Tan
2026-09-09 02:48:34 +00:00
parent 0530392821
commit 939da0c203
69 changed files with 603 additions and 255 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+8 -2
View File
@@ -368,11 +368,14 @@ Three reasons:
### Template test tiers
Paid-tier cost and speed estimates below predate the current model defaults.
See [eval defaults and overrides](CONTRIBUTING.md#testing--evals).
| Tier | What | Cost | Speed |
|------|------|------|-------|
| 1 — Static validation | Parse every `$B` command in SKILL.md and validate it against the registry; pin the Aside contract sentences and the render wrapper's option mapping | Free | <2s |
| 2 — E2E via `claude -p` | Spawn real Claude session, run each skill, check for errors | ~$3.85 | ~20min |
| 3 — LLM-as-judge | Sonnet scores docs on clarity/completeness/actionability | ~$0.15 | ~30s |
| 3 — LLM-as-judge | `claude-fable-5-1` by default scores docs on clarity/completeness/actionability | ~$0.15 | ~30s |
Tier 1 runs on every `bun run test`. Tiers 2+3 are gated behind `EVALS=1`. The idea is: catch 95% of issues for free, use LLMs only for judgment calls.
@@ -482,11 +485,14 @@ The `EvalCollector` accumulates test results and writes them in two ways:
### Test tiers
Paid-tier cost and speed estimates below predate the current model defaults.
See [eval defaults and overrides](CONTRIBUTING.md#testing--evals).
| Tier | What | Cost | Speed |
|------|------|------|-------|
| 1 — Static validation | Parse `$B` commands against the registry, Aside contract pins, render-wrapper pins, observability unit tests | Free | <5s |
| 2 — E2E via `claude -p` | Spawn real Claude session, run each skill, scan for errors | ~$3.85 | ~20min |
| 3 — LLM-as-judge | Sonnet scores docs on clarity/completeness/actionability | ~$0.15 | ~30s |
| 3 — LLM-as-judge | `claude-fable-5-1` by default scores docs on clarity/completeness/actionability | ~$0.15 | ~30s |
Tier 1 runs on every `bun run test`. Tiers 2+3 are gated behind `EVALS=1`. The idea: catch 95% of issues for free, use LLMs only for judgment calls and integration testing.