CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-13 08:29:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

v1.84.1.0 fix: default Codex and Claude to frontier models (#2835)

Browse Source 
* fix: default cross-model workflows to frontier models

* chore: bump version and changelog (v1.82.1.0)

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix: repair frontier eval budgets and workflow instructions

Preserve frontier models and quality thresholds while fixing truncated judge output, ordered section expansion, consent checks, QA scoring, and ship audit gates. Add regression coverage and refresh generated docs.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix: resolve workflow gaps exposed by frontier evals

Clarify plan-review ordering and fallback modes, preserve deploy readiness gates, honor configured merge methods, correct benchmark and canary contracts, and restore vendored installs on setup failure. Cover recovery with real-shell regressions.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix: use agent capture budgets for deploy evals

Multi-turn deploy and benchmark sessions were incorrectly limited to the single-call judge timeout. Use the existing capture tier and leave outer-test cleanup headroom, with a free policy regression test. Keep all behavioral assertions and frontier models unchanged.

Co-Authored-By: OpenAI Codex <noreply@openai.com>

* fix: clarify retro workflow and evaluate compare instructions

Include compare mode in the frontier judge excerpt, define metric sources and snapshot ordering, and preserve the existing prompt-size budget.

Co-authored-by: OpenAI Codex <noreply@openai.com>

* fix: make documentation release review and publication consistent

Review before commit, clarify changelog safeguards and unavailable reviewer modes, and preserve raw PR bodies across separate shell calls. Keep title sync in one shell and add regression coverage.

Co-authored-by: OpenAI Codex <noreply@openai.com>

---------

Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
2026-09-09 08:57:21 -07:00
committed by GitHub
co-authored by OpenAI Codex
parent c8f0c4e368
commit 71f6048e8a
141 changed files with 2678 additions and 1750 deletions
Download Patch File Download Diff File Expand all files Collapse all files
canary/SKILL.md
+10 -3
View File
@@ -570,6 +570,7 @@ console.log("GSTACK_STEP_OK");
Then copy the screenshot out of the printed session directory: `cp "<ASIDE_DIR>/<page-name>.jpg" .gstack/canary-reports/baselines/<page-name>.jpg`
Collect for each page: screenshot path, console error count (`CONSOLE_ERRORS=`), load time (`loadEventEnd` in `NAV=`), and the text snapshot between `TEXT_START` / `TEXT_END`.
Also run Phase 3's read-only link check for each monitored page and retain the URLs whose `LINK` status is 404. Repeat the same check each monitoring round; other HEAD failures are unknown, not broken links. Compare console messages by identity, not just count, and retain the text snapshot for evidence when a page's content disappears.
Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
@@ -582,7 +583,10 @@ Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
"/": {
"screenshot": "baselines/home.jpg",
"console_errors": 0,
"load_time_ms": 450
"console_error_messages": [],
"load_time_ms": 450,
"broken_links": [],
"text_snapshot": "<TEXT_START/END content>"
}
}
}
@@ -620,11 +624,12 @@ For each page to monitor:
Run the Phase 2 read script for each page with the screenshot saved as `pre-<page-name>.jpg`, then `cp "<ASIDE_DIR>/pre-<page-name>.jpg" .gstack/canary-reports/screenshots/`.
Record the console error count and load time for each page. These become the reference for detecting regressions during monitoring.
Save the same manifest schema as Phase 2 to `.gstack/canary-reports/pre-monitor.json`, with the screenshots' actual paths. This is a monitoring-start reference, not evidence of pre-deploy health. Use it when no baseline exists; never overwrite an existing baseline during monitoring.
### Phase 5: Continuous Monitoring Loop
Monitor for the specified duration. Every 60 seconds, check each page. Nothing persists between scripts — every check re-opens the page from its URL and captures fresh evidence:
Record the start and deadline. After each full round, wait `max(0, 60 - elapsed-round-seconds)` seconds using the host's wait tool or `sleep`. If a round exceeds 60 seconds, start the next immediately and report the actual cadence; never overlap rounds. Stop at the deadline after the current round.
```bash
aside repl '
@@ -655,7 +660,7 @@ After each check, compare results against the baseline (or pre-deploy snapshot):
**Don't cry wolf.** Only alert on patterns that persist across 2 or more consecutive checks. A single transient network blip is not an alert.
**If a CRITICAL or HIGH alert is detected**, immediately notify the user via AskUserQuestion:
**After a CRITICAL or HIGH pattern is confirmed on two consecutive checks**, immediately notify the user via AskUserQuestion. A first occurrence is pending, not yet an alert:
```
CANARY ALERT
@@ -702,6 +707,7 @@ VERDICT: [DEPLOY IS HEALTHY / DEPLOY HAS ISSUES — details above]
```
Save report to `.gstack/canary-reports/{date}-canary.md` and `.gstack/canary-reports/{date}-canary.json`.
Per-page and overall status: BROKEN if any confirmed CRITICAL alert occurred; otherwise DEGRADED if any confirmed alert occurred; otherwise HEALTHY. Note resolved incidents separately without erasing them from the run's status. JSON fields: `url`, `started_at`, `ended_at`, `status`, `pages` (URL, checks, latest metrics, status), and `alerts` (severity, URL, first_seen, confirmed_at, evidence, resolved). Unconfirmed transients go in a separate `observations` array.
Log the result for the review dashboard:
@@ -711,6 +717,7 @@ mkdir -p ~/.gstack/projects/$SLUG
```
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","url":"<url>","duration_min":<N>,"alerts":<N>}`
Append it to `~/.gstack/projects/$SLUG/canary-history.jsonl`; never overwrite history.
### Phase 7: Baseline Update
canary/SKILL.md.tmpl
+10 -3
View File
@@ -82,6 +82,7 @@ console.log("GSTACK_STEP_OK");
Then copy the screenshot out of the printed session directory: `cp "<ASIDE_DIR>/<page-name>.jpg" .gstack/canary-reports/baselines/<page-name>.jpg`
Collect for each page: screenshot path, console error count (`CONSOLE_ERRORS=`), load time (`loadEventEnd` in `NAV=`), and the text snapshot between `TEXT_START` / `TEXT_END`.
Also run Phase 3's read-only link check for each monitored page and retain the URLs whose `LINK` status is 404. Repeat the same check each monitoring round; other HEAD failures are unknown, not broken links. Compare console messages by identity, not just count, and retain the text snapshot for evidence when a page's content disappears.
Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
@@ -94,7 +95,10 @@ Save the baseline manifest to `.gstack/canary-reports/baseline.json`:
"/": {
"screenshot": "baselines/home.jpg",
"console_errors": 0,
"load_time_ms": 450
"console_error_messages": [],
"load_time_ms": 450,
"broken_links": [],
"text_snapshot": "<TEXT_START/END content>"
}
}
}
@@ -132,11 +136,12 @@ For each page to monitor:
Run the Phase 2 read script for each page with the screenshot saved as `pre-<page-name>.jpg`, then `cp "<ASIDE_DIR>/pre-<page-name>.jpg" .gstack/canary-reports/screenshots/`.
Record the console error count and load time for each page. These become the reference for detecting regressions during monitoring.
Save the same manifest schema as Phase 2 to `.gstack/canary-reports/pre-monitor.json`, with the screenshots' actual paths. This is a monitoring-start reference, not evidence of pre-deploy health. Use it when no baseline exists; never overwrite an existing baseline during monitoring.
### Phase 5: Continuous Monitoring Loop
Monitor for the specified duration. Every 60 seconds, check each page. Nothing persists between scripts — every check re-opens the page from its URL and captures fresh evidence:
Record the start and deadline. After each full round, wait `max(0, 60 - elapsed-round-seconds)` seconds using the host's wait tool or `sleep`. If a round exceeds 60 seconds, start the next immediately and report the actual cadence; never overlap rounds. Stop at the deadline after the current round.
```bash
aside repl '
@@ -167,7 +172,7 @@ After each check, compare results against the baseline (or pre-deploy snapshot):
**Don't cry wolf.** Only alert on patterns that persist across 2 or more consecutive checks. A single transient network blip is not an alert.
**If a CRITICAL or HIGH alert is detected**, immediately notify the user via AskUserQuestion:
**After a CRITICAL or HIGH pattern is confirmed on two consecutive checks**, immediately notify the user via AskUserQuestion. A first occurrence is pending, not yet an alert:
```
CANARY ALERT
@@ -214,6 +219,7 @@ VERDICT: [DEPLOY IS HEALTHY / DEPLOY HAS ISSUES — details above]
```
Save report to `.gstack/canary-reports/{date}-canary.md` and `.gstack/canary-reports/{date}-canary.json`.
Per-page and overall status: BROKEN if any confirmed CRITICAL alert occurred; otherwise DEGRADED if any confirmed alert occurred; otherwise HEALTHY. Note resolved incidents separately without erasing them from the run's status. JSON fields: `url`, `started_at`, `ended_at`, `status`, `pages` (URL, checks, latest metrics, status), and `alerts` (severity, URL, first_seen, confirmed_at, evidence, resolved). Unconfirmed transients go in a separate `observations` array.
Log the result for the review dashboard:
@@ -223,6 +229,7 @@ mkdir -p ~/.gstack/projects/$SLUG
```
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","url":"<url>","duration_min":<N>,"alerts":<N>}`
Append it to `~/.gstack/projects/$SLUG/canary-history.jsonl`; never overwrite history.
### Phase 7: Baseline Update