From be83949eb696920eacdff3224ae46fa4185b3a85 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Thu, 19 Nov 2026 19:59:36 +0800 Subject: [PATCH] Codex skill suggestions name each skill the way Codex lists it (GST-3195) (#4523) On Codex, gstack's routing hint now names each skill by the `name:` in its installed SKILL.md (`review`), the name Codex lists, so `$review` loads the suggested skill. Before, the hint said `gstack-review` (the folder name), which Codex doesn't list, so mentioning it loaded nothing. Nothing to do: the next `./setup --host codex` picks it up. Claude Code's hint is unchanged. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- ...dex-routing-hint-uses-codex-skill-names.md | 1 + docs/skill-routing-and-contracts.md | 2 +- hosts/codex/hooks/codex-skill-route-hook.ts | 4 +- lib/skill-route.ts | 9 ++-- test/codex-hooks-real-codex.test.ts | 50 ++++++++++++++++++- test/codex-hooks.test.ts | 26 ++++++++-- 6 files changed, 79 insertions(+), 13 deletions(-) create mode 100644 changelog.d/codex-routing-hint-uses-codex-skill-names.md diff --git a/changelog.d/codex-routing-hint-uses-codex-skill-names.md b/changelog.d/codex-routing-hint-uses-codex-skill-names.md new file mode 100644 index 000000000..ca8e60cec --- /dev/null +++ b/changelog.d/codex-routing-hint-uses-codex-skill-names.md @@ -0,0 +1 @@ +- **On Codex, gstack's skill suggestion now names the skill the way Codex lists it.** The routing hint used to say "gstack-review", a name Codex doesn't know, so mentioning it as `$gstack-review` loaded nothing. It now reads each skill's name from its installed `SKILL.md`, so the hint says `review` and `$review` loads the skill it suggests. diff --git a/docs/skill-routing-and-contracts.md b/docs/skill-routing-and-contracts.md index 5f6d1d34b..0ece37ec2 100644 --- a/docs/skill-routing-and-contracts.md +++ b/docs/skill-routing-and-contracts.md @@ -89,7 +89,7 @@ The hook reads `destructive_guard` every time it runs, so `gstack-config set des **Your own hooks stay as they are.** Codex 0.170 skips the whole `hooks.json` when it holds a key it does not know, your hooks included. So gstack adds no marker keys (it recognizes its groups by their command path), checks the new file against Codex's schema before moving it into place, and keeps the lock, the `hooks.json.bak.` backup and `gstack-settings-hook rollback` that `settings.json` has (`GSTACK_HOOKS_FORMAT=codex gstack-settings-hook rollback`). Setup leaves the file alone, prints why and carries on when the file does not parse or already fails Codex's schema, when `config.toml` does not parse, or when `config.toml` already declares one of gstack's hooks under `[hooks]` (Codex would run it twice). It also skips the hooks, saying why, when `codex` is missing or older than 0.170. `./setup --host codex --no-team` (plain `./setup --no-team` installs for Claude Code only, so it leaves the Codex hooks alone; `--host auto` with `codex` on your PATH also reaches them) and `gstack-uninstall` remove gstack's groups and nothing else. If one of your own hooks comes after gstack's group in the same event, removing gstack's group would shift yours to a new position and Codex would stop trusting it (`codex exec` skips an untrusted hook without asking). So gstack leaves the file unchanged and says so; delete gstack's group by hand and re-trust your hooks in `/hooks`. -**Routing on Codex** scans `${CODEX_HOME:-~/.codex}/skills`, names gstack's skills the way Codex does (`gstack-review`), and takes their triggers from the same skill in gstack's checkout, because Codex's copies keep only a name and a description. The line reads "this request fits the gstack-review skill. Read and follow the gstack-review skill before you start". A prompt that starts with `$` (a Codex skill mention) or `/` is never routed. +**Routing on Codex** scans `${CODEX_HOME:-~/.codex}/skills`, names each of gstack's skills by the `name:` in its installed `SKILL.md`, which is the name Codex lists and you mention with `$` (the `gstack-review` folder holds `name: review`, so `$review`), and takes their triggers from the same skill in gstack's checkout, because Codex's copies keep only a name and a description. A skill whose `name:` is missing or unreadable is named by its folder. The line reads "this request fits the review skill. Read and follow the review skill before you start". A prompt that starts with `$` (a Codex skill mention) or `/` is never routed. **The destructive-command check on Codex** runs the same `check-careful.sh`, but Codex has no "ask": its hook parser rejects `permissionDecision: "ask"` and lets the command run. So in an interactive Codex session, every command the check would ask about is blocked instead, and the reason tells the model to stop and ask you. If you want it, run it yourself by typing `!` and the command in Codex (Codex runs `!` commands without hooks), or turn the check off. In `codex exec`, and any session `gstack-session-kind` reports as headless or unattended, it is a warning and the command runs, as on Claude Code. The two deny shapes stay a deny everywhere. Codex already refuses some `rm -rf` commands by itself, before any hook runs. diff --git a/hosts/codex/hooks/codex-skill-route-hook.ts b/hosts/codex/hooks/codex-skill-route-hook.ts index 7703ef41c..ec556b3bc 100755 --- a/hosts/codex/hooks/codex-skill-route-hook.ts +++ b/hosts/codex/hooks/codex-skill-route-hook.ts @@ -2,8 +2,8 @@ /** * codex-skill-route-hook — the skill-routing hook on Codex (GST-3172). The same UserPromptSubmit hook as Claude * Code's (hosts/claude/hooks/skill-route-hook.ts) with host codex: it scans ${CODEX_HOME:-~/.codex}/skills, names - * skills as Codex does (gstack-review), words the hint for Codex, and never routes a prompt that starts with `$` - * (a Codex skill mention) or `/`. Same switch (skill_routing), same deadline, same trace log. + * skills as Codex lists them (by their SKILL.md name, so gstack-review/ is review), words the hint for Codex, and + * never routes a prompt that starts with `$` (a Codex skill mention) or `/`. Same switch (skill_routing), same deadline, same trace log. * Docs: docs/skill-routing-and-contracts.md#codex. */ import { runSkillRouteHook } from '../../claude/hooks/skill-route-hook.ts'; diff --git a/lib/skill-route.ts b/lib/skill-route.ts index 007484a2a..68fc7bd40 100644 --- a/lib/skill-route.ts +++ b/lib/skill-route.ts @@ -203,9 +203,10 @@ export function scanInstalledSkills(dir: string): { skills: InstalledSkill[]; pr } /** - * Codex's skills, as scanInstalledSkills reads them, with each gstack skill (a gstack- directory) named by - * its directory, the name Codex mentions it by ($gstack-review). Codex renders keep only name and description, so - * a gstack skill without triggers takes them from the same skill's Claude render in this checkout. + * Codex's skills, as scanInstalledSkills reads them. Each gstack skill (a gstack- directory) keeps the name its + * installed SKILL.md gives, the one Codex lists and mentions it by (gstack-review/ holds name: review, so $review), or + * its directory when that name is missing or unusable. Codex renders keep only name and description, so a gstack + * skill without triggers takes them from the same skill's Claude render in this checkout. */ export function scanCodexSkills(dir: string): { skills: InstalledSkill[]; problems: string[] } { const { skills, problems } = scanInstalledSkills(dir); @@ -221,7 +222,7 @@ export function scanCodexSkills(dir: string): { skills: InstalledSkill[]; proble if (fm && /^triggers:/m.test(fm)) triggers = frontmatterList(`---\n${fm}\n---`, 'triggers'); } catch { /* no Claude render beside this file: the skill routes on nothing */ } } - return { ...s, name: entry, triggers, gstack: true }; + return { ...s, name: SKILL_NAME.test(s.name) ? s.name : entry, triggers, gstack: true }; }), }; } diff --git a/test/codex-hooks-real-codex.test.ts b/test/codex-hooks-real-codex.test.ts index 54553c340..627d24761 100644 --- a/test/codex-hooks-real-codex.test.ts +++ b/test/codex-hooks-real-codex.test.ts @@ -69,6 +69,41 @@ async function codexExec(extra: string[]): Promise { return out; } +/** Run codex exec to completion against a local stub model that answers every turn at once; returns each request body. */ +async function codexAgainstStub(prompt: string): Promise { + const bodies: string[] = []; + const sse = (data: Record) => `event: ${data.type}\ndata: ${JSON.stringify(data)}\n\n`; + const server = Bun.serve({ + port: 0, hostname: '127.0.0.1', + async fetch(req) { + bodies.push(await req.text()); + const item = { type: 'message', id: 'msg_stub', role: 'assistant', status: 'completed', content: [{ type: 'output_text', text: 'stub', annotations: [] }] }; + const usage = { input_tokens: 1, input_tokens_details: null, output_tokens: 1, output_tokens_details: null, total_tokens: 2 }; + return new Response(sse({ type: 'response.created', response: { id: 'resp_stub' } }) + + sse({ type: 'response.output_item.done', output_index: 0, item }) + + sse({ type: 'response.completed', response: { id: 'resp_stub', usage } }), { headers: { 'content-type': 'text/event-stream' } }); + }, + }); + try { + const work = path.join(dir, 'work'); + fs.mkdirSync(work, { recursive: true }); + const child = spawn('codex', ['exec', '--skip-git-repo-check', '--dangerously-bypass-hook-trust', '-c', 'model_provider=gstack_stub', + '-c', `model_providers.gstack_stub={name="stub",base_url="http://127.0.0.1:${server.port}/v1",wire_api="responses"}`, prompt], { + cwd: work, stdio: ['ignore', 'ignore', 'ignore'], + env: { + PATH: process.env.PATH ?? '', HOME: dir, CODEX_HOME: home(), OPENAI_API_KEY: 'sk-not-a-real-key', SHELL: '/bin/bash', + GSTACK_STATE_ROOT: path.join(dir, 'state'), GSTACK_INSTALLED_SKILLS_DIR: path.join(home(), 'skills'), + }, + }); + const killer = setTimeout(() => child.kill('SIGKILL'), 60_000); + await new Promise(resolve => child.on('exit', resolve)); + clearTimeout(killer); + } finally { + server.stop(true); + } + return bodies; +} + describe.skipIf(!usable && !required)('real codex reads the hooks.json gstack writes', () => { test('codex 0.170 or newer is on PATH', () => { expect(version?.[0] ?? 'no codex on PATH').toMatch(/^codex-cli /); @@ -79,10 +114,23 @@ describe.skipIf(!usable && !required)('real codex reads the hooks.json gstack wr install(); const out = await codexExec(['--dangerously-bypass-hook-trust']); expect(fs.existsSync(marker())).toBe(true); - expect(fs.readFileSync(routeLog(), 'utf8')).toMatch(/event=routed skill=gstack-review .*host=codex/); + expect(fs.readFileSync(routeLog(), 'utf8')).toMatch(/event=routed skill=review .*host=codex/); expect(out).not.toContain('failed to parse hooks config'); }, 90_000); + test("the routing hint names the skill the way Codex lists it, and that $mention loads the skill's body", async () => { + install(); + const skill = path.join(home(), 'skills', 'gstack-review'); + fs.mkdirSync(skill, { recursive: true }); + fs.writeFileSync(path.join(skill, 'SKILL.md'), '---\nname: review\ndescription: Review a diff before it lands.\n---\nREAL_CODEX_SKILL_BODY\n'); + const [routed] = await codexAgainstStub('please review my changes'); + const hinted = /this request fits the ([A-Za-z0-9_:-]+) skill/.exec(routed ?? '')?.[1]; + expect(hinted).toBe('review'); + expect(routed).toContain(`- ${hinted}: Review a diff before it lands. (file: `); + expect((await codexAgainstStub(`$${hinted} look at this`)).join('\n')).toContain('REAL_CODEX_SKILL_BODY'); + expect((await codexAgainstStub('$gstack-review look at this')).join('\n')).not.toContain('REAL_CODEX_SKILL_BODY'); + }, 200_000); + test('negative controls: an unknown key makes Codex skip the whole file, and untrusted hooks never run', async () => { install(); const file = path.join(home(), 'hooks.json'); diff --git a/test/codex-hooks.test.ts b/test/codex-hooks.test.ts index b9928858c..08aa9ee29 100644 --- a/test/codex-hooks.test.ts +++ b/test/codex-hooks.test.ts @@ -256,19 +256,35 @@ describe('skill routing on Codex', () => { test("gstack's Codex skills take Codex's names and the triggers of the same skill in this checkout", () => { const { skills } = scanCodexSkills(codexSkills()); - const review = skills.find(s => s.name === 'gstack-review'); + const review = skills.find(s => s.name === 'review'); expect(review?.gstack).toBe(true); expect(review?.triggers).toContain('review my changes'); expect(skills.find(s => s.name === 'my-notes')?.triggers).toEqual([]); const decision = routePrompt(PAYLOADS.UserPromptSubmit.prompt, skills, undefined, 'codex'); - expect(decision.route?.skill).toBe('gstack-review'); + expect(decision.route?.skill).toBe('review'); + }); + + test('a Codex skill is named by its SKILL.md name, with or without skill_prefix, and by its folder when that name is unusable', () => { + const names = (frontmatterName: string | null) => { + const skills = path.join(dir, 'codex-named'); + fs.rmSync(skills, { recursive: true, force: true }); + fs.mkdirSync(path.join(skills, 'gstack-review'), { recursive: true }); + fs.writeFileSync(path.join(skills, 'gstack-review', 'SKILL.md'), '---\n' + (frontmatterName === null ? '' : 'name: ' + frontmatterName + '\n') + 'description: d\n---\n'); + const scan = scanCodexSkills(skills); + const decision = routePrompt(PAYLOADS.UserPromptSubmit.prompt, scan.skills, undefined, 'codex'); + return { name: scan.skills[0]?.name, routed: decision.route?.skill, problems: scan.problems }; + }; + expect(names('review')).toEqual({ name: 'review', routed: 'review', problems: [] }); + expect(names('gstack-review')).toEqual({ name: 'gstack-review', routed: 'gstack-review', problems: [] }); + expect(names(null)).toEqual({ name: 'gstack-review', routed: 'gstack-review', problems: [] }); + expect(names('"not a name"')).toEqual({ name: 'gstack-review', routed: 'gstack-review', problems: [] }); }); test('a $ skill mention or a / command is never routed on Codex; on Claude Code a $ prompt still routes as before', () => { const { skills } = scanCodexSkills(codexSkills()); expect(routePrompt('$gstack-review review my changes', skills, undefined, 'codex')).toMatchObject({ route: null, reason: 'skill-mention' }); expect(routePrompt('/review review my changes', skills, undefined, 'codex')).toMatchObject({ route: null, reason: 'slash-command' }); - expect(routePrompt(' $ review my changes', skills).route?.skill).toBe('gstack-review'); + expect(routePrompt(' $ review my changes', skills).route?.skill).toBe('review'); }); test("the Codex line names Codex's skill; Claude Code's line is unchanged", () => { @@ -289,8 +305,8 @@ describe('skill routing on Codex', () => { expect(r.status).toBe(0); const out = JSON.parse(r.stdout).hookSpecificOutput; expect(out.hookEventName).toBe('UserPromptSubmit'); - expect(out.additionalContext).toContain('this request fits the gstack-review skill. Read and follow the gstack-review skill'); - expect(fs.readFileSync(path.join(dir, 'state', 'hooks', 'skill-route.log'), 'utf8')).toMatch(/event=routed skill=gstack-review .*host=codex/); + expect(out.additionalContext).toContain('this request fits the review skill. Read and follow the review skill'); + expect(fs.readFileSync(path.join(dir, 'state', 'hooks', 'skill-route.log'), 'utf8')).toMatch(/event=routed skill=review .*host=codex/); expect(run({ GSTACK_SKILL_ROUTING: 'off' }).stdout).toBe(''); }); });