gstack/test/fixtures/eng-paired-suite-749df.json

{
  "sourceRevision": "749df4446b5a1abd24a590bed745efa7a5fcd43b",
  "originalOutcome": "timeout",
  "reportSha256": "ececdc15cc362ee0c46662072d12df6e27dcf89861040079e58632a9516afe62",
  "notes": [
    "Unchanged relevant R6/R7/R9/T5/T6 and terminal report excerpts plus exact native D9/D11 calls. Tests reuse the original D8 call from eng-neutral-seed-749df.json. The excerpt is not a complete reviewed plan and does not establish a paid pass."
  ],
  "windowStart": "2026-09-16T12:12:00.000Z",
  "windowEnd": "2026-09-16T12:16:00.000Z",
  "parts": [
    "# Plan: Multi-tenant Auth Refactor (reviewed)\n\nReviewed target: `PLAN.md` (\"Plan: Multi-tenant Auth Refactor\") in repo `gstack-plan-count-wWc0se`, branch `main`, commit `ce9bfe4`.\nReview: `/plan-eng-review`, 2026-09-16. Design doc: none (`/office-hours` skipped at the user's request).\nEvidence note: the repository contains only `PLAN.md` and `CLAUDE.md`; no source or tests exist to probe. Every \"runtime evidence\" entry below is **unknown** and every finding is calibrated against quoted plan text, not code.\n",
    "## Decision ledger",
    "### R6: Error contract for validateAndDispatch()\nFinding: Architecture #3 (security architecture), P1, confidence 8/10, `PLAN.md:32-33` (\"three nested try/catch blocks; each catch swallows a different error class\"), plan-eng-review\nPlan baseline: original proposal — three catches, each swallowing one error class; no stated fail-closed contract. Whether a swallowed validation error currently leads to dispatch is unknown.\nRuntime evidence: unknown (no source in repo).\nComparison grid:\n\n| Choice | Current | A Fail closed everywhere | B Fail closed for validate+policy only | C Keep swallowing as planned |\n|---|---|---|---|---|\n| R6 validation / policy error | swallowed | deny; typed `AuthError` subclass per stage, logged with tenant + request id; never dispatch | deny; typed error, logged; never dispatch | swallowed (behavior unknown) |\n| R6 dispatch-stage error | swallowed | propagate to caller unchanged (caller decides), logged | swallowed as today | swallowed |\n| Cache adapter unavailable | unknown | treat as validation failure: deny, log | deny, log | unknown |\n| Caller-visible result | unknown | allow / deny(reason) / thrown dispatch error; no silent success | allow / deny(reason); dispatch errors silent | unknown |\n| R7 rollout | pending | pending | pending | pending |\n| R8 function structure (Section 2) | 60 lines, nested | pending | pending | pending |\n\nQuestion D8:\nD8 — What happens when validateAndDispatch() hits an error?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Architecture review, security).\nELI10: The function that decides whether a request gets through has three nested try/catch blocks, and each one catches an error and moves on (PLAN.md:32-33). In an auth path, \"catch and move on\" is the dangerous direction: if a token check throws and the code keeps going, a bad token might be dispatched as if it were fine. The safe rule is fail closed: any error while validating or checking policy means deny, with a typed error that says which stage failed, and nothing is dispatched.\nStakes if we pick wrong: Fail open means an IDP hiccup or a malformed token could let a request through with no log line to find it later; fail closed means an IDP outage denies logins loudly, which is the outcome you want.\nRecommendation: A because auth must fail closed, typed errors make the three failure stages testable and greppable, and letting dispatch errors propagate keeps the broker from hiding downstream bugs.\nCompleteness: A=10/10, B=6/10, C=2/10\nPros / cons:\nA) Fail closed: validate/policy errors deny with typed errors; dispatch errors propagate (recommended)\n  ✅ No path from a swallowed error to a dispatched request; each stage has a named error class to assert on\n  ✅ Every denial is logged with tenant and request id, so a 3am incident has a trail\n  ❌ An IDP or cache outage becomes visible login failures rather than silent degradation (human: ~half day / CC: ~10 min)\nB) Fail closed for validate and policy; keep swallowing dispatch errors\n  ✅ Closes the security-relevant fail-open path\n  ✅ Smaller change to the dispatch stage as written\n  ❌ Downstream dispatch bugs stay invisible; callers cannot distinguish \"dispatched\" from \"failed silently\"\nC) Keep the three swallowing catches as planned\n  ✅ No change to the proposed function\n  ✅ Zero risk of newly visible failures\n  ❌ Unknown fail-open behavior in the auth decision path, untestable by error class\nNet: loud, typed denials vs. silent continues in the one function that must never be silent.\nHeader: Error contract\nOptions:\nA) Fail closed: validate/policy errors deny with typed errors; dispatch errors propagate (recommended)\n✅ No path from a swallowed error to a dispatched request; each stage has a named error class to assert on\n✅ Every denial is logged with tenant and request id, so a 3am incident has a trail\n❌ An IDP or cache outage becomes visible login failures rather than silent degradation (human: ~half day / CC: ~10 min)\nB) Fail closed for validate and policy; keep swallowing dispatch errors\n✅ Closes the security-relevant fail-open path\n✅ Smaller change to the dispatch stage as written\n❌ Downstream dispatch bugs stay invisible; callers cannot distinguish \"dispatched\" from \"failed silently\"\nC) Keep the three swallowing catches as planned\n✅ No change to the proposed function\n✅ Zero risk of newly visible failures\n❌ Unknown fail-open behavior in the auth decision path, untestable by error class\n\nState: approved\nActual answer: A) Fail closed: validate/policy errors deny with typed errors; dispatch errors propagate — D8 answer \"Fail closed: validate/policy errors deny with typed errors; dispatch errors propagate (recommended)\"\nAccepted scope: `validateAndDispatch()` fails closed. Any error during token validation, cache adapter access, or `evaluateRequestPolicy()` results in a deny carrying a typed error (`TokenValidationError`, `CacheUnavailableError`, `PolicyDenied`, all extending `AuthError`), logged with tenant id and request id; nothing is dispatched. Errors thrown by the dispatch stage propagate to the caller unchanged after logging. No catch swallows an error. Required proof: one unit test per stage asserting deny + error class + no dispatch, plus one asserting a dispatch-stage error reaches the caller.\nHistory: none.",
    "### R7: Rollout strategy for replacing legacyAuthFlow()\nFinding: Architecture #4, P2, confidence 8/10, `PLAN.md:36` (\"The existing `legacyAuthFlow()` will get rewritten as part of this work\"), plan-eng-review\nPlan baseline: original proposal — rewrite `legacyAuthFlow()` in place; one deploy cuts every tenant over with no rollback lever except redeploy.\nRuntime evidence: unknown; callers of `legacyAuthFlow()` and its exact signature are not visible in this repo.\nComparison grid:\n\n| Choice | Current | A Flag-routed strangler | B Parity tests, then single cutover | C Rewrite in place as planned |\n|---|---|---|---|---|\n| R7 cutover mechanism | in-place rewrite | `legacyAuthFlow()` kept; a per-tenant / percentage flag (`AUTH_BROKER_ENABLED`) routes to `AuthBroker`; legacy deleted after bake | `legacyAuthFlow()` kept until parity suite is green, then one deploy switches all callers; no flag | in-place rewrite |\n| Rollback | redeploy | flip flag, seconds | redeploy | redeploy |\n| Temporary duplication | none | two flows live for the bake window | two flows live until cutover commit | none |\n| Regression contract (R9, Test review) | pending | pending | pending | pending |\n| R4-R6 (approved) | injection, single writer, fail closed | same | same | same |\n\nQuestion D9:\nD9 — How does AuthBroker replace legacyAuthFlow() in production?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Architecture review, incremental change).\nELI10: The plan rewrites the existing login flow in place (PLAN.md:36), so the day it deploys, every tenant is on the new code and the only way back is another deploy. The alternative is to keep the old flow alive for a short bake, put a switch in front of both, and move tenants over gradually. If something is wrong for one tenant, you flip the switch back in seconds instead of paging the on-call to redeploy.\nStakes if we pick wrong: A multi-tenant auth outage with a redeploy-length rollback, or, on the other side, a flag and two code paths that someone forgets to delete.\nRecommendation: A because auth is the highest blast-radius path in the product, a flag makes the wrong choice cheap to undo (reversibility), and the delete-legacy step is a named task with a date, not an afterthought.\nCompleteness: A=10/10, B=7/10, C=3/10\nPros / cons:\nA) Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)\n  ✅ Rollback is a flag flip; canary a single internal tenant before anyone else sees the new broker\n  ✅ Parity can be checked live: same request through both paths in shadow mode during bake\n  ❌ Two auth paths coexist for the bake window and the flag plus deletion task must be tracked (human: ~1 day / CC: ~20 min)\nB) Parity tests green, then a single cutover commit (no flag)\n  ✅ No flag plumbing; the parity suite is the gate\n  ✅ Legacy deleted in the same cutover commit, no lingering duplication\n  ❌ All tenants move at once; rollback is a redeploy\nC) Rewrite legacyAuthFlow() in place, as planned\n  ✅ Smallest diff and no temporary duplication\n  ✅ Nothing to clean up afterwards\n  ❌ No regression baseline to compare against once the old code is gone; rollback is a redeploy\nNet: a flag and a scheduled deletion vs. betting the whole tenant base on one deploy of the auth path.\nHeader: Rollout\nOptions:\nA) Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)\n✅ Rollback is a flag flip; canary a single internal tenant before anyone else sees the new broker\n✅ Parity can be checked live: same request through both paths in shadow mode during bake\n❌ Two auth paths coexist for the bake window and the flag plus deletion task must be tracked (human: ~1 day / CC: ~20 min)\nB) Parity tests green, then a single cutover commit (no flag)\n✅ No flag plumbing; the parity suite is the gate\n✅ Legacy deleted in the same cutover commit, no lingering duplication\n❌ All tenants move at once; rollback is a redeploy\nC) Rewrite legacyAuthFlow() in place, as planned\n✅ Smallest diff and no temporary duplication\n✅ Nothing to clean up afterwards\n❌ No regression baseline to compare against once the old code is gone; rollback is a redeploy\n\nState: approved\nActual answer: A) Flag-routed strangler — D9 answer \"Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)\"\nAccepted scope: `legacyAuthFlow()` stays untouched during the bake as the parity oracle. A flag `AUTH_BROKER_ENABLED` (per-tenant allowlist plus percentage) routes each request to `AuthBroker.validateAndDispatch()` or `legacyAuthFlow()`. Optional shadow mode runs both and logs decision mismatches without affecting the response. A named task deletes `legacyAuthFlow()`, the flag and the shadow code after the bake. Required proof: routing tests for flag on/off/percentage, and a shadow-mode mismatch-logging test.\nHistory: none.",
    "### R9: Regression contract for legacyAuthFlow() (IRON RULE)\nFinding: Test review #1 (CRITICAL), P1, confidence 9/10, `PLAN.md:36-37` (\"The existing `legacyAuthFlow()` will get rewritten as part of this work; no regression test for the prior behavior is planned\") and `PLAN.md:23-25` (\"That coverage does not exercise legacyAuthFlow() or assert compatibility with its prior behavior\"), plan-eng-review\nPlan baseline: original proposal — no regression coverage of prior behavior; new-component tests only. D9 approved: legacy stays as parity oracle during a flag-routed bake.\nRuntime evidence: unknown; `legacyAuthFlow()` callers, signature and error behavior are not visible in this repo. Test framework: unknown (no package.json / config in repo); `*.test.ts` naming assumed, to be matched to the real repo.\nComparison grid:\n\n| Choice | Current | A Full parity matrix + E2E | B Parity on decisions; error paths new-only | C Happy path + one denial |\n|---|---|---|---|---|\n| R9 behavior to preserve | unstated | every allow/deny decision, cache write, dispatch call and IDP call count/order of `legacyAuthFlow()` across the full case matrix | allow/deny decisions across the matrix | allow on valid token, deny on expired |\n| Case matrix | none | token {valid, expired, revoked, tenant suspended, wrong audience, wrong issuer, policy-version bump, missing tenant claim, cross-tenant token, malformed header} × cache {hit, miss} × IDP {all ok, call N fails, call N times out, N=1..5} | token matrix × cache {hit, miss}; IDP failures covered only on `AuthBroker` | 2 cases |\n| Intended differences | unstated | recorded explicitly: error-path outcomes where legacy fails open (if any) are documented as D8 divergences and flagged as legacy bugs; IDP calls stay 5 sequential (D4) | same recording for decision paths only | none recorded |\n| Acceptance assertions | none | per case: decision, error class, cache write yes/no, dispatch invoked yes/no, IDP call count + order; both paths share fixtures | decision + dispatch invoked | decision only |\n| E2E login journey | none | one per tenant state (active, suspended, logged-out) through the flag router [→E2E] | none | none |\n| Required proof already approved (D7, D8, D9, D10) | — | included | included | included |\n\nQuestion D11:\nD11 — How do we prove the new broker matches legacyAuthFlow() before it replaces it?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Test review, regression rule).\nELI10: The plan replaces the existing login flow and says plainly that no test will check the new code behaves like the old one (PLAN.md:36-37). The old flow is the only spec we have. A parity suite runs the same inputs through both old and new code and asserts the same allow/deny, the same cache writes, the same dispatch, and the same five IDP calls. Where they must differ on purpose (D8 makes errors deny instead of being swallowed), the difference is written down, not discovered in production.\nStakes if we pick wrong: A tenant that could log in yesterday cannot today, or worse, one that should be locked out still gets in, and nobody can say which of the two flows is \"right\".\nRecommendation: A because with CC the full matrix is minutes of table-driven test code, auth is the one place \"happy path only\" is not acceptable, and the suite doubles as the gate for the deferred Promise.all follow-up (D4).\nCompleteness: A=10/10, B=7/10, C=4/10\nPros / cons:\nA) Full parity matrix across both flows, plus E2E login journeys (recommended)\n  ✅ Every token state, cache state and IDP failure position is asserted identically on legacy and broker from shared fixtures\n  ✅ Intended divergences (D8 fail-closed) are recorded per case, and any legacy fail-open found becomes a flagged bug\n  ❌ Largest test file in the change; table-driven, but ~100 cases to name and maintain (human: ~2 days / CC: ~30 min)\nB) Parity on allow/deny decisions; IDP error paths tested only on the new broker\n  ✅ Catches decision regressions across the token and cache matrix\n  ✅ Smaller suite; error paths still covered on the new code\n  ❌ Legacy's actual error behavior is never recorded, so fail-open differences are invisible\nC) Happy path plus one denial\n  ✅ Minutes to write, easy to read\n  ✅ Confirms the wiring works end to end\n  ❌ Misses every edge case the plan already knows about (suspension, revocation, audience, policy version)\nNet: a table of ~100 cheap cases now vs. finding out in production which flow was right.\nHeader: Regression\nOptions:\nA) Full parity matrix across both flows, plus E2E login journeys (recommended)\n✅ Every token state, cache state and IDP failure position is asserted identically on legacy and broker from shared fixtures\n✅ Intended divergences (D8 fail-closed) are recorded per case, and any legacy fail-open found becomes a flagged bug\n❌ Largest test file in the change; table-driven, but ~100 cases to name and maintain (human: ~2 days / CC: ~30 min)\nB) Parity on allow/deny decisions; IDP error paths tested only on the new broker\n✅ Catches decision regressions across the token and cache matrix\n✅ Smaller suite; error paths still covered on the new code\n❌ Legacy's actual error behavior is never recorded, so fail-open differences are invisible\nC) Happy path plus one denial\n✅ Minutes to write, easy to read\n✅ Confirms the wiring works end to end\n❌ Misses every edge case the plan already knows about (suspension, revocation, audience, policy version)\n\nState: approved\nActual answer: A) Full parity matrix across both flows, plus E2E login journeys — D11 answer \"Full parity matrix across both flows, plus E2E login journeys (recommended)\"\nAccepted scope: **Regression contract.** Behavior to preserve: every allow/deny decision, cache write, dispatch invocation and IDP call count/order of `legacyAuthFlow()`. Case matrix: token {valid, expired, revoked, tenant suspended, wrong audience, wrong issuer, policy-version bump, missing tenant claim, cross-tenant token, malformed header} × cache {hit, miss} × IDP {all ok, call N fails, call N times out, N=1..5}. Intended differences: only D8 fail-closed outcomes on error paths, recorded per case; any legacy fail-open discovered is flagged as a legacy bug in the report and not silently \"fixed\" by the parity assertion. IDP calls remain 5 sequential (D4). Acceptance assertions per case: decision, error class, cache write yes/no, dispatch invoked yes/no, IDP call count and order; shared fixtures drive both `legacyAuthFlow()` and `AuthBroker`. Plus E2E login journeys through the flag router for active, suspended and logged-out tenant states. Test files (naming to match the real repo): `auth/__tests__/parity.legacy-vs-broker.test.ts`, `auth/__tests__/broker.test.ts`, `auth/__tests__/request-policy.test.ts`, `auth/__tests__/session-mint.test.ts`, `auth/__tests__/router.test.ts`, `e2e/auth-login.e2e.ts`.\nHistory: none.",
    "## Implementation Tasks",
    "- [ ] **T5 (P1, human: ~2d / CC: ~1h)** — auth/__tests__/parity — Build the parity suite: shared fixtures drive `legacyAuthFlow()` and `AuthBroker` across token × cache × IDP matrix\n  - Surfaced by: Test review #1 CRITICAL (`PLAN.md:36-37, 23-25`) → D11\n  - Files: `auth/__tests__/parity.legacy-vs-broker.test.ts`, `auth/__tests__/fixtures/`\n  - Verify: every matrix cell asserts decision, error class, cache write y/n, dispatch y/n, IDP call count + order; intended differences limited to D8 fail-closed cases and listed per case; any legacy fail-open reported as a legacy bug",
    "- [ ] **T6 (P1, human: ~1d / CC: ~30min)** — auth/router.ts — Route by `AUTH_BROKER_ENABLED` (tenant allowlist + percentage) and leave `legacyAuthFlow()` untouched\n  - Surfaced by: Architecture #4 (`PLAN.md:36` big-bang rewrite) → D9\n  - Files: `auth/router.ts`, `auth/__tests__/router.test.ts`\n  - Verify: flag off → legacy; on → broker; percentage is deterministic per tenant; `git diff` shows no change to `legacyAuthFlow()`",
    "## GSTACK REVIEW REPORT\n\n| Review | Trigger | Why | Runs | Status | Findings |\n|--------|---------|-----|------|--------|----------|\n| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |\n| Outside Review | codex via `/plan-eng-review` | Independent 2nd opinion | 1 | DISABLED (skipped) | host: claude, outside_provider: codex, outside_status: disabled, phase: plan-review |\n| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR (PLAN) | 37 issues, 0 critical gaps |\n| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |\n| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |\n\n**OUTSIDE COVERAGE:** codex, phase plan-review, disabled (`codex_reviews` disabled in config) — skipped, 0 findings. No outside coverage for this plan.\n\n**VERDICT:** ENG CLEARED — ready to implement. CEO, Design and DX reviews not run (optional; backend-only refactor, no UI scope).\n\nNO UNRESOLVED DECISIONS"
  ],
  "calls": [
    {
      "sessionId": "a75c4f50-7964-47ef-a0b0-a830761427a9",
      "toolUseId": "toolu_01MP88ZZpDSbsfGHa8Whiyy2",
      "questions": [
        {
          "question": "D9 — How does AuthBroker replace legacyAuthFlow() in production?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Architecture review, incremental change).\nELI10: The plan rewrites the existing login flow in place (PLAN.md:36), so the day it deploys, every tenant is on the new code and the only way back is another deploy. The alternative is to keep the old flow alive for a short bake, put a switch in front of both, and move tenants over gradually. If something is wrong for one tenant, you flip the switch back in seconds instead of paging the on-call to redeploy.\nStakes if we pick wrong: A multi-tenant auth outage with a redeploy-length rollback, or, on the other side, a flag and two code paths that someone forgets to delete.\nRecommendation: A because auth is the highest blast-radius path in the product, a flag makes the wrong choice cheap to undo (reversibility), and the delete-legacy step is a named task with a date, not an afterthought.\nCompleteness: A=10/10, B=7/10, C=3/10\nPros / cons:\nA) Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)\n  ✅ Rollback is a flag flip; canary a single internal tenant before anyone else sees the new broker\n  ✅ Parity can be checked live: same request through both paths in shadow mode during bake\n  ❌ Two auth paths coexist for the bake window and the flag plus deletion task must be tracked (human: ~1 day / CC: ~20 min)\nB) Parity tests green, then a single cutover commit (no flag)\n  ✅ No flag plumbing; the parity suite is the gate\n  ✅ Legacy deleted in the same cutover commit, no lingering duplication\n  ❌ All tenants move at once; rollback is a redeploy\nC) Rewrite legacyAuthFlow() in place, as planned\n  ✅ Smallest diff and no temporary duplication\n  ✅ Nothing to clean up afterwards\n  ❌ No regression baseline to compare against once the old code is gone; rollback is a redeploy\nNet: a flag and a scheduled deletion vs. betting the whole tenant base on one deploy of the auth path.",
          "header": "Rollout",
          "multiSelect": false,
          "options": [
            {
              "label": "Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)",
              "description": "✅ Rollback is a flag flip; canary a single internal tenant before anyone else sees the new broker\n✅ Parity can be checked live: same request through both paths in shadow mode during bake\n❌ Two auth paths coexist for the bake window and the flag plus deletion task must be tracked (human: ~1 day / CC: ~20 min)"
            },
            {
              "label": "Parity tests green, then a single cutover commit (no flag)",
              "description": "✅ No flag plumbing; the parity suite is the gate\n✅ Legacy deleted in the same cutover commit, no lingering duplication\n❌ All tenants move at once; rollback is a redeploy"
            },
            {
              "label": "Rewrite legacyAuthFlow() in place, as planned",
              "description": "✅ Smallest diff and no temporary duplication\n✅ Nothing to clean up afterwards\n❌ No regression baseline to compare against once the old code is gone; rollback is a redeploy"
            }
          ]
        }
      ],
      "answered": true,
      "failed": false,
      "answers": {
        "D9 — How does AuthBroker replace legacyAuthFlow() in production?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Architecture review, incremental change).\nELI10: The plan rewrites the existing login flow in place (PLAN.md:36), so the day it deploys, every tenant is on the new code and the only way back is another deploy. The alternative is to keep the old flow alive for a short bake, put a switch in front of both, and move tenants over gradually. If something is wrong for one tenant, you flip the switch back in seconds instead of paging the on-call to redeploy.\nStakes if we pick wrong: A multi-tenant auth outage with a redeploy-length rollback, or, on the other side, a flag and two code paths that someone forgets to delete.\nRecommendation: A because auth is the highest blast-radius path in the product, a flag makes the wrong choice cheap to undo (reversibility), and the delete-legacy step is a named task with a date, not an afterthought.\nCompleteness: A=10/10, B=7/10, C=3/10\nPros / cons:\nA) Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)\n  ✅ Rollback is a flag flip; canary a single internal tenant before anyone else sees the new broker\n  ✅ Parity can be checked live: same request through both paths in shadow mode during bake\n  ❌ Two auth paths coexist for the bake window and the flag plus deletion task must be tracked (human: ~1 day / CC: ~20 min)\nB) Parity tests green, then a single cutover commit (no flag)\n  ✅ No flag plumbing; the parity suite is the gate\n  ✅ Legacy deleted in the same cutover commit, no lingering duplication\n  ❌ All tenants move at once; rollback is a redeploy\nC) Rewrite legacyAuthFlow() in place, as planned\n  ✅ Smallest diff and no temporary duplication\n  ✅ Nothing to clean up afterwards\n  ❌ No regression baseline to compare against once the old code is gone; rollback is a redeploy\nNet: a flag and a scheduled deletion vs. betting the whole tenant base on one deploy of the auth path.": "Flag-routed strangler: keep legacy, route by flag, delete after bake (recommended)"
      },
      "unansweredQuestionIndices": [],
      "answeredAt": "2026-09-16T12:12:58.114Z"
    },
    {
      "sessionId": "a75c4f50-7964-47ef-a0b0-a830761427a9",
      "toolUseId": "toolu_01BRiwGZ7fU4AL1hWRJN1Yf9",
      "questions": [
        {
          "question": "D11 — How do we prove the new broker matches legacyAuthFlow() before it replaces it?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Test review, regression rule).\nELI10: The plan replaces the existing login flow and says plainly that no test will check the new code behaves like the old one (PLAN.md:36-37). The old flow is the only spec we have. A parity suite runs the same inputs through both old and new code and asserts the same allow/deny, the same cache writes, the same dispatch, and the same five IDP calls. Where they must differ on purpose (D8 makes errors deny instead of being swallowed), the difference is written down, not discovered in production.\nStakes if we pick wrong: A tenant that could log in yesterday cannot today, or worse, one that should be locked out still gets in, and nobody can say which of the two flows is \"right\".\nRecommendation: A because with CC the full matrix is minutes of table-driven test code, auth is the one place \"happy path only\" is not acceptable, and the suite doubles as the gate for the deferred Promise.all follow-up (D4).\nCompleteness: A=10/10, B=7/10, C=4/10\nPros / cons:\nA) Full parity matrix across both flows, plus E2E login journeys (recommended)\n  ✅ Every token state, cache state and IDP failure position is asserted identically on legacy and broker from shared fixtures\n  ✅ Intended divergences (D8 fail-closed) are recorded per case, and any legacy fail-open found becomes a flagged bug\n  ❌ Largest test file in the change; table-driven, but ~100 cases to name and maintain (human: ~2 days / CC: ~30 min)\nB) Parity on allow/deny decisions; IDP error paths tested only on the new broker\n  ✅ Catches decision regressions across the token and cache matrix\n  ✅ Smaller suite; error paths still covered on the new code\n  ❌ Legacy's actual error behavior is never recorded, so fail-open differences are invisible\nC) Happy path plus one denial\n  ✅ Minutes to write, easy to read\n  ✅ Confirms the wiring works end to end\n  ❌ Misses every edge case the plan already knows about (suspension, revocation, audience, policy version)\nNet: a table of ~100 cheap cases now vs. finding out in production which flow was right.",
          "header": "Regression",
          "multiSelect": false,
          "options": [
            {
              "label": "Full parity matrix across both flows, plus E2E login journeys (recommended)",
              "description": "✅ Every token state, cache state and IDP failure position is asserted identically on legacy and broker from shared fixtures\n✅ Intended divergences (D8 fail-closed) are recorded per case, and any legacy fail-open found becomes a flagged bug\n❌ Largest test file in the change; table-driven, but ~100 cases to name and maintain (human: ~2 days / CC: ~30 min)"
            },
            {
              "label": "Parity on allow/deny decisions; IDP error paths tested only on the new broker",
              "description": "✅ Catches decision regressions across the token and cache matrix\n✅ Smaller suite; error paths still covered on the new code\n❌ Legacy's actual error behavior is never recorded, so fail-open differences are invisible"
            },
            {
              "label": "Happy path plus one denial",
              "description": "✅ Minutes to write, easy to read\n✅ Confirms the wiring works end to end\n❌ Misses every edge case the plan already knows about (suspension, revocation, audience, policy version)"
            }
          ]
        }
      ],
      "answered": true,
      "failed": false,
      "answers": {
        "D11 — How do we prove the new broker matches legacyAuthFlow() before it replaces it?\nProject/branch/task: main branch, /plan-eng-review of PLAN.md \"Multi-tenant Auth Refactor\" (Test review, regression rule).\nELI10: The plan replaces the existing login flow and says plainly that no test will check the new code behaves like the old one (PLAN.md:36-37). The old flow is the only spec we have. A parity suite runs the same inputs through both old and new code and asserts the same allow/deny, the same cache writes, the same dispatch, and the same five IDP calls. Where they must differ on purpose (D8 makes errors deny instead of being swallowed), the difference is written down, not discovered in production.\nStakes if we pick wrong: A tenant that could log in yesterday cannot today, or worse, one that should be locked out still gets in, and nobody can say which of the two flows is \"right\".\nRecommendation: A because with CC the full matrix is minutes of table-driven test code, auth is the one place \"happy path only\" is not acceptable, and the suite doubles as the gate for the deferred Promise.all follow-up (D4).\nCompleteness: A=10/10, B=7/10, C=4/10\nPros / cons:\nA) Full parity matrix across both flows, plus E2E login journeys (recommended)\n  ✅ Every token state, cache state and IDP failure position is asserted identically on legacy and broker from shared fixtures\n  ✅ Intended divergences (D8 fail-closed) are recorded per case, and any legacy fail-open found becomes a flagged bug\n  ❌ Largest test file in the change; table-driven, but ~100 cases to name and maintain (human: ~2 days / CC: ~30 min)\nB) Parity on allow/deny decisions; IDP error paths tested only on the new broker\n  ✅ Catches decision regressions across the token and cache matrix\n  ✅ Smaller suite; error paths still covered on the new code\n  ❌ Legacy's actual error behavior is never recorded, so fail-open differences are invisible\nC) Happy path plus one denial\n  ✅ Minutes to write, easy to read\n  ✅ Confirms the wiring works end to end\n  ❌ Misses every edge case the plan already knows about (suspension, revocation, audience, policy version)\nNet: a table of ~100 cheap cases now vs. finding out in production which flow was right.": "Full parity matrix across both flows, plus E2E login journeys (recommended)"
      },
      "unansweredQuestionIndices": [],
      "answeredAt": "2026-09-16T12:15:38.996Z"
    }
  ]
}