test: derive keeps the gap at 3 events; changelog fragment for #3436

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-11-02 09:44:20 +00:00
1 parent 8e2d889331
commit eb9b765835
2 files changed
+22

No files matched your search

@@ -0,0 +1,8 @@
**/plan-tune no longer reports a mismatch between your stated and observed preferences when it has no observed answers behind it (#3436).** `--check-mismatch` compared a declared value against an inferred value that had never moved from its 0.5 starting point, so a strong declaration with no matching answers printed `scope_appetite: declared 0.9 vs inferred 0.5 (gap 0.4)` even though no logged answer touched that dimension.
### Fixed
- **`bin/gstack-developer-profile`**: `--check-mismatch`, `--gap` and the gap that `--derive` stores compare a dimension only when at least 3 logged answers moved it. For a profile with per-dimension counts, this floor replaces the old 10-answer overall minimum. Below it, `--check-mismatch` prints `no observed evidence` or `too little evidence (n of 3 events)` instead of a gap, and `--gap` lists the dimension under a new `low_evidence` field. A profile derived before `contributing_events` existed keeps the old behavior until its next derive.
- **/plan-tune "Show gap"**: says "not enough observed answers yet" for those dimensions instead of calling them drift.
### Tests
- `test/gstack-developer-profile.test.ts`: a declared dimension with no evidence is not flagged by `--check-mismatch` or `--gap` (both failed on main), 1 or 2 events report too little evidence, 3 events are compared, and `--derive` stores no gap without evidence.
+14
View File
@@ -679,6 +679,20 @@ describe('gstack-developer-profile --check-mismatch', () => {
expect(JSON.parse(runDev('--gap').stdout).gap.scope_appetite).toBeCloseTo(0.45, 2);
});
test('derive stores the gap for a dimension with 3 contributing events', () => {
for (let i = 0; i < 3; i++) {
logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: `s${i}`, ts: `2026-10-2${i}T12:00:00Z` });
}
expect(runDev('--derive').status).toBe(0);
const file = path.join(tmpHome, 'developer-profile.json');
const p = readProfile() as any;
expect(p.inferred.contributing_events.architecture_care).toBe(3);
p.declared = { architecture_care: 0.1 };
fs.writeFileSync(file, JSON.stringify(p));
expect(runDev('--derive').status).toBe(0);
expect((readProfile() as any).gap.architecture_care).toBeGreaterThan(0.3);
});
test('derive stores no gap for a dimension without evidence', () => {
deriveWithOneSignal();
expect(runDev('--derive').status).toBe(0);