fix(plan-tune): keep the 10-answer floor in --check-mismatch beside the per-dimension floor

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-11-02 09:54:31 +00:00
1 parent 77a8eafab5
commit 205379a830
3 files changed
+20 -5

No files matched your search

+1 -1
View File
@@ -736,7 +736,7 @@ do_check_mismatch() {
const sampleSize = (p.inferred && p.inferred.sample_size) || 0;
const contributing = p.inferred && p.inferred.contributing_events;
if (!contributing && sampleSize < 10) {
if (sampleSize < 10) {
console.log('MISMATCH: not enough data (' + sampleSize + ' events; need 10+)');
process.exit(0);
}
@@ -1,8 +1,8 @@
**/plan-tune no longer reports a mismatch between your stated and observed preferences when it has no observed answers behind it (#3436).** `--check-mismatch` compared a declared value against an inferred value that had never moved from its 0.5 starting point, so a strong declaration with no matching answers printed `scope_appetite: declared 0.9 vs inferred 0.5 (gap 0.4)` even though no logged answer touched that dimension.
### Fixed
- **`bin/gstack-developer-profile`**: `--check-mismatch`, `--gap` and the gap that `--derive` stores compare a dimension only when at least 3 logged answers moved it. For a profile with per-dimension counts, this floor replaces the old 10-answer overall minimum. Below it, `--check-mismatch` prints `no observed evidence` or `too little evidence (n of 3 events)` instead of a gap, and `--gap` lists the dimension under a new `low_evidence` field. A profile derived before `contributing_events` existed keeps the old behavior until its next derive.
- **`bin/gstack-developer-profile`**: `--check-mismatch`, `--gap` and the gap that `--derive` stores compare a dimension only when at least 3 logged answers moved it. `--check-mismatch` still needs 10 logged answers overall before it reports anything. Below the 3-answer floor, `--check-mismatch` prints `no observed evidence` or `too little evidence (n of 3 events)` instead of a gap, and `--gap` lists the dimension under a new `low_evidence` field. A profile derived before `contributing_events` existed keeps the old behavior until its next derive.
- **/plan-tune "Show gap"**: says "not enough observed answers yet" for those dimensions instead of calling them drift.
### Tests
- `test/gstack-developer-profile.test.ts`: a declared dimension with no evidence is not flagged by `--check-mismatch` or `--gap` (both failed on main), 1 or 2 events report too little evidence, 3 events are compared, and `--derive` stores no gap without evidence.
- `test/gstack-developer-profile.test.ts`: a declared dimension with no evidence is not flagged by `--check-mismatch` or `--gap` (both failed on main), 1 or 2 events report too little evidence, 3 events are compared, three same-day answers that agree with the declaration still report not enough data, and `--derive` stores no gap without evidence.
+17 -2
View File
@@ -657,7 +657,7 @@ describe('gstack-developer-profile --check-mismatch', () => {
const p = readProfile() as any;
p.declared = { scope_appetite: 0.9 };
p.inferred.values.scope_appetite = 0.45;
p.inferred.sample_size = 2;
p.inferred.sample_size = 10;
p.inferred.contributing_events = { scope_appetite: 2 };
fs.writeFileSync(file, JSON.stringify(p));
const r = runDev('--check-mismatch');
@@ -672,13 +672,28 @@ describe('gstack-developer-profile --check-mismatch', () => {
const p = readProfile() as any;
p.declared = { scope_appetite: 0.9 };
p.inferred.values.scope_appetite = 0.45;
p.inferred.sample_size = 3;
p.inferred.sample_size = 10;
p.inferred.contributing_events = { scope_appetite: 3 };
fs.writeFileSync(file, JSON.stringify(p));
expect(runDev('--check-mismatch').stdout).toContain('scope_appetite: declared 0.9 vs inferred 0.45 (gap 0.45)');
expect(JSON.parse(runDev('--gap').stdout).gap.scope_appetite).toBeCloseTo(0.45, 2);
});
test('three same-day answers that agree with the declaration are not a mismatch', () => {
for (let i = 0; i < 3; i++) {
logQuestion({ skill: 'plan-ceo-review', question_id: 'plan-ceo-review-mode', question_summary: 'mode?', user_choice: 'expand', session_id: `s${i}`, ts: '2026-10-20T12:00:00Z' });
}
expect(runDev('--derive').status).toBe(0);
const file = path.join(tmpHome, 'developer-profile.json');
const p = readProfile() as any;
expect(p.inferred.contributing_events.scope_appetite).toBe(3);
p.declared = { scope_appetite: 0.9 };
fs.writeFileSync(file, JSON.stringify(p));
const r = runDev('--check-mismatch');
expect(r.stdout).toContain('not enough data');
expect(r.stdout).not.toContain('scope_appetite: declared');
});
test('derive stores the gap for a dimension with 3 contributing events', () => {
for (let i = 0; i < 3; i++) {
logQuestion({ skill: 'plan-eng-review', question_id: 'plan-eng-review-arch-finding', question_summary: 'arch?', user_choice: 'fix-now', session_id: `s${i}`, ts: `2026-10-2${i}T12:00:00Z` });