mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-15 04:47:23 +01:00
feat(cso-eval): daily precision counts confirmed incidental issues and leaves out declared fixture instrumentation
This commit is contained in:
1 parent
d804fbb2fc
commit
d561177ef4
3 files changed
+46
-5
No files matched your search
@@ -294,6 +294,18 @@ requalified catalog or withdraws the catalog's runtimes in the same pull
|
||||
request. Independently, the helper refuses to select a runtime recorded under
|
||||
another isolation policy.
|
||||
|
||||
The daily precision gate (`dailyPrecision95` in
|
||||
[`cso-eval.ts`](../scripts/cso-eval.ts)) divides true positives by reported
|
||||
findings. Hypotheses count in neither term. A finding the evaluator marks
|
||||
`instrumentation` also counts in neither term: it targets the evaluator's
|
||||
declared fixture instrumentation, such as the canary overlay files or the egress
|
||||
probe route. The seeded issue adds one true positive per cell, however many
|
||||
times it is restated. A finding the evaluator marks `incidental` is a different
|
||||
issue that a unanimous judge panel confirmed is real in that source. It adds one
|
||||
true positive per distinct `incidentalKey` in its cell, never counts toward
|
||||
recall, and needs no seeded match. Recall still measures only the seeded issue,
|
||||
and the thresholds are unchanged.
|
||||
|
||||
The judge observes held-out attacks in one boot: a `VerificationRequest`'s
|
||||
`security` field is either one assertion or an array of up to eight assertions
|
||||
with distinct names, judged in a single before/after pair. The before phase is
|
||||
|
||||
Reference in new issue
Block a user