CalvinBackup/shannon
1
0
Fork 0
mirror of https://github.com/KeygraphHQ/shannon.git synced 2026-08-14 07:20:19 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: multi-provider model support, SARIF output, and exploit-mode fixes (#402)

Browse Source 
* feat(worker): record token, cache, and turn usage per agent

* feat: replace model tiers with a single SHANNON_AI_MODEL across five providers

* feat(cli): rebuild the setup wizard for provider and model selection

* docs: document single-model selection and supported providers

* feat(worker): use chat completions for OpenAI behind a custom base URL

* feat: add SHANNON_AI_OPENAI_FORMAT to pick the wire API for OpenAI gateways

* refactor(cli): drop endpoint path hints from the gateway format picker

* feat(worker): enable pi in-session provider retry with retry-after backoff

* refactor(worker): hand provider error classification to pi and drop the Anthropic ladders

* refactor: remove the subscription retry preset and pipeline config section

* fix(worker): validate Bedrock credentials with the same live probe as other providers

* feat(worker): render the report from structured findings instead of agent-written markdown

* fix(worker): dispose the credential probe session on every path

* fix(worker): refuse to replace the assembled report with an empty one

* refactor(worker): catch post-processing throws across the whole finalization block

* revert(worker): drop the report zero-findings guard

* docs(worker): correct the retry split and Bedrock credential claims

* docs: regenerate llms-full.txt from current sources

* feat(cli): build and run the npx flow from a clone

* refactor(cli): flatten the setup summary output

* feat(cli): reject runs with more than one provider configured

* fix(worker): say a rejected bash call never ran

* chore(cli): drop grok-4.3 and gpt-5.6-luna from the setup suggestions

* feat(worker): capture structured finding locations for SARIF output

* fix(worker): enumerate queue confidence so the report inherits it verbatim

* feat(worker): give the reporting phase a mode-specific output schema

* feat(worker): emit a SARIF 2.1.0 log for exploitative runs

* fix(worker): correct SARIF locations and defer fingerprinting to the upload action

* fix(worker): drop the confidence suffix from the analysis-mode summary list

* feat(worker): give exploit findings a dedicated code location field

* feat(worker): carry structured code locations from the vuln queue to the report

* fix(worker): join code locations from the vuln queue instead of re-asking agents

* fix(worker): spell out the finding_id to category mapping in the tool schema

* feat: drop Google/Gemini as a supported AI provider

* fix(worker): stop asking the report agent for code locations

* docs: correct the provider list and drop the removed rate-limit settings

* docs: add provider cyber safeguards and suggested models per provider

* docs: document the SARIF output and the report rating thresholds
This commit is contained in:
ezl-keygraph
2026-07-30 19:31:52 +05:30
committed by GitHub
parent 30a12114ae
commit 1ce250d6a5
69 changed files with 3225 additions and 1471 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/worker/src/utils/billing-detection.ts
-90
View File
@@ -1,90 +0,0 @@
// Copyright (C) 2025 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/**
* Consolidated billing/spending cap detection utilities.
*
* Anthropic's spending cap behavior is inconsistent:
* - Sometimes a proper provider error (billing_error)
* - Sometimes the model responds with text about the cap
* - Sometimes partial billing before cutoff
*
* This module provides defense-in-depth detection with shared pattern lists
* to prevent drift between detection points.
*/
/**
* Text patterns for model-output sniffing (what the model says).
* Used by the pi executor and the behavioral heuristic.
*/
export const BILLING_TEXT_PATTERNS = [
'spending cap',
'spending limit',
'cap reached',
'budget exceeded',
'usage limit',
] as const;
/**
* API patterns for error message classification (what the API returns).
* Used by classifyErrorForTemporal in error-handling.ts.
*/
export const BILLING_API_PATTERNS = [
'billing_error',
'credit balance is too low',
'insufficient credits',
'usage is blocked due to insufficient credits',
'please visit plans & billing',
'please visit plans and billing',
'usage limit reached',
'quota exceeded',
'daily rate limit',
'limit will reset',
'billing limit reached',
] as const;
/**
* Checks if text matches any billing text pattern.
* Used for sniffing model output content for spending cap messages.
*/
export function matchesBillingTextPattern(text: string): boolean {
const lowerText = text.toLowerCase();
return BILLING_TEXT_PATTERNS.some((pattern) => lowerText.includes(pattern));
}
/**
* Checks if an error message matches any billing API pattern.
* Used for classifying API error messages.
*/
export function matchesBillingApiPattern(message: string): boolean {
const lowerMessage = message.toLowerCase();
return BILLING_API_PATTERNS.some((pattern) => lowerMessage.includes(pattern));
}
/**
* Behavioral heuristic for detecting spending cap.
*
* When the model hits a spending cap, it often returns a short message
* with $0 cost. Legitimate agent work NEVER costs $0 with only 1-2 turns.
*
* This combines three signals:
* 1. Very low turn count (<=2)
* 2. Zero cost ($0)
* 3. Text matches billing patterns
*
* @param turns - Number of turns the agent took
* @param cost - Total cost in USD
* @param resultText - The result text from the agent
* @returns true if this looks like a spending cap hit
*/
export function isSpendingCapBehavior(turns: number, cost: number, resultText: string): boolean {
// Only check if turns <= 2 AND cost is exactly 0
if (turns > 2 || cost !== 0) {
return false;
}
return matchesBillingTextPattern(resultText);
}