From d7ae2eaf8023d63e699910a877daafdf38fa480d Mon Sep 17 00:00:00 2001 From: Matteo Meucci Date: Mon, 17 Nov 2025 19:01:00 +0100 Subject: [PATCH] Update 4.5_Appendix_E.md --- Document/content/4.5_Appendix_E.md | 38 ------------------------------ 1 file changed, 38 deletions(-) diff --git a/Document/content/4.5_Appendix_E.md b/Document/content/4.5_Appendix_E.md index b0ee26a..7f5e725 100644 --- a/Document/content/4.5_Appendix_E.md +++ b/Document/content/4.5_Appendix_E.md @@ -97,8 +97,6 @@ AI System Architectural Components & Data (Note): Note: Component identifiers correspond to the SAIF numbering scheme illustrated in the threat model diagram within this guide. ---- - ## (2) User Input **Summary:** User Input is the front door of the system, every downstream component depends on it. Without strong input validation, filtering, and limits, it becomes the main vector for prompt injection, data leakage, DoS, and toxicity propagation. @@ -148,8 +146,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated - Use contextual and sentiment filters on incoming requests. - Escalate high-risk or policy-violating cases to human review workflows. ---- - ## (3) User Output **Summary:** The last mile to users/connected systems; without control, it’s a vector for excessive agency, prompt leakage, misinformation, and unsafe rendering. @@ -198,8 +194,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated - Sanitize HTML/Markdown through allowlisted elements and attributes. - Disable unsafe embeds, links, and inline scripts in all rendering environments. ---- - ## (4) Application **Summary:** The orchestration brain that manages sessions, APIs, and business logic. Weak validation, error handling, or access controls at this layer can cascade into systemic compromise across the entire application stack. @@ -276,8 +270,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated - Require secondary approvals or human-in-the-loop validation for high-impact actions. - Log and audit all agent-initiated operations for accountability. ---- - ## (5) Agent / Plugin **Summary:** Extended arms of the system; vulnerable to IPIJ, secrets handling, tampering, excessive actions, and unsafe workflows. @@ -322,8 +314,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Maintain strict tool allowlists and egress proxy controls; enforce validation of content types and schema for external responses. ---- - ## (6) External Sources **Summary:** Bridges to the outside world; unverified data can inject poison, trigger unsafe actions, or spread misinformation. @@ -376,8 +366,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Establish data provenance and reputation scoring mechanisms; perform adversarial sample and anomaly testing; apply cryptographic integrity checks on datasets and model artifacts throughout the pipeline. ---- - ## (7) Input Handling **Summary:** The filter layer; weak parsing/schema enforcement lets adversarial inputs/injections slip through. @@ -422,8 +410,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Use domain-based allowlists with outbound proxy enforcement; validate and enforce safe content types for all retrieved or external resources. ---- - ## (8) Output Handling **Summary:** Safety gate before delivery; failure here leaks sensitive data, misinformation, and unsafe content. @@ -484,8 +470,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Restrict actions to allowlisted commands; apply authorization and policy checks before execution; require explicit human confirmation for high-impact operations. ---- - ## (9) Model **Summary:** The core intelligence; targeted by injection, poisoning, theft, inversion, DoS, and unsafe outputs. @@ -562,8 +546,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Integrate toxicity and bias post-filters; ground model outputs in verified sources; restrict actionable outputs via policy enforcement; require approvals for high-risk autonomous actions. ---- - ## (10) Model Storage Infrastructure **Summary:** Crown jewels at rest — must be encrypted, signed, and access-controlled. @@ -600,8 +582,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Use WORM (Write Once, Read Many) or immutable storage for production models; perform integrity verification on model load; restrict access to service accounts with strict RBAC and scoped tokens. ---- - ## (11) Model Serving Infrastructure **Summary:** Execution gateway; must resist poisoning, theft, DoS, and unsafe outputs. @@ -638,8 +618,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Cap input request sizes and token lengths; configure quotas and throttling at the API gateway; use circuit breakers and autoscaling for load protection; validate input buffers and parsers to prevent overflow or runaway generation. ---- - ## (12) Evaluation **Summary:** Where model quality and trustworthiness are validated; weak evaluation enables unsafe, biased, or manipulated outputs to pass undetected. @@ -716,8 +694,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Isolate evaluation from training environments; enforce CSRF protection in evaluation tools; validate external dependencies and ensure reproducible runs. ---- - ## (13) Training & Tuning **Summary:** Where knowledge is forged; poor data embeds lasting bias and backdoors. @@ -762,8 +738,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Encrypt model checkpoints, logs, and gradient data with strong key management (KMS); apply RBAC and access scoping to all storage locations; conduct regular permission audits and integrity checks across training infrastructure. ---- - ## (14) Model Frameworks & Code **Summary:** ML runtime backbone; supply chain or unsafe integrations taint the system. @@ -792,8 +766,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Disable or sandbox dynamic `eval` and code-generation features; restrict plugin or module loading to trusted sources; isolate untrusted or experimental code in containers; harden deserialization routines and enforce strict content-type validation. ---- - ## (15) Data Storage Infrastructure **Summary:** Knowledge vault; poisoning/tampering/leaks here undermine integrity & confidentiality. @@ -830,8 +802,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Enforce quotas and rate limits for data ingestion; validate and harden file parsers and buffer handling; apply throttling and back-pressure controls for high-volume writes or uploads. ---- - ## (16) Training Data **Summary:** Root of trust; compromise propagates to all downstream behavior. @@ -868,8 +838,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Maintain signed and version-controlled datasets; apply provenance and reputation scoring for all data sources; perform golden-set cross-validation to detect data drift or contamination. ---- - ## (17) Data Filtering & Processing **Summary:** Gatekeeper stage; weak validation lets poisoned/sensitive data pass. @@ -914,8 +882,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Enforce data size quotas and schema validation; apply ingestion rate limits and back-pressure controls; monitor for pipeline anomalies and memory spikes in ETL workloads. ---- - ## (18) Data Sources **Summary:** Entry point of truth; without provenance checks, they introduce poisoned/unsafe content. @@ -952,8 +918,6 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Recommendations:** Apply reliability and reputation scoring for all data sources; cross-reference new data against ground-truth sets; perform continuous drift and consistency monitoring across ingestion pipelines. ---- - ## (19) External Sources **Summary:** Outside the trust boundary; major vectors for poisoning, leakage, and misinformation. @@ -989,5 +953,3 @@ Note: Component identifiers correspond to the SAIF numbering scheme illustrated **Rationale:** External feeds and open data sources may provide low-reliability or adversarial content that misguides training or inference outputs, spreading false narratives or bias. **Recommendations:** Assign reliability scores and reputation metrics to external sources; validate information against ground-truth datasets; require human review for high-impact or public-facing data feeds. - ----