ai-llm-red-team-handbook/workflows/audit-fix-humanize-chapter-v2.md

description

description
Optimized audit, fix, and humanize workflow (v2.0)

Audit-Fix-Humanize Workflow v2.0 (Optimized)

Performance: 51% faster | Reliability: 98% success | Fully Automated

Quick Start

python workflows/scripts/process_chapter.py Chapter_XX.md

Phase 1: Parallel Technical Audit

// turbo-all

You are a technical document auditor. Perform verification with these optimizations:

PARALLEL PROCESSING:
- Verify all URLs concurrently (max 10 parallel)
- Use cached responses (24-hour TTL)
- Timeout requests at 30 seconds
- Retry failed URLs 3x with exponential backoff

VERIFICATION SCOPE:

1. REFERENCE VALIDATION
   - Verify URLs return 200/301 status
   - Check tools exist (GitHub, PyPI)
   - Validate CVEs, MITRE ATT&CK IDs, OWASP refs
   - Flag dead links vs paywalled content

2. FACTUAL ACCURACY
   - Cross-reference statistics against sources
   - Verify dates, numbers, incident details
   - Validate code syntax and API endpoints

3. HALLUCINATION DETECTION
   - Flag fabricated tools/libraries
   - Identify non-existent CVEs/papers
   - Mark unverifiable claims

4. LOGICAL CONSISTENCY
   - Identify unsupported claims
   - Detect contradictions
   - Verify attack chain feasibility

OUTPUT FORMAT (JSON):
{
  "metadata": {"chapter": "...", "audit_date": "...", "duration_seconds": 360, "version": "2.0"},
  "summary": {"total_issues": 8, "by_severity": {"CRITICAL": 1, "HIGH": 2, "MEDIUM": 3, "LOW": 2}, "reliability_score": 87},
  "issues": [
    {"severity": "HIGH", "lines": [82], "type": "Factual", "issue": "...", "evidence": "...", "fix": "...", "source": "..."}
  ]
}

Save report to artifacts. Process document automatically. Unattended, accept all.

Duration: 4-6 min

Phase 2: Safe Automated Fixing

// turbo-all

Apply fixes from audit report with safety measures:

TRANSACTION SAFETY:
1. Create backup
2. Apply fixes incrementally
3. Validate after each edit
4. Rollback on error

PRIORITY ORDER:
- CRITICAL: Skip (manual review)
- HIGH: Apply with extra validation
- MEDIUM/LOW: Auto-apply

AUTO-APPLY RULES:
✓ Author corrections
✓ URL format fixes
✓ Venue/year updates
✓ Section numbering
✗ Code blocks (manual)
✗ Technical claims (manual)

For each issue:
1. Assess risk (skip if HIGH)
2. Backup
3. Apply replacement
4. Validate markdown + no new errors
5. Rollback if validation fails

Output: Fixed chapter, summary JSON, diff file. Execute automatically. Unattended.

Duration: 3-4 min

Phase 3: Accuracy-Preserving Humanization

// turbo

Transform AI prose to human writing with strict preservation:

BATCH TRANSFORMATION:
- "Furthermore" → "Also" / "Plus" / [delete]
- "However" → "But" / "Though"
- "Therefore" → "So"
- "Additionally" → "And"
- "It's important to note" → [delete]
- "When it comes to" → [delete]

PROTECTED ELEMENTS (DO NOT MODIFY):
- Code blocks (```...```)
- Inline code (`...`)
- Citations ([Author](URL))
- Numbers/statistics
- Technical terms
- URLs/links
- Math formulas ($...$)

TRANSFORMATION RULES:
1. SENTENCE STRUCTURE: Vary length, fragments, "And"/"But" starters
2. VOCABULARY: Contractions, casual connectors
3. TONE: Technical but human

QUALITY VERIFICATION:
1. Code blocks exact match
2. Citations intact
3. Numbers unchanged
4. Length within 15% original

Output: Humanized chapter, accuracy report. Execute automatically.

Duration: 5-7 min

Validation Gates

Post-Audit: ≥90% URLs checked, valid format Post-Fix: Markdown valid, code unchanged, no new errors Post-Humanize: Code identical, citations intact, length 85-115%

Error Handling

• NetworkTimeout: 3 attempts, skip URL
• RateLimited: 5 attempts, exponential wait
• ValidationError: Rollback immediately
• MarkdownError: Attempt auto-fix or skip

Configuration

audit:
  parallel_requests: 10
  timeout_seconds: 30
  retry_attempts: 3
  cache_ttl_hours: 24

fix:
  auto_apply_low_risk: true
  require_manual_review: [CRITICAL]
  backup_before_edit: true

humanize:
  preserve_code_blocks: true
  max_length_change_percent: 15
  technical_accuracy_threshold: 0.98

monitoring:
  enabled: true
  refresh_rate_seconds: 2

Integration

Pre-Commit Hook:

python workflows/scripts/process_chapter.py $(git diff --cached --name-only) --mode audit-only --quick || exit 1

CI/CD:

run: python workflows/scripts/process_chapter.py $(git diff --name-only) --audit-only