mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-07-06 16:57:51 +02:00
Release v0.7.3 - Android workflows, LiteLLM integration, ARM64 support (#32)
* ci: add worker validation and Docker build checks Add automated validation to prevent worker-related issues: **Worker Validation Script:** - New script: .github/scripts/validate-workers.sh - Validates all workers in docker-compose.yml exist - Checks required files: Dockerfile, requirements.txt, worker.py - Verifies files are tracked by git (not gitignored) - Detects gitignore issues that could hide workers **CI Workflow Updates:** - Added validate-workers job (runs on every PR) - Added build-workers job (runs if workers/ modified) - Uses Docker Buildx for caching - Validates Docker images build successfully - Updated test-summary to check validation results **PR Template:** - New pull request template with comprehensive checklist - Specific section for worker-related changes - Reminds contributors to validate worker files - Includes documentation and changelog reminders These checks would have caught the secrets worker gitignore issue. Implements Phase 1 improvements from CI/CD quality assessment. * fix: add dev branch to test workflow triggers The test workflow was configured for 'develop' but the actual branch is named 'dev'. This caused tests not to run on PRs to dev branch. Now tests will run on: - PRs to: main, master, dev, develop - Pushes to: main, master, dev, develop, feature/** * fix: properly detect worker file changes in CI The previous condition used invalid GitHub context field. Now uses git diff to properly detect changes to workers/ or docker-compose.yml. Behavior: - Job always runs the check step - Detects if workers/ or docker-compose.yml modified - Only builds Docker images if workers actually changed - Shows clear skip message when no worker changes detected * feat: Add Python SAST workflow with three security analysis tools Implements Issue #5 - Python SAST workflow that combines: - Dependency scanning (pip-audit) for CVE detection - Security linting (Bandit) for vulnerability patterns - Type checking (Mypy) for type safety issues ## Changes **New Modules:** - `DependencyScanner`: Scans Python dependencies for known CVEs using pip-audit - `BanditAnalyzer`: Analyzes Python code for security issues using Bandit - `MypyAnalyzer`: Checks Python code for type safety issues using Mypy **New Workflow:** - `python_sast`: Temporal workflow that orchestrates all three SAST tools - Runs tools in parallel for fast feedback (3-5 min vs hours for fuzzing) - Generates unified SARIF report with findings from all tools - Supports configurable severity/confidence thresholds **Updates:** - Added SAST dependencies to Python worker (bandit, pip-audit, mypy) - Updated module __init__.py files to export new analyzers - Added type_errors.py test file to vulnerable_app for Mypy validation ## Testing Workflow tested successfully on vulnerable_app: - ✅ Bandit: Detected 9 security issues (command injection, unsafe functions) - ✅ Mypy: Detected 5 type errors - ✅ DependencyScanner: Ran successfully (no CVEs in test dependencies) - ✅ SARIF export: Generated valid SARIF with 14 total findings * fix: Remove unused imports to pass linter * fix: resolve live monitoring bug, remove deprecated parameters, and auto-start Python worker - Fix live monitoring style error by calling _live_monitor() helper directly - Remove default_parameters duplication from 10 workflow metadata files - Remove deprecated volume_mode parameter from 26 files across CLI, SDK, backend, and docs - Configure Python worker to start automatically with docker compose up - Clean up constants, validation, completion, and example files Fixes # - Live monitoring now works correctly with --live flag - Workflow metadata follows JSON Schema standard - Cleaner codebase without deprecated volume_mode - Python worker (most commonly used) starts by default * fix: resolve linter errors and optimize CI worker builds - Remove unused Literal import from backend findings model - Remove unnecessary f-string prefixes in CLI findings command - Optimize GitHub Actions to build only modified workers - Detect specific worker changes (python, secrets, rust, android, ossfuzz) - Build only changed workers instead of all 5 - Build all workers if docker-compose.yml changes - Significantly reduces CI build time * feat: Add Android static analysis workflow with Jadx, OpenGrep, and MobSF Comprehensive Android security testing workflow converted from Prefect to Temporal architecture: Modules (3): - JadxDecompiler: APK to Java source code decompilation - OpenGrepAndroid: Static analysis with Android-specific security rules - MobSFScanner: Comprehensive mobile security framework integration Custom Rules (13): - clipboard-sensitive-data, hardcoded-secrets, insecure-data-storage - insecure-deeplink, insecure-logging, intent-redirection - sensitive_data_sharedPreferences, sqlite-injection - vulnerable-activity, vulnerable-content-provider, vulnerable-service - webview-javascript-enabled, webview-load-arbitrary-url Workflow: - 6-phase Temporal workflow: download → Jadx → OpenGrep → MobSF → SARIF → upload - 4 activities: decompile_with_jadx, scan_with_opengrep, scan_with_mobsf, generate_android_sarif - SARIF output combining findings from all security tools Docker Worker: - ARM64 Mac compatibility via amd64 platform emulation - Pre-installed: Android SDK, Jadx 1.4.7, OpenGrep 1.45.0, MobSF 3.9.7 - MobSF runs as background service with API key auto-generation - Added aiohttp for async HTTP communication Test APKs: - BeetleBug.apk and shopnest.apk for workflow validation * fix(android): correct activity names and MobSF API key generation - Fix activity names in workflow.py (get_target, upload_results, cleanup_cache) - Fix MobSF API key generation in Dockerfile startup script (cut delimiter) - Update activity parameter signatures to match actual implementations - Workflow now executes successfully with Jadx and OpenGrep * feat: add platform-aware worker architecture with ARM64 support Implement platform-specific Dockerfile selection and graceful tool degradation to support both x86_64 and ARM64 (Apple Silicon) platforms. **Backend Changes:** - Add system info API endpoint (/system/info) exposing host filesystem paths - Add FUZZFORGE_HOST_ROOT environment variable to backend service - Add graceful degradation in MobSF activity for ARM64 platforms **CLI Changes:** - Implement multi-strategy path resolution (backend API, .fuzzforge marker, env var) - Add platform detection (linux/amd64 vs linux/arm64) - Add worker metadata.yaml reading for platform capabilities - Auto-select appropriate Dockerfile based on detected platform - Pass platform-specific env vars to docker-compose **Worker Changes:** - Create workers/android/metadata.yaml defining platform capabilities - Rename Dockerfile -> Dockerfile.amd64 (full toolchain with MobSF) - Create Dockerfile.arm64 (excludes MobSF due to Rosetta 2 incompatibility) - Update docker-compose.yml to use ${ANDROID_DOCKERFILE} variable **Workflow Changes:** - Handle MobSF "skipped" status gracefully in workflow - Log clear warnings when tools are unavailable on platform **Key Features:** - Automatic platform detection and Dockerfile selection - Graceful degradation when tools unavailable (MobSF on ARM64) - Works from any directory (backend API provides paths) - Manual override via environment variables - Clear user feedback about platform and selected Dockerfile **Benefits:** - Android workflow now works on Apple Silicon Macs - No code changes needed for other workflows - Convention established for future platform-specific workers Closes: MobSF Rosetta 2 incompatibility issue Implements: Platform-aware worker architecture (Option B) * fix: make MobSFScanner import conditional for ARM64 compatibility - Add try-except block to conditionally import MobSFScanner in modules/android/__init__.py - Allows Android worker to start on ARM64 without MobSF dependencies (aiohttp) - MobSF activity gracefully skips on ARM64 with clear warning message - Remove workflow path detection logic (not needed - workflows receive directories) Platform-aware architecture fully functional on ARM64: - CLI detects ARM64 and selects Dockerfile.arm64 automatically - Worker builds and runs without MobSF on ARM64 - Jadx successfully decompiles APKs (4145 files from BeetleBug.apk) - OpenGrep finds security vulnerabilities (8 issues found) - MobSF gracefully skips with warning on ARM64 - Graceful degradation working as designed Tested with: ff workflow run android_static_analysis test_projects/android_test/ \ --wait --no-interactive apk_path=BeetleBug.apk decompile_apk=true Results: 8 security findings (1 ERROR, 7 WARNINGS) * docs: update CHANGELOG with Android workflow and ARM64 support Added [Unreleased] section documenting: - Android Static Analysis Workflow (Jadx, OpenGrep, MobSF) - Platform-Aware Worker Architecture with ARM64 support - Python SAST Workflow - CI/CD improvements and worker validation - CLI enhancements - Bug fixes and technical changes Fixed date typo: 2025-01-16 → 2025-10-16 * fix: resolve linter errors in Android modules - Remove unused imports from mobsf_scanner.py (asyncio, hashlib, json, Optional) - Remove unused variables from opengrep_android.py (start_col, end_col) - Remove duplicate Path import from workflow.py * ci: support multi-platform Dockerfiles in worker validation Updated worker validation script to accept both: - Single Dockerfile pattern (existing workers) - Multi-platform Dockerfile pattern (Dockerfile.amd64, Dockerfile.arm64, etc.) This enables platform-aware worker architectures like the Android worker which uses different Dockerfiles for x86_64 and ARM64 platforms. * Feature/litellm proxy (#27) * feat: seed governance config and responses routing * Add env-configurable timeout for proxy providers * Integrate LiteLLM OTEL collector and update docs * Make .env.litellm optional for LiteLLM proxy * Add LiteLLM proxy integration with model-agnostic virtual keys Changes: - Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50) - Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion - All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions) - Bootstrap handles database/env mismatch after docker prune by deleting stale aliases - CLI and Cognee configured to use LiteLLM proxy with virtual keys - Added comprehensive documentation in volumes/env/README.md Technical details: - task-agent entrypoint waits for keys in .env file before starting uvicorn - Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY - Removed hardcoded API keys from docker-compose.yml - All services route through http://localhost:10999 proxy * Fix CLI not loading virtual keys from global .env Project .env files with empty OPENAI_API_KEY values were overriding the global virtual keys. Updated _load_env_file_if_exists to only override with non-empty values. * Fix agent executor not passing API key to LiteLLM The agent was initializing LiteLlm without api_key or api_base, causing authentication errors when using the LiteLLM proxy. Now reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment variables and passes them to LiteLlm constructor. * Auto-populate project .env with virtual key from global config When running 'ff init', the command now checks for a global volumes/env/.env file and automatically uses the OPENAI_API_KEY virtual key if found. This ensures projects work with LiteLLM proxy out of the box without manual key configuration. * docs: Update README with LiteLLM configuration instructions Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy. * Refactor workflow parameters to use JSON Schema defaults Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable. * Remove .env.example from task_agent * Fix MDX syntax error in llm-proxy.md * fix: apply default parameters from metadata.yaml automatically Fixed TemporalManager.run_workflow() to correctly apply default parameter values from workflow metadata.yaml files when parameters are not provided by the caller. Previous behavior: - When workflow_params was empty {}, the condition `if workflow_params and 'parameters' in metadata` would fail - Parameters would not be extracted from schema, resulting in workflows receiving only target_id with no other parameters New behavior: - Removed the `workflow_params and` requirement from the condition - Now explicitly checks for defaults in parameter spec - Applies defaults from metadata.yaml automatically when param not provided - Workflows receive all parameters with proper fallback: provided value > metadata default > None This makes metadata.yaml the single source of truth for parameter defaults, removing the need for workflows to implement defensive default handling. Affected workflows: - llm_secret_detection (was failing with KeyError) - All other workflows now benefit from automatic default application Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com> * fix: add default values to llm_analysis workflow parameters Resolves validation error where agent_url was None when not explicitly provided. The TemporalManager applies defaults from metadata.yaml, not from module input schemas, so all parameters need defaults in the workflow metadata. Changes: - Add default agent_url, llm_model (gpt-5-mini), llm_provider (openai) - Expand file_patterns to 45 comprehensive patterns covering code, configs, secrets, and Docker files - Increase default limits: max_files (10), max_file_size (100KB), timeout (90s) * refactor: replace .env.example with .env.template in documentation - Remove volumes/env/.env.example file - Update all documentation references to use .env.template instead - Update bootstrap script error message - Update .gitignore comment * feat(cli): add worker management commands with improved progress feedback Add comprehensive CLI commands for managing Temporal workers: - ff worker list - List workers with status and uptime - ff worker start <name> - Start specific worker with optional rebuild - ff worker stop - Safely stop all workers without affecting core services Improvements: - Live progress display during worker startup with Rich Status spinner - Real-time elapsed time counter and container state updates - Health check status tracking (starting → unhealthy → healthy) - Helpful contextual hints at 10s, 30s, 60s intervals - Better timeout messages showing last known state Worker management enhancements: - Use 'docker compose' (space) instead of 'docker-compose' (hyphen) - Stop workers individually with 'docker stop' to avoid stopping core services - Platform detection and Dockerfile selection (ARM64/AMD64) Documentation: - Updated docker-setup.md with CLI commands as primary method - Created comprehensive cli-reference.md with all commands and examples - Added worker management best practices * fix: MobSF scanner now properly parses files dict structure MobSF returns 'files' as a dict (not list): {"filename": "line_numbers"} The parser was treating it as a list, causing zero findings to be extracted. Now properly iterates over the dict and creates one finding per affected file with correct line numbers and metadata (CWE, OWASP, MASVS, CVSS). Fixed in both code_analysis and behaviour sections. * chore: bump version to 0.7.3 * docs: fix broken documentation links in cli-reference * chore: add worker startup documentation and cleanup .gitignore - Add workflow-to-worker mapping tables across documentation - Update troubleshooting guide with worker requirements section - Enhance getting started guide with worker examples - Add quick reference to docker setup guide - Add WEEK_SUMMARY*.md pattern to .gitignore * docs: update CHANGELOG with missing versions and recent changes - Add Unreleased section for post-v0.7.3 documentation updates - Add v0.7.2 entry with bug fixes and worker improvements - Document that v0.7.1 was re-tagged as v0.7.2 - Fix v0.6.0 date to "Undocumented" (no tag exists) - Add version comparison links for easier navigation * chore: bump all package versions to 0.7.3 for consistency * Update GitHub link to fuzzforge_ai --------- Co-authored-by: Songbird99 <150154823+Songbird99@users.noreply.github.com> Co-authored-by: Songbird <Songbirdx99@gmail.com>
This commit is contained in:
tduhamel42
@@ -22,7 +22,6 @@
|
||||
"parameters": {
|
||||
"workflow_name": "string",
|
||||
"target_path": "string",
|
||||
"volume_mode": "string (ro|rw)",
|
||||
"parameters": "object"
|
||||
}
|
||||
},
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "backend"
|
||||
version = "0.7.0"
|
||||
version = "0.7.3"
|
||||
description = "FuzzForge OSS backend"
|
||||
authors = []
|
||||
readme = "README.md"
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
"""
|
||||
System information endpoints for FuzzForge API.
|
||||
|
||||
Provides system configuration and filesystem paths to CLI for worker management.
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import Dict
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
router = APIRouter(prefix="/system", tags=["system"])
|
||||
|
||||
|
||||
@router.get("/info")
|
||||
async def get_system_info() -> Dict[str, str]:
|
||||
"""
|
||||
Get system information including host filesystem paths.
|
||||
|
||||
This endpoint exposes paths needed by the CLI to manage workers via docker-compose.
|
||||
The FUZZFORGE_HOST_ROOT environment variable is set by docker-compose and points
|
||||
to the FuzzForge installation directory on the host machine.
|
||||
|
||||
Returns:
|
||||
Dictionary containing:
|
||||
- host_root: Absolute path to FuzzForge root on host
|
||||
- docker_compose_path: Path to docker-compose.yml on host
|
||||
- workers_dir: Path to workers directory on host
|
||||
"""
|
||||
host_root = os.getenv("FUZZFORGE_HOST_ROOT", "")
|
||||
|
||||
return {
|
||||
"host_root": host_root,
|
||||
"docker_compose_path": f"{host_root}/docker-compose.yml" if host_root else "",
|
||||
"workers_dir": f"{host_root}/workers" if host_root else "",
|
||||
}
|
||||
@@ -43,6 +43,42 @@ ALLOWED_CONTENT_TYPES = [
|
||||
router = APIRouter(prefix="/workflows", tags=["workflows"])
|
||||
|
||||
|
||||
def extract_defaults_from_json_schema(metadata: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Extract default parameter values from JSON Schema format.
|
||||
|
||||
Converts from:
|
||||
parameters:
|
||||
properties:
|
||||
param_name:
|
||||
default: value
|
||||
|
||||
To:
|
||||
{param_name: value}
|
||||
|
||||
Args:
|
||||
metadata: Workflow metadata dictionary
|
||||
|
||||
Returns:
|
||||
Dictionary of parameter defaults
|
||||
"""
|
||||
defaults = {}
|
||||
|
||||
# Check if there's a legacy default_parameters field
|
||||
if "default_parameters" in metadata:
|
||||
defaults.update(metadata["default_parameters"])
|
||||
|
||||
# Extract defaults from JSON Schema parameters
|
||||
parameters = metadata.get("parameters", {})
|
||||
properties = parameters.get("properties", {})
|
||||
|
||||
for param_name, param_spec in properties.items():
|
||||
if "default" in param_spec:
|
||||
defaults[param_name] = param_spec["default"]
|
||||
|
||||
return defaults
|
||||
|
||||
|
||||
def create_structured_error_response(
|
||||
error_type: str,
|
||||
message: str,
|
||||
@@ -164,7 +200,7 @@ async def get_workflow_metadata(
|
||||
author=metadata.get("author"),
|
||||
tags=metadata.get("tags", []),
|
||||
parameters=metadata.get("parameters", {}),
|
||||
default_parameters=metadata.get("default_parameters", {}),
|
||||
default_parameters=extract_defaults_from_json_schema(metadata),
|
||||
required_modules=metadata.get("required_modules", [])
|
||||
)
|
||||
|
||||
@@ -221,7 +257,7 @@ async def submit_workflow(
|
||||
# Merge default parameters with user parameters
|
||||
workflow_info = temporal_mgr.workflows[workflow_name]
|
||||
metadata = workflow_info.metadata or {}
|
||||
defaults = metadata.get("default_parameters", {})
|
||||
defaults = extract_defaults_from_json_schema(metadata)
|
||||
user_params = submission.parameters or {}
|
||||
workflow_params = {**defaults, **user_params}
|
||||
|
||||
@@ -450,7 +486,7 @@ async def upload_and_submit_workflow(
|
||||
# Merge default parameters with user parameters
|
||||
workflow_info = temporal_mgr.workflows.get(workflow_name)
|
||||
metadata = workflow_info.metadata or {}
|
||||
defaults = metadata.get("default_parameters", {})
|
||||
defaults = extract_defaults_from_json_schema(metadata)
|
||||
workflow_params = {**defaults, **workflow_params}
|
||||
|
||||
# Start workflow execution
|
||||
@@ -617,11 +653,8 @@ async def get_workflow_parameters(
|
||||
else:
|
||||
param_definitions = parameters_schema
|
||||
|
||||
# Add default values to the schema
|
||||
default_params = metadata.get("default_parameters", {})
|
||||
for param_name, param_schema in param_definitions.items():
|
||||
if isinstance(param_schema, dict) and param_name in default_params:
|
||||
param_schema["default"] = default_params[param_name]
|
||||
# Extract default values from JSON Schema
|
||||
default_params = extract_defaults_from_json_schema(metadata)
|
||||
|
||||
return {
|
||||
"workflow": workflow_name,
|
||||
|
||||
+3
-16
@@ -24,7 +24,7 @@ from fastmcp.server.http import create_sse_app
|
||||
|
||||
from src.temporal.manager import TemporalManager
|
||||
from src.core.setup import setup_result_storage, validate_infrastructure
|
||||
from src.api import workflows, runs, fuzzing
|
||||
from src.api import workflows, runs, fuzzing, system
|
||||
|
||||
from fastmcp import FastMCP
|
||||
|
||||
@@ -76,6 +76,7 @@ app = FastAPI(
|
||||
app.include_router(workflows.router)
|
||||
app.include_router(runs.router)
|
||||
app.include_router(fuzzing.router)
|
||||
app.include_router(system.router)
|
||||
|
||||
|
||||
def get_temporal_status() -> Dict[str, Any]:
|
||||
@@ -212,14 +213,6 @@ def _lookup_workflow(workflow_name: str):
|
||||
metadata = info.metadata
|
||||
defaults = metadata.get("default_parameters", {})
|
||||
default_target_path = metadata.get("default_target_path") or defaults.get("target_path")
|
||||
supported_modes = metadata.get("supported_volume_modes") or ["ro", "rw"]
|
||||
if not isinstance(supported_modes, list) or not supported_modes:
|
||||
supported_modes = ["ro", "rw"]
|
||||
default_volume_mode = (
|
||||
metadata.get("default_volume_mode")
|
||||
or defaults.get("volume_mode")
|
||||
or supported_modes[0]
|
||||
)
|
||||
return {
|
||||
"name": workflow_name,
|
||||
"version": metadata.get("version", "0.6.0"),
|
||||
@@ -229,9 +222,7 @@ def _lookup_workflow(workflow_name: str):
|
||||
"parameters": metadata.get("parameters", {}),
|
||||
"default_parameters": metadata.get("default_parameters", {}),
|
||||
"required_modules": metadata.get("required_modules", []),
|
||||
"supported_volume_modes": supported_modes,
|
||||
"default_target_path": default_target_path,
|
||||
"default_volume_mode": default_volume_mode
|
||||
"default_target_path": default_target_path
|
||||
}
|
||||
|
||||
|
||||
@@ -256,10 +247,6 @@ async def list_workflows_mcp() -> Dict[str, Any]:
|
||||
"description": metadata.get("description", ""),
|
||||
"author": metadata.get("author"),
|
||||
"tags": metadata.get("tags", []),
|
||||
"supported_volume_modes": metadata.get("supported_volume_modes", ["ro", "rw"]),
|
||||
"default_volume_mode": metadata.get("default_volume_mode")
|
||||
or defaults.get("volume_mode")
|
||||
or "ro",
|
||||
"default_target_path": metadata.get("default_target_path")
|
||||
or defaults.get("target_path")
|
||||
})
|
||||
|
||||
@@ -14,7 +14,7 @@ Models for workflow findings and submissions
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
from typing import Dict, Any, Optional, Literal, List
|
||||
from typing import Dict, Any, Optional, List
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
@@ -73,10 +73,6 @@ class WorkflowMetadata(BaseModel):
|
||||
default_factory=list,
|
||||
description="Required module names"
|
||||
)
|
||||
supported_volume_modes: List[Literal["ro", "rw"]] = Field(
|
||||
default=["ro", "rw"],
|
||||
description="Supported volume mount modes"
|
||||
)
|
||||
|
||||
|
||||
class WorkflowListItem(BaseModel):
|
||||
|
||||
@@ -187,12 +187,28 @@ class TemporalManager:
|
||||
|
||||
# Add parameters in order based on metadata schema
|
||||
# This ensures parameters match the workflow signature order
|
||||
if workflow_params and 'parameters' in workflow_info.metadata:
|
||||
# Apply defaults from metadata.yaml if parameter not provided
|
||||
if 'parameters' in workflow_info.metadata:
|
||||
param_schema = workflow_info.metadata['parameters'].get('properties', {})
|
||||
logger.debug(f"Found {len(param_schema)} parameters in schema")
|
||||
# Iterate parameters in schema order and add values
|
||||
for param_name in param_schema.keys():
|
||||
param_value = workflow_params.get(param_name)
|
||||
param_spec = param_schema[param_name]
|
||||
|
||||
# Use provided param, or fall back to default from metadata
|
||||
if workflow_params and param_name in workflow_params:
|
||||
param_value = workflow_params[param_name]
|
||||
logger.debug(f"Using provided value for {param_name}: {param_value}")
|
||||
elif 'default' in param_spec:
|
||||
param_value = param_spec['default']
|
||||
logger.debug(f"Using default for {param_name}: {param_value}")
|
||||
else:
|
||||
param_value = None
|
||||
logger.debug(f"No value or default for {param_name}, using None")
|
||||
|
||||
workflow_args.append(param_value)
|
||||
else:
|
||||
logger.debug("No 'parameters' section found in workflow metadata")
|
||||
|
||||
# Determine task queue from workflow vertical
|
||||
vertical = workflow_info.metadata.get("vertical", "default")
|
||||
|
||||
@@ -10,5 +10,7 @@
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from .security_analyzer import SecurityAnalyzer
|
||||
from .bandit_analyzer import BanditAnalyzer
|
||||
from .mypy_analyzer import MypyAnalyzer
|
||||
|
||||
__all__ = ["SecurityAnalyzer"]
|
||||
__all__ = ["SecurityAnalyzer", "BanditAnalyzer", "MypyAnalyzer"]
|
||||
@@ -0,0 +1,328 @@
|
||||
"""
|
||||
Bandit Analyzer Module - Analyzes Python code for security issues using Bandit
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class BanditAnalyzer(BaseModule):
|
||||
"""
|
||||
Analyzes Python code for security issues using Bandit.
|
||||
|
||||
This module:
|
||||
- Runs Bandit security linter on Python files
|
||||
- Detects common security issues (SQL injection, hardcoded secrets, etc.)
|
||||
- Reports findings with severity levels
|
||||
"""
|
||||
|
||||
# Severity mapping from Bandit levels to our standard
|
||||
SEVERITY_MAP = {
|
||||
"LOW": "low",
|
||||
"MEDIUM": "medium",
|
||||
"HIGH": "high"
|
||||
}
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
"""Get module metadata"""
|
||||
return ModuleMetadata(
|
||||
name="bandit_analyzer",
|
||||
version="1.0.0",
|
||||
description="Analyzes Python code for security issues using Bandit",
|
||||
author="FuzzForge Team",
|
||||
category="analyzer",
|
||||
tags=["python", "security", "bandit", "sast"],
|
||||
input_schema={
|
||||
"severity_level": {
|
||||
"type": "string",
|
||||
"enum": ["low", "medium", "high"],
|
||||
"description": "Minimum severity level to report",
|
||||
"default": "low"
|
||||
},
|
||||
"confidence_level": {
|
||||
"type": "string",
|
||||
"enum": ["low", "medium", "high"],
|
||||
"description": "Minimum confidence level to report",
|
||||
"default": "medium"
|
||||
},
|
||||
"exclude_tests": {
|
||||
"type": "boolean",
|
||||
"description": "Exclude test files from analysis",
|
||||
"default": True
|
||||
},
|
||||
"skip_ids": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "List of Bandit test IDs to skip",
|
||||
"default": []
|
||||
}
|
||||
},
|
||||
output_schema={
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"description": "List of security issues found by Bandit"
|
||||
}
|
||||
},
|
||||
requires_workspace=True
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate module configuration"""
|
||||
severity = config.get("severity_level", "low")
|
||||
if severity not in ["low", "medium", "high"]:
|
||||
raise ValueError("severity_level must be one of: low, medium, high")
|
||||
|
||||
confidence = config.get("confidence_level", "medium")
|
||||
if confidence not in ["low", "medium", "high"]:
|
||||
raise ValueError("confidence_level must be one of: low, medium, high")
|
||||
|
||||
skip_ids = config.get("skip_ids", [])
|
||||
if not isinstance(skip_ids, list):
|
||||
raise ValueError("skip_ids must be a list")
|
||||
|
||||
return True
|
||||
|
||||
async def _run_bandit(
|
||||
self,
|
||||
workspace: Path,
|
||||
severity_level: str,
|
||||
confidence_level: str,
|
||||
exclude_tests: bool,
|
||||
skip_ids: List[str]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Run Bandit on the workspace.
|
||||
|
||||
Args:
|
||||
workspace: Path to workspace
|
||||
severity_level: Minimum severity to report
|
||||
confidence_level: Minimum confidence to report
|
||||
exclude_tests: Whether to exclude test files
|
||||
skip_ids: List of test IDs to skip
|
||||
|
||||
Returns:
|
||||
Bandit JSON output as dict
|
||||
"""
|
||||
try:
|
||||
# Build bandit command
|
||||
cmd = [
|
||||
"bandit",
|
||||
"-r", str(workspace),
|
||||
"-f", "json",
|
||||
"-ll", # Report all findings (we'll filter later)
|
||||
]
|
||||
|
||||
# Add exclude patterns for test files
|
||||
if exclude_tests:
|
||||
cmd.extend(["-x", "*/test_*.py,*/tests/*,*_test.py"])
|
||||
|
||||
# Add skip IDs if specified
|
||||
if skip_ids:
|
||||
cmd.extend(["-s", ",".join(skip_ids)])
|
||||
|
||||
logger.info(f"Running Bandit on: {workspace}")
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
|
||||
# Bandit returns non-zero if issues found, which is expected
|
||||
if process.returncode not in [0, 1]:
|
||||
logger.error(f"Bandit failed: {stderr.decode()}")
|
||||
return {"results": []}
|
||||
|
||||
# Parse JSON output
|
||||
result = json.loads(stdout.decode())
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error running Bandit: {e}")
|
||||
return {"results": []}
|
||||
|
||||
def _should_include_finding(
|
||||
self,
|
||||
issue: Dict[str, Any],
|
||||
min_severity: str,
|
||||
min_confidence: str
|
||||
) -> bool:
|
||||
"""
|
||||
Determine if a Bandit issue should be included based on severity/confidence.
|
||||
|
||||
Args:
|
||||
issue: Bandit issue dict
|
||||
min_severity: Minimum severity threshold
|
||||
min_confidence: Minimum confidence threshold
|
||||
|
||||
Returns:
|
||||
True if issue should be included
|
||||
"""
|
||||
severity_order = ["low", "medium", "high"]
|
||||
issue_severity = issue.get("issue_severity", "LOW").lower()
|
||||
issue_confidence = issue.get("issue_confidence", "LOW").lower()
|
||||
|
||||
severity_meets_threshold = severity_order.index(issue_severity) >= severity_order.index(min_severity)
|
||||
confidence_meets_threshold = severity_order.index(issue_confidence) >= severity_order.index(min_confidence)
|
||||
|
||||
return severity_meets_threshold and confidence_meets_threshold
|
||||
|
||||
def _convert_to_findings(
|
||||
self,
|
||||
bandit_result: Dict[str, Any],
|
||||
workspace: Path,
|
||||
min_severity: str,
|
||||
min_confidence: str
|
||||
) -> List[ModuleFinding]:
|
||||
"""
|
||||
Convert Bandit results to ModuleFindings.
|
||||
|
||||
Args:
|
||||
bandit_result: Bandit JSON output
|
||||
workspace: Workspace path for relative paths
|
||||
min_severity: Minimum severity to include
|
||||
min_confidence: Minimum confidence to include
|
||||
|
||||
Returns:
|
||||
List of ModuleFindings
|
||||
"""
|
||||
findings = []
|
||||
|
||||
for issue in bandit_result.get("results", []):
|
||||
# Filter by severity and confidence
|
||||
if not self._should_include_finding(issue, min_severity, min_confidence):
|
||||
continue
|
||||
|
||||
# Extract issue details
|
||||
test_id = issue.get("test_id", "B000")
|
||||
test_name = issue.get("test_name", "unknown")
|
||||
issue_text = issue.get("issue_text", "No description")
|
||||
severity = self.SEVERITY_MAP.get(issue.get("issue_severity", "LOW"), "low")
|
||||
|
||||
# File location
|
||||
filename = issue.get("filename", "")
|
||||
line_number = issue.get("line_number", 0)
|
||||
code = issue.get("code", "")
|
||||
|
||||
# Try to get relative path
|
||||
try:
|
||||
file_path = Path(filename)
|
||||
rel_path = file_path.relative_to(workspace)
|
||||
except (ValueError, TypeError):
|
||||
rel_path = Path(filename).name
|
||||
|
||||
# Create finding
|
||||
finding = self.create_finding(
|
||||
title=f"{test_name} ({test_id})",
|
||||
description=issue_text,
|
||||
severity=severity,
|
||||
category="security-issue",
|
||||
file_path=str(rel_path),
|
||||
line_start=line_number,
|
||||
line_end=line_number,
|
||||
code_snippet=code.strip() if code else None,
|
||||
recommendation=f"Review and fix the security issue identified by Bandit test {test_id}",
|
||||
metadata={
|
||||
"test_id": test_id,
|
||||
"test_name": test_name,
|
||||
"confidence": issue.get("issue_confidence", "LOW").lower(),
|
||||
"cwe": issue.get("issue_cwe", {}).get("id") if issue.get("issue_cwe") else None,
|
||||
"more_info": issue.get("more_info", "")
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
return findings
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""
|
||||
Execute the Bandit analyzer module.
|
||||
|
||||
Args:
|
||||
config: Module configuration
|
||||
workspace: Path to workspace
|
||||
|
||||
Returns:
|
||||
ModuleResult with security findings
|
||||
"""
|
||||
start_time = time.time()
|
||||
metadata = self.get_metadata()
|
||||
|
||||
# Validate inputs
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
# Get configuration
|
||||
severity_level = config.get("severity_level", "low")
|
||||
confidence_level = config.get("confidence_level", "medium")
|
||||
exclude_tests = config.get("exclude_tests", True)
|
||||
skip_ids = config.get("skip_ids", [])
|
||||
|
||||
# Run Bandit
|
||||
logger.info("Starting Bandit analysis...")
|
||||
bandit_result = await self._run_bandit(
|
||||
workspace,
|
||||
severity_level,
|
||||
confidence_level,
|
||||
exclude_tests,
|
||||
skip_ids
|
||||
)
|
||||
|
||||
# Convert to findings
|
||||
findings = self._convert_to_findings(
|
||||
bandit_result,
|
||||
workspace,
|
||||
severity_level,
|
||||
confidence_level
|
||||
)
|
||||
|
||||
# Calculate summary
|
||||
severity_counts = {}
|
||||
for finding in findings:
|
||||
sev = finding.severity
|
||||
severity_counts[sev] = severity_counts.get(sev, 0) + 1
|
||||
|
||||
execution_time = time.time() - start_time
|
||||
|
||||
return ModuleResult(
|
||||
module=metadata.name,
|
||||
version=metadata.version,
|
||||
status="success",
|
||||
execution_time=execution_time,
|
||||
findings=findings,
|
||||
summary={
|
||||
"total_issues": len(findings),
|
||||
"by_severity": severity_counts,
|
||||
"files_analyzed": len(set(f.file_path for f in findings if f.file_path))
|
||||
},
|
||||
metadata={
|
||||
"bandit_version": bandit_result.get("generated_at", "unknown"),
|
||||
"metrics": bandit_result.get("metrics", {})
|
||||
}
|
||||
)
|
||||
@@ -0,0 +1,269 @@
|
||||
"""
|
||||
Mypy Analyzer Module - Analyzes Python code for type safety issues using Mypy
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MypyAnalyzer(BaseModule):
|
||||
"""
|
||||
Analyzes Python code for type safety issues using Mypy.
|
||||
|
||||
This module:
|
||||
- Runs Mypy type checker on Python files
|
||||
- Detects type errors and inconsistencies
|
||||
- Reports findings with configurable strictness
|
||||
"""
|
||||
|
||||
# Map Mypy error codes to severity
|
||||
ERROR_SEVERITY_MAP = {
|
||||
"error": "medium",
|
||||
"note": "info"
|
||||
}
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
"""Get module metadata"""
|
||||
return ModuleMetadata(
|
||||
name="mypy_analyzer",
|
||||
version="1.0.0",
|
||||
description="Analyzes Python code for type safety issues using Mypy",
|
||||
author="FuzzForge Team",
|
||||
category="analyzer",
|
||||
tags=["python", "type-checking", "mypy", "sast"],
|
||||
input_schema={
|
||||
"strict_mode": {
|
||||
"type": "boolean",
|
||||
"description": "Enable strict type checking",
|
||||
"default": False
|
||||
},
|
||||
"ignore_missing_imports": {
|
||||
"type": "boolean",
|
||||
"description": "Ignore errors about missing imports",
|
||||
"default": True
|
||||
},
|
||||
"follow_imports": {
|
||||
"type": "string",
|
||||
"enum": ["normal", "silent", "skip", "error"],
|
||||
"description": "How to handle imports",
|
||||
"default": "silent"
|
||||
}
|
||||
},
|
||||
output_schema={
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"description": "List of type errors found by Mypy"
|
||||
}
|
||||
},
|
||||
requires_workspace=True
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate module configuration"""
|
||||
follow_imports = config.get("follow_imports", "silent")
|
||||
if follow_imports not in ["normal", "silent", "skip", "error"]:
|
||||
raise ValueError("follow_imports must be one of: normal, silent, skip, error")
|
||||
|
||||
return True
|
||||
|
||||
async def _run_mypy(
|
||||
self,
|
||||
workspace: Path,
|
||||
strict_mode: bool,
|
||||
ignore_missing_imports: bool,
|
||||
follow_imports: str
|
||||
) -> str:
|
||||
"""
|
||||
Run Mypy on the workspace.
|
||||
|
||||
Args:
|
||||
workspace: Path to workspace
|
||||
strict_mode: Enable strict checking
|
||||
ignore_missing_imports: Ignore missing import errors
|
||||
follow_imports: How to handle imports
|
||||
|
||||
Returns:
|
||||
Mypy output as string
|
||||
"""
|
||||
try:
|
||||
# Build mypy command
|
||||
cmd = [
|
||||
"mypy",
|
||||
str(workspace),
|
||||
"--show-column-numbers",
|
||||
"--no-error-summary",
|
||||
f"--follow-imports={follow_imports}"
|
||||
]
|
||||
|
||||
if strict_mode:
|
||||
cmd.append("--strict")
|
||||
|
||||
if ignore_missing_imports:
|
||||
cmd.append("--ignore-missing-imports")
|
||||
|
||||
logger.info(f"Running Mypy on: {workspace}")
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
|
||||
# Mypy returns non-zero if errors found, which is expected
|
||||
output = stdout.decode()
|
||||
return output
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error running Mypy: {e}")
|
||||
return ""
|
||||
|
||||
def _parse_mypy_output(self, output: str, workspace: Path) -> List[ModuleFinding]:
|
||||
"""
|
||||
Parse Mypy output and convert to findings.
|
||||
|
||||
Mypy output format:
|
||||
file.py:10:5: error: Incompatible return value type [return-value]
|
||||
file.py:15: note: See https://...
|
||||
|
||||
Args:
|
||||
output: Mypy stdout
|
||||
workspace: Workspace path for relative paths
|
||||
|
||||
Returns:
|
||||
List of ModuleFindings
|
||||
"""
|
||||
findings = []
|
||||
|
||||
# Regex to parse mypy output lines
|
||||
# Format: filename:line:column: level: message [error-code]
|
||||
pattern = r'^(.+?):(\d+)(?::(\d+))?: (error|note): (.+?)(?:\s+\[([^\]]+)\])?$'
|
||||
|
||||
for line in output.splitlines():
|
||||
match = re.match(pattern, line.strip())
|
||||
if not match:
|
||||
continue
|
||||
|
||||
filename, line_num, column, level, message, error_code = match.groups()
|
||||
|
||||
# Convert to relative path
|
||||
try:
|
||||
file_path = Path(filename)
|
||||
rel_path = file_path.relative_to(workspace)
|
||||
except (ValueError, TypeError):
|
||||
rel_path = Path(filename).name
|
||||
|
||||
# Skip if it's just a note (unless it's a standalone note)
|
||||
if level == "note" and not error_code:
|
||||
continue
|
||||
|
||||
# Map severity
|
||||
severity = self.ERROR_SEVERITY_MAP.get(level, "medium")
|
||||
|
||||
# Create finding
|
||||
title = f"Type error: {error_code or 'type-issue'}"
|
||||
description = message
|
||||
|
||||
finding = self.create_finding(
|
||||
title=title,
|
||||
description=description,
|
||||
severity=severity,
|
||||
category="type-error",
|
||||
file_path=str(rel_path),
|
||||
line_start=int(line_num),
|
||||
line_end=int(line_num),
|
||||
recommendation="Review and fix the type inconsistency or add appropriate type annotations",
|
||||
metadata={
|
||||
"error_code": error_code or "unknown",
|
||||
"column": int(column) if column else None,
|
||||
"level": level
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
return findings
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""
|
||||
Execute the Mypy analyzer module.
|
||||
|
||||
Args:
|
||||
config: Module configuration
|
||||
workspace: Path to workspace
|
||||
|
||||
Returns:
|
||||
ModuleResult with type checking findings
|
||||
"""
|
||||
start_time = time.time()
|
||||
metadata = self.get_metadata()
|
||||
|
||||
# Validate inputs
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
# Get configuration
|
||||
strict_mode = config.get("strict_mode", False)
|
||||
ignore_missing_imports = config.get("ignore_missing_imports", True)
|
||||
follow_imports = config.get("follow_imports", "silent")
|
||||
|
||||
# Run Mypy
|
||||
logger.info("Starting Mypy analysis...")
|
||||
mypy_output = await self._run_mypy(
|
||||
workspace,
|
||||
strict_mode,
|
||||
ignore_missing_imports,
|
||||
follow_imports
|
||||
)
|
||||
|
||||
# Parse output to findings
|
||||
findings = self._parse_mypy_output(mypy_output, workspace)
|
||||
|
||||
# Calculate summary
|
||||
error_code_counts = {}
|
||||
for finding in findings:
|
||||
code = finding.metadata.get("error_code", "unknown")
|
||||
error_code_counts[code] = error_code_counts.get(code, 0) + 1
|
||||
|
||||
execution_time = time.time() - start_time
|
||||
|
||||
return ModuleResult(
|
||||
module=metadata.name,
|
||||
version=metadata.version,
|
||||
status="success",
|
||||
execution_time=execution_time,
|
||||
findings=findings,
|
||||
summary={
|
||||
"total_errors": len(findings),
|
||||
"by_error_code": error_code_counts,
|
||||
"files_with_errors": len(set(f.file_path for f in findings if f.file_path))
|
||||
},
|
||||
metadata={
|
||||
"strict_mode": strict_mode,
|
||||
"ignore_missing_imports": ignore_missing_imports
|
||||
}
|
||||
)
|
||||
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Android Security Analysis Modules
|
||||
|
||||
Modules for Android application security testing:
|
||||
- JadxDecompiler: APK decompilation using Jadx
|
||||
- MobSFScanner: Mobile security analysis using MobSF
|
||||
- OpenGrepAndroid: Static analysis using OpenGrep/Semgrep with Android-specific rules
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from .jadx_decompiler import JadxDecompiler
|
||||
from .opengrep_android import OpenGrepAndroid
|
||||
|
||||
# MobSF is optional (not available on ARM64 platform)
|
||||
try:
|
||||
from .mobsf_scanner import MobSFScanner
|
||||
__all__ = ["JadxDecompiler", "MobSFScanner", "OpenGrepAndroid"]
|
||||
except ImportError:
|
||||
# MobSF dependencies not available (e.g., ARM64 platform)
|
||||
MobSFScanner = None
|
||||
__all__ = ["JadxDecompiler", "OpenGrepAndroid"]
|
||||
@@ -0,0 +1,15 @@
|
||||
rules:
|
||||
- id: clipboard-sensitive-data
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Sensitive data may be copied to the clipboard."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: security
|
||||
area: clipboard
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
pattern: "$CLIPBOARD.setPrimaryClip($CLIP)"
|
||||
@@ -0,0 +1,23 @@
|
||||
rules:
|
||||
- id: hardcoded-secrets
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Possible hardcoded secret found in variable '$NAME'."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M2
|
||||
category: secrets
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
patterns:
|
||||
- pattern-either:
|
||||
- pattern: 'String $NAME = "$VAL";'
|
||||
- pattern: 'final String $NAME = "$VAL";'
|
||||
- pattern: 'private String $NAME = "$VAL";'
|
||||
- pattern: 'public static String $NAME = "$VAL";'
|
||||
- pattern: 'static final String $NAME = "$VAL";'
|
||||
- pattern-regex: "$NAME =~ /(?i).*(api|key|token|secret|pass|auth|session|bearer|access|private).*/"
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
rules:
|
||||
- id: insecure-data-storage
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Potential insecure data storage (external storage)."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M2
|
||||
category: security
|
||||
area: storage
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
pattern-either:
|
||||
- pattern: "$CTX.openFileOutput($NAME, $MODE)"
|
||||
- pattern: "Environment.getExternalStorageDirectory()"
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: insecure-deeplink
|
||||
severity: WARNING
|
||||
languages: [xml]
|
||||
message: "Potential insecure deeplink found in intent-filter."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: component
|
||||
area: manifest
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/AndroidManifest.xml"
|
||||
pattern: |
|
||||
<intent-filter>
|
||||
@@ -0,0 +1,21 @@
|
||||
rules:
|
||||
- id: insecure-logging
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Sensitive data logged via Android Log API."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M2
|
||||
category: logging
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
patterns:
|
||||
- pattern-either:
|
||||
- pattern: "Log.d($TAG, $MSG)"
|
||||
- pattern: "Log.e($TAG, $MSG)"
|
||||
- pattern: "System.out.println($MSG)"
|
||||
- pattern-regex: "$MSG =~ /(?i).*(password|token|secret|api|auth|session).*/"
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
rules:
|
||||
- id: intent-redirection
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Potential intent redirection: using getIntent().getExtras() without validation."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: intent
|
||||
area: intercomponent
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
pattern: "$ACT.getIntent().getExtras()"
|
||||
@@ -0,0 +1,18 @@
|
||||
rules:
|
||||
- id: sensitive-data-in-shared-preferences
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Sensitive data may be stored in SharedPreferences. Please review the key '$KEY'."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M2
|
||||
category: security
|
||||
area: storage
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
patterns:
|
||||
- pattern: "$EDITOR.putString($KEY, $VAL);"
|
||||
- pattern-regex: "$KEY =~ /(?i).*(username|password|pass|token|auth_token|api_key|secret|sessionid|email).*/"
|
||||
@@ -0,0 +1,21 @@
|
||||
rules:
|
||||
- id: sqlite-injection
|
||||
severity: ERROR
|
||||
languages: [java]
|
||||
message: "Possible SQL injection: concatenated input in rawQuery or execSQL."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M7
|
||||
category: injection
|
||||
area: database
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
patterns:
|
||||
- pattern-either:
|
||||
- pattern: "$DB.rawQuery($QUERY, ...)"
|
||||
- pattern: "$DB.execSQL($QUERY)"
|
||||
- pattern-regex: "$QUERY =~ /.*\".*\".*\\+.*/"
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: vulnerable-activity
|
||||
severity: WARNING
|
||||
languages: [xml]
|
||||
message: "Activity exported without permission."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: component
|
||||
area: manifest
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/AndroidManifest.xml"
|
||||
pattern: |
|
||||
<activity android:exported="true"
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: vulnerable-content-provider
|
||||
severity: WARNING
|
||||
languages: [xml]
|
||||
message: "ContentProvider exported without permission."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: component
|
||||
area: manifest
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/AndroidManifest.xml"
|
||||
pattern: |
|
||||
<provider android:exported="true"
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: vulnerable-service
|
||||
severity: WARNING
|
||||
languages: [xml]
|
||||
message: "Service exported without permission."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
category: component
|
||||
area: manifest
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/AndroidManifest.xml"
|
||||
pattern: |
|
||||
<service android:exported="true"
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: webview-javascript-enabled
|
||||
severity: ERROR
|
||||
languages: [java]
|
||||
message: "WebView with JavaScript enabled can be dangerous if loading untrusted content."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M7
|
||||
category: webview
|
||||
area: ui
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
pattern: "$W.getSettings().setJavaScriptEnabled(true)"
|
||||
@@ -0,0 +1,16 @@
|
||||
rules:
|
||||
- id: webview-load-arbitrary-url
|
||||
severity: WARNING
|
||||
languages: [java]
|
||||
message: "Loading unvalidated URL in WebView may cause open redirect or XSS."
|
||||
metadata:
|
||||
authors:
|
||||
- Guerric ELOI (FuzzingLabs)
|
||||
owasp-mobile: M7
|
||||
category: webview
|
||||
area: ui
|
||||
verification-level: [L1]
|
||||
paths:
|
||||
include:
|
||||
- "**/*.java"
|
||||
pattern: "$W.loadUrl($URL)"
|
||||
@@ -0,0 +1,270 @@
|
||||
"""
|
||||
Jadx APK Decompilation Module
|
||||
|
||||
Decompiles Android APK files to Java source code using Jadx.
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import asyncio
|
||||
import shutil
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleResult
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class JadxDecompiler(BaseModule):
|
||||
"""Module for decompiling APK files to Java source code using Jadx"""
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
return ModuleMetadata(
|
||||
name="jadx_decompiler",
|
||||
version="1.5.0",
|
||||
description="Android APK decompilation using Jadx - converts DEX bytecode to Java source",
|
||||
author="FuzzForge Team",
|
||||
category="android",
|
||||
tags=["android", "jadx", "decompilation", "reverse", "apk"],
|
||||
input_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"apk_path": {
|
||||
"type": "string",
|
||||
"description": "Path to the APK to decompile (absolute or relative to workspace)",
|
||||
},
|
||||
"output_dir": {
|
||||
"type": "string",
|
||||
"description": "Directory (relative to workspace) where Jadx output should be written",
|
||||
"default": "jadx_output",
|
||||
},
|
||||
"overwrite": {
|
||||
"type": "boolean",
|
||||
"description": "Overwrite existing output directory if present",
|
||||
"default": True,
|
||||
},
|
||||
"threads": {
|
||||
"type": "integer",
|
||||
"description": "Number of Jadx decompilation threads",
|
||||
"default": 4,
|
||||
"minimum": 1,
|
||||
"maximum": 32,
|
||||
},
|
||||
"decompiler_args": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Additional arguments passed directly to Jadx",
|
||||
"default": [],
|
||||
},
|
||||
},
|
||||
"required": ["apk_path"],
|
||||
},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"output_dir": {
|
||||
"type": "string",
|
||||
"description": "Path to decompiled output directory",
|
||||
},
|
||||
"source_dir": {
|
||||
"type": "string",
|
||||
"description": "Path to decompiled Java sources",
|
||||
},
|
||||
"resource_dir": {
|
||||
"type": "string",
|
||||
"description": "Path to extracted resources",
|
||||
},
|
||||
"java_files": {
|
||||
"type": "integer",
|
||||
"description": "Number of Java files decompiled",
|
||||
},
|
||||
},
|
||||
},
|
||||
requires_workspace=True,
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate module configuration"""
|
||||
apk_path = config.get("apk_path")
|
||||
if not apk_path:
|
||||
raise ValueError("'apk_path' must be provided for Jadx decompilation")
|
||||
|
||||
threads = config.get("threads", 4)
|
||||
if not isinstance(threads, int) or threads < 1 or threads > 32:
|
||||
raise ValueError("threads must be between 1 and 32")
|
||||
|
||||
return True
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""
|
||||
Execute Jadx decompilation on an APK file.
|
||||
|
||||
Args:
|
||||
config: Configuration dict with apk_path, output_dir, etc.
|
||||
workspace: Workspace directory path
|
||||
|
||||
Returns:
|
||||
ModuleResult with decompilation summary and metadata
|
||||
"""
|
||||
self.start_timer()
|
||||
|
||||
try:
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
workspace = workspace.resolve()
|
||||
|
||||
# Resolve APK path
|
||||
apk_path = Path(config["apk_path"])
|
||||
if not apk_path.is_absolute():
|
||||
apk_path = (workspace / apk_path).resolve()
|
||||
|
||||
if not apk_path.exists():
|
||||
raise ValueError(f"APK not found: {apk_path}")
|
||||
|
||||
if apk_path.is_dir():
|
||||
raise ValueError(f"APK path must be a file, not a directory: {apk_path}")
|
||||
|
||||
logger.info(f"Decompiling APK: {apk_path}")
|
||||
|
||||
# Resolve output directory
|
||||
output_dir = Path(config.get("output_dir", "jadx_output"))
|
||||
if not output_dir.is_absolute():
|
||||
output_dir = (workspace / output_dir).resolve()
|
||||
|
||||
# Handle existing output directory
|
||||
if output_dir.exists():
|
||||
if config.get("overwrite", True):
|
||||
logger.info(f"Removing existing output directory: {output_dir}")
|
||||
shutil.rmtree(output_dir)
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Output directory already exists: {output_dir}. Set overwrite=true to replace it."
|
||||
)
|
||||
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Build Jadx command
|
||||
threads = str(config.get("threads", 4))
|
||||
extra_args = config.get("decompiler_args", []) or []
|
||||
|
||||
cmd = [
|
||||
"jadx",
|
||||
"--threads-count",
|
||||
threads,
|
||||
"--deobf", # Deobfuscate code
|
||||
"--output-dir",
|
||||
str(output_dir),
|
||||
]
|
||||
cmd.extend(extra_args)
|
||||
cmd.append(str(apk_path))
|
||||
|
||||
logger.info(f"Running Jadx: {' '.join(cmd)}")
|
||||
|
||||
# Execute Jadx
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
cwd=str(workspace),
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
stdout_str = stdout.decode(errors="ignore") if stdout else ""
|
||||
stderr_str = stderr.decode(errors="ignore") if stderr else ""
|
||||
|
||||
if stdout_str:
|
||||
logger.debug(f"Jadx stdout: {stdout_str[:200]}...")
|
||||
if stderr_str:
|
||||
logger.debug(f"Jadx stderr: {stderr_str[:200]}...")
|
||||
|
||||
if process.returncode != 0:
|
||||
error_output = stderr_str or stdout_str or "No error output"
|
||||
raise RuntimeError(
|
||||
f"Jadx failed with exit code {process.returncode}: {error_output[:500]}"
|
||||
)
|
||||
|
||||
# Verify output structure
|
||||
source_dir = output_dir / "sources"
|
||||
resource_dir = output_dir / "resources"
|
||||
|
||||
if not source_dir.exists():
|
||||
logger.warning(
|
||||
f"Jadx sources directory not found at expected path: {source_dir}"
|
||||
)
|
||||
# Use output_dir as fallback
|
||||
source_dir = output_dir
|
||||
|
||||
# Count decompiled Java files
|
||||
java_files = 0
|
||||
if source_dir.exists():
|
||||
java_files = sum(1 for _ in source_dir.rglob("*.java"))
|
||||
logger.info(f"Decompiled {java_files} Java files")
|
||||
|
||||
# Log sample files for debugging
|
||||
sample_files = []
|
||||
for idx, file_path in enumerate(source_dir.rglob("*.java")):
|
||||
sample_files.append(str(file_path.relative_to(workspace)))
|
||||
if idx >= 4:
|
||||
break
|
||||
if sample_files:
|
||||
logger.debug(f"Sample Java files: {sample_files}")
|
||||
|
||||
# Create summary
|
||||
summary = {
|
||||
"output_dir": str(output_dir),
|
||||
"source_dir": str(source_dir if source_dir.exists() else output_dir),
|
||||
"resource_dir": str(
|
||||
resource_dir if resource_dir.exists() else output_dir
|
||||
),
|
||||
"java_files": java_files,
|
||||
"apk_name": apk_path.name,
|
||||
"apk_size_bytes": apk_path.stat().st_size,
|
||||
}
|
||||
|
||||
metadata = {
|
||||
"apk_path": str(apk_path),
|
||||
"output_dir": str(output_dir),
|
||||
"source_dir": summary["source_dir"],
|
||||
"resource_dir": summary["resource_dir"],
|
||||
"threads": threads,
|
||||
"decompiler": "jadx",
|
||||
"decompiler_version": "1.5.0",
|
||||
}
|
||||
|
||||
logger.info(
|
||||
f"✓ Jadx decompilation completed: {java_files} Java files generated"
|
||||
)
|
||||
|
||||
return self.create_result(
|
||||
findings=[], # Jadx doesn't generate findings, only decompiles
|
||||
status="success",
|
||||
summary=summary,
|
||||
metadata=metadata,
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"Jadx decompilation failed: {exc}", exc_info=True)
|
||||
return self.create_result(
|
||||
findings=[],
|
||||
status="failed",
|
||||
error=str(exc),
|
||||
metadata={"decompiler": "jadx", "apk_path": config.get("apk_path")},
|
||||
)
|
||||
@@ -0,0 +1,437 @@
|
||||
"""
|
||||
MobSF Scanner Module
|
||||
|
||||
Mobile Security Framework (MobSF) integration for comprehensive Android app security analysis.
|
||||
Performs static analysis on APK files including permissions, manifest analysis, code analysis, and behavior checks.
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import logging
|
||||
import os
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List
|
||||
import aiohttp
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MobSFScanner(BaseModule):
|
||||
"""Mobile Security Framework (MobSF) scanner module for Android applications"""
|
||||
|
||||
SEVERITY_MAP = {
|
||||
"dangerous": "critical",
|
||||
"high": "high",
|
||||
"warning": "medium",
|
||||
"medium": "medium",
|
||||
"low": "low",
|
||||
"info": "low",
|
||||
"secure": "low",
|
||||
}
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
return ModuleMetadata(
|
||||
name="mobsf_scanner",
|
||||
version="3.9.7",
|
||||
description="Comprehensive Android security analysis using Mobile Security Framework (MobSF)",
|
||||
author="FuzzForge Team",
|
||||
category="android",
|
||||
tags=["mobile", "android", "mobsf", "sast", "scanner", "security"],
|
||||
input_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"mobsf_url": {
|
||||
"type": "string",
|
||||
"description": "MobSF server URL",
|
||||
"default": "http://localhost:8877",
|
||||
},
|
||||
"file_path": {
|
||||
"type": "string",
|
||||
"description": "Path to the APK file to scan (absolute or relative to workspace)",
|
||||
},
|
||||
"api_key": {
|
||||
"type": "string",
|
||||
"description": "MobSF API key (if not provided, will try MOBSF_API_KEY env var)",
|
||||
"default": None,
|
||||
},
|
||||
"rescan": {
|
||||
"type": "boolean",
|
||||
"description": "Force rescan even if file was previously analyzed",
|
||||
"default": False,
|
||||
},
|
||||
},
|
||||
"required": ["file_path"],
|
||||
},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"description": "Security findings from MobSF analysis"
|
||||
},
|
||||
"scan_hash": {"type": "string"},
|
||||
"total_findings": {"type": "integer"},
|
||||
"severity_counts": {"type": "object"},
|
||||
}
|
||||
},
|
||||
requires_workspace=True,
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate module configuration"""
|
||||
if "mobsf_url" in config and not isinstance(config["mobsf_url"], str):
|
||||
raise ValueError("mobsf_url must be a string")
|
||||
|
||||
file_path = config.get("file_path")
|
||||
if not file_path:
|
||||
raise ValueError("file_path is required for MobSF scanning")
|
||||
|
||||
return True
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""
|
||||
Execute MobSF security analysis on an APK file.
|
||||
|
||||
Args:
|
||||
config: Configuration dict with file_path, mobsf_url, api_key
|
||||
workspace: Workspace directory path
|
||||
|
||||
Returns:
|
||||
ModuleResult with security findings from MobSF
|
||||
"""
|
||||
self.start_timer()
|
||||
|
||||
try:
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
# Get configuration
|
||||
mobsf_url = config.get("mobsf_url", "http://localhost:8877")
|
||||
file_path_str = config["file_path"]
|
||||
rescan = config.get("rescan", False)
|
||||
|
||||
# Get API key from config or environment
|
||||
api_key = config.get("api_key") or os.environ.get("MOBSF_API_KEY", "")
|
||||
if not api_key:
|
||||
logger.warning("No MobSF API key provided. Some functionality may be limited.")
|
||||
|
||||
# Resolve APK file path
|
||||
file_path = Path(file_path_str)
|
||||
if not file_path.is_absolute():
|
||||
file_path = (workspace / file_path).resolve()
|
||||
|
||||
if not file_path.exists():
|
||||
raise FileNotFoundError(f"APK file not found: {file_path}")
|
||||
|
||||
if not file_path.is_file():
|
||||
raise ValueError(f"APK path must be a file: {file_path}")
|
||||
|
||||
logger.info(f"Starting MobSF scan of APK: {file_path}")
|
||||
|
||||
# Upload and scan APK
|
||||
scan_hash = await self._upload_file(mobsf_url, file_path, api_key)
|
||||
logger.info(f"APK uploaded to MobSF with hash: {scan_hash}")
|
||||
|
||||
# Start scan
|
||||
await self._start_scan(mobsf_url, scan_hash, api_key, rescan=rescan)
|
||||
logger.info(f"MobSF scan completed for hash: {scan_hash}")
|
||||
|
||||
# Get JSON results
|
||||
scan_results = await self._get_json_results(mobsf_url, scan_hash, api_key)
|
||||
|
||||
# Parse results into findings
|
||||
findings = self._parse_scan_results(scan_results, file_path)
|
||||
|
||||
# Create summary
|
||||
summary = self._create_summary(findings, scan_hash)
|
||||
|
||||
logger.info(f"✓ MobSF scan completed: {len(findings)} findings")
|
||||
|
||||
return self.create_result(
|
||||
findings=findings,
|
||||
status="success",
|
||||
summary=summary,
|
||||
metadata={
|
||||
"tool": "mobsf",
|
||||
"tool_version": "3.9.7",
|
||||
"scan_hash": scan_hash,
|
||||
"apk_file": str(file_path),
|
||||
"mobsf_url": mobsf_url,
|
||||
}
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
logger.error(f"MobSF scanner failed: {exc}", exc_info=True)
|
||||
return self.create_result(
|
||||
findings=[],
|
||||
status="failed",
|
||||
error=str(exc),
|
||||
metadata={"tool": "mobsf", "file_path": config.get("file_path")}
|
||||
)
|
||||
|
||||
async def _upload_file(self, mobsf_url: str, file_path: Path, api_key: str) -> str:
|
||||
"""
|
||||
Upload APK file to MobSF server.
|
||||
|
||||
Returns:
|
||||
Scan hash for the uploaded file
|
||||
"""
|
||||
headers = {'X-Mobsf-Api-Key': api_key} if api_key else {}
|
||||
|
||||
# Create multipart form data
|
||||
filename = file_path.name
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
with open(file_path, 'rb') as f:
|
||||
data = aiohttp.FormData()
|
||||
data.add_field('file',
|
||||
f,
|
||||
filename=filename,
|
||||
content_type='application/vnd.android.package-archive')
|
||||
|
||||
async with session.post(
|
||||
f"{mobsf_url}/api/v1/upload",
|
||||
headers=headers,
|
||||
data=data,
|
||||
timeout=aiohttp.ClientTimeout(total=300)
|
||||
) as response:
|
||||
if response.status != 200:
|
||||
error_text = await response.text()
|
||||
raise Exception(f"Failed to upload file to MobSF: {error_text}")
|
||||
|
||||
result = await response.json()
|
||||
scan_hash = result.get('hash')
|
||||
if not scan_hash:
|
||||
raise Exception(f"MobSF upload failed: {result}")
|
||||
|
||||
return scan_hash
|
||||
|
||||
async def _start_scan(self, mobsf_url: str, scan_hash: str, api_key: str, rescan: bool = False) -> Dict[str, Any]:
|
||||
"""
|
||||
Start MobSF scan for uploaded file.
|
||||
|
||||
Returns:
|
||||
Scan result dictionary
|
||||
"""
|
||||
headers = {'X-Mobsf-Api-Key': api_key} if api_key else {}
|
||||
data = {
|
||||
'hash': scan_hash,
|
||||
're_scan': '1' if rescan else '0'
|
||||
}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(
|
||||
f"{mobsf_url}/api/v1/scan",
|
||||
headers=headers,
|
||||
data=data,
|
||||
timeout=aiohttp.ClientTimeout(total=600) # 10 minutes for scan
|
||||
) as response:
|
||||
if response.status != 200:
|
||||
error_text = await response.text()
|
||||
raise Exception(f"MobSF scan failed: {error_text}")
|
||||
|
||||
result = await response.json()
|
||||
return result
|
||||
|
||||
async def _get_json_results(self, mobsf_url: str, scan_hash: str, api_key: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Retrieve JSON scan results from MobSF.
|
||||
|
||||
Returns:
|
||||
Scan results dictionary
|
||||
"""
|
||||
headers = {'X-Mobsf-Api-Key': api_key} if api_key else {}
|
||||
data = {'hash': scan_hash}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(
|
||||
f"{mobsf_url}/api/v1/report_json",
|
||||
headers=headers,
|
||||
data=data,
|
||||
timeout=aiohttp.ClientTimeout(total=60)
|
||||
) as response:
|
||||
if response.status != 200:
|
||||
error_text = await response.text()
|
||||
raise Exception(f"Failed to retrieve MobSF results: {error_text}")
|
||||
|
||||
return await response.json()
|
||||
|
||||
def _parse_scan_results(self, scan_data: Dict[str, Any], apk_path: Path) -> List[ModuleFinding]:
|
||||
"""Parse MobSF JSON results into standardized findings"""
|
||||
findings = []
|
||||
|
||||
# Parse permissions
|
||||
if 'permissions' in scan_data:
|
||||
for perm_name, perm_attrs in scan_data['permissions'].items():
|
||||
if isinstance(perm_attrs, dict):
|
||||
severity = self.SEVERITY_MAP.get(
|
||||
perm_attrs.get('status', '').lower(), 'low'
|
||||
)
|
||||
|
||||
finding = self.create_finding(
|
||||
title=f"Android Permission: {perm_name}",
|
||||
description=perm_attrs.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-permission",
|
||||
metadata={
|
||||
'permission': perm_name,
|
||||
'status': perm_attrs.get('status'),
|
||||
'info': perm_attrs.get('info'),
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
# Parse manifest analysis
|
||||
if 'manifest_analysis' in scan_data:
|
||||
manifest_findings = scan_data['manifest_analysis'].get('manifest_findings', [])
|
||||
for item in manifest_findings:
|
||||
if isinstance(item, dict):
|
||||
severity = self.SEVERITY_MAP.get(item.get('severity', '').lower(), 'medium')
|
||||
|
||||
finding = self.create_finding(
|
||||
title=item.get('title') or item.get('name') or "Manifest Issue",
|
||||
description=item.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-manifest",
|
||||
metadata={
|
||||
'rule': item.get('rule'),
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
# Parse code analysis
|
||||
if 'code_analysis' in scan_data:
|
||||
code_findings = scan_data['code_analysis'].get('findings', {})
|
||||
for finding_name, finding_data in code_findings.items():
|
||||
if isinstance(finding_data, dict):
|
||||
metadata_dict = finding_data.get('metadata', {})
|
||||
severity = self.SEVERITY_MAP.get(
|
||||
metadata_dict.get('severity', '').lower(), 'medium'
|
||||
)
|
||||
|
||||
# MobSF returns 'files' as a dict: {filename: line_numbers}
|
||||
files_dict = finding_data.get('files', {})
|
||||
|
||||
# Create a finding for each affected file
|
||||
if isinstance(files_dict, dict) and files_dict:
|
||||
for file_path, line_numbers in files_dict.items():
|
||||
finding = self.create_finding(
|
||||
title=finding_name,
|
||||
description=metadata_dict.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-code-analysis",
|
||||
file_path=file_path,
|
||||
line_number=line_numbers, # Can be string like "28" or "65,81"
|
||||
metadata={
|
||||
'cwe': metadata_dict.get('cwe'),
|
||||
'owasp': metadata_dict.get('owasp'),
|
||||
'masvs': metadata_dict.get('masvs'),
|
||||
'cvss': metadata_dict.get('cvss'),
|
||||
'ref': metadata_dict.get('ref'),
|
||||
'line_numbers': line_numbers,
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
else:
|
||||
# Fallback: create one finding without file info
|
||||
finding = self.create_finding(
|
||||
title=finding_name,
|
||||
description=metadata_dict.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-code-analysis",
|
||||
metadata={
|
||||
'cwe': metadata_dict.get('cwe'),
|
||||
'owasp': metadata_dict.get('owasp'),
|
||||
'masvs': metadata_dict.get('masvs'),
|
||||
'cvss': metadata_dict.get('cvss'),
|
||||
'ref': metadata_dict.get('ref'),
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
# Parse behavior analysis
|
||||
if 'behaviour' in scan_data:
|
||||
for key, value in scan_data['behaviour'].items():
|
||||
if isinstance(value, dict):
|
||||
metadata_dict = value.get('metadata', {})
|
||||
labels = metadata_dict.get('label', [])
|
||||
label = labels[0] if labels else 'Unknown Behavior'
|
||||
|
||||
severity = self.SEVERITY_MAP.get(
|
||||
metadata_dict.get('severity', '').lower(), 'medium'
|
||||
)
|
||||
|
||||
# MobSF returns 'files' as a dict: {filename: line_numbers}
|
||||
files_dict = value.get('files', {})
|
||||
|
||||
# Create a finding for each affected file
|
||||
if isinstance(files_dict, dict) and files_dict:
|
||||
for file_path, line_numbers in files_dict.items():
|
||||
finding = self.create_finding(
|
||||
title=f"Behavior: {label}",
|
||||
description=metadata_dict.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-behavior",
|
||||
file_path=file_path,
|
||||
line_number=line_numbers,
|
||||
metadata={
|
||||
'line_numbers': line_numbers,
|
||||
'behavior_key': key,
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
else:
|
||||
# Fallback: create one finding without file info
|
||||
finding = self.create_finding(
|
||||
title=f"Behavior: {label}",
|
||||
description=metadata_dict.get('description', 'No description'),
|
||||
severity=severity,
|
||||
category="android-behavior",
|
||||
metadata={
|
||||
'behavior_key': key,
|
||||
'tool': 'mobsf',
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
logger.debug(f"Parsed {len(findings)} findings from MobSF results")
|
||||
return findings
|
||||
|
||||
def _create_summary(self, findings: List[ModuleFinding], scan_hash: str) -> Dict[str, Any]:
|
||||
"""Create analysis summary"""
|
||||
severity_counter = Counter()
|
||||
category_counter = Counter()
|
||||
|
||||
for finding in findings:
|
||||
severity_counter[finding.severity] += 1
|
||||
category_counter[finding.category] += 1
|
||||
|
||||
return {
|
||||
"scan_hash": scan_hash,
|
||||
"total_findings": len(findings),
|
||||
"severity_counts": dict(severity_counter),
|
||||
"category_counts": dict(category_counter),
|
||||
}
|
||||
@@ -0,0 +1,440 @@
|
||||
"""
|
||||
OpenGrep Android Static Analysis Module
|
||||
|
||||
Pattern-based static analysis for Android applications using OpenGrep/Semgrep
|
||||
with Android-specific security rules.
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleFinding, ModuleResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OpenGrepAndroid(BaseModule):
|
||||
"""OpenGrep static analysis module specialized for Android security"""
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
"""Get module metadata"""
|
||||
return ModuleMetadata(
|
||||
name="opengrep_android",
|
||||
version="1.45.0",
|
||||
description="Android-focused static analysis using OpenGrep/Semgrep with custom security rules for Java/Kotlin",
|
||||
author="FuzzForge Team",
|
||||
category="android",
|
||||
tags=["sast", "android", "opengrep", "semgrep", "java", "kotlin", "security"],
|
||||
input_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"config": {
|
||||
"type": "string",
|
||||
"enum": ["auto", "p/security-audit", "p/owasp-top-ten", "p/cwe-top-25"],
|
||||
"default": "auto",
|
||||
"description": "Rule configuration to use"
|
||||
},
|
||||
"custom_rules_path": {
|
||||
"type": "string",
|
||||
"description": "Path to a directory containing custom OpenGrep rules (Android-specific rules recommended)",
|
||||
"default": None,
|
||||
},
|
||||
"languages": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Specific languages to analyze (defaults to java, kotlin for Android)",
|
||||
"default": ["java", "kotlin"],
|
||||
},
|
||||
"include_patterns": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "File patterns to include",
|
||||
"default": [],
|
||||
},
|
||||
"exclude_patterns": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "File patterns to exclude",
|
||||
"default": [],
|
||||
},
|
||||
"max_target_bytes": {
|
||||
"type": "integer",
|
||||
"default": 1000000,
|
||||
"description": "Maximum file size to analyze (bytes)"
|
||||
},
|
||||
"timeout": {
|
||||
"type": "integer",
|
||||
"default": 300,
|
||||
"description": "Analysis timeout in seconds"
|
||||
},
|
||||
"severity": {
|
||||
"type": "array",
|
||||
"items": {"type": "string", "enum": ["ERROR", "WARNING", "INFO"]},
|
||||
"default": ["ERROR", "WARNING", "INFO"],
|
||||
"description": "Minimum severity levels to report"
|
||||
},
|
||||
"confidence": {
|
||||
"type": "array",
|
||||
"items": {"type": "string", "enum": ["HIGH", "MEDIUM", "LOW"]},
|
||||
"default": ["HIGH", "MEDIUM", "LOW"],
|
||||
"description": "Minimum confidence levels to report"
|
||||
}
|
||||
}
|
||||
},
|
||||
output_schema={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"description": "Security findings from OpenGrep analysis"
|
||||
},
|
||||
"total_findings": {"type": "integer"},
|
||||
"severity_counts": {"type": "object"},
|
||||
"files_analyzed": {"type": "integer"},
|
||||
}
|
||||
},
|
||||
requires_workspace=True,
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate configuration"""
|
||||
timeout = config.get("timeout", 300)
|
||||
if not isinstance(timeout, int) or timeout < 30 or timeout > 3600:
|
||||
raise ValueError("Timeout must be between 30 and 3600 seconds")
|
||||
|
||||
max_bytes = config.get("max_target_bytes", 1000000)
|
||||
if not isinstance(max_bytes, int) or max_bytes < 1000 or max_bytes > 10000000:
|
||||
raise ValueError("max_target_bytes must be between 1000 and 10000000")
|
||||
|
||||
custom_rules_path = config.get("custom_rules_path")
|
||||
if custom_rules_path:
|
||||
rules_path = Path(custom_rules_path)
|
||||
if not rules_path.exists():
|
||||
logger.warning(f"Custom rules path does not exist: {custom_rules_path}")
|
||||
|
||||
return True
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""Execute OpenGrep static analysis on Android code"""
|
||||
self.start_timer()
|
||||
|
||||
try:
|
||||
# Validate inputs
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
logger.info(f"Running OpenGrep Android analysis on {workspace}")
|
||||
|
||||
# Build opengrep command
|
||||
cmd = ["opengrep", "scan", "--json"]
|
||||
|
||||
# Add configuration
|
||||
custom_rules_path = config.get("custom_rules_path")
|
||||
use_custom_rules = False
|
||||
if custom_rules_path and Path(custom_rules_path).exists():
|
||||
cmd.extend(["--config", custom_rules_path])
|
||||
use_custom_rules = True
|
||||
logger.info(f"Using custom Android rules from: {custom_rules_path}")
|
||||
else:
|
||||
config_type = config.get("config", "auto")
|
||||
if config_type == "auto":
|
||||
cmd.extend(["--config", "auto"])
|
||||
else:
|
||||
cmd.extend(["--config", config_type])
|
||||
|
||||
# Add timeout
|
||||
cmd.extend(["--timeout", str(config.get("timeout", 300))])
|
||||
|
||||
# Add max target bytes
|
||||
cmd.extend(["--max-target-bytes", str(config.get("max_target_bytes", 1000000))])
|
||||
|
||||
# Add languages if specified (but NOT when using custom rules)
|
||||
languages = config.get("languages", ["java", "kotlin"])
|
||||
if languages and not use_custom_rules:
|
||||
langs = ",".join(languages)
|
||||
cmd.extend(["--lang", langs])
|
||||
logger.debug(f"Analyzing languages: {langs}")
|
||||
|
||||
# Add include patterns
|
||||
include_patterns = config.get("include_patterns", [])
|
||||
for pattern in include_patterns:
|
||||
cmd.extend(["--include", pattern])
|
||||
|
||||
# Add exclude patterns
|
||||
exclude_patterns = config.get("exclude_patterns", [])
|
||||
for pattern in exclude_patterns:
|
||||
cmd.extend(["--exclude", pattern])
|
||||
|
||||
# Add severity filter if single level requested
|
||||
severity_levels = config.get("severity", ["ERROR", "WARNING", "INFO"])
|
||||
if severity_levels and len(severity_levels) == 1:
|
||||
cmd.extend(["--severity", severity_levels[0]])
|
||||
|
||||
# Disable metrics collection
|
||||
cmd.append("--disable-version-check")
|
||||
cmd.append("--no-git-ignore")
|
||||
|
||||
# Add target directory
|
||||
cmd.append(str(workspace))
|
||||
|
||||
logger.debug(f"Running command: {' '.join(cmd)}")
|
||||
|
||||
# Run OpenGrep
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
cwd=workspace
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
|
||||
# Parse results
|
||||
findings = []
|
||||
if process.returncode in [0, 1]: # 0 = no findings, 1 = findings found
|
||||
findings = self._parse_opengrep_output(stdout.decode(), workspace, config)
|
||||
logger.info(f"OpenGrep found {len(findings)} potential security issues")
|
||||
else:
|
||||
error_msg = stderr.decode()
|
||||
logger.error(f"OpenGrep failed: {error_msg}")
|
||||
return self.create_result(
|
||||
findings=[],
|
||||
status="failed",
|
||||
error=f"OpenGrep execution failed (exit code {process.returncode}): {error_msg[:500]}"
|
||||
)
|
||||
|
||||
# Create summary
|
||||
summary = self._create_summary(findings)
|
||||
|
||||
return self.create_result(
|
||||
findings=findings,
|
||||
status="success",
|
||||
summary=summary,
|
||||
metadata={
|
||||
"tool": "opengrep",
|
||||
"tool_version": "1.45.0",
|
||||
"languages": languages,
|
||||
"custom_rules": bool(custom_rules_path),
|
||||
}
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"OpenGrep Android module failed: {e}", exc_info=True)
|
||||
return self.create_result(
|
||||
findings=[],
|
||||
status="failed",
|
||||
error=str(e)
|
||||
)
|
||||
|
||||
def _parse_opengrep_output(self, output: str, workspace: Path, config: Dict[str, Any]) -> List[ModuleFinding]:
|
||||
"""Parse OpenGrep JSON output into findings"""
|
||||
findings = []
|
||||
|
||||
if not output.strip():
|
||||
return findings
|
||||
|
||||
try:
|
||||
data = json.loads(output)
|
||||
results = data.get("results", [])
|
||||
logger.debug(f"OpenGrep returned {len(results)} raw results")
|
||||
|
||||
# Get filtering criteria
|
||||
allowed_severities = set(config.get("severity", ["ERROR", "WARNING", "INFO"]))
|
||||
allowed_confidences = set(config.get("confidence", ["HIGH", "MEDIUM", "LOW"]))
|
||||
|
||||
for result in results:
|
||||
# Extract basic info
|
||||
rule_id = result.get("check_id", "unknown")
|
||||
message = result.get("message", "")
|
||||
extra = result.get("extra", {})
|
||||
severity = extra.get("severity", "INFO").upper()
|
||||
|
||||
# File location info
|
||||
path_info = result.get("path", "")
|
||||
start_line = result.get("start", {}).get("line", 0)
|
||||
end_line = result.get("end", {}).get("line", 0)
|
||||
|
||||
# Code snippet
|
||||
lines = extra.get("lines", "")
|
||||
|
||||
# Metadata
|
||||
rule_metadata = extra.get("metadata", {})
|
||||
cwe = rule_metadata.get("cwe", [])
|
||||
owasp = rule_metadata.get("owasp", [])
|
||||
confidence = extra.get("confidence", rule_metadata.get("confidence", "MEDIUM")).upper()
|
||||
|
||||
# Apply severity filter
|
||||
if severity not in allowed_severities:
|
||||
continue
|
||||
|
||||
# Apply confidence filter
|
||||
if confidence not in allowed_confidences:
|
||||
continue
|
||||
|
||||
# Make file path relative to workspace
|
||||
if path_info:
|
||||
try:
|
||||
rel_path = Path(path_info).relative_to(workspace)
|
||||
path_info = str(rel_path)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
# Map severity to our standard levels
|
||||
finding_severity = self._map_severity(severity)
|
||||
|
||||
# Create finding
|
||||
finding = self.create_finding(
|
||||
title=f"Android Security: {rule_id}",
|
||||
description=message or f"OpenGrep rule {rule_id} triggered",
|
||||
severity=finding_severity,
|
||||
category=self._get_category(rule_id, extra),
|
||||
file_path=path_info if path_info else None,
|
||||
line_start=start_line if start_line > 0 else None,
|
||||
line_end=end_line if end_line > 0 and end_line != start_line else None,
|
||||
code_snippet=lines.strip() if lines else None,
|
||||
recommendation=self._get_recommendation(rule_id, extra),
|
||||
metadata={
|
||||
"rule_id": rule_id,
|
||||
"opengrep_severity": severity,
|
||||
"confidence": confidence,
|
||||
"cwe": cwe,
|
||||
"owasp": owasp,
|
||||
"fix": extra.get("fix", ""),
|
||||
"impact": extra.get("impact", ""),
|
||||
"likelihood": extra.get("likelihood", ""),
|
||||
"references": extra.get("references", []),
|
||||
"tool": "opengrep",
|
||||
}
|
||||
)
|
||||
|
||||
findings.append(finding)
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning(f"Failed to parse OpenGrep output: {e}. Output snippet: {output[:200]}...")
|
||||
except Exception as e:
|
||||
logger.warning(f"Error processing OpenGrep results: {e}", exc_info=True)
|
||||
|
||||
return findings
|
||||
|
||||
def _map_severity(self, opengrep_severity: str) -> str:
|
||||
"""Map OpenGrep severity to our standard severity levels"""
|
||||
severity_map = {
|
||||
"ERROR": "high",
|
||||
"WARNING": "medium",
|
||||
"INFO": "low"
|
||||
}
|
||||
return severity_map.get(opengrep_severity.upper(), "medium")
|
||||
|
||||
def _get_category(self, rule_id: str, extra: Dict[str, Any]) -> str:
|
||||
"""Determine finding category based on rule and metadata"""
|
||||
rule_metadata = extra.get("metadata", {})
|
||||
cwe_list = rule_metadata.get("cwe", [])
|
||||
owasp_list = rule_metadata.get("owasp", [])
|
||||
|
||||
rule_lower = rule_id.lower()
|
||||
|
||||
# Android-specific categories
|
||||
if "injection" in rule_lower or "sql" in rule_lower:
|
||||
return "injection"
|
||||
elif "intent" in rule_lower:
|
||||
return "android-intent"
|
||||
elif "webview" in rule_lower:
|
||||
return "android-webview"
|
||||
elif "deeplink" in rule_lower:
|
||||
return "android-deeplink"
|
||||
elif "storage" in rule_lower or "sharedpreferences" in rule_lower:
|
||||
return "android-storage"
|
||||
elif "logging" in rule_lower or "log" in rule_lower:
|
||||
return "android-logging"
|
||||
elif "clipboard" in rule_lower:
|
||||
return "android-clipboard"
|
||||
elif "activity" in rule_lower or "service" in rule_lower or "provider" in rule_lower:
|
||||
return "android-component"
|
||||
elif "crypto" in rule_lower or "encrypt" in rule_lower:
|
||||
return "cryptography"
|
||||
elif "hardcode" in rule_lower or "secret" in rule_lower:
|
||||
return "secrets"
|
||||
elif "auth" in rule_lower:
|
||||
return "authentication"
|
||||
elif cwe_list:
|
||||
return f"cwe-{cwe_list[0]}"
|
||||
elif owasp_list:
|
||||
return f"owasp-{owasp_list[0].replace(' ', '-').lower()}"
|
||||
else:
|
||||
return "android-security"
|
||||
|
||||
def _get_recommendation(self, rule_id: str, extra: Dict[str, Any]) -> str:
|
||||
"""Generate recommendation based on rule and metadata"""
|
||||
fix_suggestion = extra.get("fix", "")
|
||||
if fix_suggestion:
|
||||
return fix_suggestion
|
||||
|
||||
rule_lower = rule_id.lower()
|
||||
|
||||
# Android-specific recommendations
|
||||
if "injection" in rule_lower or "sql" in rule_lower:
|
||||
return "Use parameterized queries or Room database with type-safe queries to prevent SQL injection."
|
||||
elif "intent" in rule_lower:
|
||||
return "Validate all incoming Intent data and use explicit Intents when possible to prevent Intent manipulation attacks."
|
||||
elif "webview" in rule_lower and "javascript" in rule_lower:
|
||||
return "Disable JavaScript in WebView if not needed, or implement proper JavaScript interfaces with @JavascriptInterface annotation."
|
||||
elif "deeplink" in rule_lower:
|
||||
return "Validate all deeplink URLs and sanitize user input to prevent deeplink hijacking attacks."
|
||||
elif "storage" in rule_lower or "sharedpreferences" in rule_lower:
|
||||
return "Encrypt sensitive data before storing in SharedPreferences or use EncryptedSharedPreferences for Android API 23+."
|
||||
elif "logging" in rule_lower:
|
||||
return "Remove sensitive data from logs in production builds. Use ProGuard/R8 to strip logging statements."
|
||||
elif "clipboard" in rule_lower:
|
||||
return "Avoid placing sensitive data on the clipboard. If necessary, clear clipboard data when no longer needed."
|
||||
elif "crypto" in rule_lower:
|
||||
return "Use modern cryptographic algorithms (AES-GCM, RSA-OAEP) and Android Keystore for key management."
|
||||
elif "hardcode" in rule_lower or "secret" in rule_lower:
|
||||
return "Remove hardcoded secrets. Use Android Keystore, environment variables, or secure configuration management."
|
||||
else:
|
||||
return "Review this Android security issue and apply appropriate fixes based on Android security best practices."
|
||||
|
||||
def _create_summary(self, findings: List[ModuleFinding]) -> Dict[str, Any]:
|
||||
"""Create analysis summary"""
|
||||
severity_counts = {"critical": 0, "high": 0, "medium": 0, "low": 0}
|
||||
category_counts = {}
|
||||
rule_counts = {}
|
||||
|
||||
for finding in findings:
|
||||
# Count by severity
|
||||
severity_counts[finding.severity] += 1
|
||||
|
||||
# Count by category
|
||||
category = finding.category
|
||||
category_counts[category] = category_counts.get(category, 0) + 1
|
||||
|
||||
# Count by rule
|
||||
rule_id = finding.metadata.get("rule_id", "unknown")
|
||||
rule_counts[rule_id] = rule_counts.get(rule_id, 0) + 1
|
||||
|
||||
return {
|
||||
"total_findings": len(findings),
|
||||
"severity_counts": severity_counts,
|
||||
"category_counts": category_counts,
|
||||
"top_rules": dict(sorted(rule_counts.items(), key=lambda x: x[1], reverse=True)[:10]),
|
||||
"files_analyzed": len(set(f.file_path for f in findings if f.file_path))
|
||||
}
|
||||
@@ -10,5 +10,6 @@
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from .file_scanner import FileScanner
|
||||
from .dependency_scanner import DependencyScanner
|
||||
|
||||
__all__ = ["FileScanner"]
|
||||
__all__ = ["FileScanner", "DependencyScanner"]
|
||||
@@ -0,0 +1,302 @@
|
||||
"""
|
||||
Dependency Scanner Module - Scans Python dependencies for known vulnerabilities using pip-audit
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List
|
||||
|
||||
try:
|
||||
from toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
try:
|
||||
from modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
except ImportError:
|
||||
from src.toolbox.modules.base import BaseModule, ModuleMetadata, ModuleResult, ModuleFinding
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DependencyScanner(BaseModule):
|
||||
"""
|
||||
Scans Python dependencies for known vulnerabilities using pip-audit.
|
||||
|
||||
This module:
|
||||
- Discovers dependency files (requirements.txt, pyproject.toml, setup.py, Pipfile)
|
||||
- Runs pip-audit to check for vulnerable dependencies
|
||||
- Reports CVEs with severity and affected versions
|
||||
"""
|
||||
|
||||
def get_metadata(self) -> ModuleMetadata:
|
||||
"""Get module metadata"""
|
||||
return ModuleMetadata(
|
||||
name="dependency_scanner",
|
||||
version="1.0.0",
|
||||
description="Scans Python dependencies for known vulnerabilities",
|
||||
author="FuzzForge Team",
|
||||
category="scanner",
|
||||
tags=["dependencies", "cve", "vulnerabilities", "pip-audit"],
|
||||
input_schema={
|
||||
"dependency_files": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "List of dependency files to scan (auto-discovered if empty)",
|
||||
"default": []
|
||||
},
|
||||
"ignore_vulns": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "List of vulnerability IDs to ignore",
|
||||
"default": []
|
||||
}
|
||||
},
|
||||
output_schema={
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"description": "List of vulnerable dependencies with CVE information"
|
||||
}
|
||||
},
|
||||
requires_workspace=True
|
||||
)
|
||||
|
||||
def validate_config(self, config: Dict[str, Any]) -> bool:
|
||||
"""Validate module configuration"""
|
||||
dep_files = config.get("dependency_files", [])
|
||||
if not isinstance(dep_files, list):
|
||||
raise ValueError("dependency_files must be a list")
|
||||
|
||||
ignore_vulns = config.get("ignore_vulns", [])
|
||||
if not isinstance(ignore_vulns, list):
|
||||
raise ValueError("ignore_vulns must be a list")
|
||||
|
||||
return True
|
||||
|
||||
def _discover_dependency_files(self, workspace: Path) -> List[Path]:
|
||||
"""
|
||||
Discover Python dependency files in workspace.
|
||||
|
||||
Returns:
|
||||
List of discovered dependency file paths
|
||||
"""
|
||||
dependency_patterns = [
|
||||
"requirements.txt",
|
||||
"*requirements*.txt",
|
||||
"pyproject.toml",
|
||||
"setup.py",
|
||||
"Pipfile",
|
||||
"poetry.lock"
|
||||
]
|
||||
|
||||
found_files = []
|
||||
for pattern in dependency_patterns:
|
||||
found_files.extend(workspace.rglob(pattern))
|
||||
|
||||
# Deduplicate and return
|
||||
unique_files = list(set(found_files))
|
||||
logger.info(f"Discovered {len(unique_files)} dependency files")
|
||||
return unique_files
|
||||
|
||||
async def _run_pip_audit(self, file_path: Path) -> Dict[str, Any]:
|
||||
"""
|
||||
Run pip-audit on a specific dependency file.
|
||||
|
||||
Args:
|
||||
file_path: Path to dependency file
|
||||
|
||||
Returns:
|
||||
pip-audit JSON output as dict
|
||||
"""
|
||||
try:
|
||||
# Run pip-audit with JSON output
|
||||
cmd = [
|
||||
"pip-audit",
|
||||
"--requirement", str(file_path),
|
||||
"--format", "json",
|
||||
"--progress-spinner", "off"
|
||||
]
|
||||
|
||||
logger.info(f"Running pip-audit on: {file_path.name}")
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
|
||||
# pip-audit returns 0 if no vulns, 1 if vulns found
|
||||
if process.returncode not in [0, 1]:
|
||||
logger.error(f"pip-audit failed: {stderr.decode()}")
|
||||
return {"dependencies": []}
|
||||
|
||||
# Parse JSON output
|
||||
result = json.loads(stdout.decode())
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error running pip-audit on {file_path}: {e}")
|
||||
return {"dependencies": []}
|
||||
|
||||
def _convert_to_findings(
|
||||
self,
|
||||
audit_result: Dict[str, Any],
|
||||
file_path: Path,
|
||||
workspace: Path,
|
||||
ignore_vulns: List[str]
|
||||
) -> List[ModuleFinding]:
|
||||
"""
|
||||
Convert pip-audit results to ModuleFindings.
|
||||
|
||||
Args:
|
||||
audit_result: pip-audit JSON output
|
||||
file_path: Path to scanned file
|
||||
workspace: Workspace path for relative path calculation
|
||||
ignore_vulns: List of vulnerability IDs to ignore
|
||||
|
||||
Returns:
|
||||
List of ModuleFindings
|
||||
"""
|
||||
findings = []
|
||||
|
||||
# pip-audit format: {"dependencies": [{package, version, vulns: []}]}
|
||||
for dep in audit_result.get("dependencies", []):
|
||||
package_name = dep.get("name", "unknown")
|
||||
package_version = dep.get("version", "unknown")
|
||||
vulnerabilities = dep.get("vulns", [])
|
||||
|
||||
for vuln in vulnerabilities:
|
||||
vuln_id = vuln.get("id", "UNKNOWN")
|
||||
|
||||
# Skip if in ignore list
|
||||
if vuln_id in ignore_vulns:
|
||||
logger.debug(f"Ignoring vulnerability: {vuln_id}")
|
||||
continue
|
||||
|
||||
description = vuln.get("description", "No description available")
|
||||
fix_versions = vuln.get("fix_versions", [])
|
||||
|
||||
# Map CVSS scores to severity
|
||||
# pip-audit doesn't always provide CVSS, so we default to medium
|
||||
severity = "medium"
|
||||
|
||||
# Try to get relative path
|
||||
try:
|
||||
rel_path = file_path.relative_to(workspace)
|
||||
except ValueError:
|
||||
rel_path = file_path
|
||||
|
||||
recommendation = f"Upgrade {package_name} to a fixed version: {', '.join(fix_versions)}" if fix_versions else f"Check for updates to {package_name}"
|
||||
|
||||
finding = self.create_finding(
|
||||
title=f"Vulnerable dependency: {package_name} ({vuln_id})",
|
||||
description=f"{description}\n\nAffected package: {package_name} {package_version}",
|
||||
severity=severity,
|
||||
category="vulnerable-dependency",
|
||||
file_path=str(rel_path),
|
||||
recommendation=recommendation,
|
||||
metadata={
|
||||
"cve_id": vuln_id,
|
||||
"package": package_name,
|
||||
"installed_version": package_version,
|
||||
"fix_versions": fix_versions,
|
||||
"aliases": vuln.get("aliases", []),
|
||||
"link": vuln.get("link", "")
|
||||
}
|
||||
)
|
||||
findings.append(finding)
|
||||
|
||||
return findings
|
||||
|
||||
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
|
||||
"""
|
||||
Execute the dependency scanning module.
|
||||
|
||||
Args:
|
||||
config: Module configuration
|
||||
workspace: Path to workspace
|
||||
|
||||
Returns:
|
||||
ModuleResult with vulnerability findings
|
||||
"""
|
||||
start_time = time.time()
|
||||
metadata = self.get_metadata()
|
||||
|
||||
# Validate inputs
|
||||
self.validate_config(config)
|
||||
self.validate_workspace(workspace)
|
||||
|
||||
# Get configuration
|
||||
specified_files = config.get("dependency_files", [])
|
||||
ignore_vulns = config.get("ignore_vulns", [])
|
||||
|
||||
# Discover or use specified dependency files
|
||||
if specified_files:
|
||||
dep_files = [workspace / f for f in specified_files]
|
||||
else:
|
||||
dep_files = self._discover_dependency_files(workspace)
|
||||
|
||||
if not dep_files:
|
||||
logger.warning("No dependency files found in workspace")
|
||||
return ModuleResult(
|
||||
module=metadata.name,
|
||||
version=metadata.version,
|
||||
status="success",
|
||||
execution_time=time.time() - start_time,
|
||||
findings=[],
|
||||
summary={
|
||||
"total_files": 0,
|
||||
"total_vulnerabilities": 0,
|
||||
"vulnerable_packages": 0
|
||||
}
|
||||
)
|
||||
|
||||
# Scan each dependency file
|
||||
all_findings = []
|
||||
files_scanned = 0
|
||||
|
||||
for dep_file in dep_files:
|
||||
if not dep_file.exists():
|
||||
logger.warning(f"Dependency file not found: {dep_file}")
|
||||
continue
|
||||
|
||||
logger.info(f"Scanning dependencies in: {dep_file.name}")
|
||||
audit_result = await self._run_pip_audit(dep_file)
|
||||
findings = self._convert_to_findings(audit_result, dep_file, workspace, ignore_vulns)
|
||||
|
||||
all_findings.extend(findings)
|
||||
files_scanned += 1
|
||||
|
||||
# Calculate summary
|
||||
unique_packages = len(set(f.metadata.get("package") for f in all_findings))
|
||||
|
||||
execution_time = time.time() - start_time
|
||||
|
||||
return ModuleResult(
|
||||
module=metadata.name,
|
||||
version=metadata.version,
|
||||
status="success",
|
||||
execution_time=execution_time,
|
||||
findings=all_findings,
|
||||
summary={
|
||||
"total_files": files_scanned,
|
||||
"total_vulnerabilities": len(all_findings),
|
||||
"vulnerable_packages": unique_packages
|
||||
},
|
||||
metadata={
|
||||
"scanned_files": [str(f.name) for f in dep_files if f.exists()]
|
||||
}
|
||||
)
|
||||
@@ -107,7 +107,8 @@ class LLMSecretDetectorModule(BaseModule):
|
||||
)
|
||||
|
||||
agent_url = config.get("agent_url")
|
||||
if not agent_url or not isinstance(agent_url, str):
|
||||
# agent_url is optional - will have default from metadata.yaml
|
||||
if agent_url is not None and not isinstance(agent_url, str):
|
||||
raise ValueError("agent_url must be a valid URL string")
|
||||
|
||||
max_files = config.get("max_files", 20)
|
||||
@@ -131,14 +132,14 @@ class LLMSecretDetectorModule(BaseModule):
|
||||
|
||||
logger.info(f"Starting LLM secret detection in workspace: {workspace}")
|
||||
|
||||
# Extract configuration
|
||||
agent_url = config.get("agent_url", "http://fuzzforge-task-agent:8000/a2a/litellm_agent")
|
||||
llm_model = config.get("llm_model", "gpt-4o-mini")
|
||||
llm_provider = config.get("llm_provider", "openai")
|
||||
file_patterns = config.get("file_patterns", ["*.py", "*.js", "*.ts", "*.java", "*.go", "*.env", "*.yaml", "*.yml", "*.json", "*.xml", "*.ini", "*.sql", "*.properties", "*.sh", "*.bat", "*.config", "*.conf", "*.toml", "*id_rsa*", "*.txt"])
|
||||
max_files = config.get("max_files", 20)
|
||||
max_file_size = config.get("max_file_size", 30000)
|
||||
timeout = config.get("timeout", 30) # Reduced from 45s
|
||||
# Extract configuration (defaults come from metadata.yaml via API)
|
||||
agent_url = config["agent_url"]
|
||||
llm_model = config["llm_model"]
|
||||
llm_provider = config["llm_provider"]
|
||||
file_patterns = config["file_patterns"]
|
||||
max_files = config["max_files"]
|
||||
max_file_size = config["max_file_size"]
|
||||
timeout = config["timeout"]
|
||||
|
||||
# Find files to analyze
|
||||
# Skip files that are unlikely to contain secrets
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""
|
||||
Android Static Analysis Workflow
|
||||
|
||||
Comprehensive Android application security testing combining:
|
||||
- Jadx APK decompilation
|
||||
- OpenGrep/Semgrep static analysis with Android-specific rules
|
||||
- MobSF mobile security framework analysis
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from .workflow import AndroidStaticAnalysisWorkflow
|
||||
from .activities import (
|
||||
decompile_with_jadx_activity,
|
||||
scan_with_opengrep_activity,
|
||||
scan_with_mobsf_activity,
|
||||
generate_android_sarif_activity,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"AndroidStaticAnalysisWorkflow",
|
||||
"decompile_with_jadx_activity",
|
||||
"scan_with_opengrep_activity",
|
||||
"scan_with_mobsf_activity",
|
||||
"generate_android_sarif_activity",
|
||||
]
|
||||
@@ -0,0 +1,213 @@
|
||||
"""
|
||||
Android Static Analysis Workflow Activities
|
||||
|
||||
Activities for the Android security testing workflow:
|
||||
- decompile_with_jadx_activity: Decompile APK using Jadx
|
||||
- scan_with_opengrep_activity: Analyze code with OpenGrep/Semgrep
|
||||
- scan_with_mobsf_activity: Scan APK with MobSF
|
||||
- generate_android_sarif_activity: Generate combined SARIF report
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from temporalio import activity
|
||||
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Add toolbox to path for module imports
|
||||
sys.path.insert(0, '/app/toolbox')
|
||||
|
||||
|
||||
@activity.defn(name="decompile_with_jadx")
|
||||
async def decompile_with_jadx_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Decompile Android APK to Java source code using Jadx.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: JadxDecompiler configuration
|
||||
|
||||
Returns:
|
||||
Decompilation results dictionary
|
||||
"""
|
||||
logger.info(f"Activity: decompile_with_jadx (workspace={workspace_path})")
|
||||
|
||||
try:
|
||||
from modules.android import JadxDecompiler
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
decompiler = JadxDecompiler()
|
||||
result = await decompiler.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ Jadx decompilation completed: "
|
||||
f"{result.summary.get('java_files', 0)} Java files generated"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Jadx decompilation failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="scan_with_opengrep")
|
||||
async def scan_with_opengrep_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Analyze Android code for security issues using OpenGrep/Semgrep.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: OpenGrepAndroid configuration
|
||||
|
||||
Returns:
|
||||
Analysis results dictionary
|
||||
"""
|
||||
logger.info(f"Activity: scan_with_opengrep (workspace={workspace_path})")
|
||||
|
||||
try:
|
||||
from modules.android import OpenGrepAndroid
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
analyzer = OpenGrepAndroid()
|
||||
result = await analyzer.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ OpenGrep analysis completed: "
|
||||
f"{result.summary.get('total_findings', 0)} security issues found"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"OpenGrep analysis failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="scan_with_mobsf")
|
||||
async def scan_with_mobsf_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Analyze Android APK for security issues using MobSF.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: MobSFScanner configuration
|
||||
|
||||
Returns:
|
||||
Scan results dictionary (or skipped status if MobSF unavailable)
|
||||
"""
|
||||
logger.info(f"Activity: scan_with_mobsf (workspace={workspace_path})")
|
||||
|
||||
# Check if MobSF is installed (graceful degradation for ARM64 platform)
|
||||
mobsf_path = Path("/app/mobsf")
|
||||
if not mobsf_path.exists():
|
||||
logger.warning("MobSF not installed on this platform (ARM64/Rosetta limitation)")
|
||||
return {
|
||||
"status": "skipped",
|
||||
"findings": [],
|
||||
"summary": {
|
||||
"total_findings": 0,
|
||||
"skip_reason": "MobSF unavailable on ARM64 platform (Rosetta 2 incompatibility)"
|
||||
}
|
||||
}
|
||||
|
||||
try:
|
||||
from modules.android import MobSFScanner
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
scanner = MobSFScanner()
|
||||
result = await scanner.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ MobSF scan completed: "
|
||||
f"{result.summary.get('total_findings', 0)} findings"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"MobSF scan failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="generate_android_sarif")
|
||||
async def generate_android_sarif_activity(
|
||||
jadx_result: dict,
|
||||
opengrep_result: dict,
|
||||
mobsf_result: dict,
|
||||
config: dict,
|
||||
workspace_path: str
|
||||
) -> dict:
|
||||
"""
|
||||
Generate combined SARIF report from all Android security findings.
|
||||
|
||||
Args:
|
||||
jadx_result: Jadx decompilation results
|
||||
opengrep_result: OpenGrep analysis results
|
||||
mobsf_result: MobSF scan results (may be None if disabled)
|
||||
config: Reporter configuration
|
||||
workspace_path: Workspace path
|
||||
|
||||
Returns:
|
||||
SARIF report dictionary
|
||||
"""
|
||||
logger.info("Activity: generate_android_sarif")
|
||||
|
||||
try:
|
||||
from modules.reporter import SARIFReporter
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
|
||||
# Collect all findings
|
||||
all_findings = []
|
||||
all_findings.extend(opengrep_result.get("findings", []))
|
||||
|
||||
if mobsf_result:
|
||||
all_findings.extend(mobsf_result.get("findings", []))
|
||||
|
||||
# Prepare reporter config
|
||||
reporter_config = {
|
||||
**(config or {}),
|
||||
"findings": all_findings,
|
||||
"tool_name": "FuzzForge Android Static Analysis",
|
||||
"tool_version": "1.0.0",
|
||||
"metadata": {
|
||||
"jadx_version": "1.5.0",
|
||||
"opengrep_version": "1.45.0",
|
||||
"mobsf_version": "3.9.7",
|
||||
"java_files_decompiled": jadx_result.get("summary", {}).get("java_files", 0),
|
||||
}
|
||||
}
|
||||
|
||||
reporter = SARIFReporter()
|
||||
result = await reporter.execute(reporter_config, workspace)
|
||||
|
||||
sarif_report = result.dict().get("sarif", {})
|
||||
|
||||
logger.info(f"✓ SARIF report generated with {len(all_findings)} findings")
|
||||
|
||||
return sarif_report
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"SARIF report generation failed: {e}", exc_info=True)
|
||||
raise
|
||||
@@ -0,0 +1,172 @@
|
||||
name: android_static_analysis
|
||||
version: "1.0.0"
|
||||
vertical: android
|
||||
description: "Comprehensive Android application security testing using Jadx decompilation, OpenGrep static analysis, and MobSF mobile security framework"
|
||||
author: "FuzzForge Team"
|
||||
tags:
|
||||
- "android"
|
||||
- "mobile"
|
||||
- "static-analysis"
|
||||
- "security"
|
||||
- "opengrep"
|
||||
- "semgrep"
|
||||
- "mobsf"
|
||||
- "jadx"
|
||||
- "apk"
|
||||
- "sarif"
|
||||
|
||||
# Workspace isolation mode
|
||||
# Using "shared" mode for read-only APK analysis (no file modifications except decompilation output)
|
||||
workspace_isolation: "shared"
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
apk_path:
|
||||
type: string
|
||||
description: "Path to the APK file to analyze (relative to uploaded target or absolute within workspace)"
|
||||
default: ""
|
||||
|
||||
decompile_apk:
|
||||
type: boolean
|
||||
description: "Whether to decompile APK with Jadx before OpenGrep analysis"
|
||||
default: true
|
||||
|
||||
jadx_config:
|
||||
type: object
|
||||
description: "Jadx decompiler configuration"
|
||||
properties:
|
||||
output_dir:
|
||||
type: string
|
||||
description: "Output directory for decompiled sources"
|
||||
default: "jadx_output"
|
||||
overwrite:
|
||||
type: boolean
|
||||
description: "Overwrite existing decompilation output"
|
||||
default: true
|
||||
threads:
|
||||
type: integer
|
||||
description: "Number of decompilation threads"
|
||||
default: 4
|
||||
minimum: 1
|
||||
maximum: 32
|
||||
decompiler_args:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "Additional Jadx arguments"
|
||||
default: []
|
||||
|
||||
opengrep_config:
|
||||
type: object
|
||||
description: "OpenGrep/Semgrep static analysis configuration"
|
||||
properties:
|
||||
config:
|
||||
type: string
|
||||
enum: ["auto", "p/security-audit", "p/owasp-top-ten", "p/cwe-top-25"]
|
||||
description: "Preset OpenGrep ruleset (ignored if custom_rules_path is set)"
|
||||
default: "auto"
|
||||
custom_rules_path:
|
||||
type: string
|
||||
description: "Path to custom OpenGrep rules directory (use Android-specific rules for best results)"
|
||||
default: "/app/toolbox/modules/android/custom_rules"
|
||||
languages:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "Programming languages to analyze (defaults to java, kotlin for Android)"
|
||||
default: ["java", "kotlin"]
|
||||
include_patterns:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "File patterns to include in scan"
|
||||
default: []
|
||||
exclude_patterns:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "File patterns to exclude from scan"
|
||||
default: []
|
||||
max_target_bytes:
|
||||
type: integer
|
||||
description: "Maximum file size to analyze (bytes)"
|
||||
default: 1000000
|
||||
timeout:
|
||||
type: integer
|
||||
description: "Analysis timeout in seconds"
|
||||
default: 300
|
||||
severity:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enum: ["ERROR", "WARNING", "INFO"]
|
||||
description: "Severity levels to include in results"
|
||||
default: ["ERROR", "WARNING", "INFO"]
|
||||
confidence:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enum: ["HIGH", "MEDIUM", "LOW"]
|
||||
description: "Confidence levels to include in results"
|
||||
default: ["HIGH", "MEDIUM", "LOW"]
|
||||
|
||||
mobsf_config:
|
||||
type: object
|
||||
description: "MobSF scanner configuration"
|
||||
properties:
|
||||
enabled:
|
||||
type: boolean
|
||||
description: "Enable MobSF analysis (requires APK file)"
|
||||
default: true
|
||||
mobsf_url:
|
||||
type: string
|
||||
description: "MobSF server URL"
|
||||
default: "http://localhost:8877"
|
||||
api_key:
|
||||
type: string
|
||||
description: "MobSF API key (if not provided, uses MOBSF_API_KEY env var)"
|
||||
default: null
|
||||
rescan:
|
||||
type: boolean
|
||||
description: "Force rescan even if APK was previously analyzed"
|
||||
default: false
|
||||
|
||||
reporter_config:
|
||||
type: object
|
||||
description: "SARIF reporter configuration"
|
||||
properties:
|
||||
include_code_flows:
|
||||
type: boolean
|
||||
description: "Include code flow information in SARIF output"
|
||||
default: false
|
||||
logical_id:
|
||||
type: string
|
||||
description: "Custom identifier for the SARIF report"
|
||||
default: null
|
||||
|
||||
output_schema:
|
||||
type: object
|
||||
properties:
|
||||
sarif:
|
||||
type: object
|
||||
description: "SARIF-formatted findings from all Android security tools"
|
||||
summary:
|
||||
type: object
|
||||
description: "Android security analysis summary"
|
||||
properties:
|
||||
total_findings:
|
||||
type: integer
|
||||
decompiled_java_files:
|
||||
type: integer
|
||||
description: "Number of Java files decompiled by Jadx"
|
||||
opengrep_findings:
|
||||
type: integer
|
||||
description: "Findings from OpenGrep/Semgrep analysis"
|
||||
mobsf_findings:
|
||||
type: integer
|
||||
description: "Findings from MobSF analysis"
|
||||
severity_distribution:
|
||||
type: object
|
||||
category_distribution:
|
||||
type: object
|
||||
@@ -0,0 +1,289 @@
|
||||
"""
|
||||
Android Static Analysis Workflow - Temporal Version
|
||||
|
||||
Comprehensive security testing for Android applications using Jadx, OpenGrep, and MobSF.
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from datetime import timedelta
|
||||
from typing import Dict, Any, Optional
|
||||
from pathlib import Path
|
||||
|
||||
from temporalio import workflow
|
||||
from temporalio.common import RetryPolicy
|
||||
|
||||
# Import activity interfaces (will be executed by worker)
|
||||
with workflow.unsafe.imports_passed_through():
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class AndroidStaticAnalysisWorkflow:
|
||||
"""
|
||||
Android Static Application Security Testing workflow.
|
||||
|
||||
This workflow:
|
||||
1. Downloads target (APK) from MinIO
|
||||
2. (Optional) Decompiles APK using Jadx
|
||||
3. Runs OpenGrep/Semgrep static analysis on decompiled code
|
||||
4. (Optional) Runs MobSF comprehensive security scan
|
||||
5. Generates a SARIF report with all findings
|
||||
6. Uploads results to MinIO
|
||||
7. Cleans up cache
|
||||
"""
|
||||
|
||||
@workflow.run
|
||||
async def run(
|
||||
self,
|
||||
target_id: str,
|
||||
apk_path: Optional[str] = None,
|
||||
decompile_apk: bool = True,
|
||||
jadx_config: Optional[Dict[str, Any]] = None,
|
||||
opengrep_config: Optional[Dict[str, Any]] = None,
|
||||
mobsf_config: Optional[Dict[str, Any]] = None,
|
||||
reporter_config: Optional[Dict[str, Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Main workflow execution.
|
||||
|
||||
Args:
|
||||
target_id: UUID of the uploaded target (APK) in MinIO
|
||||
apk_path: Path to APK file within target (if target is not a single APK)
|
||||
decompile_apk: Whether to decompile APK with Jadx before OpenGrep
|
||||
jadx_config: Configuration for Jadx decompiler
|
||||
opengrep_config: Configuration for OpenGrep analyzer
|
||||
mobsf_config: Configuration for MobSF scanner
|
||||
reporter_config: Configuration for SARIF reporter
|
||||
|
||||
Returns:
|
||||
Dictionary containing SARIF report and summary
|
||||
"""
|
||||
workflow_id = workflow.info().workflow_id
|
||||
|
||||
workflow.logger.info(
|
||||
f"Starting AndroidStaticAnalysisWorkflow "
|
||||
f"(workflow_id={workflow_id}, target_id={target_id})"
|
||||
)
|
||||
|
||||
# Default configurations
|
||||
if not jadx_config:
|
||||
jadx_config = {
|
||||
"output_dir": "jadx_output",
|
||||
"overwrite": True,
|
||||
"threads": 4,
|
||||
"decompiler_args": []
|
||||
}
|
||||
|
||||
if not opengrep_config:
|
||||
opengrep_config = {
|
||||
"config": "auto",
|
||||
"custom_rules_path": "/app/toolbox/modules/android/custom_rules",
|
||||
"languages": ["java", "kotlin"],
|
||||
"severity": ["ERROR", "WARNING", "INFO"],
|
||||
"confidence": ["HIGH", "MEDIUM", "LOW"],
|
||||
"timeout": 300,
|
||||
}
|
||||
|
||||
if not mobsf_config:
|
||||
mobsf_config = {
|
||||
"enabled": True,
|
||||
"mobsf_url": "http://localhost:8877",
|
||||
"api_key": None,
|
||||
"rescan": False,
|
||||
}
|
||||
|
||||
if not reporter_config:
|
||||
reporter_config = {
|
||||
"include_code_flows": False
|
||||
}
|
||||
|
||||
# Activity retry policy
|
||||
retry_policy = RetryPolicy(
|
||||
initial_interval=timedelta(seconds=1),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=3,
|
||||
backoff_coefficient=2.0,
|
||||
)
|
||||
|
||||
# Phase 0: Download target from MinIO
|
||||
workflow.logger.info(f"Phase 0: Downloading target from MinIO (target_id={target_id})")
|
||||
workspace_path = await workflow.execute_activity(
|
||||
"get_target",
|
||||
args=[target_id, workflow.info().workflow_id, "shared"],
|
||||
start_to_close_timeout=timedelta(minutes=10),
|
||||
retry_policy=retry_policy,
|
||||
)
|
||||
workflow.logger.info(f"✓ Target downloaded to: {workspace_path}")
|
||||
|
||||
# Handle case where workspace_path is a file (single APK upload)
|
||||
# vs. a directory containing files
|
||||
workspace_path_obj = Path(workspace_path)
|
||||
|
||||
# Determine actual workspace directory and APK path
|
||||
if apk_path:
|
||||
# User explicitly provided apk_path
|
||||
actual_apk_path = apk_path
|
||||
# workspace_path could be either a file or directory
|
||||
# If it's a file and apk_path matches the filename, use parent as workspace
|
||||
if workspace_path_obj.name == apk_path:
|
||||
workspace_path = str(workspace_path_obj.parent)
|
||||
workflow.logger.info(f"Adjusted workspace to parent directory: {workspace_path}")
|
||||
else:
|
||||
# No apk_path provided - check if workspace_path is an APK file
|
||||
if workspace_path_obj.suffix.lower() == '.apk' or workspace_path_obj.name.endswith('.apk'):
|
||||
# workspace_path is the APK file itself
|
||||
actual_apk_path = workspace_path_obj.name
|
||||
workspace_path = str(workspace_path_obj.parent)
|
||||
workflow.logger.info(f"Detected single APK file: {actual_apk_path}, workspace: {workspace_path}")
|
||||
else:
|
||||
# workspace_path is a directory, need to find APK within it
|
||||
actual_apk_path = None
|
||||
workflow.logger.info("Workspace is a directory, APK detection will be handled by modules")
|
||||
|
||||
# Phase 1: Jadx decompilation (if enabled and APK provided)
|
||||
jadx_result = None
|
||||
analysis_workspace = workspace_path
|
||||
|
||||
if decompile_apk and actual_apk_path:
|
||||
workflow.logger.info(f"Phase 1: Decompiling APK with Jadx (apk={actual_apk_path})")
|
||||
|
||||
jadx_activity_config = {
|
||||
**jadx_config,
|
||||
"apk_path": actual_apk_path
|
||||
}
|
||||
|
||||
jadx_result = await workflow.execute_activity(
|
||||
"decompile_with_jadx",
|
||||
args=[workspace_path, jadx_activity_config],
|
||||
start_to_close_timeout=timedelta(minutes=15),
|
||||
retry_policy=retry_policy,
|
||||
)
|
||||
|
||||
if jadx_result.get("status") == "success":
|
||||
# Use decompiled sources as workspace for OpenGrep
|
||||
source_dir = jadx_result.get("summary", {}).get("source_dir")
|
||||
if source_dir:
|
||||
analysis_workspace = source_dir
|
||||
workflow.logger.info(
|
||||
f"✓ Jadx decompiled {jadx_result.get('summary', {}).get('java_files', 0)} Java files"
|
||||
)
|
||||
else:
|
||||
workflow.logger.warning(f"Jadx decompilation failed: {jadx_result.get('error')}")
|
||||
else:
|
||||
workflow.logger.info("Phase 1: Jadx decompilation skipped")
|
||||
|
||||
# Phase 2: OpenGrep static analysis
|
||||
workflow.logger.info(f"Phase 2: OpenGrep analysis on {analysis_workspace}")
|
||||
|
||||
opengrep_result = await workflow.execute_activity(
|
||||
"scan_with_opengrep",
|
||||
args=[analysis_workspace, opengrep_config],
|
||||
start_to_close_timeout=timedelta(minutes=20),
|
||||
retry_policy=retry_policy,
|
||||
)
|
||||
|
||||
workflow.logger.info(
|
||||
f"✓ OpenGrep completed: {opengrep_result.get('summary', {}).get('total_findings', 0)} findings"
|
||||
)
|
||||
|
||||
# Phase 3: MobSF analysis (if enabled and APK provided)
|
||||
mobsf_result = None
|
||||
|
||||
if mobsf_config.get("enabled", True) and actual_apk_path:
|
||||
workflow.logger.info(f"Phase 3: MobSF scan on APK: {actual_apk_path}")
|
||||
|
||||
mobsf_activity_config = {
|
||||
**mobsf_config,
|
||||
"file_path": actual_apk_path
|
||||
}
|
||||
|
||||
try:
|
||||
mobsf_result = await workflow.execute_activity(
|
||||
"scan_with_mobsf",
|
||||
args=[workspace_path, mobsf_activity_config],
|
||||
start_to_close_timeout=timedelta(minutes=30),
|
||||
retry_policy=RetryPolicy(
|
||||
maximum_attempts=2 # MobSF can be flaky, limit retries
|
||||
),
|
||||
)
|
||||
|
||||
# Handle skipped or completed status
|
||||
if mobsf_result.get("status") == "skipped":
|
||||
workflow.logger.warning(
|
||||
f"⚠️ MobSF skipped: {mobsf_result.get('summary', {}).get('skip_reason', 'Unknown reason')}"
|
||||
)
|
||||
else:
|
||||
workflow.logger.info(
|
||||
f"✓ MobSF completed: {mobsf_result.get('summary', {}).get('total_findings', 0)} findings"
|
||||
)
|
||||
except Exception as e:
|
||||
workflow.logger.warning(f"MobSF scan failed (continuing without it): {e}")
|
||||
mobsf_result = None
|
||||
else:
|
||||
workflow.logger.info("Phase 3: MobSF scan skipped (disabled or no APK)")
|
||||
|
||||
# Phase 4: Generate SARIF report
|
||||
workflow.logger.info("Phase 4: Generating SARIF report")
|
||||
|
||||
sarif_report = await workflow.execute_activity(
|
||||
"generate_android_sarif",
|
||||
args=[jadx_result or {}, opengrep_result, mobsf_result, reporter_config, workspace_path],
|
||||
start_to_close_timeout=timedelta(minutes=5),
|
||||
retry_policy=retry_policy,
|
||||
)
|
||||
|
||||
# Phase 5: Upload results to MinIO
|
||||
workflow.logger.info("Phase 5: Uploading results to MinIO")
|
||||
|
||||
result_url = await workflow.execute_activity(
|
||||
"upload_results",
|
||||
args=[workflow.info().workflow_id, sarif_report, "sarif"],
|
||||
start_to_close_timeout=timedelta(minutes=10),
|
||||
retry_policy=retry_policy,
|
||||
)
|
||||
|
||||
workflow.logger.info(f"✓ Results uploaded: {result_url}")
|
||||
|
||||
# Phase 6: Cleanup cache
|
||||
workflow.logger.info("Phase 6: Cleaning up cache")
|
||||
|
||||
await workflow.execute_activity(
|
||||
"cleanup_cache",
|
||||
args=[workspace_path, "shared"],
|
||||
start_to_close_timeout=timedelta(minutes=5),
|
||||
retry_policy=RetryPolicy(maximum_attempts=1), # Don't retry cleanup
|
||||
)
|
||||
|
||||
# Calculate summary
|
||||
total_findings = len(sarif_report.get("runs", [{}])[0].get("results", []))
|
||||
|
||||
summary = {
|
||||
"workflow": "android_static_analysis",
|
||||
"target_id": target_id,
|
||||
"total_findings": total_findings,
|
||||
"decompiled_java_files": (jadx_result or {}).get("summary", {}).get("java_files", 0) if jadx_result else 0,
|
||||
"opengrep_findings": opengrep_result.get("summary", {}).get("total_findings", 0),
|
||||
"mobsf_findings": mobsf_result.get("summary", {}).get("total_findings", 0) if mobsf_result else 0,
|
||||
"result_url": result_url,
|
||||
}
|
||||
|
||||
workflow.logger.info(
|
||||
f"✅ AndroidStaticAnalysisWorkflow completed successfully: {total_findings} findings"
|
||||
)
|
||||
|
||||
return {
|
||||
"sarif": sarif_report,
|
||||
"summary": summary,
|
||||
}
|
||||
@@ -16,11 +16,6 @@ tags:
|
||||
# - "copy-on-write": Download once, copy for each run (balances performance and isolation)
|
||||
workspace_isolation: "isolated"
|
||||
|
||||
default_parameters:
|
||||
target_file: null
|
||||
max_iterations: 1000000
|
||||
timeout_seconds: 1800
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -16,12 +16,6 @@ tags:
|
||||
# - "copy-on-write": Download once, copy for each run (balances performance and isolation)
|
||||
workspace_isolation: "isolated"
|
||||
|
||||
default_parameters:
|
||||
target_name: null
|
||||
max_iterations: 1000000
|
||||
timeout_seconds: 1800
|
||||
sanitizer: "address"
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -30,13 +30,5 @@ parameters:
|
||||
default: false
|
||||
description: "Scan files without Git context"
|
||||
|
||||
default_parameters:
|
||||
scan_mode: "detect"
|
||||
redact: true
|
||||
no_git: false
|
||||
|
||||
required_modules:
|
||||
- "gitleaks"
|
||||
|
||||
supported_volume_modes:
|
||||
- "ro"
|
||||
|
||||
@@ -13,38 +13,84 @@ tags:
|
||||
# Workspace isolation mode
|
||||
workspace_isolation: "shared"
|
||||
|
||||
default_parameters:
|
||||
agent_url: "http://fuzzforge-task-agent:8000/a2a/litellm_agent"
|
||||
llm_model: "gpt-5-mini"
|
||||
llm_provider: "openai"
|
||||
max_files: 5
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
agent_url:
|
||||
type: string
|
||||
description: "A2A agent endpoint URL"
|
||||
default: "http://fuzzforge-task-agent:8000/a2a/litellm_agent"
|
||||
llm_model:
|
||||
type: string
|
||||
description: "LLM model to use (e.g., gpt-4o-mini, claude-3-5-sonnet)"
|
||||
default: "gpt-5-mini"
|
||||
llm_provider:
|
||||
type: string
|
||||
description: "LLM provider (openai, anthropic, etc.)"
|
||||
default: "openai"
|
||||
file_patterns:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "File patterns to analyze (e.g., ['*.py', '*.js'])"
|
||||
default:
|
||||
- "*.py"
|
||||
- "*.js"
|
||||
- "*.ts"
|
||||
- "*.jsx"
|
||||
- "*.tsx"
|
||||
- "*.java"
|
||||
- "*.go"
|
||||
- "*.rs"
|
||||
- "*.c"
|
||||
- "*.cpp"
|
||||
- "*.h"
|
||||
- "*.hpp"
|
||||
- "*.cs"
|
||||
- "*.php"
|
||||
- "*.rb"
|
||||
- "*.swift"
|
||||
- "*.kt"
|
||||
- "*.scala"
|
||||
- "*.env"
|
||||
- "*.yaml"
|
||||
- "*.yml"
|
||||
- "*.json"
|
||||
- "*.xml"
|
||||
- "*.ini"
|
||||
- "*.sql"
|
||||
- "*.properties"
|
||||
- "*.sh"
|
||||
- "*.bat"
|
||||
- "*.ps1"
|
||||
- "*.config"
|
||||
- "*.conf"
|
||||
- "*.toml"
|
||||
- "*id_rsa*"
|
||||
- "*id_dsa*"
|
||||
- "*id_ecdsa*"
|
||||
- "*id_ed25519*"
|
||||
- "*.pem"
|
||||
- "*.key"
|
||||
- "*.pub"
|
||||
- "*.txt"
|
||||
- "*.md"
|
||||
- "Dockerfile"
|
||||
- "docker-compose.yml"
|
||||
- ".gitignore"
|
||||
- ".dockerignore"
|
||||
description: "File patterns to analyze for security issues and secrets"
|
||||
max_files:
|
||||
type: integer
|
||||
description: "Maximum number of files to analyze"
|
||||
default: 10
|
||||
max_file_size:
|
||||
type: integer
|
||||
description: "Maximum file size in bytes"
|
||||
default: 100000
|
||||
timeout:
|
||||
type: integer
|
||||
description: "Timeout per file in seconds"
|
||||
default: 90
|
||||
|
||||
output_schema:
|
||||
type: object
|
||||
|
||||
@@ -30,14 +30,42 @@ parameters:
|
||||
type: integer
|
||||
default: 20
|
||||
|
||||
default_parameters:
|
||||
agent_url: "http://fuzzforge-task-agent:8000/a2a/litellm_agent"
|
||||
llm_model: "gpt-5-mini"
|
||||
llm_provider: "openai"
|
||||
max_files: 20
|
||||
max_file_size:
|
||||
type: integer
|
||||
default: 30000
|
||||
description: "Maximum file size in bytes"
|
||||
|
||||
timeout:
|
||||
type: integer
|
||||
default: 30
|
||||
description: "Timeout per file in seconds"
|
||||
|
||||
file_patterns:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
default:
|
||||
- "*.py"
|
||||
- "*.js"
|
||||
- "*.ts"
|
||||
- "*.java"
|
||||
- "*.go"
|
||||
- "*.env"
|
||||
- "*.yaml"
|
||||
- "*.yml"
|
||||
- "*.json"
|
||||
- "*.xml"
|
||||
- "*.ini"
|
||||
- "*.sql"
|
||||
- "*.properties"
|
||||
- "*.sh"
|
||||
- "*.bat"
|
||||
- "*.config"
|
||||
- "*.conf"
|
||||
- "*.toml"
|
||||
- "*id_rsa*"
|
||||
- "*.txt"
|
||||
description: "File patterns to scan for secrets"
|
||||
|
||||
required_modules:
|
||||
- "llm_secret_detector"
|
||||
|
||||
supported_volume_modes:
|
||||
- "ro"
|
||||
|
||||
@@ -17,6 +17,7 @@ class LlmSecretDetectionWorkflow:
|
||||
llm_model: Optional[str] = None,
|
||||
llm_provider: Optional[str] = None,
|
||||
max_files: Optional[int] = None,
|
||||
max_file_size: Optional[int] = None,
|
||||
timeout: Optional[int] = None,
|
||||
file_patterns: Optional[list] = None
|
||||
) -> Dict[str, Any]:
|
||||
@@ -67,6 +68,8 @@ class LlmSecretDetectionWorkflow:
|
||||
config["llm_provider"] = llm_provider
|
||||
if max_files:
|
||||
config["max_files"] = max_files
|
||||
if max_file_size:
|
||||
config["max_file_size"] = max_file_size
|
||||
if timeout:
|
||||
config["timeout"] = timeout
|
||||
if file_patterns:
|
||||
|
||||
@@ -16,13 +16,6 @@ tags:
|
||||
# OSS-Fuzz campaigns use isolated mode for safe concurrent campaigns
|
||||
workspace_isolation: "isolated"
|
||||
|
||||
default_parameters:
|
||||
project_name: null
|
||||
campaign_duration_hours: 1
|
||||
override_engine: null
|
||||
override_sanitizer: null
|
||||
max_iterations: null
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
required:
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
@@ -0,0 +1,191 @@
|
||||
"""
|
||||
Python SAST Workflow Activities
|
||||
|
||||
Activities specific to the Python SAST workflow:
|
||||
- scan_dependencies_activity: Scan Python dependencies for CVEs using pip-audit
|
||||
- analyze_with_bandit_activity: Analyze Python code for security issues using Bandit
|
||||
- analyze_with_mypy_activity: Analyze Python code for type safety using Mypy
|
||||
- generate_python_sast_sarif_activity: Generate SARIF report from all findings
|
||||
"""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from temporalio import activity
|
||||
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Add toolbox to path for module imports
|
||||
sys.path.insert(0, '/app/toolbox')
|
||||
|
||||
|
||||
@activity.defn(name="scan_dependencies")
|
||||
async def scan_dependencies_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Scan Python dependencies for known vulnerabilities using pip-audit.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: DependencyScanner configuration
|
||||
|
||||
Returns:
|
||||
Scanner results dictionary
|
||||
"""
|
||||
logger.info(f"Activity: scan_dependencies (workspace={workspace_path})")
|
||||
|
||||
try:
|
||||
from modules.scanner import DependencyScanner
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
scanner = DependencyScanner()
|
||||
result = await scanner.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ Dependency scanning completed: "
|
||||
f"{result.summary.get('total_vulnerabilities', 0)} vulnerabilities found"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Dependency scanning failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="analyze_with_bandit")
|
||||
async def analyze_with_bandit_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Analyze Python code for security issues using Bandit.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: BanditAnalyzer configuration
|
||||
|
||||
Returns:
|
||||
Analysis results dictionary
|
||||
"""
|
||||
logger.info(f"Activity: analyze_with_bandit (workspace={workspace_path})")
|
||||
|
||||
try:
|
||||
from modules.analyzer import BanditAnalyzer
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
analyzer = BanditAnalyzer()
|
||||
result = await analyzer.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ Bandit analysis completed: "
|
||||
f"{result.summary.get('total_issues', 0)} security issues found"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Bandit analysis failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="analyze_with_mypy")
|
||||
async def analyze_with_mypy_activity(workspace_path: str, config: dict) -> dict:
|
||||
"""
|
||||
Analyze Python code for type safety issues using Mypy.
|
||||
|
||||
Args:
|
||||
workspace_path: Path to the workspace directory
|
||||
config: MypyAnalyzer configuration
|
||||
|
||||
Returns:
|
||||
Analysis results dictionary
|
||||
"""
|
||||
logger.info(f"Activity: analyze_with_mypy (workspace={workspace_path})")
|
||||
|
||||
try:
|
||||
from modules.analyzer import MypyAnalyzer
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
if not workspace.exists():
|
||||
raise FileNotFoundError(f"Workspace not found: {workspace_path}")
|
||||
|
||||
analyzer = MypyAnalyzer()
|
||||
result = await analyzer.execute(config, workspace)
|
||||
|
||||
logger.info(
|
||||
f"✓ Mypy analysis completed: "
|
||||
f"{result.summary.get('total_errors', 0)} type errors found"
|
||||
)
|
||||
return result.dict()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Mypy analysis failed: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
|
||||
@activity.defn(name="generate_python_sast_sarif")
|
||||
async def generate_python_sast_sarif_activity(
|
||||
dependency_results: dict,
|
||||
bandit_results: dict,
|
||||
mypy_results: dict,
|
||||
config: dict,
|
||||
workspace_path: str
|
||||
) -> dict:
|
||||
"""
|
||||
Generate SARIF report from all SAST analysis results.
|
||||
|
||||
Args:
|
||||
dependency_results: Results from dependency scanner
|
||||
bandit_results: Results from Bandit analyzer
|
||||
mypy_results: Results from Mypy analyzer
|
||||
config: Reporter configuration
|
||||
workspace_path: Path to the workspace
|
||||
|
||||
Returns:
|
||||
SARIF report dictionary
|
||||
"""
|
||||
logger.info("Activity: generate_python_sast_sarif")
|
||||
|
||||
try:
|
||||
from modules.reporter import SARIFReporter
|
||||
|
||||
workspace = Path(workspace_path)
|
||||
|
||||
# Combine findings from all modules
|
||||
all_findings = []
|
||||
|
||||
# Add dependency scanner findings
|
||||
dependency_findings = dependency_results.get("findings", [])
|
||||
all_findings.extend(dependency_findings)
|
||||
|
||||
# Add Bandit findings
|
||||
bandit_findings = bandit_results.get("findings", [])
|
||||
all_findings.extend(bandit_findings)
|
||||
|
||||
# Add Mypy findings
|
||||
mypy_findings = mypy_results.get("findings", [])
|
||||
all_findings.extend(mypy_findings)
|
||||
|
||||
# Prepare reporter config
|
||||
reporter_config = {
|
||||
**config,
|
||||
"findings": all_findings,
|
||||
"tool_name": "FuzzForge Python SAST",
|
||||
"tool_version": "1.0.0"
|
||||
}
|
||||
|
||||
reporter = SARIFReporter()
|
||||
result = await reporter.execute(reporter_config, workspace)
|
||||
|
||||
# Extract SARIF from result
|
||||
sarif = result.dict().get("sarif", {})
|
||||
|
||||
logger.info(f"✓ SARIF report generated with {len(all_findings)} findings")
|
||||
return sarif
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"SARIF report generation failed: {e}", exc_info=True)
|
||||
raise
|
||||
@@ -0,0 +1,110 @@
|
||||
name: python_sast
|
||||
version: "1.0.0"
|
||||
vertical: python
|
||||
description: "Python Static Application Security Testing (SAST) workflow combining dependency scanning (pip-audit), security linting (Bandit), and type checking (Mypy)"
|
||||
author: "FuzzForge Team"
|
||||
tags:
|
||||
- "python"
|
||||
- "sast"
|
||||
- "security"
|
||||
- "type-checking"
|
||||
- "dependencies"
|
||||
- "bandit"
|
||||
- "mypy"
|
||||
- "pip-audit"
|
||||
- "sarif"
|
||||
|
||||
# Workspace isolation mode (system-level configuration)
|
||||
# Using "shared" mode for read-only SAST analysis (no file modifications)
|
||||
workspace_isolation: "shared"
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
dependency_config:
|
||||
type: object
|
||||
description: "Dependency scanner (pip-audit) configuration"
|
||||
properties:
|
||||
dependency_files:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "List of dependency files to scan (auto-discovered if empty)"
|
||||
default: []
|
||||
ignore_vulns:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "List of vulnerability IDs to ignore"
|
||||
default: []
|
||||
bandit_config:
|
||||
type: object
|
||||
description: "Bandit security analyzer configuration"
|
||||
properties:
|
||||
severity_level:
|
||||
type: string
|
||||
enum: ["low", "medium", "high"]
|
||||
description: "Minimum severity level to report"
|
||||
default: "low"
|
||||
confidence_level:
|
||||
type: string
|
||||
enum: ["low", "medium", "high"]
|
||||
description: "Minimum confidence level to report"
|
||||
default: "medium"
|
||||
exclude_tests:
|
||||
type: boolean
|
||||
description: "Exclude test files from analysis"
|
||||
default: true
|
||||
skip_ids:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: "List of Bandit test IDs to skip"
|
||||
default: []
|
||||
mypy_config:
|
||||
type: object
|
||||
description: "Mypy type checker configuration"
|
||||
properties:
|
||||
strict_mode:
|
||||
type: boolean
|
||||
description: "Enable strict type checking"
|
||||
default: false
|
||||
ignore_missing_imports:
|
||||
type: boolean
|
||||
description: "Ignore errors about missing imports"
|
||||
default: true
|
||||
follow_imports:
|
||||
type: string
|
||||
enum: ["normal", "silent", "skip", "error"]
|
||||
description: "How to handle imports"
|
||||
default: "silent"
|
||||
reporter_config:
|
||||
type: object
|
||||
description: "SARIF reporter configuration"
|
||||
properties:
|
||||
include_code_flows:
|
||||
type: boolean
|
||||
description: "Include code flow information"
|
||||
default: false
|
||||
|
||||
output_schema:
|
||||
type: object
|
||||
properties:
|
||||
sarif:
|
||||
type: object
|
||||
description: "SARIF-formatted SAST findings from all tools"
|
||||
summary:
|
||||
type: object
|
||||
description: "SAST execution summary"
|
||||
properties:
|
||||
total_findings:
|
||||
type: integer
|
||||
vulnerabilities:
|
||||
type: integer
|
||||
description: "CVEs found in dependencies"
|
||||
security_issues:
|
||||
type: integer
|
||||
description: "Security issues found by Bandit"
|
||||
type_errors:
|
||||
type: integer
|
||||
description: "Type errors found by Mypy"
|
||||
@@ -0,0 +1,265 @@
|
||||
"""
|
||||
Python SAST Workflow - Temporal Version
|
||||
|
||||
Static Application Security Testing for Python projects using multiple tools.
|
||||
"""
|
||||
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from datetime import timedelta
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
from temporalio import workflow
|
||||
from temporalio.common import RetryPolicy
|
||||
|
||||
# Import activity interfaces (will be executed by worker)
|
||||
with workflow.unsafe.imports_passed_through():
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class PythonSastWorkflow:
|
||||
"""
|
||||
Python Static Application Security Testing workflow.
|
||||
|
||||
This workflow:
|
||||
1. Downloads target from MinIO
|
||||
2. Runs dependency scanning (pip-audit for CVEs)
|
||||
3. Runs security linting (Bandit for security issues)
|
||||
4. Runs type checking (Mypy for type safety)
|
||||
5. Generates a SARIF report with all findings
|
||||
6. Uploads results to MinIO
|
||||
7. Cleans up cache
|
||||
"""
|
||||
|
||||
@workflow.run
|
||||
async def run(
|
||||
self,
|
||||
target_id: str,
|
||||
dependency_config: Optional[Dict[str, Any]] = None,
|
||||
bandit_config: Optional[Dict[str, Any]] = None,
|
||||
mypy_config: Optional[Dict[str, Any]] = None,
|
||||
reporter_config: Optional[Dict[str, Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Main workflow execution.
|
||||
|
||||
Args:
|
||||
target_id: UUID of the uploaded target in MinIO
|
||||
dependency_config: Configuration for dependency scanner
|
||||
bandit_config: Configuration for Bandit analyzer
|
||||
mypy_config: Configuration for Mypy analyzer
|
||||
reporter_config: Configuration for SARIF reporter
|
||||
|
||||
Returns:
|
||||
Dictionary containing SARIF report and summary
|
||||
"""
|
||||
workflow_id = workflow.info().workflow_id
|
||||
|
||||
workflow.logger.info(
|
||||
f"Starting PythonSASTWorkflow "
|
||||
f"(workflow_id={workflow_id}, target_id={target_id})"
|
||||
)
|
||||
|
||||
# Default configurations
|
||||
if not dependency_config:
|
||||
dependency_config = {
|
||||
"dependency_files": [], # Auto-discover
|
||||
"ignore_vulns": []
|
||||
}
|
||||
|
||||
if not bandit_config:
|
||||
bandit_config = {
|
||||
"severity_level": "low",
|
||||
"confidence_level": "medium",
|
||||
"exclude_tests": True,
|
||||
"skip_ids": []
|
||||
}
|
||||
|
||||
if not mypy_config:
|
||||
mypy_config = {
|
||||
"strict_mode": False,
|
||||
"ignore_missing_imports": True,
|
||||
"follow_imports": "silent"
|
||||
}
|
||||
|
||||
if not reporter_config:
|
||||
reporter_config = {
|
||||
"include_code_flows": False
|
||||
}
|
||||
|
||||
results = {
|
||||
"workflow_id": workflow_id,
|
||||
"target_id": target_id,
|
||||
"status": "running",
|
||||
"steps": []
|
||||
}
|
||||
|
||||
try:
|
||||
# Get run ID for workspace isolation (using shared mode for read-only analysis)
|
||||
run_id = workflow.info().run_id
|
||||
|
||||
# Step 1: Download target from MinIO
|
||||
workflow.logger.info("Step 1: Downloading target from MinIO")
|
||||
target_path = await workflow.execute_activity(
|
||||
"get_target",
|
||||
args=[target_id, run_id, "shared"], # target_id, run_id, workspace_isolation
|
||||
start_to_close_timeout=timedelta(minutes=5),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=1),
|
||||
maximum_interval=timedelta(seconds=30),
|
||||
maximum_attempts=3
|
||||
)
|
||||
)
|
||||
results["steps"].append({
|
||||
"step": "download_target",
|
||||
"status": "success",
|
||||
"target_path": target_path
|
||||
})
|
||||
workflow.logger.info(f"✓ Target downloaded to: {target_path}")
|
||||
|
||||
# Step 2: Dependency scanning (pip-audit)
|
||||
workflow.logger.info("Step 2: Scanning dependencies for vulnerabilities")
|
||||
dependency_results = await workflow.execute_activity(
|
||||
"scan_dependencies",
|
||||
args=[target_path, dependency_config],
|
||||
start_to_close_timeout=timedelta(minutes=10),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=2),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=2
|
||||
)
|
||||
)
|
||||
results["steps"].append({
|
||||
"step": "dependency_scanning",
|
||||
"status": "success",
|
||||
"vulnerabilities": dependency_results.get("summary", {}).get("total_vulnerabilities", 0)
|
||||
})
|
||||
workflow.logger.info(
|
||||
f"✓ Dependency scanning completed: "
|
||||
f"{dependency_results.get('summary', {}).get('total_vulnerabilities', 0)} vulnerabilities"
|
||||
)
|
||||
|
||||
# Step 3: Security linting (Bandit)
|
||||
workflow.logger.info("Step 3: Analyzing security issues with Bandit")
|
||||
bandit_results = await workflow.execute_activity(
|
||||
"analyze_with_bandit",
|
||||
args=[target_path, bandit_config],
|
||||
start_to_close_timeout=timedelta(minutes=10),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=2),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=2
|
||||
)
|
||||
)
|
||||
results["steps"].append({
|
||||
"step": "bandit_analysis",
|
||||
"status": "success",
|
||||
"issues": bandit_results.get("summary", {}).get("total_issues", 0)
|
||||
})
|
||||
workflow.logger.info(
|
||||
f"✓ Bandit analysis completed: "
|
||||
f"{bandit_results.get('summary', {}).get('total_issues', 0)} security issues"
|
||||
)
|
||||
|
||||
# Step 4: Type checking (Mypy)
|
||||
workflow.logger.info("Step 4: Type checking with Mypy")
|
||||
mypy_results = await workflow.execute_activity(
|
||||
"analyze_with_mypy",
|
||||
args=[target_path, mypy_config],
|
||||
start_to_close_timeout=timedelta(minutes=10),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=2),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=2
|
||||
)
|
||||
)
|
||||
results["steps"].append({
|
||||
"step": "mypy_analysis",
|
||||
"status": "success",
|
||||
"type_errors": mypy_results.get("summary", {}).get("total_errors", 0)
|
||||
})
|
||||
workflow.logger.info(
|
||||
f"✓ Mypy analysis completed: "
|
||||
f"{mypy_results.get('summary', {}).get('total_errors', 0)} type errors"
|
||||
)
|
||||
|
||||
# Step 5: Generate SARIF report
|
||||
workflow.logger.info("Step 5: Generating SARIF report")
|
||||
sarif_report = await workflow.execute_activity(
|
||||
"generate_python_sast_sarif",
|
||||
args=[dependency_results, bandit_results, mypy_results, reporter_config, target_path],
|
||||
start_to_close_timeout=timedelta(minutes=5)
|
||||
)
|
||||
results["steps"].append({
|
||||
"step": "report_generation",
|
||||
"status": "success"
|
||||
})
|
||||
|
||||
# Count total findings in SARIF
|
||||
total_findings = 0
|
||||
if sarif_report and "runs" in sarif_report:
|
||||
total_findings = len(sarif_report["runs"][0].get("results", []))
|
||||
|
||||
workflow.logger.info(f"✓ SARIF report generated with {total_findings} findings")
|
||||
|
||||
# Step 6: Upload results to MinIO
|
||||
workflow.logger.info("Step 6: Uploading results")
|
||||
try:
|
||||
results_url = await workflow.execute_activity(
|
||||
"upload_results",
|
||||
args=[workflow_id, sarif_report, "sarif"],
|
||||
start_to_close_timeout=timedelta(minutes=2)
|
||||
)
|
||||
results["results_url"] = results_url
|
||||
workflow.logger.info(f"✓ Results uploaded to: {results_url}")
|
||||
except Exception as e:
|
||||
workflow.logger.warning(f"Failed to upload results: {e}")
|
||||
results["results_url"] = None
|
||||
|
||||
# Step 7: Cleanup cache
|
||||
workflow.logger.info("Step 7: Cleaning up cache")
|
||||
try:
|
||||
await workflow.execute_activity(
|
||||
"cleanup_cache",
|
||||
args=[target_path, "shared"], # target_path, workspace_isolation
|
||||
start_to_close_timeout=timedelta(minutes=1)
|
||||
)
|
||||
workflow.logger.info("✓ Cache cleaned up (skipped for shared mode)")
|
||||
except Exception as e:
|
||||
workflow.logger.warning(f"Cache cleanup failed: {e}")
|
||||
|
||||
# Mark workflow as successful
|
||||
results["status"] = "success"
|
||||
results["sarif"] = sarif_report
|
||||
results["summary"] = {
|
||||
"total_findings": total_findings,
|
||||
"vulnerabilities": dependency_results.get("summary", {}).get("total_vulnerabilities", 0),
|
||||
"security_issues": bandit_results.get("summary", {}).get("total_issues", 0),
|
||||
"type_errors": mypy_results.get("summary", {}).get("total_errors", 0)
|
||||
}
|
||||
workflow.logger.info(f"✓ Workflow completed successfully: {workflow_id}")
|
||||
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
workflow.logger.error(f"Workflow failed: {e}")
|
||||
results["status"] = "error"
|
||||
results["error"] = str(e)
|
||||
results["steps"].append({
|
||||
"step": "error",
|
||||
"status": "failed",
|
||||
"error": str(e)
|
||||
})
|
||||
raise
|
||||
@@ -18,11 +18,6 @@ tags:
|
||||
# Using "shared" mode for read-only security analysis (no file modifications)
|
||||
workspace_isolation: "shared"
|
||||
|
||||
default_parameters:
|
||||
scanner_config: {}
|
||||
analyzer_config: {}
|
||||
reporter_config: {}
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -23,12 +23,5 @@ parameters:
|
||||
default: 10
|
||||
description: "Maximum directory depth to scan"
|
||||
|
||||
default_parameters:
|
||||
verify: true
|
||||
max_depth: 10
|
||||
|
||||
required_modules:
|
||||
- "trufflehog"
|
||||
|
||||
supported_volume_modes:
|
||||
- "ro"
|
||||
|
||||
Reference in New Issue
Block a user