diff --git a/README.md b/README.md index e0857c0..25fdc33 100644 --- a/README.md +++ b/README.md @@ -1,664 +1,429 @@ -# NeuroSploit v2 +# NeuroSploit v3 -![NeuroSploitv2](https://img.shields.io/badge/NeuroSploitv2-AI--Powered%20Pentesting-blueviolet) -![Version](https://img.shields.io/badge/Version-2.0.0-blue) +![NeuroSploit](https://img.shields.io/badge/NeuroSploit-AI--Powered%20Pentesting-blueviolet) +![Version](https://img.shields.io/badge/Version-3.0.0-blue) ![License](https://img.shields.io/badge/License-MIT-green) -![Python](https://img.shields.io/badge/Python-3.8+-yellow) +![Python](https://img.shields.io/badge/Python-3.10+-yellow) +![React](https://img.shields.io/badge/React-18-61dafb) -**AI-Powered Penetration Testing Framework with Adaptive Intelligence** +**AI-Powered Penetration Testing Platform with Web GUI** -NeuroSploit v2 is an advanced security assessment framework that combines reconnaissance tools with adaptive AI analysis. It intelligently collects data, analyzes attack surfaces, and performs targeted security testing using LLM-powered decision making. +NeuroSploit v3 is an advanced security assessment platform that combines AI-driven vulnerability testing with a modern web interface. It uses prompt-driven testing to dynamically determine what vulnerabilities to test based on natural language instructions. --- -## What's New in v2 +## What's New in v3 -- **Adaptive AI Mode** - AI automatically determines if context is sufficient; runs tools only when needed -- **3 Execution Modes** - CLI, Interactive, and guided Experience/Wizard mode -- **Consolidated Recon** - All reconnaissance outputs merged into a single context file -- **Context-Based Analysis** - Analyze pre-collected recon data without re-running tools -- **Professional Reports** - Auto-generated HTML reports with charts and findings +- **Web GUI** - Modern React interface for scan management, real-time monitoring, and reports +- **Dynamic Vulnerability Engine** - Tests 50+ vulnerability types based on prompt analysis +- **Prompt-Driven Testing** - AI extracts vulnerability types from natural language prompts +- **Real-time Dashboard** - WebSocket-powered live updates during scans +- **Multiple Input Modes** - Single URL, comma-separated URLs, or file upload +- **Preset Prompts** - Ready-to-use security testing profiles +- **Export Reports** - HTML, PDF, and JSON export formats +- **Docker Deployment** - One-command deployment with Docker Compose --- ## Table of Contents -- [Features](#features) -- [Installation](#installation) - [Quick Start](#quick-start) -- [3 Execution Modes](#3-execution-modes) -- [Workflow](#workflow) -- [Adaptive AI Mode](#adaptive-ai-mode) -- [Configuration](#configuration) -- [CLI Reference](#cli-reference) -- [Agent Roles](#agent-roles) -- [Built-in Tools](#built-in-tools) -- [Output Files](#output-files) -- [Examples](#examples) +- [Features](#features) - [Architecture](#architecture) +- [Web GUI](#web-gui) +- [API Reference](#api-reference) +- [Vulnerability Engine](#vulnerability-engine) +- [Configuration](#configuration) +- [Development](#development) - [Security Notice](#security-notice) --- +## Quick Start + +### Option 1: Docker (Recommended) + +```bash +# Clone repository +git clone https://github.com/your-org/NeuroSploitv2.git +cd NeuroSploitv2 + +# Copy environment file and add your API keys +cp .env.example .env +nano .env # Add ANTHROPIC_API_KEY or OPENAI_API_KEY + +# Start with Docker Compose +./start.sh +# or +docker-compose up -d +``` + +Access the web interface at **http://localhost:3000** + +### Option 2: Manual Setup + +```bash +# Backend +cd backend +python3 -m venv venv +source venv/bin/activate +pip install -r requirements.txt +uvicorn backend.main:app --host 0.0.0.0 --port 8000 + +# Frontend (new terminal) +cd frontend +npm install +npm run dev +``` + +--- + ## Features ### Core Capabilities | Feature | Description | |---------|-------------| -| **Adaptive AI** | Automatically runs tools when context is insufficient | -| **Multi-Mode** | CLI, Interactive, and Wizard execution modes | -| **Consolidated Recon** | All tool outputs merged into single context file | -| **Multi-LLM Support** | Claude, OpenAI, Gemini, Ollama, LM Studio | -| **Professional Reports** | HTML reports with charts and findings | -| **Extensible** | Custom agents, tools, and prompts | - -### Security Testing - -| Category | Tests | -|----------|-------| -| **Injection** | SQL Injection, XSS, Command Injection, Template Injection | -| **File Attacks** | LFI, Path Traversal, File Upload, XXE | -| **Server-Side** | SSRF, RCE, Deserialization | -| **Authentication** | Auth Bypass, IDOR, Session Issues, JWT | -| **Reconnaissance** | Subdomain Enum, Port Scan, Tech Detection, URL Collection | - -### Reconnaissance Tools - -| Tool | Purpose | -|------|---------| -| subfinder, amass, assetfinder | Subdomain enumeration | -| httpx, httprobe | HTTP probing | -| gau, waybackurls, waymore | URL collection | -| katana, gospider | Web crawling | -| naabu, nmap | Port scanning | -| nuclei | Vulnerability scanning | - ---- - -## Installation - -### Prerequisites - -```bash -# Python 3.8+ -python3 --version - -# Install dependencies -pip3 install -r requirements.txt -``` - -### Setup - -```bash -# Clone repository -git clone https://github.com/CyberSecurityUP/NeuroSploit -cd NeuroSploitv2 - -# Create config from example -cp config/config-example.json config/config.json - -# Edit with your LLM API keys -nano config/config.json - -# Create required directories -mkdir -p results reports logs - -# Install security tools (recommended) -python3 neurosploit.py --install-tools -``` - -### Environment Variables - -```bash -# Set in .bashrc, .zshrc, or .env -export ANTHROPIC_API_KEY="your_key" -export OPENAI_API_KEY="your_key" -export GEMINI_API_KEY="your_key" -``` - ---- - -## Quick Start - -### Option 1: Wizard Mode (Recommended for beginners) - -```bash -python3 neurosploit.py -e -``` - -Follow the guided prompts to configure your scan. - -### Option 2: Two-Step Workflow - -```bash -# Step 1: Run reconnaissance -python3 neurosploit.py --recon example.com - -# Step 2: AI analysis -python3 neurosploit.py --input "Find XSS and SQLi vulnerabilities" \ - -cf results/context_*.json \ - --llm-profile claude_opus_default -``` - -### Option 3: Interactive Mode - -```bash -python3 neurosploit.py -i -``` - ---- - -## 3 Execution Modes - -### 1. CLI Mode - -Direct command-line execution with all parameters: - -```bash -# Reconnaissance -python3 neurosploit.py --recon example.com - -# AI Analysis with context -python3 neurosploit.py --input "Analyze for XSS and SQLi" \ - -cf results/context_X.json \ - --llm-profile claude_opus_default - -# Full pentest scan -python3 neurosploit.py --scan https://example.com - -# Quick scan -python3 neurosploit.py --quick-scan https://example.com -``` - -### 2. Interactive Mode (`-i`) - -REPL interface with tab completion: - -```bash -python3 neurosploit.py -i -``` - -``` - ╔═══════════════════════════════════════════════════════════╗ - ║ NeuroSploitv2 - AI Offensive Security ║ - ║ Interactive Mode ║ - ╚═══════════════════════════════════════════════════════════╝ - -NeuroSploit> help -NeuroSploit> recon example.com -NeuroSploit> analyze results/context_X.json -NeuroSploit> scan https://example.com -NeuroSploit> experience -NeuroSploit> exit -``` - -**Available Commands:** - -| Command | Description | -|---------|-------------| -| `recon ` | Run full reconnaissance | -| `analyze ` | LLM analysis of context file | -| `scan ` | Full pentest with tools | -| `quick_scan ` | Fast essential checks | -| `experience` / `wizard` | Start guided setup | -| `set_agent ` | Set default agent role | -| `set_profile ` | Set LLM profile | -| `list_roles` | Show available agents | -| `list_profiles` | Show LLM profiles | -| `check_tools` | Check installed tools | -| `install_tools` | Install required tools | -| `discover_ollama` | Find local Ollama models | - -### 3. Experience/Wizard Mode (`-e`) - -Guided step-by-step configuration: - -```bash -python3 neurosploit.py -e -``` - -``` - ╔═══════════════════════════════════════════════════════════╗ - ║ NEUROSPLOIT - EXPERIENCE MODE (WIZARD) ║ - ║ Step-by-step Configuration ║ - ╚═══════════════════════════════════════════════════════════╝ - -[STEP 1/6] Choose Operation Mode --------------------------------------------------- - 1. AI Analysis - Analyze recon context with LLM (no tools) - 2. Full Scan - Run real pentest tools + AI analysis - 3. Quick Scan - Fast essential checks + AI analysis - 4. Recon Only - Run reconnaissance tools, save context - -[STEP 2/6] Set Target -[STEP 3/6] Context File -[STEP 4/6] LLM Profile -[STEP 5/6] Agent Role -[STEP 6/6] Custom Prompt - -============================================================ - CONFIGURATION SUMMARY -============================================================ - Mode: analysis - Target: example.com - Context File: results/context_20240115.json - LLM Profile: claude_opus_default - Agent Role: bug_bounty_hunter - Prompt: Find XSS and SQLi vulnerabilities... -============================================================ - - Execute with this configuration? [Y/n]: -``` - ---- - -## Workflow - -### Recommended Workflow - -``` -┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ -│ STEP 1 │ │ STEP 2 │ │ STEP 3 │ -│ RECON │────▶│ AI ANALYSIS │────▶│ REPORT │ -│ │ │ │ │ │ -│ - Subdomains │ │ - Adaptive AI │ │ - HTML Report │ -│ - URLs │ │ - Auto-test │ │ - JSON Results │ -│ - Ports │ │ - if needed │ │ - Findings │ -│ - Technologies │ │ │ │ │ -└─────────────────┘ └─────────────────┘ └─────────────────┘ -``` - -### Step 1: Reconnaissance - -```bash -python3 neurosploit.py --recon example.com -``` - -Runs all discovery tools and consolidates output: - -- **Subdomain Enumeration**: subfinder, amass, assetfinder -- **HTTP Probing**: httpx, httprobe -- **URL Collection**: gau, waybackurls, waymore -- **Web Crawling**: katana, gospider -- **Port Scanning**: naabu, nmap -- **Vulnerability Scanning**: nuclei - -**Output:** `results/context_YYYYMMDD_HHMMSS.json` - -### Step 2: AI Analysis - -```bash -python3 neurosploit.py --input "Test for SQL injection and XSS" \ - -cf results/context_X.json \ - --llm-profile claude_opus_default -``` - -The Adaptive AI: -1. Analyzes your request -2. Checks if context has sufficient data -3. Runs additional tests if needed -4. Provides comprehensive analysis - ---- - -## Adaptive AI Mode - -The AI automatically determines if context data is sufficient: - -``` -====================================================================== - NEUROSPLOIT ADAPTIVE AI - BUG_BOUNTY_HUNTER -====================================================================== - Mode: Adaptive (LLM + Tools when needed) - Target: testphp.vulnweb.com - Context loaded with: - - Subdomains: 1 - - URLs: 12085 - - URLs with params: 10989 -====================================================================== - -[PHASE 1] Analyzing Context Sufficiency --------------------------------------------------- - [*] User wants: xss, sqli - [*] Data sufficient: No - [*] Missing: XSS test results, SQL injection evidence - -[PHASE 2] Collecting Missing Data --------------------------------------------------- - [!] Context insufficient for: XSS test results - [*] Running tools to collect data... - - [XSS] Running XSS tests... - [>] curl: -s -k "http://target.com/search?q=%3Cscript%3Ealert(1)%3C/script%3E" - [!] FOUND: XSS - - [SQLi] Running SQL Injection tests... - [>] curl: -s -k "http://target.com/product?id=1'" - [!] FOUND: SQL Injection - - [+] Ran 15 tool commands to fill context gaps - -[PHASE 3] AI Analysis --------------------------------------------------- - [*] Generating final analysis with collected evidence... - [+] Analysis complete -``` - -### How It Works - -| Scenario | AI Action | -|----------|-----------| -| Context has XSS evidence | LLM-only analysis (no tools) | -| Context missing XSS evidence | Run XSS tests, then analyze | -| User asks for port scan | Check context, run nmap if missing | -| General analysis request | Use available context data | - -### Supported Auto-Tests - -When context is insufficient, AI can automatically run: - -| Test | Trigger Keywords | -|------|------------------| -| XSS | xss, cross-site, reflected, stored | -| SQLi | sqli, sql, injection, database | -| LFI | lfi, file, inclusion, traversal | -| SSRF | ssrf, server-side, request | -| RCE | rce, command, execution, shell | -| Crawl | crawl, discover, spider, urls | -| Port Scan | port, scan, nmap, service | - ---- - -## Configuration - -### config/config.json - -```json -{ - "llm": { - "default_profile": "claude_opus_default", - "profiles": { - "claude_opus_default": { - "provider": "claude", - "model": "claude-sonnet-4-20250514", - "api_key": "${ANTHROPIC_API_KEY}", - "temperature": 0.7, - "max_tokens": 8192, - "guardrails_enabled": true, - "hallucination_mitigation_strategy": "grounding" - }, - "ollama_local": { - "provider": "ollama", - "model": "llama3:8b", - "api_key": "", - "temperature": 0.7 - }, - "gpt_4o": { - "provider": "gpt", - "model": "gpt-4o", - "api_key": "${OPENAI_API_KEY}", - "temperature": 0.7 - } - } - }, - "agent_roles": { - "bug_bounty_hunter": { - "enabled": true, - "description": "Aggressive bug bounty hunting", - "llm_profile": "claude_opus_default", - "tools_allowed": ["subfinder", "nuclei", "sqlmap"] - }, - "red_team_agent": { - "enabled": true, - "description": "Red team operations specialist" - } - }, - "tools": { - "nmap": "/usr/bin/nmap", - "sqlmap": "/usr/bin/sqlmap", - "nuclei": "/usr/local/bin/nuclei" - } -} -``` - -### LLM Providers - -| Provider | Config Value | Notes | -|----------|--------------|-------| -| Claude (Anthropic) | `"provider": "claude"` | Best for security analysis | -| OpenAI | `"provider": "gpt"` | GPT-4, GPT-4o | -| Google | `"provider": "gemini"` | Gemini Pro | -| Ollama | `"provider": "ollama"` | Local models | -| LM Studio | `"provider": "lmstudio"` | Local with OpenAI API | - ---- - -## CLI Reference - -``` -usage: neurosploit.py [-h] [--recon TARGET] [--context-file FILE] - [--target TARGET] [--scan TARGET] [--quick-scan TARGET] - [--install-tools] [--check-tools] [-r AGENT_ROLE] [-i] - [-e] [--input INPUT] [--llm-profile LLM_PROFILE] - -NeuroSploitv2 - AI-Powered Penetration Testing Framework - -Arguments: - --recon TARGET Run FULL RECON on target - --context-file, -cf Load recon context from JSON file - --target, -t Specify target URL/domain - --scan TARGET Run FULL pentest scan with tools - --quick-scan TARGET Run QUICK pentest scan - --install-tools Install required security tools - --check-tools Check status of installed tools - -r, --agent-role Agent role to execute (optional) - -i, --interactive Start interactive mode - -e, --experience Start wizard mode (guided setup) - --input Input prompt for the AI agent - --llm-profile LLM profile to use - --list-agents List available agent roles - --list-profiles List LLM profiles - -v, --verbose Enable verbose output -``` - ---- - -## Agent Roles - -Predefined agents in `config.json` with prompts in `prompts/`: - -| Agent | Description | -|-------|-------------| -| `bug_bounty_hunter` | Web app vulnerabilities, high-impact findings | -| `red_team_agent` | Simulated attack campaigns | -| `blue_team_agent` | Threat detection and response | -| `exploit_expert` | Exploitation strategies and payloads | -| `pentest_generalist` | Broad penetration testing | -| `owasp_expert` | OWASP Top 10 assessment | -| `malware_analyst` | Malware examination and IOCs | - -### Custom Agents - -1. Create prompt file: `prompts/my_agent.md` -2. Add to config: - -```json -"agent_roles": { - "my_agent": { - "enabled": true, - "description": "My custom agent", - "llm_profile": "claude_opus_default" - } -} -``` - ---- - -## Built-in Tools - -### Reconnaissance - -| Tool | File | Features | -|------|------|----------| -| OSINT Collector | `tools/recon/osint_collector.py` | IP resolution, tech detection, email patterns | -| Subdomain Finder | `tools/recon/subdomain_finder.py` | CT logs, DNS brute-force | -| DNS Enumerator | `tools/recon/dns_enumerator.py` | A, AAAA, MX, NS, TXT, CNAME | -| Full Recon Runner | `tools/recon/recon_tools.py` | Orchestrates all recon tools | - -### Post-Exploitation - -| Tool | File | Features | -|------|------|----------| -| SMB Lateral | `tools/lateral_movement/smb_lateral.py` | Share enum, pass-the-hash | -| SSH Lateral | `tools/lateral_movement/ssh_lateral.py` | SSH tunnels, key enum | -| Cron Persistence | `tools/persistence/cron_persistence.py` | Linux persistence | -| Registry Persistence | `tools/persistence/registry_persistence.py` | Windows persistence | - ---- - -## Output Files - -| File | Location | Description | -|------|----------|-------------| -| Context JSON | `results/context_*.json` | Consolidated recon data | -| Context TXT | `results/context_*.txt` | Human-readable context | -| Campaign JSON | `results/campaign_*.json` | Full execution results | -| HTML Report | `reports/report_*.html` | Professional report with charts | - -### HTML Report Features - -- Executive summary -- Severity statistics with charts -- Risk score calculation -- Vulnerability details with PoCs -- Remediation recommendations -- Modern dark theme UI - ---- - -## Examples - -### Basic Recon - -```bash -# Domain recon -python3 neurosploit.py --recon example.com - -# URL recon -python3 neurosploit.py --recon https://example.com -``` - -### AI Analysis - -```bash -# Specific vulnerability analysis -python3 neurosploit.py --input "Find SQL injection and XSS vulnerabilities. Provide PoC with CVSS scores." \ - -cf results/context_20240115.json \ - --llm-profile claude_opus_default - -# Comprehensive assessment -python3 neurosploit.py --input "Perform comprehensive security assessment. Analyze attack surface, test for OWASP Top 10, prioritize critical findings." \ - -cf results/context_X.json -``` - -### Pentest Scan - -```bash -# Full scan with context -python3 neurosploit.py --scan https://example.com -cf results/context_X.json - -# Quick scan -python3 neurosploit.py --quick-scan https://example.com -r bug_bounty_hunter -``` - -### Wizard Mode - -```bash -python3 neurosploit.py -e -# Follow interactive prompts... -``` +| **Dynamic Testing** | 50+ vulnerability types across 10 categories | +| **Prompt-Driven** | AI extracts test types from natural language | +| **Web Interface** | Modern React dashboard with real-time updates | +| **Multiple Inputs** | Single URL, bulk URLs, or file upload | +| **Preset Prompts** | Bug Bounty, OWASP Top 10, API Security, and more | +| **Export Reports** | HTML, PDF, JSON with professional styling | +| **WebSocket Updates** | Real-time scan progress and findings | +| **Docker Ready** | One-command deployment | + +### Vulnerability Categories + +| Category | Vulnerability Types | +|----------|---------------------| +| **Injection** | XSS (Reflected/Stored/DOM), SQLi, NoSQLi, Command Injection, SSTI, LDAP, XPath | +| **File Access** | LFI, RFI, Path Traversal, File Upload, XXE | +| **Request Forgery** | SSRF, CSRF, Cloud Metadata Access | +| **Authentication** | Auth Bypass, JWT Manipulation, Session Fixation, OAuth Flaws | +| **Authorization** | IDOR, BOLA, BFLA, Privilege Escalation | +| **API Security** | Rate Limiting, Mass Assignment, GraphQL Injection | +| **Logic Flaws** | Race Conditions, Business Logic, Workflow Bypass | +| **Client-Side** | CORS Misconfiguration, Clickjacking, Open Redirect, WebSocket | +| **Info Disclosure** | Error Disclosure, Source Code Exposure, Debug Endpoints | +| **Infrastructure** | Security Headers, SSL/TLS Issues, HTTP Methods | --- ## Architecture ``` -NeuroSploitv2/ -├── neurosploit.py # Main entry point -├── config/ -│ ├── config.json # Configuration -│ └── config-example.json # Example config -├── agents/ -│ └── base_agent.py # Adaptive AI agent -├── core/ -│ ├── llm_manager.py # LLM provider abstraction -│ ├── context_builder.py # Recon consolidation -│ ├── pentest_executor.py # Tool execution -│ ├── report_generator.py # Report generation -│ └── tool_installer.py # Tool installation -├── tools/ -│ ├── recon/ -│ │ ├── recon_tools.py # Advanced recon -│ │ ├── osint_collector.py # OSINT gathering -│ │ ├── subdomain_finder.py # Subdomain enum -│ │ └── dns_enumerator.py # DNS enumeration -│ ├── lateral_movement/ -│ │ ├── smb_lateral.py # SMB techniques -│ │ └── ssh_lateral.py # SSH techniques -│ └── persistence/ -│ ├── cron_persistence.py # Linux persistence -│ └── registry_persistence.py # Windows persistence -├── prompts/ -│ ├── library.json # Prompt library -│ └── *.md # Agent prompts -├── results/ # Output directory -├── reports/ # Generated reports -└── logs/ # Log files +NeuroSploitv3/ +├── backend/ # FastAPI Backend +│ ├── api/v1/ # REST API endpoints +│ │ ├── scans.py # Scan CRUD operations +│ │ ├── targets.py # Target validation +│ │ ├── prompts.py # Preset prompts +│ │ ├── reports.py # Report generation +│ │ ├── dashboard.py # Dashboard stats +│ │ └── vulnerabilities.py # Vulnerability management +│ ├── core/ +│ │ ├── vuln_engine/ # Dynamic vulnerability testing +│ │ │ ├── engine.py # Main testing engine +│ │ │ ├── registry.py # Vulnerability registry +│ │ │ ├── payload_generator.py +│ │ │ └── testers/ # Category-specific testers +│ │ ├── prompt_engine/ # Prompt parsing +│ │ │ └── parser.py # Extract vuln types from prompts +│ │ └── report_engine/ # Report generation +│ │ └── generator.py # HTML/PDF/JSON export +│ ├── models/ # SQLAlchemy ORM models +│ ├── schemas/ # Pydantic validation schemas +│ ├── services/ # Business logic +│ └── main.py # FastAPI app entry +│ +├── frontend/ # React Frontend +│ ├── src/ +│ │ ├── pages/ # Page components +│ │ │ ├── HomePage.tsx # Dashboard +│ │ │ ├── NewScanPage.tsx # Create scan +│ │ │ ├── ScanDetailsPage.tsx +│ │ │ ├── ReportsPage.tsx +│ │ │ └── ReportViewPage.tsx +│ │ ├── components/ # Reusable components +│ │ ├── services/ # API client +│ │ └── store/ # Zustand state +│ └── package.json +│ +├── docker/ # Docker configuration +│ ├── Dockerfile.backend +│ ├── Dockerfile.frontend +│ └── nginx.conf +│ +├── docker-compose.yml +├── start.sh +└── .env.example ``` --- -## Security Features +## Web GUI -- **Secure Tool Execution**: `shlex` parsing, no shell injection -- **Input Validation**: Tool paths and arguments validated -- **Timeout Protection**: 60-second default timeout -- **Permission System**: Agent-based tool access control -- **Error Handling**: Comprehensive logging +### Dashboard (Home Page) + +- **Stats Overview** - Total scans, vulnerabilities by severity, success rate +- **Severity Distribution** - Visual chart of critical/high/medium/low findings +- **Recent Scans** - Quick access to latest scan results +- **Recent Findings** - Latest discovered vulnerabilities + +### New Scan Page + +**Target Input Modes:** +- **Single URL** - Enter one target URL +- **Multiple URLs** - Comma-separated list +- **File Upload** - Upload .txt file with URLs (one per line) + +**Prompt Options:** +- **Preset Prompts** - Select from ready-to-use profiles: + - Full Penetration Test + - OWASP Top 10 + - API Security Assessment + - Bug Bounty Hunter + - Quick Security Scan + - Authentication Testing +- **Custom Prompt** - Write your own testing instructions +- **No Prompt** - Run all available tests + +### Scan Details Page + +- **Progress Bar** - Real-time scan progress +- **Discovered Endpoints** - List of found paths and URLs +- **Vulnerabilities** - Real-time findings with severity badges +- **Activity Log** - Live scan events via WebSocket + +### Reports Page + +- **Report List** - All generated reports with metadata +- **View Report** - In-browser HTML viewer +- **Export Options** - Download as HTML, PDF, or JSON +- **Delete Reports** - Remove old reports --- -## Troubleshooting +## API Reference -### LLM Connection Issues +### Base URL -```bash -# Check API key -echo $ANTHROPIC_API_KEY - -# Test with local Ollama -python3 neurosploit.py -i -NeuroSploit> discover_ollama +``` +http://localhost:8000/api/v1 ``` -### Missing Tools +### Endpoints -```bash -# Check status -python3 neurosploit.py --check-tools +#### Scans -# Install -python3 neurosploit.py --install-tools +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/scans` | Create new scan | +| `GET` | `/scans` | List all scans | +| `GET` | `/scans/{id}` | Get scan details | +| `POST` | `/scans/{id}/start` | Start scan execution | +| `POST` | `/scans/{id}/stop` | Stop running scan | +| `DELETE` | `/scans/{id}` | Delete scan | +| `GET` | `/scans/{id}/endpoints` | Get discovered endpoints | +| `GET` | `/scans/{id}/vulnerabilities` | Get found vulnerabilities | + +#### Targets + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/targets/validate` | Validate URL(s) | +| `POST` | `/targets/upload` | Upload URL file | + +#### Prompts + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/prompts/presets` | List preset prompts | +| `GET` | `/prompts/presets/{id}` | Get preset details | +| `POST` | `/prompts/parse` | Parse custom prompt | + +#### Reports + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/reports` | List all reports | +| `GET` | `/reports/{id}` | Get report details | +| `GET` | `/reports/{id}/download` | Download report | +| `DELETE` | `/reports/{id}` | Delete report | + +#### Dashboard + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/dashboard/stats` | Get dashboard statistics | +| `GET` | `/dashboard/recent-scans` | Get recent scans | +| `GET` | `/dashboard/recent-findings` | Get recent vulnerabilities | + +### WebSocket + +``` +ws://localhost:8000/ws/{scan_id} ``` -### Permission Issues +**Events:** +- `scan_started` - Scan has begun +- `scan_progress` - Progress update (percentage) +- `endpoint_found` - New endpoint discovered +- `vulnerability_found` - New vulnerability found +- `scan_completed` - Scan finished +- `scan_error` - Error occurred + +--- + +## Vulnerability Engine + +### How It Works + +1. **Prompt Parsing** - User prompt analyzed for vulnerability keywords +2. **Type Extraction** - Relevant vulnerability types identified +3. **Tester Selection** - Appropriate testers loaded from registry +4. **Payload Generation** - Context-aware payloads generated +5. **Testing Execution** - Tests run against target endpoints +6. **Finding Reporting** - Results sent via WebSocket in real-time + +### Prompt Examples + +``` +"Test for SQL injection and XSS vulnerabilities" +→ Extracts: sql_injection, xss_reflected, xss_stored + +"Check for OWASP Top 10 issues" +→ Extracts: All major vulnerability types + +"Look for authentication bypass and IDOR" +→ Extracts: auth_bypass, idor, bola + +"Find server-side request forgery and file inclusion" +→ Extracts: ssrf, lfi, rfi, path_traversal +``` + +### Adding Custom Testers + +Create a new tester in `backend/core/vuln_engine/testers/`: + +```python +from .base_tester import BaseTester, TestResult + +class MyCustomTester(BaseTester): + """Custom vulnerability tester""" + + async def test(self, url: str, endpoint: str, params: dict) -> list[TestResult]: + results = [] + # Your testing logic here + return results +``` + +Register in `backend/core/vuln_engine/registry.py`: + +```python +VULNERABILITY_REGISTRY["my_custom_vuln"] = { + "name": "My Custom Vulnerability", + "category": "custom", + "severity": "high", + "tester": "MyCustomTester", + # ... +} +``` + +--- + +## Configuration + +### Environment Variables ```bash -mkdir -p results reports logs -chmod 755 results reports logs +# .env file + +# LLM API Keys (at least one required for AI-powered testing) +ANTHROPIC_API_KEY=your-anthropic-api-key +OPENAI_API_KEY=your-openai-api-key + +# Database (default is SQLite) +DATABASE_URL=sqlite+aiosqlite:///./data/neurosploit.db + +# Server Configuration +HOST=0.0.0.0 +PORT=8000 +DEBUG=false ``` +### Preset Prompts + +Available presets in `/api/v1/prompts/presets`: + +| ID | Name | Description | +|----|------|-------------| +| `full_pentest` | Full Penetration Test | Comprehensive testing across all categories | +| `owasp_top10` | OWASP Top 10 | Focus on OWASP Top 10 vulnerabilities | +| `api_security` | API Security | API-specific security testing | +| `bug_bounty` | Bug Bounty Hunter | High-impact findings for bounty programs | +| `quick_scan` | Quick Security Scan | Fast essential security checks | +| `auth_testing` | Authentication Testing | Auth and session security | + +--- + +## Development + +### Backend Development + +```bash +cd backend +python3 -m venv venv +source venv/bin/activate +pip install -r requirements.txt + +# Run with hot reload +uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000 + +# API docs available at http://localhost:8000/docs +``` + +### Frontend Development + +```bash +cd frontend +npm install +npm run dev + +# Build for production +npm run build +``` + +### Running Tests + +```bash +# Backend tests +cd backend +pytest + +# Frontend tests +cd frontend +npm test +``` + +--- + +## Upgrading from v2 + +v3 is a complete rewrite with a new architecture. Key differences: + +| Feature | v2 | v3 | +|---------|----|----| +| Interface | CLI only | Web GUI + API | +| Vulnerability Testing | Hardcoded (XSS, SQLi, LFI) | Dynamic 50+ types | +| Test Selection | Manual | Prompt-driven | +| Progress Updates | Terminal output | WebSocket real-time | +| Reports | HTML file | Web viewer + export | +| Deployment | Python script | Docker Compose | + +**Migration:** v3 is a separate installation. Your v2 configurations and results are not compatible. + --- ## Security Notice @@ -688,18 +453,15 @@ MIT License - See [LICENSE](LICENSE) for details. ## Acknowledgements +### Technologies +- FastAPI, SQLAlchemy, Pydantic +- React, TypeScript, TailwindCSS, Zustand +- Docker, Nginx + ### LLM Providers - Anthropic Claude - OpenAI GPT -- Google Gemini -- Ollama -- LM Studio - -### Security Tools -- Nmap, Nuclei, SQLMap -- Subfinder, Amass, httpx -- Katana, Gospider, gau --- -**NeuroSploit v2** - *Intelligent Adaptive Security Testing* +**NeuroSploit v3** - *AI-Powered Penetration Testing Platform* diff --git a/install_tools.sh b/install_tools.sh new file mode 100644 index 0000000..db452e5 --- /dev/null +++ b/install_tools.sh @@ -0,0 +1,544 @@ +#!/bin/bash +# +# NeuroSploit v2 - Reconnaissance Tools Installer +# Installs all required tools for advanced reconnaissance +# + +set -e + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' # No Color + +# Banner +echo -e "${CYAN}" +echo "╔═══════════════════════════════════════════════════════════════╗" +echo "║ NEUROSPLOIT v2 - TOOLS INSTALLER ║" +echo "║ Advanced Reconnaissance Tools Setup ║" +echo "╚═══════════════════════════════════════════════════════════════╝" +echo -e "${NC}" + +# Detect OS +detect_os() { + if [[ "$OSTYPE" == "darwin"* ]]; then + OS="macos" + PKG_MANAGER="brew" + elif [ -f /etc/debian_version ]; then + OS="debian" + PKG_MANAGER="apt" + elif [ -f /etc/redhat-release ]; then + OS="redhat" + PKG_MANAGER="dnf" + elif [ -f /etc/arch-release ]; then + OS="arch" + PKG_MANAGER="pacman" + else + OS="unknown" + PKG_MANAGER="unknown" + fi + echo -e "${BLUE}[*] Detected OS: ${OS} (Package Manager: ${PKG_MANAGER})${NC}" +} + +# Check if command exists +command_exists() { + command -v "$1" &> /dev/null +} + +# Print status +print_status() { + if command_exists "$1"; then + echo -e " ${GREEN}[✓]${NC} $1 - installed" + return 0 + else + echo -e " ${RED}[✗]${NC} $1 - not found" + return 1 + fi +} + +# Install Go if not present +install_go() { + if command_exists go; then + echo -e "${GREEN}[✓] Go is already installed${NC}" + return 0 + fi + + echo -e "${YELLOW}[*] Installing Go...${NC}" + + if [ "$OS" == "macos" ]; then + brew install go + elif [ "$OS" == "debian" ]; then + sudo apt update && sudo apt install -y golang-go + elif [ "$OS" == "redhat" ]; then + sudo dnf install -y golang + elif [ "$OS" == "arch" ]; then + sudo pacman -S --noconfirm go + else + # Manual installation + GO_VERSION="1.21.5" + wget "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" + sudo rm -rf /usr/local/go + sudo tar -C /usr/local -xzf "go${GO_VERSION}.linux-amd64.tar.gz" + rm "go${GO_VERSION}.linux-amd64.tar.gz" + export PATH=$PATH:/usr/local/go/bin + echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc + echo 'export PATH=$PATH:$(go env GOPATH)/bin' >> ~/.bashrc + fi + + # Set GOPATH + export GOPATH=$HOME/go + export PATH=$PATH:$GOPATH/bin +} + +# Install Rust if not present +install_rust() { + if command_exists cargo; then + echo -e "${GREEN}[✓] Rust is already installed${NC}" + return 0 + fi + + echo -e "${YELLOW}[*] Installing Rust...${NC}" + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y + source "$HOME/.cargo/env" +} + +# Install Python packages +install_python_packages() { + echo -e "${BLUE}[*] Installing Python packages...${NC}" + + pip3 install --upgrade pip 2>/dev/null || pip install --upgrade pip + + # Core packages + pip3 install requests dnspython urllib3 2>/dev/null || pip install requests dnspython urllib3 + + # Security tools + pip3 install wafw00f 2>/dev/null || echo -e "${YELLOW} [!] wafw00f installation failed, try: pip install wafw00f${NC}" + pip3 install paramspider 2>/dev/null || echo -e "${YELLOW} [!] paramspider installation failed${NC}" +} + +# Install tool via Go +install_go_tool() { + local tool_name=$1 + local repo=$2 + + if command_exists "$tool_name"; then + echo -e " ${GREEN}[✓]${NC} $tool_name - already installed" + return 0 + fi + + echo -e " ${YELLOW}[~]${NC} Installing $tool_name..." + go install "$repo@latest" 2>/dev/null + + if command_exists "$tool_name"; then + echo -e " ${GREEN}[✓]${NC} $tool_name - installed successfully" + else + echo -e " ${RED}[✗]${NC} $tool_name - installation failed" + fi +} + +# Install tool via Cargo (Rust) +install_cargo_tool() { + local tool_name=$1 + local crate_name=${2:-$tool_name} + + if command_exists "$tool_name"; then + echo -e " ${GREEN}[✓]${NC} $tool_name - already installed" + return 0 + fi + + echo -e " ${YELLOW}[~]${NC} Installing $tool_name..." + cargo install "$crate_name" 2>/dev/null + + if command_exists "$tool_name"; then + echo -e " ${GREEN}[✓]${NC} $tool_name - installed successfully" + else + echo -e " ${RED}[✗]${NC} $tool_name - installation failed" + fi +} + +# Install system packages +install_system_packages() { + echo -e "${BLUE}[*] Installing system packages...${NC}" + + if [ "$OS" == "macos" ]; then + brew update + brew install nmap curl wget jq git python3 2>/dev/null || true + brew install feroxbuster 2>/dev/null || true + brew install nikto 2>/dev/null || true + brew install whatweb 2>/dev/null || true + + elif [ "$OS" == "debian" ]; then + sudo apt update + sudo apt install -y nmap curl wget jq git python3 python3-pip dnsutils whois + sudo apt install -y nikto whatweb 2>/dev/null || true + + elif [ "$OS" == "redhat" ]; then + sudo dnf install -y nmap curl wget jq git python3 python3-pip bind-utils whois + + elif [ "$OS" == "arch" ]; then + sudo pacman -Syu --noconfirm nmap curl wget jq git python python-pip dnsutils whois + sudo pacman -S --noconfirm nikto whatweb 2>/dev/null || true + fi +} + +# Install Go-based tools +install_go_tools() { + echo -e "\n${BLUE}[*] Installing Go-based reconnaissance tools...${NC}" + + # Ensure Go paths are set + export GOPATH=${GOPATH:-$HOME/go} + export PATH=$PATH:$GOPATH/bin + + # ProjectDiscovery tools + install_go_tool "subfinder" "github.com/projectdiscovery/subfinder/v2/cmd/subfinder" + install_go_tool "httpx" "github.com/projectdiscovery/httpx/cmd/httpx" + install_go_tool "nuclei" "github.com/projectdiscovery/nuclei/v3/cmd/nuclei" + install_go_tool "naabu" "github.com/projectdiscovery/naabu/v2/cmd/naabu" + install_go_tool "katana" "github.com/projectdiscovery/katana/cmd/katana" + install_go_tool "dnsx" "github.com/projectdiscovery/dnsx/cmd/dnsx" + install_go_tool "shuffledns" "github.com/projectdiscovery/shuffledns/cmd/shuffledns" + + # Other Go tools + install_go_tool "amass" "github.com/owasp-amass/amass/v4/..." + install_go_tool "assetfinder" "github.com/tomnomnom/assetfinder" + install_go_tool "waybackurls" "github.com/tomnomnom/waybackurls" + install_go_tool "gau" "github.com/lc/gau/v2/cmd/gau" + install_go_tool "httprobe" "github.com/tomnomnom/httprobe" + install_go_tool "ffuf" "github.com/ffuf/ffuf/v2" + install_go_tool "gobuster" "github.com/OJ/gobuster/v3" + install_go_tool "gospider" "github.com/jaeles-project/gospider" + install_go_tool "hakrawler" "github.com/hakluke/hakrawler" + install_go_tool "subjack" "github.com/haccer/subjack" + install_go_tool "gowitness" "github.com/sensepost/gowitness" + install_go_tool "findomain" "github.com/Findomain/Findomain" +} + +# Install Rust-based tools +install_rust_tools() { + echo -e "\n${BLUE}[*] Installing Rust-based tools...${NC}" + + source "$HOME/.cargo/env" 2>/dev/null || true + + install_cargo_tool "rustscan" "rustscan" + install_cargo_tool "feroxbuster" "feroxbuster" +} + +# Install Nuclei templates +install_nuclei_templates() { + echo -e "\n${BLUE}[*] Updating Nuclei templates...${NC}" + + if command_exists nuclei; then + nuclei -update-templates 2>/dev/null || echo -e "${YELLOW} [!] Template update failed, run manually: nuclei -update-templates${NC}" + echo -e " ${GREEN}[✓]${NC} Nuclei templates updated" + else + echo -e " ${RED}[✗]${NC} Nuclei not installed, skipping templates" + fi +} + +# Install SecLists +install_seclists() { + echo -e "\n${BLUE}[*] Checking SecLists...${NC}" + + SECLISTS_PATH="/opt/wordlists/SecLists" + + if [ -d "$SECLISTS_PATH" ]; then + echo -e " ${GREEN}[✓]${NC} SecLists already installed at $SECLISTS_PATH" + return 0 + fi + + echo -e " ${YELLOW}[~]${NC} Installing SecLists..." + sudo mkdir -p /opt/wordlists + sudo git clone --depth 1 https://github.com/danielmiessler/SecLists.git "$SECLISTS_PATH" 2>/dev/null || { + echo -e " ${RED}[✗]${NC} SecLists installation failed" + return 1 + } + + # Create symlinks for common wordlists + sudo ln -sf "$SECLISTS_PATH/Discovery/Web-Content/common.txt" /opt/wordlists/common.txt 2>/dev/null + sudo ln -sf "$SECLISTS_PATH/Discovery/Web-Content/raft-medium-directories.txt" /opt/wordlists/directories.txt 2>/dev/null + sudo ln -sf "$SECLISTS_PATH/Discovery/DNS/subdomains-top1million-5000.txt" /opt/wordlists/subdomains.txt 2>/dev/null + + echo -e " ${GREEN}[✓]${NC} SecLists installed" +} + +# Install additional tools via package managers or manual +install_additional_tools() { + echo -e "\n${BLUE}[*] Installing additional tools...${NC}" + + # wafw00f + if ! command_exists wafw00f; then + echo -e " ${YELLOW}[~]${NC} Installing wafw00f..." + pip3 install wafw00f 2>/dev/null || pip install wafw00f 2>/dev/null + fi + print_status "wafw00f" + + # paramspider + if ! command_exists paramspider; then + echo -e " ${YELLOW}[~]${NC} Installing paramspider..." + pip3 install paramspider 2>/dev/null || { + git clone https://github.com/devanshbatham/ParamSpider.git /tmp/paramspider 2>/dev/null + cd /tmp/paramspider && pip3 install . 2>/dev/null + cd - + } + fi + print_status "paramspider" + + # whatweb + if ! command_exists whatweb; then + if [ "$OS" == "macos" ]; then + brew install whatweb 2>/dev/null + elif [ "$OS" == "debian" ]; then + sudo apt install -y whatweb 2>/dev/null + fi + fi + print_status "whatweb" + + # nikto + if ! command_exists nikto; then + if [ "$OS" == "macos" ]; then + brew install nikto 2>/dev/null + elif [ "$OS" == "debian" ]; then + sudo apt install -y nikto 2>/dev/null + fi + fi + print_status "nikto" + + # sqlmap + if ! command_exists sqlmap; then + echo -e " ${YELLOW}[~]${NC} Installing sqlmap..." + if [ "$OS" == "macos" ]; then + brew install sqlmap 2>/dev/null + elif [ "$OS" == "debian" ]; then + sudo apt install -y sqlmap 2>/dev/null + else + pip3 install sqlmap 2>/dev/null + fi + fi + print_status "sqlmap" + + # eyewitness + if ! command_exists eyewitness; then + echo -e " ${YELLOW}[~]${NC} Installing EyeWitness..." + git clone https://github.com/RedSiege/EyeWitness.git /opt/EyeWitness 2>/dev/null || true + if [ -d "/opt/EyeWitness" ]; then + cd /opt/EyeWitness/Python/setup + sudo ./setup.sh 2>/dev/null || true + sudo ln -sf /opt/EyeWitness/Python/EyeWitness.py /usr/local/bin/eyewitness 2>/dev/null + cd - + fi + fi + print_status "eyewitness" + + # wpscan + if ! command_exists wpscan; then + echo -e " ${YELLOW}[~]${NC} Installing wpscan..." + if [ "$OS" == "macos" ]; then + brew install wpscan 2>/dev/null + else + sudo gem install wpscan 2>/dev/null || true + fi + fi + print_status "wpscan" + + # dirsearch + if ! command_exists dirsearch; then + echo -e " ${YELLOW}[~]${NC} Installing dirsearch..." + pip3 install dirsearch 2>/dev/null || { + git clone https://github.com/maurosoria/dirsearch.git /opt/dirsearch 2>/dev/null + sudo ln -sf /opt/dirsearch/dirsearch.py /usr/local/bin/dirsearch 2>/dev/null + } + fi + print_status "dirsearch" + + # massdns (for shuffledns/puredns) + if ! command_exists massdns; then + echo -e " ${YELLOW}[~]${NC} Installing massdns..." + git clone https://github.com/blechschmidt/massdns.git /tmp/massdns 2>/dev/null + cd /tmp/massdns && make 2>/dev/null && sudo make install 2>/dev/null + cd - + fi + print_status "massdns" + + # puredns + if ! command_exists puredns; then + echo -e " ${YELLOW}[~]${NC} Installing puredns..." + go install github.com/d3mondev/puredns/v2@latest 2>/dev/null + fi + print_status "puredns" + + # waymore + if ! command_exists waymore; then + echo -e " ${YELLOW}[~]${NC} Installing waymore..." + pip3 install waymore 2>/dev/null || pip install waymore 2>/dev/null + fi + print_status "waymore" +} + +# Check all tools status +check_tools_status() { + echo -e "\n${CYAN}═══════════════════════════════════════════════════════════════${NC}" + echo -e "${CYAN} TOOLS STATUS SUMMARY ${NC}" + echo -e "${CYAN}═══════════════════════════════════════════════════════════════${NC}\n" + + echo -e "${BLUE}[Subdomain Enumeration]${NC}" + print_status "subfinder" + print_status "amass" + print_status "assetfinder" + print_status "findomain" + print_status "puredns" + print_status "shuffledns" + print_status "massdns" + + echo -e "\n${BLUE}[HTTP Probing]${NC}" + print_status "httpx" + print_status "httprobe" + + echo -e "\n${BLUE}[URL Collection]${NC}" + print_status "gau" + print_status "waybackurls" + print_status "waymore" + print_status "hakrawler" + + echo -e "\n${BLUE}[Web Crawling]${NC}" + print_status "katana" + print_status "gospider" + + echo -e "\n${BLUE}[Directory Bruteforce]${NC}" + print_status "feroxbuster" + print_status "gobuster" + print_status "ffuf" + print_status "dirsearch" + + echo -e "\n${BLUE}[Port Scanning]${NC}" + print_status "rustscan" + print_status "naabu" + print_status "nmap" + + echo -e "\n${BLUE}[Vulnerability Scanning]${NC}" + print_status "nuclei" + print_status "nikto" + print_status "sqlmap" + print_status "wpscan" + + echo -e "\n${BLUE}[WAF Detection]${NC}" + print_status "wafw00f" + + echo -e "\n${BLUE}[Parameter Discovery]${NC}" + print_status "paramspider" + + echo -e "\n${BLUE}[Fingerprinting]${NC}" + print_status "whatweb" + + echo -e "\n${BLUE}[Screenshot]${NC}" + print_status "gowitness" + print_status "eyewitness" + + echo -e "\n${BLUE}[Subdomain Takeover]${NC}" + print_status "subjack" + + echo -e "\n${BLUE}[DNS Tools]${NC}" + print_status "dnsx" + print_status "dig" + + echo -e "\n${BLUE}[Utilities]${NC}" + print_status "curl" + print_status "wget" + print_status "jq" + print_status "git" + + echo -e "\n${BLUE}[Wordlists]${NC}" + if [ -d "/opt/wordlists/SecLists" ]; then + echo -e " ${GREEN}[✓]${NC} SecLists - installed at /opt/wordlists/SecLists" + else + echo -e " ${RED}[✗]${NC} SecLists - not found" + fi +} + +# Update PATH +update_path() { + echo -e "\n${BLUE}[*] Updating PATH...${NC}" + + # Add Go bin to PATH + if ! grep -q 'GOPATH' ~/.bashrc 2>/dev/null; then + echo 'export GOPATH=$HOME/go' >> ~/.bashrc + echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.bashrc + fi + + if ! grep -q 'GOPATH' ~/.zshrc 2>/dev/null; then + echo 'export GOPATH=$HOME/go' >> ~/.zshrc 2>/dev/null || true + echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.zshrc 2>/dev/null || true + fi + + # Add Cargo bin to PATH + if ! grep -q '.cargo/bin' ~/.bashrc 2>/dev/null; then + echo 'export PATH=$PATH:$HOME/.cargo/bin' >> ~/.bashrc + fi + + # Source for current session + export GOPATH=$HOME/go + export PATH=$PATH:$GOPATH/bin:$HOME/.cargo/bin + + echo -e " ${GREEN}[✓]${NC} PATH updated" +} + +# Main installation function +main() { + echo -e "${BLUE}[*] Starting NeuroSploit tools installation...${NC}\n" + + detect_os + + # Parse arguments + INSTALL_ALL=false + CHECK_ONLY=false + + while [[ "$#" -gt 0 ]]; do + case $1 in + --all) INSTALL_ALL=true ;; + --check) CHECK_ONLY=true ;; + --help|-h) + echo "Usage: $0 [OPTIONS]" + echo "" + echo "Options:" + echo " --all Install all tools (full installation)" + echo " --check Only check tool status, don't install" + echo " --help Show this help message" + echo "" + exit 0 + ;; + *) echo "Unknown parameter: $1"; exit 1 ;; + esac + shift + done + + if [ "$CHECK_ONLY" = true ]; then + check_tools_status + exit 0 + fi + + # Installation steps + install_system_packages + install_go + install_rust + install_python_packages + install_go_tools + install_rust_tools + install_additional_tools + install_seclists + install_nuclei_templates + update_path + + # Final status check + check_tools_status + + echo -e "\n${GREEN}═══════════════════════════════════════════════════════════════${NC}" + echo -e "${GREEN} INSTALLATION COMPLETE! ${NC}" + echo -e "${GREEN}═══════════════════════════════════════════════════════════════${NC}" + echo -e "\n${YELLOW}[!] Please restart your terminal or run: source ~/.bashrc${NC}" + echo -e "${YELLOW}[!] Some tools may require sudo privileges to run${NC}\n" +} + +# Run main +main "$@" diff --git a/neurosploit.py b/neurosploit.py index 8d4f29a..66e8dfa 100644 --- a/neurosploit.py +++ b/neurosploit.py @@ -37,6 +37,23 @@ from core.context_builder import ReconContextBuilder from agents.base_agent import BaseAgent from tools.recon.recon_tools import FullReconRunner +# Import AI Agents +try: + from backend.core.ai_pentest_agent import AIPentestAgent +except ImportError: + AIPentestAgent = None + +try: + from backend.core.autonomous_agent import AutonomousAgent, OperationMode + from backend.core.task_library import get_task_library, Task, TaskCategory +except ImportError: + AutonomousAgent = None + OperationMode = None + get_task_library = None + Task = None + TaskCategory = None + + class Completer: def __init__(self, neurosploit): self.neurosploit = neurosploit @@ -44,7 +61,12 @@ class Completer: "help", "run_agent", "config", "list_roles", "list_profiles", "set_profile", "set_agent", "discover_ollama", "install_tools", "scan", "quick_scan", "recon", "full_recon", "check_tools", - "experience", "wizard", "analyze", "exit", "quit" + "experience", "wizard", "analyze", "agent", "ai_agent", + # New autonomous agent modes + "pentest", "full_auto", "recon_only", "prompt_only", "analyze_only", + # Task library + "tasks", "task", "list_tasks", "create_task", "run_task", + "exit", "quit" ] self.agent_roles = list(self.neurosploit.config.get('agent_roles', {}).keys()) self.llm_profiles = list(self.neurosploit.config.get('llm', {}).get('profiles', {}).keys()) @@ -950,6 +972,465 @@ CONTEXTO DE RECON: "context_text_file": recon_results.get('context_text_file', '') } + def run_ai_agent( + self, + target: str, + prompt_file: Optional[str] = None, + context_file: Optional[str] = None, + llm_profile: Optional[str] = None + ) -> Dict: + """ + Run the AI Offensive Security Agent. + + This is an autonomous agent that: + - Uses LLM for intelligent vulnerability testing + - Confirms vulnerabilities with AI (no false positives) + - Uses recon data to inform testing + - Accepts custom .md prompt files + - Generates PoC code + + Args: + target: Target URL to test + prompt_file: Optional .md file with custom testing instructions + context_file: Optional recon context JSON file + llm_profile: Optional LLM profile to use + """ + if not AIPentestAgent: + print("[!] AI Agent not available. Check backend installation.") + return {"error": "AI Agent not installed"} + + print(f"\n{'='*70}") + print(" NEUROSPLOIT AI OFFENSIVE SECURITY AGENT") + print(f"{'='*70}") + print(f"\n[*] Target: {target}") + if prompt_file: + print(f"[*] Prompt file: {prompt_file}") + if context_file: + print(f"[*] Context file: {context_file}") + print(f"[*] Session ID: {self.session_id}") + print() + + # Load recon context if provided + recon_context = None + if context_file: + from core.context_builder import load_context_from_file + recon_context = load_context_from_file(context_file) + if recon_context: + print(f"[+] Loaded recon context: {len(recon_context.get('data', {}).get('endpoints', []))} endpoints") + + # Initialize LLM manager + profile = llm_profile or self.config.get('llm', {}).get('default_profile') + self._initialize_llm_manager(profile) + + # Run the agent + import asyncio + + async def run_agent(): + async def log_callback(level: str, message: str): + prefix = { + "info": "[*]", + "warning": "[!]", + "error": "[X]", + "debug": "[D]", + }.get(level, "[*]") + print(f"{prefix} {message}") + + async with AIPentestAgent( + target=target, + llm_manager=self.llm_manager_instance, + log_callback=log_callback, + prompt_file=prompt_file, + recon_context=recon_context, + config=self.config, + max_depth=5 + ) as agent: + report = await agent.run() + return report + + try: + report = asyncio.run(run_agent()) + except Exception as e: + logger.error(f"Agent error: {e}") + import traceback + traceback.print_exc() + return {"error": str(e)} + + # Save results + if report and report.get("findings"): + result_file = f"results/agent_{self.session_id}.json" + with open(result_file, 'w') as f: + json.dump(report, f, indent=2, default=str) + print(f"\n[+] Results saved: {result_file}") + + # Generate HTML report + self._generate_agent_report(report) + + print(f"\n{'='*70}") + print("[+] AI AGENT COMPLETE!") + print(f" Vulnerabilities found: {len(report.get('findings', []))}") + print(f"{'='*70}\n") + + return report + + def run_autonomous_agent( + self, + target: str, + mode: str = "full_auto", + task_id: Optional[str] = None, + prompt: Optional[str] = None, + prompt_file: Optional[str] = None, + context_file: Optional[str] = None, + llm_profile: Optional[str] = None + ) -> Dict: + """ + Run the Autonomous AI Security Agent. + + Modes: + - recon_only: Just reconnaissance, no testing + - full_auto: Complete workflow (Recon -> Analyze -> Test -> Report) + - prompt_only: AI decides everything based on prompt (HIGH TOKEN USAGE!) + - analyze_only: Analysis of provided data, no active testing + + Args: + target: Target URL/domain + mode: Operation mode + task_id: Task from library to execute + prompt: Custom prompt + prompt_file: Path to .md prompt file + context_file: Path to recon context JSON + llm_profile: LLM profile to use + """ + if not AutonomousAgent: + print("[!] Autonomous Agent not available. Check installation.") + return {"error": "Agent not installed"} + + print(f"\n{'='*70}") + print(" NEUROSPLOIT AUTONOMOUS AI AGENT") + print(f"{'='*70}") + print(f"\n[*] Target: {target}") + print(f"[*] Mode: {mode.upper()}") + + # Warning for prompt_only mode + if mode == "prompt_only": + print("\n[!] WARNING: PROMPT-ONLY MODE") + print("[!] This mode uses significantly more tokens than other modes.") + print("[!] The AI will decide what tools to use based on your prompt.\n") + + # Load task from library + task = None + if task_id and get_task_library: + library = get_task_library() + task = library.get_task(task_id) + if task: + print(f"[*] Task: {task.name}") + prompt = task.prompt + else: + print(f"[!] Task not found: {task_id}") + + # Load prompt from file + if prompt_file: + print(f"[*] Prompt file: {prompt_file}") + try: + path = Path(prompt_file) + for search in [path, Path("prompts") / path, Path("prompts/md_library") / path]: + if search.exists(): + prompt = search.read_text() + break + except Exception as e: + print(f"[!] Error loading prompt file: {e}") + + # Load recon context + recon_context = None + if context_file: + from core.context_builder import load_context_from_file + recon_context = load_context_from_file(context_file) + if recon_context: + print(f"[+] Loaded context: {context_file}") + + # Get operation mode + mode_map = { + "recon_only": OperationMode.RECON_ONLY, + "full_auto": OperationMode.FULL_AUTO, + "prompt_only": OperationMode.PROMPT_ONLY, + "analyze_only": OperationMode.ANALYZE_ONLY, + } + op_mode = mode_map.get(mode, OperationMode.FULL_AUTO) + + # Initialize LLM + profile = llm_profile or self.config.get('llm', {}).get('default_profile') + self._initialize_llm_manager(profile) + + print(f"[*] Session: {self.session_id}\n") + + # Run agent + import asyncio + + async def run(): + async def log_cb(level: str, message: str): + prefix = {"info": "[*]", "warning": "[!]", "error": "[X]", "debug": "[D]"}.get(level, "[*]") + print(f"{prefix} {message}") + + async def progress_cb(progress: int, message: str): + bar = "=" * (progress // 5) + ">" + " " * (20 - progress // 5) + print(f"\r[{bar}] {progress}% - {message}", end="", flush=True) + if progress == 100: + print() + + async with AutonomousAgent( + target=target, + mode=op_mode, + llm_manager=self.llm_manager_instance, + log_callback=log_cb, + progress_callback=progress_cb, + task=task, + custom_prompt=prompt, + recon_context=recon_context, + config=self.config, + prompt_file=prompt_file + ) as agent: + return await agent.run() + + try: + report = asyncio.run(run()) + except Exception as e: + logger.error(f"Agent error: {e}") + import traceback + traceback.print_exc() + return {"error": str(e)} + + # Save results + result_file = f"results/autonomous_{self.session_id}.json" + with open(result_file, 'w') as f: + json.dump(report, f, indent=2, default=str) + print(f"\n[+] Results saved: {result_file}") + + # Generate HTML report + if report.get("findings"): + self._generate_autonomous_report(report) + + return report + + def list_tasks(self): + """List all available tasks from library""" + if not get_task_library: + print("[!] Task library not available") + return + + library = get_task_library() + tasks = library.list_tasks() + + print(f"\n{'='*70}") + print(" TASK LIBRARY") + print(f"{'='*70}\n") + + # Group by category + by_category = {} + for task in tasks: + cat = task.category + if cat not in by_category: + by_category[cat] = [] + by_category[cat].append(task) + + for category, cat_tasks in by_category.items(): + print(f"[{category.upper()}]") + for task in cat_tasks: + preset = " (preset)" if task.is_preset else "" + print(f" {task.id:<25} - {task.name}{preset}") + print() + + print(f"Total: {len(tasks)} tasks") + print("\nUse: run_task ") + + def create_task(self, name: str, prompt: str, category: str = "custom"): + """Create a new task in the library""" + if not get_task_library or not Task: + print("[!] Task library not available") + return + + library = get_task_library() + task = Task( + id=f"custom_{datetime.now().strftime('%Y%m%d_%H%M%S')}", + name=name, + description=prompt[:100], + category=category, + prompt=prompt, + is_preset=False + ) + library.create_task(task) + print(f"[+] Task created: {task.id}") + return task + + def _generate_autonomous_report(self, report: Dict): + """Generate HTML report from autonomous agent results""" + from core.report_generator import ReportGenerator + + # Convert to scan format + scan_data = { + "target": report.get("target", ""), + "scan_started": report.get("scan_date", ""), + "scan_completed": datetime.now().isoformat(), + "vulnerabilities": [], + "technologies": report.get("summary", {}).get("technologies", []), + } + + for finding in report.get("findings", []): + vuln = { + "title": finding.get("title", "Unknown"), + "severity": finding.get("severity", "medium"), + "description": finding.get("description", ""), + "technical_details": finding.get("technical_details", ""), + "affected_endpoint": finding.get("endpoint", ""), + "payload": finding.get("payload", ""), + "evidence": finding.get("evidence", ""), + "impact": finding.get("impact", ""), + "poc_code": finding.get("poc_code", ""), + "exploitation_steps": finding.get("exploitation_steps", []), + "remediation": finding.get("remediation", ""), + "references": finding.get("references", []), + } + + # Add CVSS + if finding.get("cvss"): + cvss = finding["cvss"] + vuln["cvss_score"] = cvss.get("score", 0) + vuln["cvss_vector"] = cvss.get("vector", "") + + # Add CWE + if finding.get("cwe_id"): + vuln["cwe_id"] = finding["cwe_id"] + + scan_data["vulnerabilities"].append(vuln) + + # Generate LLM analysis summary + summary = report.get("summary", {}) + llm_analysis = f""" +## Autonomous AI Agent Assessment Report + +**Target:** {report.get('target', '')} +**Mode:** {report.get('mode', 'full_auto').upper()} +**Scan Date:** {report.get('scan_date', '')} + +### Executive Summary +Risk Level: **{summary.get('risk_level', 'UNKNOWN')}** + +### Findings Summary +| Severity | Count | +|----------|-------| +| Critical | {summary.get('critical', 0)} | +| High | {summary.get('high', 0)} | +| Medium | {summary.get('medium', 0)} | +| Low | {summary.get('low', 0)} | +| Info | {summary.get('info', 0)} | + +**Total Findings:** {summary.get('total_findings', 0)} +**Endpoints Tested:** {summary.get('endpoints_tested', 0)} + +### Technologies Detected +{', '.join(summary.get('technologies', [])) or 'None detected'} + +### Detailed Findings +""" + for i, finding in enumerate(report.get("findings", []), 1): + cvss_info = "" + if finding.get("cvss"): + cvss_info = f"**CVSS:** {finding['cvss'].get('score', 'N/A')} ({finding['cvss'].get('vector', '')})" + + llm_analysis += f""" +--- +#### {i}. {finding.get('title', 'Unknown')} [{finding.get('severity', 'medium').upper()}] +{cvss_info} +**CWE:** {finding.get('cwe_id', 'N/A')} +**Endpoint:** `{finding.get('endpoint', 'N/A')}` + +**Description:** +{finding.get('description', 'No description')} + +**Impact:** +{finding.get('impact', 'No impact assessment')} + +**Evidence:** +``` +{finding.get('evidence', 'No evidence')} +``` + +**Proof of Concept:** +```python +{finding.get('poc_code', '# No PoC available')} +``` + +**Remediation:** +{finding.get('remediation', 'No remediation provided')} + +""" + + # Recommendations + if report.get("recommendations"): + llm_analysis += "\n### Recommendations\n" + for rec in report["recommendations"]: + llm_analysis += f"- {rec}\n" + + report_gen = ReportGenerator(scan_data, llm_analysis) + html_report = report_gen.save_report("reports") + print(f"[+] HTML Report: {html_report}") + + def _generate_agent_report(self, report: Dict): + """Generate HTML report from AI agent results""" + from core.report_generator import ReportGenerator + + # Convert agent report to scan format + scan_data = { + "target": report.get("target", ""), + "scan_started": report.get("scan_date", ""), + "scan_completed": datetime.now().isoformat(), + "vulnerabilities": [], + "technologies": report.get("summary", {}).get("technologies", []), + } + + for finding in report.get("findings", []): + scan_data["vulnerabilities"].append({ + "title": f"{finding['type'].upper()} - {finding['severity'].upper()}", + "severity": finding["severity"], + "description": finding.get("evidence", ""), + "affected_endpoint": finding.get("endpoint", ""), + "payload": finding.get("payload", ""), + "poc_code": finding.get("poc_code", ""), + "exploitation_steps": finding.get("exploitation_steps", []), + "llm_analysis": finding.get("llm_analysis", ""), + }) + + # Generate LLM analysis summary + llm_analysis = f""" +## AI Agent Analysis Summary + +**Target:** {report.get('target', '')} +**Scan Date:** {report.get('scan_date', '')} +**LLM Enabled:** {report.get('llm_enabled', False)} + +### Summary +- Total Endpoints: {report.get('summary', {}).get('total_endpoints', 0)} +- Total Parameters: {report.get('summary', {}).get('total_parameters', 0)} +- Vulnerabilities Found: {report.get('summary', {}).get('total_vulnerabilities', 0)} + - Critical: {report.get('summary', {}).get('critical', 0)} + - High: {report.get('summary', {}).get('high', 0)} + - Medium: {report.get('summary', {}).get('medium', 0)} + - Low: {report.get('summary', {}).get('low', 0)} + +### Findings +""" + for i, finding in enumerate(report.get("findings", []), 1): + llm_analysis += f""" +#### {i}. {finding['type'].upper()} [{finding['severity'].upper()}] +- **Endpoint:** {finding.get('endpoint', '')} +- **Payload:** `{finding.get('payload', '')}` +- **Evidence:** {finding.get('evidence', '')} +- **Confidence:** {finding.get('confidence', 'medium')} +- **LLM Analysis:** {finding.get('llm_analysis', 'N/A')} +""" + + report_gen = ReportGenerator(scan_data, llm_analysis) + html_report = report_gen.save_report("reports") + print(f"[+] HTML Report: {html_report}") + def check_tools_status(self): """Check and display status of all pentest tools""" print("\n" + "="*60) @@ -1138,6 +1619,181 @@ CONTEXTO DE RECON: else: print("Usage: analyze ") print(" Then enter your analysis prompt") + elif cmd.startswith('agent ') or cmd.startswith('ai_agent '): + # AI Agent command + # Format: agent [--prompt ] [--context ] + parts = cmd.split() + if len(parts) >= 2: + target = parts[1].strip().strip('"') + prompt_file = None + context_file = None + + # Parse optional arguments + i = 2 + while i < len(parts): + if parts[i] in ['--prompt', '-p'] and i + 1 < len(parts): + prompt_file = parts[i + 1].strip().strip('"') + i += 2 + elif parts[i] in ['--context', '-c'] and i + 1 < len(parts): + context_file = parts[i + 1].strip().strip('"') + i += 2 + else: + i += 1 + + # Get LLM profile + llm_profile = self.config.get('llm', {}).get('default_profile') + self.run_ai_agent(target, prompt_file, context_file, llm_profile) + else: + print("Usage: agent [--prompt ] [--context ]") + print("") + print("Examples:") + print(" agent https://example.com") + print(" agent https://example.com --prompt bug_bounty.md") + print(" agent https://example.com --context results/context_X.json") + print("") + print("The AI Agent will:") + print(" 1. Use LLM for intelligent vulnerability testing") + print(" 2. Confirm findings with AI (no false positives)") + print(" 3. Generate PoC code for exploits") + print(" 4. Use recon data if context file provided") + + # === NEW AUTONOMOUS AGENT MODES === + elif cmd.startswith('pentest ') or cmd.startswith('full_auto '): + # Full autonomous pentest mode + parts = cmd.split() + if len(parts) >= 2: + target = parts[1].strip().strip('"') + task_id = None + prompt_file = None + context_file = None + + i = 2 + while i < len(parts): + if parts[i] in ['--task', '-t'] and i + 1 < len(parts): + task_id = parts[i + 1].strip() + i += 2 + elif parts[i] in ['--prompt', '-p'] and i + 1 < len(parts): + prompt_file = parts[i + 1].strip().strip('"') + i += 2 + elif parts[i] in ['--context', '-c'] and i + 1 < len(parts): + context_file = parts[i + 1].strip().strip('"') + i += 2 + else: + i += 1 + + self.run_autonomous_agent(target, "full_auto", task_id, None, prompt_file, context_file) + else: + print("Usage: pentest [--task ] [--prompt ] [--context ]") + print("") + print("Full autonomous pentest: Recon -> Analyze -> Test -> Report") + + elif cmd.startswith('recon_only '): + # Recon-only mode + parts = cmd.split() + if len(parts) >= 2: + target = parts[1].strip().strip('"') + self.run_autonomous_agent(target, "recon_only") + else: + print("Usage: recon_only ") + print("Just reconnaissance, no vulnerability testing") + + elif cmd.startswith('prompt_only '): + # Prompt-only mode (high token usage) + parts = cmd.split() + if len(parts) >= 2: + target = parts[1].strip().strip('"') + prompt = None + prompt_file = None + + i = 2 + while i < len(parts): + if parts[i] in ['--prompt', '-p'] and i + 1 < len(parts): + prompt_file = parts[i + 1].strip().strip('"') + i += 2 + else: + i += 1 + + if not prompt_file: + print("Enter your prompt (end with empty line):") + lines = [] + while True: + line = input() + if not line: + break + lines.append(line) + prompt = "\n".join(lines) + + print("\n[!] WARNING: PROMPT-ONLY MODE uses more tokens!") + self.run_autonomous_agent(target, "prompt_only", None, prompt, prompt_file) + else: + print("Usage: prompt_only [--prompt ]") + print("") + print("WARNING: This mode uses significantly more tokens!") + print("The AI will decide what tools to use based on your prompt.") + + elif cmd.startswith('analyze_only '): + # Analyze-only mode + parts = cmd.split() + if len(parts) >= 2: + target = parts[1].strip().strip('"') + context_file = None + + i = 2 + while i < len(parts): + if parts[i] in ['--context', '-c'] and i + 1 < len(parts): + context_file = parts[i + 1].strip().strip('"') + i += 2 + else: + i += 1 + + self.run_autonomous_agent(target, "analyze_only", None, None, None, context_file) + else: + print("Usage: analyze_only [--context ]") + print("Analysis only, no active testing") + + # === TASK LIBRARY COMMANDS === + elif cmd in ['tasks', 'list_tasks']: + self.list_tasks() + + elif cmd.startswith('run_task '): + parts = cmd.split() + if len(parts) >= 3: + task_id = parts[1].strip() + target = parts[2].strip().strip('"') + context_file = None + + i = 3 + while i < len(parts): + if parts[i] in ['--context', '-c'] and i + 1 < len(parts): + context_file = parts[i + 1].strip().strip('"') + i += 2 + else: + i += 1 + + self.run_autonomous_agent(target, "full_auto", task_id, None, None, context_file) + else: + print("Usage: run_task [--context ]") + print("Use 'tasks' to list available tasks") + + elif cmd.startswith('create_task'): + print("Create a new task for the library") + name = input("Task name: ").strip() + if not name: + print("Cancelled") + continue + print("Enter task prompt (end with empty line):") + lines = [] + while True: + line = input() + if not line: + break + lines.append(line) + prompt = "\n".join(lines) + if prompt: + self.create_task(name, prompt) + else: + print("Cancelled - no prompt provided") + else: print("Unknown command. Type 'help' for available commands.") except KeyboardInterrupt: @@ -1236,9 +1892,32 @@ TOOL MANAGEMENT: install_tools - Install required pentest tools check_tools - Check which tools are installed -AGENT COMMANDS (AI Analysis): - run_agent "" - Execute AI agent with input - set_agent - Set default agent for AI analysis +AUTONOMOUS AI AGENT (Like PentAGI): + pentest - Full auto: Recon -> Analyze -> Test -> Report + pentest --task - Use preset task from library + recon_only - Just reconnaissance, no testing + prompt_only - AI decides everything (HIGH TOKEN USAGE!) + analyze_only -c - Analysis only, no active testing + + The autonomous agent generates: + - CVSS scores with vector strings + - Detailed descriptions and impact analysis + - Working PoC code + - Remediation recommendations + - Professional HTML reports + +TASK LIBRARY: + tasks / list_tasks - List all available tasks + run_task - Run a task from the library + create_task - Create and save a new task + + Preset tasks include: full_bug_bounty, vuln_owasp_top10, + vuln_api_security, recon_full, etc. + +LEGACY AGENT: + agent - Simple AI agent (basic testing) + run_agent "" - Execute an agent role + set_agent - Set default agent CONFIGURATION: list_roles - List all available agent roles @@ -1263,6 +1942,8 @@ EXAMPLES: analyze results/context_X.json - LLM analysis of context file scan https://example.com - Full pentest scan quick_scan 192.168.1.1 - Quick vulnerability check + agent https://target.com - AI Agent pentest (uses LLM) + agent https://target.com -p bug_bounty.md -c context.json ======================================================================= """) @@ -1324,6 +2005,26 @@ EXAMPLES: parser.add_argument('--quick-scan', metavar='TARGET', help='Run QUICK pentest scan on target') + # Autonomous AI Agent options + parser.add_argument('--pentest', metavar='TARGET', + help='Run full autonomous pentest: Recon -> Analyze -> Test -> Report') + parser.add_argument('--recon-only', metavar='TARGET', + help='Run reconnaissance only, no vulnerability testing') + parser.add_argument('--prompt-only', metavar='TARGET', + help='AI decides everything based on prompt (WARNING: High token usage!)') + parser.add_argument('--analyze-only', metavar='TARGET', + help='Analysis only mode, no active testing') + parser.add_argument('--task', metavar='TASK_ID', + help='Task ID from library to execute') + parser.add_argument('--prompt-file', '-pf', metavar='FILE', + help='Custom .md prompt file for AI agent') + parser.add_argument('--list-tasks', action='store_true', + help='List all available tasks from library') + + # Legacy AI Agent options + parser.add_argument('--agent', metavar='TARGET', + help='Run simple AI Agent on target') + # Tool management parser.add_argument('--install-tools', action='store_true', help='Install required pentest tools (nmap, sqlmap, nuclei, etc.)') @@ -1393,6 +2094,62 @@ EXAMPLES: print(f"[+] Loaded context from: {args.context_file}") framework.execute_real_scan(args.quick_scan, scan_type="quick", agent_role=agent_role, recon_context=context) + # Handle Autonomous Pentest (Full Auto) + elif args.pentest: + framework.run_autonomous_agent( + target=args.pentest, + mode="full_auto", + task_id=args.task, + prompt_file=args.prompt_file, + context_file=args.context_file, + llm_profile=args.llm_profile + ) + + # Handle Recon Only + elif args.recon_only: + framework.run_autonomous_agent( + target=args.recon_only, + mode="recon_only", + llm_profile=args.llm_profile + ) + + # Handle Prompt Only (High Token Usage Warning) + elif args.prompt_only: + print("\n" + "!"*70) + print(" WARNING: PROMPT-ONLY MODE") + print(" This mode uses significantly more tokens than other modes.") + print(" The AI will decide what tools to use based on your prompt.") + print("!"*70 + "\n") + framework.run_autonomous_agent( + target=args.prompt_only, + mode="prompt_only", + prompt_file=args.prompt_file, + context_file=args.context_file, + llm_profile=args.llm_profile + ) + + # Handle Analyze Only + elif args.analyze_only: + framework.run_autonomous_agent( + target=args.analyze_only, + mode="analyze_only", + context_file=args.context_file, + llm_profile=args.llm_profile + ) + + # Handle List Tasks + elif args.list_tasks: + framework.list_tasks() + + # Handle Legacy AI Agent + elif args.agent: + framework.run_ai_agent( + target=args.agent, + prompt_file=args.prompt_file, + context_file=args.context_file, + llm_profile=args.llm_profile + ) + # Handle list commands elif args.list_agents: framework.list_agent_roles() diff --git a/start.sh b/start.sh new file mode 100644 index 0000000..be10426 --- /dev/null +++ b/start.sh @@ -0,0 +1,104 @@ +#!/bin/bash + +# NeuroSploit v3 Startup Script + +echo "================================================" +echo " NeuroSploit v3 - AI-Powered Penetration Testing" +echo "================================================" +echo "" + +# Check for .env file +if [ ! -f ".env" ]; then + echo "[!] No .env file found. Creating from .env.example..." + cp .env.example .env + echo "" + echo "==========================================" + echo " IMPORTANT: Configure your API key!" + echo "==========================================" + echo "" + echo "Edit the .env file and add your Claude API key:" + echo " ANTHROPIC_API_KEY=sk-ant-..." + echo "" + echo "Get your API key at: https://console.anthropic.com/" + echo "" + read -p "Press Enter to continue (or Ctrl+C to exit and configure)..." + echo "" +fi + +# Check if API key is configured +if grep -q "^ANTHROPIC_API_KEY=$" .env 2>/dev/null || grep -q "^ANTHROPIC_API_KEY=your-" .env 2>/dev/null; then + echo "[WARNING] ANTHROPIC_API_KEY not configured in .env" + echo "The AI agent will not work without an API key!" + echo "" +fi + +# Check for lite mode +COMPOSE_FILE="docker-compose.yml" +if [ "$1" = "--lite" ] || [ "$1" = "-l" ]; then + echo "[INFO] Using LITE mode (faster build, no security tools)" + COMPOSE_FILE="docker-compose.lite.yml" +fi + +# Check if docker-compose is available +if command -v docker-compose &> /dev/null; then + echo "Starting with Docker Compose..." + docker-compose -f $COMPOSE_FILE up -d + echo "" + echo "NeuroSploit is starting..." + echo " - Backend API: http://localhost:8000" + echo " - Web Interface: http://localhost:3000" + echo " - API Docs: http://localhost:8000/api/docs" + echo " - LLM Status: http://localhost:8000/api/v1/agent/status" + echo "" + echo "Run 'docker-compose logs -f' to view logs" + echo "" + echo "To check if LLM is configured:" + echo " curl http://localhost:8000/api/v1/agent/status" +elif command -v docker &> /dev/null && command -v docker compose &> /dev/null; then + echo "Starting with Docker Compose (v2)..." + docker compose -f $COMPOSE_FILE up -d + echo "" + echo "NeuroSploit is starting..." + echo " - Backend API: http://localhost:8000" + echo " - Web Interface: http://localhost:3000" + echo " - API Docs: http://localhost:8000/api/docs" + echo " - LLM Status: http://localhost:8000/api/v1/agent/status" +else + echo "Docker not found. Starting manually..." + echo "" + + # Start backend + echo "Starting backend..." + cd backend + if [ ! -d "venv" ]; then + python3 -m venv venv + source venv/bin/activate + pip install -r requirements.txt + else + source venv/bin/activate + fi + python -m uvicorn backend.main:app --host 0.0.0.0 --port 8000 & + BACKEND_PID=$! + cd .. + + # Start frontend + echo "Starting frontend..." + cd frontend + if [ ! -d "node_modules" ]; then + npm install + fi + npm run dev & + FRONTEND_PID=$! + cd .. + + echo "" + echo "NeuroSploit is running:" + echo " - Backend API: http://localhost:8000" + echo " - Web Interface: http://localhost:3000" + echo "" + echo "Press Ctrl+C to stop" + + # Wait for Ctrl+C + trap "kill $BACKEND_PID $FRONTEND_PID 2>/dev/null" EXIT + wait +fi diff --git a/tools/recon/recon_tools.py b/tools/recon/recon_tools.py index 26eefbd..5f52938 100644 --- a/tools/recon/recon_tools.py +++ b/tools/recon/recon_tools.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """ -Reconnaissance Tools - Network scanning, web recon, OSINT, DNS enumeration -Ferramentas avancadas de recon com suporte a multiplas ferramentas externas +NeuroSploit Advanced Reconnaissance Module +Deep enumeration with multiple tools and techniques """ import subprocess @@ -11,11 +11,18 @@ import socket import requests import shutil import os +import sys import concurrent.futures -from typing import Dict, List, Optional, Set, Tuple +import hashlib +import base64 +import tempfile +import time +from typing import Dict, List, Optional, Set, Tuple, Any +from collections import defaultdict import logging -from urllib.parse import urlparse, parse_qs +from urllib.parse import urlparse, parse_qs, urljoin, quote from pathlib import Path +from dataclasses import dataclass, field try: import dns.resolver @@ -24,26 +31,115 @@ except ImportError: logger = logging.getLogger(__name__) +# Disable SSL warnings +import urllib3 +urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) + + +# ============================================================================= +# CONFIGURATION +# ============================================================================= + +SECLISTS_BASE = "/opt/wordlists/SecLists" +WORDLISTS = { + "directories_small": f"{SECLISTS_BASE}/Discovery/Web-Content/directory-list-2.3-small.txt", + "directories_medium": f"{SECLISTS_BASE}/Discovery/Web-Content/raft-medium-directories.txt", + "directories_big": f"{SECLISTS_BASE}/Discovery/Web-Content/directory-list-2.3-big.txt", + "common": f"{SECLISTS_BASE}/Discovery/Web-Content/common.txt", + "subdomains_small": f"{SECLISTS_BASE}/Discovery/DNS/subdomains-top1million-5000.txt", + "subdomains_medium": f"{SECLISTS_BASE}/Discovery/DNS/subdomains-top1million-20000.txt", + "subdomains_big": f"{SECLISTS_BASE}/Discovery/DNS/subdomains-top1million-110000.txt", + "dns_jhaddix": f"{SECLISTS_BASE}/Discovery/DNS/dns-Jhaddix.txt", + "params": f"{SECLISTS_BASE}/Discovery/Web-Content/burp-parameter-names.txt", + "api_endpoints": f"{SECLISTS_BASE}/Discovery/Web-Content/api/api-endpoints.txt", + "backup_files": f"{SECLISTS_BASE}/Discovery/Web-Content/Common-DB-Backups.txt", +} + +# Common ports for fast scan +COMMON_PORTS = "21,22,23,25,53,80,110,111,135,139,143,443,445,993,995,1433,1521,2049,3306,3389,5432,5900,6379,8000,8080,8443,8888,9000,9200,27017" +TOP_1000_PORTS = "1-1000" +FULL_PORTS = "1-65535" + +# Patterns for sensitive data extraction +SECRET_PATTERNS = { + "aws_key": r"(?:AKIA|A3T|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}", + "aws_secret": r"(?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z/+]{40}['\"]", + "github_token": r"ghp_[a-zA-Z0-9]{36}|github_pat_[a-zA-Z0-9]{22}_[a-zA-Z0-9]{59}", + "google_api": r"AIza[0-9A-Za-z\\-_]{35}", + "slack_token": r"xox[baprs]-[0-9]{10,13}-[0-9]{10,13}[a-zA-Z0-9-]*", + "jwt": r"eyJ[A-Za-z0-9-_=]+\.eyJ[A-Za-z0-9-_=]+\.?[A-Za-z0-9-_.+/=]*", + "private_key": r"-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----", + "password_field": r"(?i)(?:password|passwd|pwd|secret|token|api_key|apikey|auth)[\s]*[=:]\s*['\"]?[^\s'\"]+", + "internal_ip": r"(?:10|172\.(?:1[6-9]|2[0-9]|3[01])|192\.168)\.\d{1,3}\.\d{1,3}", + "s3_bucket": r"(?:s3://|s3\.amazonaws\.com/|s3-[\w-]+\.amazonaws\.com/)[\w.-]+", + "firebase": r"https://[\w-]+\.firebaseio\.com", + "bearer_token": r"(?i)bearer\s+[a-zA-Z0-9\-_.~+/]+=*", +} + +# CNAME records indicating potential takeover +TAKEOVER_CNAMES = { + "github.io": "GitHub Pages", + "herokuapp.com": "Heroku", + "pantheonsite.io": "Pantheon", + "domains.tumblr.com": "Tumblr", + "wpengine.com": "WP Engine", + "ghost.io": "Ghost", + "myshopify.com": "Shopify", + "surge.sh": "Surge.sh", + "bitbucket.io": "Bitbucket", + "freshdesk.com": "Freshdesk", + "zendesk.com": "Zendesk", + "uservoice.com": "UserVoice", + "teamwork.com": "Teamwork", + "helpjuice.com": "Helpjuice", + "helpscoutdocs.com": "HelpScout", + "feedpress.me": "Feedpress", + "readme.io": "Readme.io", + "statuspage.io": "Statuspage", + "azurewebsites.net": "Azure", + "cloudapp.net": "Azure", + "trafficmanager.net": "Azure", + "blob.core.windows.net": "Azure Blob", + "cloudfront.net": "AWS CloudFront", + "s3.amazonaws.com": "AWS S3", + "elasticbeanstalk.com": "AWS Elastic Beanstalk", + "amazonaws.com": "AWS", + "storage.googleapis.com": "Google Cloud Storage", + "appspot.com": "Google App Engine", + "firebaseapp.com": "Firebase", + "netlify.app": "Netlify", + "vercel.app": "Vercel", + "now.sh": "Vercel", + "fly.dev": "Fly.io", + "render.com": "Render", +} + + +# ============================================================================= +# UTILITY FUNCTIONS +# ============================================================================= def check_tool(tool_name: str) -> Tuple[bool, Optional[str]]: - """Verifica se uma ferramenta esta instalada.""" + """Check if a tool is installed and return its path.""" path = shutil.which(tool_name) return (path is not None, path) -def run_tool(cmd: List[str], timeout: int = 300) -> Dict: - """Executa uma ferramenta e retorna o resultado.""" +def run_tool(cmd: List[str], timeout: int = 300, stdin_data: str = None) -> Dict: + """Execute a tool and return structured results.""" result = { "tool": cmd[0] if cmd else "unknown", "command": " ".join(cmd), "success": False, "stdout": "", "stderr": "", - "exit_code": -1 + "exit_code": -1, + "timed_out": False } - if not shutil.which(cmd[0]): - result["stderr"] = f"Tool '{cmd[0]}' not found" + tool_path = shutil.which(cmd[0]) + if not tool_path: + result["stderr"] = f"Tool '{cmd[0]}' not found in PATH" return result try: @@ -51,7 +147,8 @@ def run_tool(cmd: List[str], timeout: int = 300) -> Dict: cmd, capture_output=True, text=True, - timeout=timeout + timeout=timeout, + input=stdin_data ) result["stdout"] = proc.stdout result["stderr"] = proc.stderr @@ -59,523 +156,298 @@ def run_tool(cmd: List[str], timeout: int = 300) -> Dict: result["success"] = proc.returncode == 0 except subprocess.TimeoutExpired: result["stderr"] = f"Timeout after {timeout}s" + result["timed_out"] = True except Exception as e: result["stderr"] = str(e) return result -class NetworkScanner: - """Network scanning and port enumeration""" - - def __init__(self, config: Dict): - self.config = config - self.nmap_path = config.get('tools', {}).get('nmap', '/usr/bin/nmap') - - def scan(self, target: str) -> Dict: - """Perform comprehensive network scan""" - logger.info(f"Scanning target: {target}") - - results = { - "target": target, - "hosts": {}, - "summary": {} - } - - try: - # Quick scan for open ports - quick_scan = self._nmap_scan(target, "-sS -T4 -p-") - results["hosts"].update(quick_scan) - - # Service version detection - if results["hosts"]: - version_scan = self._nmap_scan(target, "-sV -sC") - results["hosts"].update(version_scan) - - # Vulnerability scan - vuln_scan = self._nmap_vuln_scan(target) - results["vulnerabilities"] = vuln_scan - - results["summary"] = self._generate_summary(results["hosts"]) - - except Exception as e: - logger.error(f"Network scan error: {e}") - results["error"] = str(e) - - return results - - def _nmap_scan(self, target: str, options: str) -> Dict: - """Execute nmap scan""" - try: - cmd = f"{self.nmap_path} {options} {target} -oX -" - result = subprocess.run( - cmd.split(), - capture_output=True, - timeout=300, - text=True - ) - - return self._parse_nmap_output(result.stdout) - except Exception as e: - logger.error(f"Nmap scan error: {e}") - return {} - - def _nmap_vuln_scan(self, target: str) -> List[Dict]: - """Scan for vulnerabilities using NSE scripts""" - try: - cmd = f"{self.nmap_path} --script vuln {target}" - result = subprocess.run( - cmd.split(), - capture_output=True, - timeout=600, - text=True - ) - - return self._parse_vuln_output(result.stdout) - except Exception as e: - logger.error(f"Vulnerability scan error: {e}") - return [] - - def _parse_nmap_output(self, output: str) -> Dict: - """Parse nmap XML output""" - hosts = {} - - # Simple parsing - in production, use proper XML parser - ip_pattern = r'(\d+\.\d+\.\d+\.\d+)' - port_pattern = r'(\d+)/tcp\s+open\s+(\S+)' - - current_ip = None - for line in output.split('\n'): - ip_match = re.search(ip_pattern, line) - if ip_match and 'Nmap scan report' in line: - current_ip = ip_match.group(1) - hosts[current_ip] = { - "ip": current_ip, - "open_ports": [], - "os": "unknown" - } - - port_match = re.search(port_pattern, line) - if port_match and current_ip: - hosts[current_ip]["open_ports"].append({ - "port": int(port_match.group(1)), - "service": port_match.group(2), - "version": "unknown" - }) - - return hosts - - def _parse_vuln_output(self, output: str) -> List[Dict]: - """Parse vulnerability scan output""" - vulnerabilities = [] - - # Extract CVEs and vulnerability info - cve_pattern = r'(CVE-\d{4}-\d+)' - for match in re.finditer(cve_pattern, output): - vulnerabilities.append({ - "cve": match.group(1), - "severity": "unknown" - }) - - return vulnerabilities - - def _generate_summary(self, hosts: Dict) -> Dict: - """Generate scan summary""" - total_hosts = len(hosts) - total_ports = sum(len(h.get("open_ports", [])) for h in hosts.values()) - services = set() - - for host in hosts.values(): - for port in host.get("open_ports", []): - services.add(port.get("service")) - - return { - "total_hosts": total_hosts, - "total_open_ports": total_ports, - "unique_services": list(services) - } +def get_wordlist(name: str, fallback: str = None) -> Optional[str]: + """Get wordlist path, checking if it exists.""" + path = WORDLISTS.get(name) + if path and os.path.exists(path): + return path + if fallback and os.path.exists(fallback): + return fallback + return None -class WebRecon: - """Web application reconnaissance""" - - def __init__(self, config: Dict): - self.config = config - - def analyze(self, url: str) -> Dict: - """Analyze web application""" - logger.info(f"Analyzing web application: {url}") - - results = { - "url": url, - "technologies": [], - "headers": {}, - "security_headers": {}, - "endpoints": [], - "forms": [], - "vulnerabilities": [] - } - - try: - # Technology detection - results["technologies"] = self._detect_technologies(url) - - # Header analysis - results["headers"], results["security_headers"] = self._analyze_headers(url) - - # Endpoint discovery - results["endpoints"] = self._discover_endpoints(url) - - # Form detection - results["forms"] = self._detect_forms(url) - - # Quick vulnerability checks - results["vulnerabilities"] = self._check_vulnerabilities(url) - - except Exception as e: - logger.error(f"Web recon error: {e}") - results["error"] = str(e) - - return results - - def _detect_technologies(self, url: str) -> List[str]: - """Detect web technologies""" - technologies = [] - - try: - response = requests.get(url, timeout=10, verify=False) - - # Check headers for technology indicators - server = response.headers.get('Server', '') - if server: - technologies.append(f"Server: {server}") - - x_powered_by = response.headers.get('X-Powered-By', '') - if x_powered_by: - technologies.append(f"X-Powered-By: {x_powered_by}") - - # Check content for framework indicators - content = response.text.lower() - if 'wordpress' in content: - technologies.append("WordPress") - if 'joomla' in content: - technologies.append("Joomla") - if 'drupal' in content: - technologies.append("Drupal") - if 'django' in content: - technologies.append("Django") - if 'laravel' in content: - technologies.append("Laravel") - - except Exception as e: - logger.error(f"Technology detection error: {e}") - - return technologies - - def _analyze_headers(self, url: str) -> tuple: - """Analyze HTTP headers""" - headers = {} - security_headers = {} - - try: - response = requests.head(url, timeout=10, verify=False) - headers = dict(response.headers) - - # Check for security headers - security_checks = [ - 'X-Frame-Options', - 'X-Content-Type-Options', - 'Strict-Transport-Security', - 'Content-Security-Policy', - 'X-XSS-Protection' - ] - - for header in security_checks: - if header in headers: - security_headers[header] = headers[header] - else: - security_headers[header] = "Missing" - - except Exception as e: - logger.error(f"Header analysis error: {e}") - - return headers, security_headers - - def _discover_endpoints(self, url: str) -> List[str]: - """Discover endpoints using common paths""" - endpoints = [] - common_paths = [ - '/admin', '/login', '/api', '/config', '/backup', - '/admin.php', '/phpinfo.php', '/info.php', - '/robots.txt', '/sitemap.xml', '/.git', '/.env' - ] - - parsed = urlparse(url) - base_url = f"{parsed.scheme}://{parsed.netloc}" - - for path in common_paths: - try: - response = requests.head( - f"{base_url}{path}", - timeout=5, - verify=False, - allow_redirects=False - ) - - if response.status_code < 400: - endpoints.append(path) - except: - continue - - return endpoints - - def _detect_forms(self, url: str) -> List[Dict]: - """Detect forms on webpage""" - forms = [] - - try: - response = requests.get(url, timeout=10, verify=False) - - # Simple form detection - form_pattern = r']*>(.*?)' - for match in re.finditer(form_pattern, response.text, re.DOTALL): - forms.append({ - "action": re.search(r'action=["\']([^"\']+)["\']', match.group(0)), - "method": re.search(r'method=["\']([^"\']+)["\']', match.group(0)) - }) - except Exception as e: - logger.error(f"Form detection error: {e}") - - return forms - - def _check_vulnerabilities(self, url: str) -> List[str]: - """Quick vulnerability checks""" - vulnerabilities = [] - - try: - # SQL Injection test - test_url = f"{url}?id=1'" - response = requests.get(test_url, timeout=10, verify=False) - if 'sql' in response.text.lower() or 'mysql' in response.text.lower(): - vulnerabilities.append("Potential SQL Injection") - - # XSS test - test_url = f"{url}?q=" - response = requests.get(test_url, timeout=10, verify=False) - if '' in response.text: - vulnerabilities.append("Potential XSS") - - except Exception as e: - logger.error(f"Vulnerability check error: {e}") - - return vulnerabilities +def extract_domain(target: str) -> str: + """Extract domain from URL or return as-is.""" + if target.startswith(('http://', 'https://')): + return urlparse(target).netloc + return target -class OSINTCollector: - """Open Source Intelligence collection""" - - def __init__(self, config: Dict): - self.config = config - - def collect(self, target: str) -> Dict: - """Collect OSINT data""" - logger.info(f"Collecting OSINT for: {target}") - - return { - "target": target, - "emails": self._find_emails(target), - "social_media": self._find_social_media(target), - "data_breaches": self._check_breaches(target), - "metadata": self._collect_metadata(target) - } - - def _find_emails(self, target: str) -> List[str]: - """Find email addresses""" - # Placeholder - would use theHarvester or similar - return [] - - def _find_social_media(self, target: str) -> Dict: - """Find social media profiles""" - return {} - - def _check_breaches(self, target: str) -> List[str]: - """Check for data breaches""" - return [] - - def _collect_metadata(self, target: str) -> Dict: - """Collect metadata""" - return {} +def make_url(host: str, scheme: str = "https") -> str: + """Ensure host has proper URL format.""" + if host.startswith(('http://', 'https://')): + return host + return f"{scheme}://{host}" -class DNSEnumerator: - """DNS enumeration""" - - def __init__(self, config: Dict): - self.config = config - - def enumerate(self, domain: str) -> Dict: - """Enumerate DNS records""" - logger.info(f"Enumerating DNS for: {domain}") - - records = { - "domain": domain, - "A": [], - "AAAA": [], - "MX": [], - "NS": [], - "TXT": [], - "SOA": [] - } - - record_types = ['A', 'AAAA', 'MX', 'NS', 'TXT', 'SOA'] - - for rtype in record_types: - try: - answers = dns.resolver.resolve(domain, rtype) - records[rtype] = [str(rdata) for rdata in answers] - except: - continue - - return records +def print_phase(phase_num: int, title: str): + """Print phase header.""" + print(f"\n{'='*60}") + print(f"[PHASE {phase_num}] {title}") + print(f"{'='*60}") -class SubdomainFinder: - """Subdomain discovery""" - - def __init__(self, config: Dict): - self.config = config - - def find(self, domain: str) -> List[str]: - """Find subdomains""" - logger.info(f"Finding subdomains for: {domain}") - - subdomains = [] - common_subdomains = [ - 'www', 'mail', 'ftp', 'admin', 'test', 'dev', - 'staging', 'api', 'blog', 'shop', 'portal' - ] - - for sub in common_subdomains: - subdomain = f"{sub}.{domain}" - try: - socket.gethostbyname(subdomain) - subdomains.append(subdomain) - except: - continue - - return subdomains +def print_result(icon: str, msg: str): + """Print formatted result.""" + print(f" {icon} {msg}") -# ============================================================================ -# ADVANCED RECON TOOLS -# ============================================================================ +# ============================================================================= +# ADVANCED SUBDOMAIN ENUMERATION +# ============================================================================= class AdvancedSubdomainEnum: - """Advanced subdomain enumeration using multiple tools.""" + """Deep subdomain enumeration using multiple tools and techniques.""" - TOOLS = ['subfinder', 'amass', 'assetfinder', 'findomain'] + TOOLS = ['subfinder', 'amass', 'assetfinder', 'findomain', 'puredns', 'shuffledns'] - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 300) + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 300) - def enumerate(self, domain: str) -> Dict: - """Enumerate subdomains using all available tools.""" - logger.info(f"[*] Enumerating subdomains for: {domain}") - print(f"[*] Enumerating subdomains for: {domain}") + def enumerate(self, domain: str, depth: str = "medium") -> Dict: + """ + Enumerate subdomains with multiple tools. + + Args: + domain: Target domain + depth: quick, medium, deep + """ + logger.info(f"[*] Subdomain enumeration for: {domain}") + print(f"[*] Starting subdomain enumeration for: {domain}") + print(f" Depth: {depth}") all_subdomains: Set[str] = set() - results = {"domain": domain, "subdomains": [], "by_tool": {}} + results = { + "domain": domain, + "subdomains": [], + "by_tool": {}, + "crt_sh": [], + "dns_bruteforce": [] + } - for tool in self.TOOLS: + # 1. Certificate Transparency (crt.sh) - Always run first (passive) + print_result("[~]", "Querying Certificate Transparency logs (crt.sh)...") + crt_subs = self._crtsh_enum(domain) + results["crt_sh"] = crt_subs + all_subdomains.update(crt_subs) + print_result("[+]", f"crt.sh: {len(crt_subs)} subdomains") + + # 2. Run enumeration tools + tools_to_run = self.TOOLS if depth == "deep" else self.TOOLS[:4] + + for tool in tools_to_run: installed, _ = check_tool(tool) if not installed: - logger.warning(f" [-] {tool} not installed, skipping...") continue - print(f" [*] Running {tool}...") + print_result("[~]", f"Running {tool}...") tool_subs = self._run_tool(tool, domain) results["by_tool"][tool] = tool_subs all_subdomains.update(tool_subs) - print(f" [+] {tool}: {len(tool_subs)} subdomains") + print_result("[+]", f"{tool}: {len(tool_subs)} subdomains") + + # 3. DNS Bruteforce (for deep mode) + if depth == "deep": + wordlist = get_wordlist("subdomains_medium") + if wordlist: + print_result("[~]", "Running DNS bruteforce...") + brute_subs = self._dns_bruteforce(domain, wordlist) + results["dns_bruteforce"] = brute_subs + all_subdomains.update(brute_subs) + print_result("[+]", f"Bruteforce: {len(brute_subs)} subdomains") + + # 4. Permutation/mutation (for deep mode) + if depth == "deep" and all_subdomains: + print_result("[~]", "Generating permutations...") + perms = self._generate_permutations(list(all_subdomains)[:100], domain) + valid_perms = self._resolve_subdomains(perms) + all_subdomains.update(valid_perms) + print_result("[+]", f"Permutations: {len(valid_perms)} valid") results["subdomains"] = sorted(list(all_subdomains)) results["total"] = len(all_subdomains) - print(f"[+] Total unique subdomains: {len(all_subdomains)}") + print_result("[✓]", f"Total unique subdomains: {len(all_subdomains)}") return results + def _crtsh_enum(self, domain: str) -> List[str]: + """Query crt.sh Certificate Transparency logs.""" + subdomains = set() + try: + url = f"https://crt.sh/?q=%.{domain}&output=json" + resp = requests.get(url, timeout=30) + if resp.status_code == 200: + data = resp.json() + for entry in data: + name = entry.get("name_value", "") + for sub in name.split("\n"): + sub = sub.strip().lower() + if sub and "*" not in sub and domain in sub: + subdomains.add(sub) + except Exception as e: + logger.warning(f"crt.sh error: {e}") + return list(subdomains) + def _run_tool(self, tool: str, domain: str) -> List[str]: - """Run a specific tool.""" + """Run specific subdomain enumeration tool.""" subdomains = [] - if tool == "subfinder": - result = run_tool(["subfinder", "-d", domain, "-silent"], self.timeout) - elif tool == "amass": - result = run_tool(["amass", "enum", "-passive", "-d", domain], self.timeout) - elif tool == "assetfinder": - result = run_tool(["assetfinder", "--subs-only", domain], self.timeout) - elif tool == "findomain": - result = run_tool(["findomain", "-t", domain, "-q"], self.timeout) - else: + cmd_map = { + "subfinder": ["subfinder", "-d", domain, "-silent", "-all"], + "amass": ["amass", "enum", "-passive", "-d", domain, "-silent"], + "assetfinder": ["assetfinder", "--subs-only", domain], + "findomain": ["findomain", "-t", domain, "-q"], + "puredns": ["puredns", "bruteforce", get_wordlist("subdomains_small") or "", domain, "-q"], + "shuffledns": ["shuffledns", "-d", domain, "-w", get_wordlist("subdomains_small") or "", "-silent"] + } + + cmd = cmd_map.get(tool) + if not cmd: return [] + result = run_tool(cmd, self.timeout) if result["stdout"]: for line in result["stdout"].strip().split('\n'): sub = line.strip().lower() - if sub and domain in sub: + if sub and domain in sub and "*" not in sub: subdomains.append(sub) return subdomains + def _dns_bruteforce(self, domain: str, wordlist: str) -> List[str]: + """DNS bruteforce using wordlist.""" + found = [] + try: + with open(wordlist, 'r') as f: + words = [w.strip() for w in f.readlines()[:5000]] + + def check_sub(word): + subdomain = f"{word}.{domain}" + try: + socket.gethostbyname(subdomain) + return subdomain + except: + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor: + results = executor.map(check_sub, words) + found = [r for r in results if r] + except Exception as e: + logger.warning(f"DNS bruteforce error: {e}") + return found + + def _generate_permutations(self, subdomains: List[str], domain: str) -> List[str]: + """Generate subdomain permutations.""" + permutations = set() + prefixes = ['dev', 'staging', 'stage', 'test', 'uat', 'qa', 'prod', 'api', 'admin', 'internal', 'private', 'beta', 'alpha', 'old', 'new', 'v1', 'v2'] + suffixes = ['-dev', '-staging', '-test', '-api', '-admin', '-internal', '2', '-v2', '-old', '-new'] + + for sub in subdomains: + parts = sub.replace(f".{domain}", "").split(".") + if parts: + base = parts[0] + for prefix in prefixes: + permutations.add(f"{prefix}.{sub}") + permutations.add(f"{prefix}-{base}.{domain}") + for suffix in suffixes: + permutations.add(f"{base}{suffix}.{domain}") + + return list(permutations)[:1000] + + def _resolve_subdomains(self, subdomains: List[str]) -> List[str]: + """Resolve subdomains to check if they exist.""" + valid = [] + + def resolve(sub): + try: + socket.gethostbyname(sub) + return sub + except: + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor: + results = executor.map(resolve, subdomains) + valid = [r for r in results if r] + + return valid + + +# ============================================================================= +# HTTP PROBING +# ============================================================================= class HttpProber: - """Check active HTTP hosts using httpx or httprobe.""" + """Advanced HTTP probing with technology detection.""" - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 120) + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 180) def probe(self, hosts: List[str]) -> Dict: - """Check which hosts are alive via HTTP/HTTPS.""" + """Probe hosts for HTTP/HTTPS with detailed info.""" logger.info(f"[*] Probing {len(hosts)} hosts...") - print(f"[*] Probing {len(hosts)} hosts via HTTP...") + print(f"[*] Probing {len(hosts)} hosts for HTTP/HTTPS...") results = { "total_input": len(hosts), "alive": [], + "details": {}, "technologies": {}, - "status_codes": {} + "status_codes": {}, + "by_status": defaultdict(list), + "redirects": [], + "interesting": [] } - # Try httpx first (more complete) - httpx_installed, _ = check_tool("httpx") - if httpx_installed: + httpx_ok, _ = check_tool("httpx") + if httpx_ok: results = self._run_httpx(hosts) else: - # Fallback to httprobe - httprobe_installed, _ = check_tool("httprobe") - if httprobe_installed: - results = self._run_httprobe(hosts) - else: - # Manual fallback with curl - results = self._manual_probe(hosts) + results = self._manual_probe(hosts) + + # Identify interesting hosts + results["interesting"] = self._identify_interesting(results) + + print_result("[+]", f"Alive hosts: {len(results['alive'])}") + print_result("[+]", f"Technologies found: {len(results['technologies'])}") + + if results["interesting"]: + print_result("[!]", f"Interesting hosts: {len(results['interesting'])}") - print(f"[+] Alive hosts: {len(results['alive'])}") return results def _run_httpx(self, hosts: List[str]) -> Dict: - """Run httpx for advanced probing.""" - results = {"total_input": len(hosts), "alive": [], "technologies": {}, "status_codes": {}} + """Run httpx with maximum output.""" + results = { + "total_input": len(hosts), + "alive": [], + "details": {}, + "technologies": {}, + "status_codes": {}, + "by_status": defaultdict(list), + "redirects": [], + "interesting": [] + } - # Create temp file with hosts - import tempfile with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: f.write('\n'.join(hosts)) hosts_file = f.name try: - cmd = ["httpx", "-l", hosts_file, "-silent", "-status-code", "-tech-detect", "-json"] + cmd = [ + "httpx", "-l", hosts_file, "-silent", + "-status-code", "-content-length", "-title", "-tech-detect", + "-web-server", "-cdn", "-follow-redirects", "-json", + "-threads", "50", "-timeout", "10" + ] result = run_tool(cmd, self.timeout) if result["stdout"]: @@ -588,15 +460,34 @@ class HttpProber: if url: results["alive"].append(url) + # Store detailed info + results["details"][url] = { + "status": data.get("status_code"), + "title": data.get("title", ""), + "server": data.get("webserver", ""), + "content_length": data.get("content_length", 0), + "technologies": data.get("tech", []), + "cdn": data.get("cdn", False), + "final_url": data.get("final_url", url) + } + + # Track redirects + if data.get("final_url") and data["final_url"] != url: + results["redirects"].append({ + "from": url, + "to": data["final_url"] + }) + # Technologies - techs = data.get("tech", []) - for tech in techs: + for tech in data.get("tech", []): results["technologies"][tech] = results["technologies"].get(tech, 0) + 1 # Status codes status = str(data.get("status_code", "")) if status: results["status_codes"][status] = results["status_codes"].get(status, 0) + 1 + results["by_status"][status].append(url) + except json.JSONDecodeError: continue finally: @@ -604,111 +495,559 @@ class HttpProber: return results - def _run_httprobe(self, hosts: List[str]) -> Dict: - """Run httprobe for basic probing.""" - results = {"total_input": len(hosts), "alive": [], "technologies": {}, "status_codes": {}} - - import tempfile - with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: - f.write('\n'.join(hosts)) - hosts_file = f.name - - try: - with open(hosts_file, 'r') as stdin: - proc = subprocess.run( - ["httprobe"], - stdin=stdin, - capture_output=True, - text=True, - timeout=self.timeout - ) - if proc.stdout: - results["alive"] = [line.strip() for line in proc.stdout.strip().split('\n') if line.strip()] - except Exception as e: - logger.error(f"httprobe error: {e}") - finally: - os.unlink(hosts_file) - - return results - def _manual_probe(self, hosts: List[str]) -> Dict: - """Manual probing using requests.""" - results = {"total_input": len(hosts), "alive": [], "technologies": {}, "status_codes": {}} + """Manual HTTP probing fallback.""" + results = { + "total_input": len(hosts), + "alive": [], + "details": {}, + "technologies": {}, + "status_codes": {}, + "by_status": defaultdict(list), + "redirects": [], + "interesting": [] + } - for host in hosts[:100]: # Limit to avoid long execution - for scheme in ['https://', 'http://']: - url = f"{scheme}{host}" if not host.startswith('http') else host + def probe_host(host): + for scheme in ['https', 'http']: + url = make_url(host, scheme) try: - resp = requests.head(url, timeout=5, verify=False, allow_redirects=True) - results["alive"].append(url) - results["status_codes"][str(resp.status_code)] = results["status_codes"].get(str(resp.status_code), 0) + 1 - break + resp = requests.get(url, timeout=10, verify=False, allow_redirects=True) + return { + "url": url, + "status": resp.status_code, + "title": re.search(r'(.*?)', resp.text, re.I), + "server": resp.headers.get("Server", ""), + "headers": dict(resp.headers) + } + except: + continue + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=30) as executor: + futures = {executor.submit(probe_host, h): h for h in hosts[:500]} + for future in concurrent.futures.as_completed(futures): + try: + data = future.result() + if data: + url = data["url"] + results["alive"].append(url) + results["details"][url] = data + status = str(data["status"]) + results["status_codes"][status] = results["status_codes"].get(status, 0) + 1 + results["by_status"][status].append(url) except: continue return results + def _identify_interesting(self, results: Dict) -> List[Dict]: + """Identify potentially interesting hosts.""" + interesting = [] + + for url, details in results.get("details", {}).items(): + reasons = [] + + # Check for interesting status codes + status = details.get("status", 0) + if status in [401, 403, 500, 502, 503]: + reasons.append(f"Status {status}") + + # Check for interesting titles + title = details.get("title", "").lower() + interesting_titles = ['admin', 'login', 'dashboard', 'panel', 'jenkins', 'gitlab', 'jira', 'confluence', 'kibana', 'grafana', 'debug', 'staging', 'internal'] + for t in interesting_titles: + if t in title: + reasons.append(f"Title contains '{t}'") + break + + # Check for interesting technologies + techs = details.get("technologies", []) + risky_techs = ['Apache Tomcat', 'Jenkins', 'GitLab', 'Jira', 'Confluence', 'Elasticsearch', 'Kibana', 'Grafana', 'phpMyAdmin', 'WordPress', 'Drupal'] + for tech in techs: + if any(rt.lower() in tech.lower() for rt in risky_techs): + reasons.append(f"Technology: {tech}") + + if reasons: + interesting.append({"url": url, "reasons": reasons}) + + return interesting + + +# ============================================================================= +# DIRECTORY BRUTEFORCE WITH FEROXBUSTER +# ============================================================================= + +class DirectoryBruter: + """Directory/file bruteforcing using feroxbuster or fallbacks.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 600) + + def bruteforce(self, target: str, wordlist_size: str = "medium", extensions: List[str] = None) -> Dict: + """ + Directory bruteforce using feroxbuster. + + Args: + target: Target URL + wordlist_size: small, medium, big + extensions: File extensions to check + """ + logger.info(f"[*] Directory bruteforce on: {target}") + print(f"[*] Starting directory bruteforce on: {target}") + + results = { + "target": target, + "directories": [], + "files": [], + "interesting": [], + "status_codes": {}, + "total": 0 + } + + # Get wordlist + wordlist_key = f"directories_{wordlist_size}" + wordlist = get_wordlist(wordlist_key, WORDLISTS.get("common")) + + if not wordlist: + print_result("[-]", "No wordlist available") + return results + + print_result("[~]", f"Using wordlist: {os.path.basename(wordlist)}") + + # Default extensions + if not extensions: + extensions = ["php", "asp", "aspx", "jsp", "html", "js", "json", "xml", "txt", "bak", "old", "conf", "config", "sql", "zip", "tar.gz", "log"] + + ferox_ok, _ = check_tool("feroxbuster") + if ferox_ok: + results = self._run_feroxbuster(target, wordlist, extensions) + else: + # Fallback to gobuster or ffuf + gobuster_ok, _ = check_tool("gobuster") + if gobuster_ok: + results = self._run_gobuster(target, wordlist, extensions) + else: + ffuf_ok, _ = check_tool("ffuf") + if ffuf_ok: + results = self._run_ffuf(target, wordlist, extensions) + else: + print_result("[-]", "No directory bruteforce tool available") + return results + + # Identify interesting findings + results["interesting"] = self._identify_interesting(results) + + print_result("[+]", f"Total found: {results['total']}") + print_result("[+]", f"Directories: {len(results['directories'])}") + print_result("[+]", f"Files: {len(results['files'])}") + if results["interesting"]: + print_result("[!]", f"Interesting: {len(results['interesting'])}") + + return results + + def _run_feroxbuster(self, target: str, wordlist: str, extensions: List[str]) -> Dict: + """Run feroxbuster for directory bruteforce.""" + results = { + "target": target, + "directories": [], + "files": [], + "interesting": [], + "status_codes": {}, + "total": 0 + } + + ext_str = ",".join(extensions) + cmd = [ + "feroxbuster", + "-u", target, + "-w", wordlist, + "-x", ext_str, + "-t", "50", + "-C", "404,400", + "--silent", + "--no-state", + "-o", "-", + "--json" + ] + + result = run_tool(cmd, self.timeout) + + if result["stdout"]: + for line in result["stdout"].strip().split('\n'): + if not line.strip() or not line.startswith('{'): + continue + try: + data = json.loads(line) + if data.get("type") == "response": + entry = { + "url": data.get("url", ""), + "status": data.get("status", 0), + "size": data.get("content_length", 0), + "words": data.get("word_count", 0), + "lines": data.get("line_count", 0) + } + + if entry["url"]: + results["total"] += 1 + status = str(entry["status"]) + results["status_codes"][status] = results["status_codes"].get(status, 0) + 1 + + if entry["url"].endswith('/'): + results["directories"].append(entry) + else: + results["files"].append(entry) + except: + continue + + return results + + def _run_gobuster(self, target: str, wordlist: str, extensions: List[str]) -> Dict: + """Run gobuster as fallback.""" + results = { + "target": target, + "directories": [], + "files": [], + "interesting": [], + "status_codes": {}, + "total": 0 + } + + ext_str = ",".join(extensions) + cmd = [ + "gobuster", "dir", + "-u", target, + "-w", wordlist, + "-x", ext_str, + "-t", "50", + "-q", + "--no-error" + ] + + result = run_tool(cmd, self.timeout) + + if result["stdout"]: + pattern = r"(\S+)\s+\(Status:\s*(\d+)\)" + for match in re.finditer(pattern, result["stdout"]): + path, status = match.groups() + entry = {"url": urljoin(target, path), "status": int(status), "size": 0} + results["total"] += 1 + results["status_codes"][status] = results["status_codes"].get(status, 0) + 1 + + if path.endswith('/'): + results["directories"].append(entry) + else: + results["files"].append(entry) + + return results + + def _run_ffuf(self, target: str, wordlist: str, extensions: List[str]) -> Dict: + """Run ffuf as fallback.""" + results = { + "target": target, + "directories": [], + "files": [], + "interesting": [], + "status_codes": {}, + "total": 0 + } + + fuzz_url = f"{target.rstrip('/')}/FUZZ" + cmd = [ + "ffuf", + "-u", fuzz_url, + "-w", wordlist, + "-t", "50", + "-mc", "200,201,204,301,302,307,308,401,403,405,500", + "-o", "-", + "-of", "json", + "-s" + ] + + result = run_tool(cmd, self.timeout) + + if result["stdout"]: + try: + data = json.loads(result["stdout"]) + for entry in data.get("results", []): + item = { + "url": entry.get("url", ""), + "status": entry.get("status", 0), + "size": entry.get("length", 0) + } + results["total"] += 1 + status = str(item["status"]) + results["status_codes"][status] = results["status_codes"].get(status, 0) + 1 + + if item["url"].endswith('/'): + results["directories"].append(item) + else: + results["files"].append(item) + except: + pass + + return results + + def _identify_interesting(self, results: Dict) -> List[Dict]: + """Identify interesting findings.""" + interesting = [] + interesting_patterns = [ + r'\.(?:bak|backup|old|orig|save|swp|tmp)$', + r'\.(?:sql|db|mdb|sqlite)$', + r'\.(?:conf|config|cfg|ini|env)$', + r'\.(?:log|logs)$', + r'(?:admin|login|dashboard|panel|console)', + r'(?:upload|uploads|files|backup)', + r'(?:api|v1|v2|graphql)', + r'(?:\.git|\.svn|\.hg)', + r'(?:phpinfo|info\.php|test\.php)', + r'(?:wp-admin|wp-content|wp-includes)', + r'(?:install|setup|config)', + ] + + all_items = results["directories"] + results["files"] + for item in all_items: + url = item.get("url", "").lower() + for pattern in interesting_patterns: + if re.search(pattern, url): + interesting.append(item) + break + + return interesting + + +# ============================================================================= +# PARAMETER SPIDER +# ============================================================================= + +class ParamSpider: + """Parameter discovery using paramspider and analysis.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 300) + + def spider(self, domain: str) -> Dict: + """Discover parameters from various sources.""" + logger.info(f"[*] Parameter discovery for: {domain}") + print(f"[*] Starting parameter discovery for: {domain}") + + results = { + "domain": domain, + "urls_with_params": [], + "unique_params": set(), + "by_param": defaultdict(list), + "interesting_params": [], + "total": 0 + } + + # Try paramspider + paramspider_ok, _ = check_tool("paramspider") + if paramspider_ok: + print_result("[~]", "Running paramspider...") + ps_results = self._run_paramspider(domain) + results["urls_with_params"].extend(ps_results) + else: + print_result("[-]", "paramspider not available, using alternative methods...") + + # Also collect from gau/waybackurls + print_result("[~]", "Collecting URLs from archives...") + archive_urls = self._collect_archive_urls(domain) + + # Parse parameters + all_urls = results["urls_with_params"] + archive_urls + for url in all_urls: + params = self._extract_params(url) + for param in params: + results["unique_params"].add(param) + results["by_param"][param].append(url) + + results["unique_params"] = list(results["unique_params"]) + results["total"] = len(all_urls) + + # Identify interesting parameters + results["interesting_params"] = self._identify_interesting_params(results["unique_params"]) + + # Convert defaultdict to regular dict for JSON serialization + results["by_param"] = dict(results["by_param"]) + + print_result("[+]", f"URLs with params: {len(all_urls)}") + print_result("[+]", f"Unique parameters: {len(results['unique_params'])}") + if results["interesting_params"]: + print_result("[!]", f"Interesting params: {', '.join(results['interesting_params'][:10])}") + + return results + + def _run_paramspider(self, domain: str) -> List[str]: + """Run paramspider tool.""" + urls = [] + + with tempfile.TemporaryDirectory() as tmpdir: + cmd = ["paramspider", "-d", domain, "-o", tmpdir, "-s"] + result = run_tool(cmd, self.timeout) + + # Read output files + for f in Path(tmpdir).glob("*.txt"): + try: + with open(f, 'r') as file: + urls.extend([line.strip() for line in file if '=' in line]) + except: + continue + + return urls + + def _collect_archive_urls(self, domain: str) -> List[str]: + """Collect URLs with parameters from archives.""" + urls = [] + + # Try gau + gau_ok, _ = check_tool("gau") + if gau_ok: + result = run_tool(["gau", "--subs", domain], self.timeout) + if result["stdout"]: + for line in result["stdout"].strip().split('\n'): + url = line.strip() + if '?' in url and '=' in url: + urls.append(url) + + # Try waybackurls + wayback_ok, _ = check_tool("waybackurls") + if wayback_ok: + result = run_tool(["waybackurls", domain], self.timeout) + if result["stdout"]: + for line in result["stdout"].strip().split('\n'): + url = line.strip() + if '?' in url and '=' in url: + urls.append(url) + + return list(set(urls)) + + def _extract_params(self, url: str) -> List[str]: + """Extract parameter names from URL.""" + params = [] + try: + parsed = urlparse(url) + query = parse_qs(parsed.query) + params = list(query.keys()) + except: + pass + return params + + def _identify_interesting_params(self, params: List[str]) -> List[str]: + """Identify potentially interesting/vulnerable parameters.""" + interesting = [] + + sqli_params = ['id', 'pid', 'uid', 'userid', 'user_id', 'item', 'itemid', 'cat', 'category', 'page', 'p', 'q', 'query', 'search', 's', 'keyword', 'order', 'sort', 'filter'] + xss_params = ['q', 'query', 'search', 's', 'keyword', 'name', 'username', 'user', 'email', 'message', 'msg', 'comment', 'text', 'content', 'title', 'desc', 'description', 'error', 'err', 'ref', 'callback', 'redirect', 'url', 'return', 'returnUrl', 'return_url', 'next', 'goto', 'dest', 'destination', 'redir'] + lfi_params = ['file', 'filename', 'path', 'filepath', 'page', 'include', 'inc', 'dir', 'document', 'doc', 'folder', 'root', 'pg', 'template', 'view'] + ssrf_params = ['url', 'uri', 'link', 'src', 'source', 'dest', 'redirect', 'uri', 'path', 'continue', 'return', 'page', 'feed', 'host', 'site', 'html', 'domain', 'callback', 'api'] + rce_params = ['cmd', 'exec', 'command', 'execute', 'ping', 'query', 'jump', 'code', 'reg', 'do', 'func', 'arg', 'option', 'load', 'process', 'step', 'read', 'function', 'req', 'feature', 'exe', 'module', 'payload', 'run', 'print'] + idor_params = ['id', 'user', 'userid', 'user_id', 'account', 'account_id', 'accountid', 'uid', 'pid', 'profile', 'profile_id', 'doc', 'document', 'order', 'order_id', 'orderid', 'invoice', 'invoice_id', 'number', 'no'] + + all_interesting = set(sqli_params + xss_params + lfi_params + ssrf_params + rce_params + idor_params) + + for param in params: + param_lower = param.lower() + if param_lower in all_interesting: + interesting.append(param) + + return interesting + + +# ============================================================================= +# URL COLLECTION +# ============================================================================= class URLCollector: - """Collect URLs using gau, waybackurls and waymore.""" + """Collect URLs using multiple passive sources.""" - TOOLS = ['gau', 'waybackurls', 'waymore'] + TOOLS = ['gau', 'waybackurls', 'waymore', 'hakrawler'] - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 300) + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 300) def collect(self, domain: str) -> Dict: """Collect URLs from passive sources.""" - logger.info(f"[*] Collecting URLs for: {domain}") - print(f"[*] Collecting historical URLs for: {domain}") + logger.info(f"[*] URL collection for: {domain}") + print(f"[*] Collecting URLs for: {domain}") all_urls: Set[str] = set() urls_with_params: Set[str] = set() js_files: Set[str] = set() - results = {"domain": domain, "urls": [], "by_tool": {}} + api_endpoints: Set[str] = set() + + results = { + "domain": domain, + "urls": [], + "urls_with_params": [], + "js_files": [], + "api_endpoints": [], + "by_tool": {}, + "by_extension": defaultdict(list), + "total": 0 + } for tool in self.TOOLS: installed, _ = check_tool(tool) if not installed: continue - print(f" [*] Running {tool}...") + print_result("[~]", f"Running {tool}...") tool_urls = self._run_tool(tool, domain) results["by_tool"][tool] = len(tool_urls) for url in tool_urls: all_urls.add(url) + + # Categorize + url_lower = url.lower() if '?' in url and '=' in url: urls_with_params.add(url) - if '.js' in url.lower(): + if '.js' in url_lower: js_files.add(url) + if any(x in url_lower for x in ['/api/', '/v1/', '/v2/', '/v3/', '/graphql', '/rest/', '/json/']): + api_endpoints.add(url) - print(f" [+] {tool}: {len(tool_urls)} URLs") + # By extension + ext_match = re.search(r'\.(\w{2,5})(?:\?|$)', url_lower) + if ext_match: + ext = ext_match.group(1) + results["by_extension"][ext].append(url) + + print_result("[+]", f"{tool}: {len(tool_urls)} URLs") results["urls"] = list(all_urls) results["urls_with_params"] = list(urls_with_params) results["js_files"] = list(js_files) + results["api_endpoints"] = list(api_endpoints) results["total"] = len(all_urls) - print(f"[+] Total unique URLs: {len(all_urls)}") - print(f"[+] URLs with params: {len(urls_with_params)}") - print(f"[+] JS files: {len(js_files)}") + results["by_extension"] = dict(results["by_extension"]) + + print_result("[✓]", f"Total unique URLs: {len(all_urls)}") + print_result("[+]", f"URLs with params: {len(urls_with_params)}") + print_result("[+]", f"JS files: {len(js_files)}") + print_result("[+]", f"API endpoints: {len(api_endpoints)}") return results def _run_tool(self, tool: str, domain: str) -> List[str]: - """Run a specific tool.""" + """Run URL collection tool.""" urls = [] - if tool == "gau": - result = run_tool(["gau", domain], self.timeout) - elif tool == "waybackurls": - result = run_tool(["waybackurls", domain], self.timeout) - elif tool == "waymore": - result = run_tool(["waymore", "-i", domain, "-mode", "U"], self.timeout) - else: + cmd_map = { + "gau": ["gau", "--subs", domain], + "waybackurls": ["waybackurls", domain], + "waymore": ["waymore", "-i", domain, "-mode", "U", "-oU", "-"], + "hakrawler": ["hakrawler", "-url", f"https://{domain}", "-subs", "-plain"] + } + + cmd = cmd_map.get(tool) + if not cmd: return [] + result = run_tool(cmd, self.timeout) if result["stdout"]: for line in result["stdout"].strip().split('\n'): url = line.strip() @@ -718,18 +1057,22 @@ class URLCollector: return urls -class WebCrawler: - """Web crawler using katana or gospider.""" +# ============================================================================= +# WEB CRAWLER +# ============================================================================= - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 300) - self.depth = config.get('crawl_depth', 3) +class WebCrawler: + """Web crawling with katana or gospider.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 300) + self.depth = self.config.get('crawl_depth', 3) def crawl(self, target: str) -> Dict: """Crawl target to discover URLs, forms, endpoints.""" logger.info(f"[*] Crawling: {target}") - print(f"[*] Starting crawl on: {target}") + print(f"[*] Crawling: {target}") results = { "target": target, @@ -737,27 +1080,51 @@ class WebCrawler: "forms": [], "js_files": [], "api_endpoints": [], - "params": [] + "params": [], + "comments": [], + "total": 0 } - # Try katana first - katana_installed, _ = check_tool("katana") - if katana_installed: + katana_ok, _ = check_tool("katana") + if katana_ok: results = self._run_katana(target) else: - # Fallback to gospider - gospider_installed, _ = check_tool("gospider") - if gospider_installed: + gospider_ok, _ = check_tool("gospider") + if gospider_ok: results = self._run_gospider(target) - print(f"[+] URLs discovered: {len(results.get('urls', []))}") + print_result("[+]", f"URLs discovered: {len(results.get('urls', []))}") + print_result("[+]", f"JS files: {len(results.get('js_files', []))}") + print_result("[+]", f"API endpoints: {len(results.get('api_endpoints', []))}") + return results def _run_katana(self, target: str) -> Dict: - """Run katana for crawling.""" - results = {"target": target, "urls": [], "forms": [], "js_files": [], "api_endpoints": [], "params": []} + """Run katana for advanced crawling.""" + results = { + "target": target, + "urls": [], + "forms": [], + "js_files": [], + "api_endpoints": [], + "params": [], + "comments": [], + "total": 0 + } + + cmd = [ + "katana", + "-u", target, + "-d", str(self.depth), + "-silent", + "-jc", # JavaScript crawling + "-kf", "all", # Known files + "-ef", "css,png,jpg,jpeg,gif,svg,ico,woff,woff2,ttf,eot", + "-ct", "60", # Concurrency + "-timeout", "10", + "-aff" # Automatic form filling + ] - cmd = ["katana", "-u", target, "-d", str(self.depth), "-silent", "-jc", "-kf", "all", "-ef", "css,png,jpg,gif,svg,ico"] result = run_tool(cmd, self.timeout) if result["stdout"]: @@ -767,10 +1134,12 @@ class WebCrawler: continue results["urls"].append(url) + results["total"] += 1 + url_lower = url.lower() - if '.js' in url.lower(): + if '.js' in url_lower: results["js_files"].append(url) - if '/api/' in url.lower() or '/v1/' in url or '/v2/' in url: + if any(x in url_lower for x in ['/api/', '/v1/', '/v2/', '/graphql', '/rest/']): results["api_endpoints"].append(url) if '?' in url and '=' in url: results["params"].append(url) @@ -778,15 +1147,32 @@ class WebCrawler: return results def _run_gospider(self, target: str) -> Dict: - """Run gospider for crawling.""" - results = {"target": target, "urls": [], "forms": [], "js_files": [], "api_endpoints": [], "params": []} + """Run gospider as fallback.""" + results = { + "target": target, + "urls": [], + "forms": [], + "js_files": [], + "api_endpoints": [], + "params": [], + "comments": [], + "total": 0 + } + + cmd = [ + "gospider", + "-s", target, + "-d", str(self.depth), + "-c", "10", + "-t", "5", + "--js", + "-q" + ] - cmd = ["gospider", "-s", target, "-d", str(self.depth), "--no-redirect", "-q"] result = run_tool(cmd, self.timeout) if result["stdout"]: for line in result["stdout"].strip().split('\n'): - # gospider output: [source] URL if ' - ' in line: url = line.split(' - ')[-1].strip() else: @@ -794,10 +1180,12 @@ class WebCrawler: if url and url.startswith(('http://', 'https://')): results["urls"].append(url) + results["total"] += 1 + url_lower = url.lower() - if '.js' in url.lower(): + if '.js' in url_lower: results["js_files"].append(url) - if '/api/' in url.lower(): + if '/api/' in url_lower: results["api_endpoints"].append(url) if '?' in url: results["params"].append(url) @@ -805,109 +1193,320 @@ class WebCrawler: return results +# ============================================================================= +# ADVANCED PORT SCANNING +# ============================================================================= + class PortScanner: - """Port scanner using naabu or nmap.""" + """Advanced port scanning with rustscan, naabu, or nmap.""" - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 600) + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 600) - def scan(self, target: str, ports: str = "1-10000") -> Dict: - """Port scan on target.""" - logger.info(f"[*] Scanning ports on: {target}") - print(f"[*] Scanning ports on: {target}") + def scan(self, target: str, scan_type: str = "quick") -> Dict: + """ + Port scan with service detection. - results = {"target": target, "open_ports": [], "by_service": {}} + Args: + target: Target host/IP + scan_type: quick (top ports), full (all ports), stealth + """ + logger.info(f"[*] Port scanning: {target}") + print(f"[*] Port scanning: {target} ({scan_type} mode)") - # Try naabu first (faster) - naabu_installed, _ = check_tool("naabu") - if naabu_installed: - results = self._run_naabu(target, ports) + results = { + "target": target, + "open_ports": [], + "services": {}, + "by_service": defaultdict(list), + "total": 0 + } + + # Determine port range based on scan type + if scan_type == "quick": + ports = COMMON_PORTS + elif scan_type == "full": + ports = FULL_PORTS else: - # Fallback to nmap - nmap_installed, _ = check_tool("nmap") - if nmap_installed: - results = self._run_nmap(target, ports) + ports = TOP_1000_PORTS - print(f"[+] Open ports: {len(results.get('open_ports', []))}") + # Try rustscan first (fastest) + rustscan_ok, _ = check_tool("rustscan") + if rustscan_ok: + print_result("[~]", "Using rustscan (fastest)...") + results = self._run_rustscan(target, ports) + else: + # Try naabu + naabu_ok, _ = check_tool("naabu") + if naabu_ok: + print_result("[~]", "Using naabu...") + results = self._run_naabu(target, ports) + else: + # Fallback to nmap + nmap_ok, _ = check_tool("nmap") + if nmap_ok: + print_result("[~]", "Using nmap...") + results = self._run_nmap(target, ports) + + # Service version detection on open ports + if results["open_ports"] and scan_type != "quick": + print_result("[~]", "Running service version detection...") + services = self._detect_services(target, results["open_ports"]) + results["services"] = services + + for port, service in services.items(): + results["by_service"][service].append(port) + + results["by_service"] = dict(results["by_service"]) + + print_result("[+]", f"Open ports: {results['total']}") + if results["services"]: + print_result("[+]", f"Services detected: {len(results['services'])}") + + return results + + def _run_rustscan(self, target: str, ports: str) -> Dict: + """Run rustscan for ultra-fast scanning.""" + results = { + "target": target, + "open_ports": [], + "services": {}, + "by_service": defaultdict(list), + "total": 0 + } + + cmd = ["rustscan", "-a", target, "-p", ports, "--ulimit", "5000", "-g"] + result = run_tool(cmd, self.timeout) + + if result["stdout"]: + # Parse rustscan output format: host -> [ports] + for line in result["stdout"].strip().split('\n'): + if '->' in line: + ports_str = line.split('->')[-1].strip().strip('[]') + for port in ports_str.split(','): + try: + p = int(port.strip()) + results["open_ports"].append({"port": p, "protocol": "tcp"}) + except: + continue + + results["total"] = len(results["open_ports"]) return results def _run_naabu(self, target: str, ports: str) -> Dict: - """Run naabu for fast port scan.""" - results = {"target": target, "open_ports": [], "by_service": {}} + """Run naabu for fast port scanning.""" + results = { + "target": target, + "open_ports": [], + "services": {}, + "by_service": defaultdict(list), + "total": 0 + } - cmd = ["naabu", "-host", target, "-p", ports, "-silent"] + cmd = ["naabu", "-host", target, "-p", ports, "-silent", "-c", "100"] result = run_tool(cmd, self.timeout) if result["stdout"]: for line in result["stdout"].strip().split('\n'): line = line.strip() if ':' in line: - host, port = line.rsplit(':', 1) try: - results["open_ports"].append({ - "host": host, - "port": int(port), - "protocol": "tcp" - }) - except ValueError: + _, port = line.rsplit(':', 1) + results["open_ports"].append({"port": int(port), "protocol": "tcp"}) + except: continue + results["total"] = len(results["open_ports"]) return results def _run_nmap(self, target: str, ports: str) -> Dict: - """Run nmap for detailed port scan.""" - results = {"target": target, "open_ports": [], "by_service": {}} + """Run nmap for port scanning.""" + results = { + "target": target, + "open_ports": [], + "services": {}, + "by_service": defaultdict(list), + "total": 0 + } - cmd = ["nmap", "-sS", "-T4", "-p", ports, "--open", target] + cmd = ["nmap", "-sS", "-T4", "-p", ports, "--open", "-Pn", target] result = run_tool(cmd, self.timeout) if result["stdout"]: port_pattern = r"(\d+)/(\w+)\s+open\s+(\S+)" for match in re.finditer(port_pattern, result["stdout"]): port_info = { - "host": target, "port": int(match.group(1)), "protocol": match.group(2), "service": match.group(3) } results["open_ports"].append(port_info) - results["by_service"][match.group(3)] = results["by_service"].get(match.group(3), 0) + 1 + results["by_service"][match.group(3)].append(int(match.group(1))) + + results["total"] = len(results["open_ports"]) + return results + + def _detect_services(self, target: str, ports: List[Dict]) -> Dict: + """Detect services on open ports using nmap.""" + services = {} + + nmap_ok, _ = check_tool("nmap") + if not nmap_ok: + return services + + port_list = ",".join([str(p["port"]) for p in ports[:50]]) # Limit to 50 ports + cmd = ["nmap", "-sV", "-p", port_list, "-Pn", target] + result = run_tool(cmd, 300) + + if result["stdout"]: + pattern = r"(\d+)/\w+\s+open\s+(\S+)\s+(.*)" + for match in re.finditer(pattern, result["stdout"]): + port = int(match.group(1)) + service = f"{match.group(2)} {match.group(3)}".strip() + services[port] = service + + return services + + +# ============================================================================= +# DNS ENUMERATION +# ============================================================================= + +class DNSEnumerator: + """Advanced DNS enumeration.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def enumerate(self, domain: str) -> Dict: + """Complete DNS enumeration.""" + logger.info(f"[*] DNS enumeration for: {domain}") + print(f"[*] DNS enumeration for: {domain}") + + results = { + "domain": domain, + "A": [], + "AAAA": [], + "MX": [], + "NS": [], + "TXT": [], + "SOA": [], + "CNAME": [], + "SRV": [], + "zone_transfer": [], + "nameservers_info": [] + } + + record_types = ['A', 'AAAA', 'MX', 'NS', 'TXT', 'SOA', 'CNAME', 'SRV'] + + if dns: + for rtype in record_types: + try: + answers = dns.resolver.resolve(domain, rtype) + results[rtype] = [str(rdata) for rdata in answers] + print_result("[+]", f"{rtype}: {len(results[rtype])} records") + except dns.resolver.NoAnswer: + pass + except dns.resolver.NXDOMAIN: + print_result("[-]", f"Domain {domain} does not exist") + break + except Exception as e: + pass + + # Try zone transfer on each nameserver + if results["NS"]: + print_result("[~]", "Attempting zone transfer...") + for ns in results["NS"]: + zt_result = self._try_zone_transfer(domain, ns.rstrip('.')) + if zt_result: + results["zone_transfer"].extend(zt_result) + print_result("[!]", f"Zone transfer successful on {ns}!") + else: + # Fallback to dig/nslookup + print_result("[~]", "Using dig/nslookup fallback...") + results = self._dig_fallback(domain) + + return results + + def _try_zone_transfer(self, domain: str, nameserver: str) -> List[str]: + """Attempt zone transfer.""" + records = [] + try: + import dns.zone + import dns.query + z = dns.zone.from_xfr(dns.query.xfr(nameserver, domain, timeout=10)) + for name, node in z.nodes.items(): + records.append(str(name)) + except Exception: + pass + return records + + def _dig_fallback(self, domain: str) -> Dict: + """Fallback using dig command.""" + results = { + "domain": domain, + "A": [], "AAAA": [], "MX": [], "NS": [], "TXT": [], "SOA": [], "CNAME": [], "SRV": [], + "zone_transfer": [], "nameservers_info": [] + } + + dig_ok, _ = check_tool("dig") + if not dig_ok: + return results + + for rtype in ['A', 'AAAA', 'MX', 'NS', 'TXT', 'CNAME']: + result = run_tool(["dig", "+short", rtype, domain], 30) + if result["stdout"]: + results[rtype] = [r.strip() for r in result["stdout"].strip().split('\n') if r.strip()] return results +# ============================================================================= +# VULNERABILITY SCANNER +# ============================================================================= + class VulnScanner: - """Vulnerability scanner using nuclei.""" + """Vulnerability scanning using nuclei.""" - def __init__(self, config: Dict): - self.config = config - self.timeout = config.get('timeout', 600) + def __init__(self, config: Dict = None): + self.config = config or {} + self.timeout = self.config.get('timeout', 900) - def scan(self, targets: List[str], templates: str = None) -> Dict: - """Vulnerability scan on targets.""" - logger.info(f"[*] Scanning vulnerabilities on {len(targets)} targets") - print(f"[*] Scanning vulnerabilities on {len(targets)} targets...") + def scan(self, targets: List[str], severity: str = "all", templates: str = None) -> Dict: + """ + Vulnerability scan with nuclei. + + Args: + targets: List of URLs to scan + severity: critical, high, medium, low, info, all + templates: Specific template path or tag + """ + logger.info(f"[*] Vulnerability scanning {len(targets)} targets") + print(f"[*] Vulnerability scanning {len(targets)} targets...") results = { "total_targets": len(targets), "vulnerabilities": [], - "by_severity": {"critical": [], "high": [], "medium": [], "low": [], "info": []} + "by_severity": {"critical": [], "high": [], "medium": [], "low": [], "info": []}, + "by_type": defaultdict(list), + "statistics": {} } - nuclei_installed, _ = check_tool("nuclei") - if not nuclei_installed: - print(" [-] nuclei not installed") + nuclei_ok, _ = check_tool("nuclei") + if not nuclei_ok: + print_result("[-]", "nuclei not installed") return results - # Create temp file with targets - import tempfile with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: f.write('\n'.join(targets)) targets_file = f.name try: - cmd = ["nuclei", "-l", targets_file, "-silent", "-nc", "-j"] + cmd = ["nuclei", "-l", targets_file, "-silent", "-nc", "-j", "-c", "50", "-bs", "25", "-rl", "150"] + + if severity != "all": + cmd.extend(["-s", severity]) if templates: cmd.extend(["-t", templates]) @@ -924,8 +1523,13 @@ class VulnScanner: "name": finding.get("info", {}).get("name", ""), "severity": finding.get("info", {}).get("severity", "info"), "url": finding.get("matched-at", ""), + "host": finding.get("host", ""), "description": finding.get("info", {}).get("description", ""), - "curl_command": finding.get("curl-command", "") + "tags": finding.get("info", {}).get("tags", []), + "reference": finding.get("info", {}).get("reference", []), + "curl_command": finding.get("curl-command", ""), + "matcher_name": finding.get("matcher-name", ""), + "extracted": finding.get("extracted-results", []) } results["vulnerabilities"].append(vuln) @@ -933,108 +1537,1096 @@ class VulnScanner: if sev in results["by_severity"]: results["by_severity"][sev].append(vuln) - print(f" [!] [{sev.upper()}] {vuln['name']} - {vuln['url']}") + # By type/tag + for tag in vuln["tags"]: + results["by_type"][tag].append(vuln) + + # Print finding + sev_icon = {"critical": "[!!]", "high": "[!]", "medium": "[*]", "low": "[+]", "info": "[i]"}.get(sev, "[?]") + print_result(sev_icon, f"[{sev.upper()}] {vuln['name']} - {vuln['url']}") + except json.JSONDecodeError: continue + finally: os.unlink(targets_file) - print(f"[+] Vulnerabilities found: {len(results['vulnerabilities'])}") + # Statistics + results["statistics"] = { + "total": len(results["vulnerabilities"]), + "critical": len(results["by_severity"]["critical"]), + "high": len(results["by_severity"]["high"]), + "medium": len(results["by_severity"]["medium"]), + "low": len(results["by_severity"]["low"]), + "info": len(results["by_severity"]["info"]) + } + results["by_type"] = dict(results["by_type"]) + + print_result("[✓]", f"Total vulnerabilities: {results['statistics']['total']}") + print_result("[!]", f"Critical: {results['statistics']['critical']} | High: {results['statistics']['high']} | Medium: {results['statistics']['medium']}") + return results -class FullReconRunner: - """Run all recon tools and consolidate results.""" +# ============================================================================= +# WAF DETECTION +# ============================================================================= + +class WAFDetector: + """Web Application Firewall detection.""" + + WAF_SIGNATURES = { + "Cloudflare": ["cf-ray", "cloudflare", "__cfduid", "cf-cache-status"], + "AWS WAF": ["x-amzn-requestid", "x-amz-cf-id"], + "Akamai": ["akamai", "akamai-ghost", "ak_bmsc"], + "Imperva/Incapsula": ["incap_ses", "visid_incap", "x-iinfo", "incapsula"], + "Sucuri": ["sucuri", "x-sucuri-id", "x-sucuri-cache"], + "F5 BIG-IP": ["bigipserver", "x-cnection", "x-wa-info"], + "Barracuda": ["barra_counter_session", "barracuda"], + "Citrix NetScaler": ["ns_af", "citrix_ns_id", "nsc_"], + "Fortinet FortiWeb": ["fortiwafsid", "fgd_icon_hash"], + "ModSecurity": ["mod_security", "modsecurity"], + "DenyAll": ["sessioncookie", "denyall"], + "StackPath": ["x-sp-", "stackpath"], + "Fastly": ["fastly", "x-fastly-request-id"], + "KeyCDN": ["keycdn", "x-edge-location"], + } def __init__(self, config: Dict = None): self.config = config or {} - def run(self, target: str, target_type: str = "domain") -> Dict: + def detect(self, target: str) -> Dict: + """Detect WAF on target.""" + logger.info(f"[*] WAF detection for: {target}") + print(f"[*] Detecting WAF on: {target}") + + results = { + "target": target, + "waf_detected": False, + "waf_name": None, + "confidence": "low", + "indicators": [], + "bypass_hints": [] + } + + # Try wafw00f first + wafw00f_ok, _ = check_tool("wafw00f") + if wafw00f_ok: + print_result("[~]", "Using wafw00f...") + wafw00f_result = self._run_wafw00f(target) + if wafw00f_result.get("waf_detected"): + results.update(wafw00f_result) + print_result("[!]", f"WAF Detected: {results['waf_name']}") + return results + + # Manual detection + print_result("[~]", "Running manual WAF detection...") + manual_result = self._manual_detection(target) + results.update(manual_result) + + if results["waf_detected"]: + print_result("[!]", f"WAF Detected: {results['waf_name']} (Confidence: {results['confidence']})") + results["bypass_hints"] = self._get_bypass_hints(results["waf_name"]) + else: + print_result("[+]", "No WAF detected") + + return results + + def _run_wafw00f(self, target: str) -> Dict: + """Run wafw00f for WAF detection.""" + result = { + "waf_detected": False, + "waf_name": None, + "confidence": "low", + "indicators": [] + } + + cmd = ["wafw00f", target, "-o", "-"] + output = run_tool(cmd, 60) + + if output["stdout"]: + if "is behind" in output["stdout"]: + match = re.search(r"is behind (.+?)(?:\s|$)", output["stdout"]) + if match: + result["waf_detected"] = True + result["waf_name"] = match.group(1).strip() + result["confidence"] = "high" + elif "No WAF" not in output["stdout"]: + result["waf_detected"] = True + result["confidence"] = "medium" + + return result + + def _manual_detection(self, target: str) -> Dict: + """Manual WAF detection via headers and behavior.""" + result = { + "waf_detected": False, + "waf_name": None, + "confidence": "low", + "indicators": [] + } + + url = make_url(target) + + try: + # Normal request + resp_normal = requests.get(url, timeout=10, verify=False) + headers_normal = {k.lower(): v.lower() for k, v in resp_normal.headers.items()} + cookies_normal = resp_normal.cookies.get_dict() + + # Check headers and cookies for WAF signatures + for waf_name, signatures in self.WAF_SIGNATURES.items(): + for sig in signatures: + sig_lower = sig.lower() + # Check headers + for header, value in headers_normal.items(): + if sig_lower in header or sig_lower in value: + result["waf_detected"] = True + result["waf_name"] = waf_name + result["indicators"].append(f"Header match: {header}") + result["confidence"] = "medium" + break + # Check cookies + for cookie_name in cookies_normal: + if sig_lower in cookie_name.lower(): + result["waf_detected"] = True + result["waf_name"] = waf_name + result["indicators"].append(f"Cookie match: {cookie_name}") + result["confidence"] = "medium" + break + if result["waf_detected"]: + break + + # Malicious request test (if no WAF detected yet) + if not result["waf_detected"]: + payloads = [ + "?id=1' OR '1'='1", + "?q=", + "?file=../../../etc/passwd", + "?cmd=;cat /etc/passwd" + ] + + for payload in payloads: + try: + resp_malicious = requests.get(f"{url}{payload}", timeout=10, verify=False) + # Check for WAF block responses + if resp_malicious.status_code in [403, 406, 429, 503]: + content = resp_malicious.text.lower() + waf_keywords = ['blocked', 'forbidden', 'denied', 'firewall', 'security', 'waf', 'captcha', 'challenge'] + if any(kw in content for kw in waf_keywords): + result["waf_detected"] = True + result["confidence"] = "medium" + result["indicators"].append(f"Blocked request: {payload}") + break + except: + continue + + except Exception as e: + logger.warning(f"WAF detection error: {e}") + + return result + + def _get_bypass_hints(self, waf_name: str) -> List[str]: + """Get WAF bypass hints.""" + hints = { + "Cloudflare": [ + "Try finding origin IP via DNS history, Shodan, or SecurityTrails", + "Use HTTP/2 specific techniques", + "Try case variation: SeLeCt instead of SELECT", + "URL encode payloads multiple times" + ], + "AWS WAF": [ + "Try unicode normalization bypass", + "Use JSON-based payloads", + "Chunk transfer encoding" + ], + "ModSecurity": [ + "Try comments in SQL: SEL/**/ECT", + "Use HPP (HTTP Parameter Pollution)", + "Try alternative encodings" + ], + "Akamai": [ + "Try cache poisoning techniques", + "Use origin IP if discoverable", + "Header injection techniques" + ] + } + return hints.get(waf_name, ["Try common bypass techniques: encoding, case variation, HPP"]) + + +# ============================================================================= +# JS FILE ANALYZER +# ============================================================================= + +class JSAnalyzer: + """JavaScript file analysis for secrets, endpoints, and sensitive info.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def analyze(self, js_urls: List[str]) -> Dict: + """Analyze JavaScript files for sensitive information.""" + logger.info(f"[*] Analyzing {len(js_urls)} JS files") + print(f"[*] Analyzing {len(js_urls)} JavaScript files for secrets...") + + results = { + "files_analyzed": 0, + "secrets": [], + "api_endpoints": [], + "domains": [], + "emails": [], + "comments": [], + "by_file": {} + } + + for url in js_urls[:50]: # Limit to 50 files + try: + file_results = self._analyze_file(url) + if file_results: + results["by_file"][url] = file_results + results["secrets"].extend(file_results.get("secrets", [])) + results["api_endpoints"].extend(file_results.get("endpoints", [])) + results["domains"].extend(file_results.get("domains", [])) + results["files_analyzed"] += 1 + except Exception as e: + logger.warning(f"Error analyzing {url}: {e}") + continue + + # Deduplicate + results["secrets"] = list(set([s["value"] if isinstance(s, dict) else s for s in results["secrets"]])) + results["api_endpoints"] = list(set(results["api_endpoints"])) + results["domains"] = list(set(results["domains"])) + + print_result("[+]", f"Files analyzed: {results['files_analyzed']}") + print_result("[!]", f"Secrets found: {len(results['secrets'])}") + print_result("[+]", f"API endpoints: {len(results['api_endpoints'])}") + + if results["secrets"]: + for secret in results["secrets"][:5]: + print_result("[!!]", f"Secret: {secret[:50]}...") + + return results + + def _analyze_file(self, url: str) -> Dict: + """Analyze single JS file.""" + results = { + "secrets": [], + "endpoints": [], + "domains": [], + "comments": [] + } + + try: + resp = requests.get(url, timeout=15, verify=False) + if resp.status_code != 200: + return results + + content = resp.text + + # Find secrets + for secret_type, pattern in SECRET_PATTERNS.items(): + matches = re.findall(pattern, content) + for match in matches: + results["secrets"].append({ + "type": secret_type, + "value": match, + "file": url + }) + + # Find API endpoints + endpoint_patterns = [ + r'["\']/(api|v[0-9]+)/[a-zA-Z0-9/_-]+["\']', + r'["\']https?://[^"\']+/api/[^"\']+["\']', + r'fetch\(["\'][^"\']+["\']', + r'axios\.(get|post|put|delete)\(["\'][^"\']+["\']', + r'\.ajax\(\{[^}]*url:\s*["\'][^"\']+["\']' + ] + + for pattern in endpoint_patterns: + matches = re.findall(pattern, content, re.I) + for match in matches: + if isinstance(match, tuple): + match = match[0] + endpoint = match.strip('"\'') + if len(endpoint) > 3: + results["endpoints"].append(endpoint) + + # Find domains/URLs + domain_pattern = r'https?://([a-zA-Z0-9][-a-zA-Z0-9]*\.)+[a-zA-Z]{2,}' + domains = re.findall(domain_pattern, content) + results["domains"] = list(set(domains))[:20] + + except Exception as e: + logger.warning(f"JS analysis error for {url}: {e}") + + return results + + +# ============================================================================= +# SUBDOMAIN TAKEOVER DETECTION +# ============================================================================= + +class TakeoverDetector: + """Detect potential subdomain takeover vulnerabilities.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def detect(self, subdomains: List[str]) -> Dict: + """Check for subdomain takeover possibilities.""" + logger.info(f"[*] Checking {len(subdomains)} subdomains for takeover") + print(f"[*] Checking {len(subdomains)} subdomains for takeover...") + + results = { + "checked": 0, + "vulnerable": [], + "potential": [], + "cname_records": {} + } + + # Try subjack if available + subjack_ok, _ = check_tool("subjack") + if subjack_ok: + print_result("[~]", "Using subjack...") + subjack_results = self._run_subjack(subdomains) + results["vulnerable"].extend(subjack_results) + + # Manual CNAME check + print_result("[~]", "Checking CNAME records...") + for subdomain in subdomains[:100]: # Limit + cname_result = self._check_cname(subdomain) + results["checked"] += 1 + + if cname_result.get("vulnerable"): + results["vulnerable"].append(cname_result) + print_result("[!!]", f"VULNERABLE: {subdomain} -> {cname_result['cname']} ({cname_result['service']})") + elif cname_result.get("potential"): + results["potential"].append(cname_result) + + if cname_result.get("cname"): + results["cname_records"][subdomain] = cname_result["cname"] + + print_result("[+]", f"Subdomains checked: {results['checked']}") + print_result("[!]", f"Vulnerable: {len(results['vulnerable'])}") + print_result("[*]", f"Potential: {len(results['potential'])}") + + return results + + def _run_subjack(self, subdomains: List[str]) -> List[Dict]: + """Run subjack for takeover detection.""" + vulnerable = [] + + with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: + f.write('\n'.join(subdomains)) + subs_file = f.name + + try: + cmd = ["subjack", "-w", subs_file, "-t", "50", "-timeout", "30", "-o", "-", "-ssl"] + result = run_tool(cmd, 300) + + if result["stdout"]: + for line in result["stdout"].strip().split('\n'): + if "[Vulnerable]" in line or "vulnerable" in line.lower(): + vulnerable.append({"subdomain": line, "source": "subjack"}) + finally: + os.unlink(subs_file) + + return vulnerable + + def _check_cname(self, subdomain: str) -> Dict: + """Check CNAME record for takeover indicators.""" + result = { + "subdomain": subdomain, + "cname": None, + "vulnerable": False, + "potential": False, + "service": None + } + + try: + if dns: + answers = dns.resolver.resolve(subdomain, 'CNAME') + for rdata in answers: + cname = str(rdata.target).rstrip('.') + result["cname"] = cname + + # Check against known takeover signatures + for pattern, service in TAKEOVER_CNAMES.items(): + if pattern in cname.lower(): + result["potential"] = True + result["service"] = service + + # Try to resolve CNAME - if it fails, likely vulnerable + try: + socket.gethostbyname(cname) + except socket.gaierror: + result["vulnerable"] = True + break + except: + pass + + return result + + +# ============================================================================= +# CORS MISCONFIGURATION CHECKER +# ============================================================================= + +class CORSChecker: + """Check for CORS misconfigurations.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def check(self, targets: List[str]) -> Dict: + """Check targets for CORS misconfigurations.""" + logger.info(f"[*] CORS check on {len(targets)} targets") + print(f"[*] Checking {len(targets)} targets for CORS misconfigurations...") + + results = { + "checked": 0, + "vulnerable": [], + "warnings": [], + "by_type": defaultdict(list) + } + + for target in targets[:50]: + url = make_url(target) + cors_result = self._check_cors(url) + results["checked"] += 1 + + if cors_result.get("vulnerable"): + results["vulnerable"].append(cors_result) + results["by_type"][cors_result["type"]].append(url) + print_result("[!]", f"CORS Vuln ({cors_result['type']}): {url}") + elif cors_result.get("warning"): + results["warnings"].append(cors_result) + + results["by_type"] = dict(results["by_type"]) + + print_result("[+]", f"Checked: {results['checked']}") + print_result("[!]", f"Vulnerable: {len(results['vulnerable'])}") + + return results + + def _check_cors(self, url: str) -> Dict: + """Check single URL for CORS misconfiguration.""" + result = { + "url": url, + "vulnerable": False, + "warning": False, + "type": None, + "details": None + } + + test_origins = [ + "https://evil.com", + "null", + f"https://{urlparse(url).netloc}.evil.com", + urlparse(url).scheme + "://" + urlparse(url).netloc.replace(".", "x"), + ] + + try: + for origin in test_origins: + headers = {"Origin": origin} + resp = requests.get(url, headers=headers, timeout=10, verify=False) + + acao = resp.headers.get("Access-Control-Allow-Origin", "") + acac = resp.headers.get("Access-Control-Allow-Credentials", "") + + # Check for vulnerable configurations + if acao == "*": + result["warning"] = True + result["type"] = "wildcard_origin" + result["details"] = "ACAO: * (wildcard)" + + if acac.lower() == "true": + result["vulnerable"] = True + result["type"] = "wildcard_with_credentials" + return result + + elif acao == origin: + result["vulnerable"] = True + result["type"] = "origin_reflection" + result["details"] = f"Origin reflected: {origin}" + + if acac.lower() == "true": + result["type"] = "origin_reflection_with_credentials" + return result + + elif acao == "null" and origin == "null": + result["vulnerable"] = True + result["type"] = "null_origin_allowed" + result["details"] = "null origin allowed" + return result + + except Exception as e: + logger.warning(f"CORS check error for {url}: {e}") + + return result + + +# ============================================================================= +# SCREENSHOT CAPTURE +# ============================================================================= + +class ScreenshotCapture: + """Capture screenshots of web targets.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + self.output_dir = config.get('screenshot_dir', '/opt/NeuroSploitv2/results/screenshots') + + def capture(self, targets: List[str]) -> Dict: + """Capture screenshots of targets.""" + logger.info(f"[*] Capturing screenshots for {len(targets)} targets") + print(f"[*] Capturing screenshots for {len(targets)} targets...") + + results = { + "captured": 0, + "failed": 0, + "screenshots": [], + "output_dir": self.output_dir + } + + # Create output directory + os.makedirs(self.output_dir, exist_ok=True) + + # Try gowitness + gowitness_ok, _ = check_tool("gowitness") + if gowitness_ok: + print_result("[~]", "Using gowitness...") + results = self._run_gowitness(targets) + else: + # Try eyewitness + eyewitness_ok, _ = check_tool("eyewitness") + if eyewitness_ok: + print_result("[~]", "Using eyewitness...") + results = self._run_eyewitness(targets) + else: + print_result("[-]", "No screenshot tool available (gowitness/eyewitness)") + + print_result("[+]", f"Screenshots captured: {results['captured']}") + print_result("[+]", f"Output directory: {results['output_dir']}") + + return results + + def _run_gowitness(self, targets: List[str]) -> Dict: + """Run gowitness for screenshots.""" + results = { + "captured": 0, + "failed": 0, + "screenshots": [], + "output_dir": self.output_dir + } + + with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: + f.write('\n'.join(targets)) + targets_file = f.name + + try: + cmd = [ + "gowitness", "file", + "-f", targets_file, + "-P", self.output_dir, + "--timeout", "30", + "-t", "10" + ] + result = run_tool(cmd, 600) + + # Count screenshots + if os.path.exists(self.output_dir): + screenshots = list(Path(self.output_dir).glob("*.png")) + results["captured"] = len(screenshots) + results["screenshots"] = [str(s) for s in screenshots[:100]] + + finally: + os.unlink(targets_file) + + return results + + def _run_eyewitness(self, targets: List[str]) -> Dict: + """Run eyewitness for screenshots.""" + results = { + "captured": 0, + "failed": 0, + "screenshots": [], + "output_dir": self.output_dir + } + + with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f: + f.write('\n'.join(targets)) + targets_file = f.name + + try: + cmd = [ + "eyewitness", + "-f", targets_file, + "-d", self.output_dir, + "--timeout", "30", + "--threads", "10", + "--no-prompt" + ] + result = run_tool(cmd, 600) + + if os.path.exists(self.output_dir): + screenshots = list(Path(self.output_dir).glob("**/*.png")) + results["captured"] = len(screenshots) + results["screenshots"] = [str(s) for s in screenshots[:100]] + + finally: + os.unlink(targets_file) + + return results + + +# ============================================================================= +# CLOUD BUCKET ENUMERATION +# ============================================================================= + +class CloudBucketEnum: + """Enumerate cloud storage buckets (S3, GCS, Azure).""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def enumerate(self, domain: str, keywords: List[str] = None) -> Dict: + """Enumerate cloud buckets based on domain and keywords.""" + logger.info(f"[*] Cloud bucket enumeration for: {domain}") + print(f"[*] Enumerating cloud buckets for: {domain}") + + results = { + "domain": domain, + "s3_buckets": [], + "gcs_buckets": [], + "azure_blobs": [], + "accessible": [], + "total": 0 + } + + # Generate bucket names + base_name = domain.replace(".", "-").replace("www-", "") + bucket_names = self._generate_bucket_names(base_name, keywords or []) + + print_result("[~]", f"Testing {len(bucket_names)} potential bucket names...") + + # Check S3 + s3_found = self._check_s3_buckets(bucket_names) + results["s3_buckets"] = s3_found + + # Check GCS + gcs_found = self._check_gcs_buckets(bucket_names) + results["gcs_buckets"] = gcs_found + + # Check Azure + azure_found = self._check_azure_blobs(bucket_names) + results["azure_blobs"] = azure_found + + results["accessible"] = [b for b in s3_found + gcs_found + azure_found if b.get("accessible")] + results["total"] = len(s3_found) + len(gcs_found) + len(azure_found) + + print_result("[+]", f"S3 buckets: {len(s3_found)}") + print_result("[+]", f"GCS buckets: {len(gcs_found)}") + print_result("[+]", f"Azure blobs: {len(azure_found)}") + + if results["accessible"]: + print_result("[!]", f"Accessible buckets: {len(results['accessible'])}") + + return results + + def _generate_bucket_names(self, base: str, keywords: List[str]) -> List[str]: + """Generate potential bucket names.""" + names = set() + + prefixes = ['', 'dev-', 'staging-', 'prod-', 'test-', 'backup-', 'assets-', 'static-', 'media-', 'uploads-', 'data-', 'files-', 'cdn-', 'img-', 'images-'] + suffixes = ['', '-dev', '-staging', '-prod', '-test', '-backup', '-assets', '-static', '-media', '-uploads', '-data', '-files', '-cdn', '-images', '-public', '-private', '-internal'] + + for prefix in prefixes: + for suffix in suffixes: + name = f"{prefix}{base}{suffix}".strip('-') + if name and 3 <= len(name) <= 63: + names.add(name) + + for keyword in keywords: + names.add(f"{base}-{keyword}") + names.add(f"{keyword}-{base}") + + return list(names)[:200] + + def _check_s3_buckets(self, names: List[str]) -> List[Dict]: + """Check for S3 buckets.""" + found = [] + + def check_bucket(name): + try: + url = f"https://{name}.s3.amazonaws.com" + resp = requests.head(url, timeout=5) + if resp.status_code in [200, 403, 301, 307]: + accessible = resp.status_code == 200 + return {"name": name, "url": url, "status": resp.status_code, "accessible": accessible} + except: + pass + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor: + results = executor.map(check_bucket, names) + found = [r for r in results if r] + + return found + + def _check_gcs_buckets(self, names: List[str]) -> List[Dict]: + """Check for Google Cloud Storage buckets.""" + found = [] + + def check_bucket(name): + try: + url = f"https://storage.googleapis.com/{name}" + resp = requests.head(url, timeout=5) + if resp.status_code in [200, 403]: + accessible = resp.status_code == 200 + return {"name": name, "url": url, "status": resp.status_code, "accessible": accessible} + except: + pass + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor: + results = executor.map(check_bucket, names) + found = [r for r in results if r] + + return found + + def _check_azure_blobs(self, names: List[str]) -> List[Dict]: + """Check for Azure Blob Storage.""" + found = [] + + def check_blob(name): + try: + url = f"https://{name}.blob.core.windows.net" + resp = requests.head(url, timeout=5) + if resp.status_code in [200, 403, 400]: + accessible = resp.status_code == 200 + return {"name": name, "url": url, "status": resp.status_code, "accessible": accessible} + except: + pass + return None + + with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor: + results = executor.map(check_blob, names) + found = [r for r in results if r] + + return found + + +# ============================================================================= +# TECHNOLOGY FINGERPRINTER +# ============================================================================= + +class TechFingerprinter: + """Advanced technology fingerprinting.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def fingerprint(self, target: str) -> Dict: + """Deep technology fingerprinting.""" + logger.info(f"[*] Fingerprinting: {target}") + print(f"[*] Technology fingerprinting: {target}") + + results = { + "target": target, + "technologies": [], + "cms": None, + "web_server": None, + "programming_language": None, + "frameworks": [], + "js_libraries": [], + "cdn": None, + "analytics": [], + "headers": {}, + "meta_tags": {} + } + + # Try whatweb first + whatweb_ok, _ = check_tool("whatweb") + if whatweb_ok: + print_result("[~]", "Running whatweb...") + whatweb_results = self._run_whatweb(target) + results.update(whatweb_results) + + # Manual fingerprinting + print_result("[~]", "Running manual fingerprinting...") + manual_results = self._manual_fingerprint(target) + + # Merge results + results["technologies"] = list(set(results.get("technologies", []) + manual_results.get("technologies", []))) + results["frameworks"] = list(set(results.get("frameworks", []) + manual_results.get("frameworks", []))) + results["js_libraries"] = list(set(results.get("js_libraries", []) + manual_results.get("js_libraries", []))) + + if not results["cms"]: + results["cms"] = manual_results.get("cms") + if not results["web_server"]: + results["web_server"] = manual_results.get("web_server") + + results["headers"] = manual_results.get("headers", {}) + results["meta_tags"] = manual_results.get("meta_tags", {}) + + print_result("[+]", f"Technologies: {len(results['technologies'])}") + if results["cms"]: + print_result("[+]", f"CMS: {results['cms']}") + if results["web_server"]: + print_result("[+]", f"Web Server: {results['web_server']}") + + return results + + def _run_whatweb(self, target: str) -> Dict: + """Run whatweb for fingerprinting.""" + results = {"technologies": [], "cms": None, "web_server": None, "frameworks": [], "js_libraries": []} + + url = make_url(target) + cmd = ["whatweb", "-a", "3", "--color=never", url] + result = run_tool(cmd, 120) + + if result["stdout"]: + # Parse whatweb output + techs = re.findall(r'\[([^\]]+)\]', result["stdout"]) + results["technologies"] = list(set(techs)) + + # Identify specific categories + cms_keywords = ['WordPress', 'Drupal', 'Joomla', 'Magento', 'Shopify', 'PrestaShop', 'OpenCart', 'TYPO3', 'Ghost'] + framework_keywords = ['Laravel', 'Django', 'Rails', 'Express', 'Spring', 'ASP.NET', 'Flask', 'FastAPI', 'Next.js', 'Nuxt'] + + for tech in results["technologies"]: + for cms in cms_keywords: + if cms.lower() in tech.lower(): + results["cms"] = cms + for fw in framework_keywords: + if fw.lower() in tech.lower(): + results["frameworks"].append(fw) + + return results + + def _manual_fingerprint(self, target: str) -> Dict: + """Manual technology fingerprinting.""" + results = { + "technologies": [], + "cms": None, + "web_server": None, + "programming_language": None, + "frameworks": [], + "js_libraries": [], + "headers": {}, + "meta_tags": {} + } + + url = make_url(target) + + try: + resp = requests.get(url, timeout=15, verify=False) + + # Headers analysis + headers = dict(resp.headers) + results["headers"] = headers + + if 'Server' in headers: + results["web_server"] = headers['Server'] + results["technologies"].append(f"Server: {headers['Server']}") + + if 'X-Powered-By' in headers: + results["programming_language"] = headers['X-Powered-By'] + results["technologies"].append(f"X-Powered-By: {headers['X-Powered-By']}") + + # Content analysis + content = resp.text.lower() + + # CMS detection + cms_signatures = { + 'WordPress': ['wp-content', 'wp-includes', 'wordpress'], + 'Drupal': ['drupal', 'sites/default/files'], + 'Joomla': ['joomla', '/components/com_'], + 'Magento': ['magento', 'mage/'], + 'Shopify': ['shopify', 'cdn.shopify'], + 'Ghost': ['ghost', 'ghost/'], + } + + for cms, sigs in cms_signatures.items(): + if any(sig in content for sig in sigs): + results["cms"] = cms + results["technologies"].append(cms) + break + + # JS Library detection + js_libs = { + 'jQuery': ['jquery', 'jquery.min.js'], + 'React': ['react', 'react.production.min.js', '__react'], + 'Vue.js': ['vue.js', 'vue.min.js', '__vue__'], + 'Angular': ['angular', 'ng-app', 'ng-controller'], + 'Bootstrap': ['bootstrap', 'bootstrap.min'], + 'Tailwind': ['tailwindcss', 'tailwind'], + } + + for lib, sigs in js_libs.items(): + if any(sig in content for sig in sigs): + results["js_libraries"].append(lib) + results["technologies"].append(lib) + + # Meta tags + meta_patterns = { + 'generator': r']*name=["\']generator["\'][^>]*content=["\']([^"\']+)["\']', + 'framework': r']*name=["\']framework["\'][^>]*content=["\']([^"\']+)["\']', + } + + for name, pattern in meta_patterns.items(): + match = re.search(pattern, content, re.I) + if match: + results["meta_tags"][name] = match.group(1) + results["technologies"].append(match.group(1)) + + except Exception as e: + logger.warning(f"Manual fingerprint error: {e}") + + return results + + +# ============================================================================= +# FULL RECON RUNNER - ORCHESTRATOR +# ============================================================================= + +class FullReconRunner: + """ + Complete reconnaissance orchestrator. + Runs all phases and consolidates results. + """ + + def __init__(self, config: Dict = None): + self.config = config or {} + + def run(self, target: str, target_type: str = "domain", depth: str = "medium") -> Dict: """ - Run full recon and return consolidated context. + Run comprehensive reconnaissance. Args: target: Target domain or URL - target_type: Target type (domain, url) + target_type: domain, url + depth: quick, medium, deep Returns: - Dict with all consolidated results + Consolidated recon results """ from core.context_builder import ReconContextBuilder print(f"\n{'='*70}") - print(" NEUROSPLOIT - ADVANCED RECON") + print(" NEUROSPLOIT v2 - ADVANCED RECONNAISSANCE ENGINE") print(f"{'='*70}") print(f"\n[*] Target: {target}") - print(f"[*] Type: {target_type}\n") + print(f"[*] Type: {target_type}") + print(f"[*] Depth: {depth}\n") # Initialize context builder ctx = ReconContextBuilder() ctx.set_target(target, target_type) - # Extract domain from target - if target_type == "url": - parsed = urlparse(target) - domain = parsed.netloc - else: - domain = target + # Extract domain + domain = extract_domain(target) if target_type == "url" else target - # 1. Subdomain Enumeration - print("\n" + "=" * 50) - print("[PHASE 1] Subdomain Enumeration") - print("=" * 50) + # ================================================================ + # PHASE 1: Subdomain Enumeration + # ================================================================ + print_phase(1, "SUBDOMAIN ENUMERATION") sub_enum = AdvancedSubdomainEnum(self.config) - sub_results = sub_enum.enumerate(domain) + sub_results = sub_enum.enumerate(domain, depth) ctx.add_subdomains(sub_results.get("subdomains", [])) ctx.add_tool_result("subdomain_enum", sub_results) - # 2. HTTP Probing - print("\n" + "=" * 50) - print("[PHASE 2] HTTP Probing") - print("=" * 50) + subdomains = sub_results.get("subdomains", [domain]) + + # ================================================================ + # PHASE 2: HTTP Probing + # ================================================================ + print_phase(2, "HTTP PROBING & TECHNOLOGY DETECTION") prober = HttpProber(self.config) - probe_results = prober.probe(sub_results.get("subdomains", [domain])) + probe_results = prober.probe(subdomains) ctx.add_live_hosts(probe_results.get("alive", [])) ctx.add_technologies(list(probe_results.get("technologies", {}).keys())) ctx.add_tool_result("http_probe", probe_results) - # 3. URL Collection - print("\n" + "=" * 50) - print("[PHASE 3] URL Collection") - print("=" * 50) + alive_hosts = probe_results.get("alive", []) + + # ================================================================ + # PHASE 3: WAF Detection + # ================================================================ + print_phase(3, "WAF DETECTION") + waf_detector = WAFDetector(self.config) + waf_result = waf_detector.detect(target) + ctx.add_tool_result("waf_detection", waf_result) + + # ================================================================ + # PHASE 4: Port Scanning + # ================================================================ + print_phase(4, "PORT SCANNING") + port_scanner = PortScanner(self.config) + scan_type = "quick" if depth == "quick" else ("full" if depth == "deep" else "quick") + port_results = port_scanner.scan(domain, scan_type) + ctx.add_open_ports(port_results.get("open_ports", [])) + ctx.add_tool_result("port_scan", port_results) + + # ================================================================ + # PHASE 5: Directory Bruteforce + # ================================================================ + if alive_hosts and depth != "quick": + print_phase(5, "DIRECTORY BRUTEFORCE") + dir_bruter = DirectoryBruter(self.config) + wordlist_size = "medium" if depth == "medium" else "big" + dir_results = dir_bruter.bruteforce(alive_hosts[0], wordlist_size) + ctx.add_interesting_paths([d.get("url", "") for d in dir_results.get("interesting", [])]) + ctx.add_tool_result("dir_bruteforce", dir_results) + + # ================================================================ + # PHASE 6: URL Collection + # ================================================================ + print_phase(6, "URL COLLECTION") url_collector = URLCollector(self.config) url_results = url_collector.collect(domain) ctx.add_urls(url_results.get("urls", [])) ctx.add_js_files(url_results.get("js_files", [])) + ctx.add_api_endpoints(url_results.get("api_endpoints", [])) ctx.add_tool_result("url_collection", url_results) - # 4. Web Crawling - print("\n" + "=" * 50) - print("[PHASE 4] Web Crawling") - print("=" * 50) - crawler = WebCrawler(self.config) - alive_hosts = probe_results.get("alive", []) + # ================================================================ + # PHASE 7: Parameter Discovery + # ================================================================ + print_phase(7, "PARAMETER DISCOVERY") + param_spider = ParamSpider(self.config) + param_results = param_spider.spider(domain) + ctx.add_tool_result("param_discovery", param_results) + + # ================================================================ + # PHASE 8: Web Crawling + # ================================================================ if alive_hosts: - crawl_target = alive_hosts[0] # Crawl first alive host - crawl_results = crawler.crawl(crawl_target) + print_phase(8, "WEB CRAWLING") + crawler = WebCrawler(self.config) + crawl_results = crawler.crawl(alive_hosts[0]) ctx.add_urls(crawl_results.get("urls", [])) ctx.add_js_files(crawl_results.get("js_files", [])) ctx.add_api_endpoints(crawl_results.get("api_endpoints", [])) ctx.add_tool_result("crawling", crawl_results) - # 5. Port Scanning - print("\n" + "=" * 50) - print("[PHASE 5] Port Scanning") - print("=" * 50) - port_scanner = PortScanner(self.config) - port_results = port_scanner.scan(domain) - ctx.add_open_ports(port_results.get("open_ports", [])) - ctx.add_tool_result("port_scan", port_results) + # ================================================================ + # PHASE 9: JavaScript Analysis + # ================================================================ + js_files = list(ctx.js_files) + if js_files: + print_phase(9, "JAVASCRIPT ANALYSIS") + js_analyzer = JSAnalyzer(self.config) + js_results = js_analyzer.analyze(js_files) + ctx.add_secrets(js_results.get("secrets", [])) + ctx.add_api_endpoints(js_results.get("api_endpoints", [])) + ctx.add_tool_result("js_analysis", js_results) - # 6. DNS Enumeration - print("\n" + "=" * 50) - print("[PHASE 6] DNS Enumeration") - print("=" * 50) + # ================================================================ + # PHASE 10: DNS Enumeration + # ================================================================ + print_phase(10, "DNS ENUMERATION") dns_enum = DNSEnumerator(self.config) dns_results = dns_enum.enumerate(domain) dns_records = [] @@ -1045,49 +2637,221 @@ class FullReconRunner: ctx.add_dns_records(dns_records) ctx.add_tool_result("dns_enum", dns_results) - # 7. Vulnerability Scanning - print("\n" + "=" * 50) - print("[PHASE 7] Vulnerability Scanning") - print("=" * 50) - vuln_scanner = VulnScanner(self.config) - scan_targets = probe_results.get("alive", [target])[:20] # Limit to 20 - vuln_results = vuln_scanner.scan(scan_targets) + # ================================================================ + # PHASE 11: Subdomain Takeover Check + # ================================================================ + if depth != "quick" and subdomains: + print_phase(11, "SUBDOMAIN TAKEOVER CHECK") + takeover = TakeoverDetector(self.config) + takeover_results = takeover.detect(subdomains[:100]) + ctx.add_tool_result("subdomain_takeover", takeover_results) + + if takeover_results.get("vulnerable"): + for v in takeover_results["vulnerable"]: + ctx.add_vulnerabilities([{ + "title": "Subdomain Takeover", + "severity": "high", + "affected_endpoint": v.get("subdomain", ""), + "description": f"Potential subdomain takeover via {v.get('service', 'unknown')}" + }]) + + # ================================================================ + # PHASE 12: CORS Misconfiguration Check + # ================================================================ + if alive_hosts and depth != "quick": + print_phase(12, "CORS MISCONFIGURATION CHECK") + cors_checker = CORSChecker(self.config) + cors_results = cors_checker.check(alive_hosts[:30]) + ctx.add_tool_result("cors_check", cors_results) + + for vuln in cors_results.get("vulnerable", []): + ctx.add_vulnerabilities([{ + "title": f"CORS Misconfiguration ({vuln.get('type', '')})", + "severity": "medium", + "affected_endpoint": vuln.get("url", ""), + "description": vuln.get("details", "") + }]) + + # ================================================================ + # PHASE 13: Cloud Bucket Enumeration + # ================================================================ + if depth == "deep": + print_phase(13, "CLOUD BUCKET ENUMERATION") + cloud_enum = CloudBucketEnum(self.config) + cloud_results = cloud_enum.enumerate(domain) + ctx.add_tool_result("cloud_buckets", cloud_results) + + for bucket in cloud_results.get("accessible", []): + ctx.add_vulnerabilities([{ + "title": "Accessible Cloud Bucket", + "severity": "high", + "affected_endpoint": bucket.get("url", ""), + "description": f"Publicly accessible cloud storage: {bucket.get('name', '')}" + }]) + + # ================================================================ + # PHASE 14: Technology Fingerprinting + # ================================================================ + print_phase(14, "TECHNOLOGY FINGERPRINTING") + fingerprinter = TechFingerprinter(self.config) + tech_results = fingerprinter.fingerprint(target) + ctx.add_technologies(tech_results.get("technologies", [])) + ctx.add_tool_result("tech_fingerprint", tech_results) + + # ================================================================ + # PHASE 15: Vulnerability Scanning + # ================================================================ + print_phase(15, "VULNERABILITY SCANNING (NUCLEI)") + vuln_scanner = VulnScanner(self.config) + scan_targets = alive_hosts[:30] if alive_hosts else [target] + severity = "all" if depth == "deep" else "critical,high,medium" + vuln_results = vuln_scanner.scan(scan_targets, severity) - vulns = [] for v in vuln_results.get("vulnerabilities", []): - vulns.append({ + ctx.add_vulnerabilities([{ "title": v.get("name", ""), "severity": v.get("severity", "info"), "affected_endpoint": v.get("url", ""), - "description": v.get("description", "") - }) - ctx.add_vulnerabilities(vulns) + "description": v.get("description", ""), + "references": v.get("reference", []) + }]) ctx.add_tool_result("vuln_scan", vuln_results) - # Identify interesting paths + # ================================================================ + # PHASE 16: Screenshot Capture (optional) + # ================================================================ + if depth == "deep" and alive_hosts: + print_phase(16, "SCREENSHOT CAPTURE") + screenshot = ScreenshotCapture(self.config) + screenshot_results = screenshot.capture(alive_hosts[:20]) + ctx.add_tool_result("screenshots", screenshot_results) + + # ================================================================ + # CONSOLIDATION + # ================================================================ + print(f"\n{'='*70}") + print("[FINAL] CONSOLIDATING RESULTS") + print(f"{'='*70}") + + # Identify interesting paths from all URLs all_urls = list(ctx.urls) ctx.add_interesting_paths(all_urls) - # Save consolidated context - print("\n" + "=" * 50) - print("[FINAL PHASE] Consolidating Context") - print("=" * 50) + # Save context saved = ctx.save() + # Print summary print(f"\n{'='*70}") - print("[+] RECON COMPLETE!") - print(f" - Subdomains: {len(ctx.subdomains)}") - print(f" - Live hosts: {len(ctx.live_hosts)}") - print(f" - URLs: {len(ctx.urls)}") - print(f" - URLs with params: {len(ctx.urls_with_params)}") - print(f" - Open ports: {len(ctx.open_ports)}") - print(f" - Vulnerabilities: {len(ctx.vulnerabilities)}") - print(f"\n[+] Context saved to: {saved['json']}") - print(f"{'='*70}\n") + print("[✓] RECONNAISSANCE COMPLETE!") + print(f"{'='*70}") + print(f""" + SUMMARY: + ───────────────────────────────────────────── + Subdomains discovered: {len(ctx.subdomains)} + Live hosts: {len(ctx.live_hosts)} + Open ports: {len(ctx.open_ports)} + URLs collected: {len(ctx.urls)} + URLs with parameters: {len(ctx.urls_with_params)} + JavaScript files: {len(ctx.js_files)} + API endpoints: {len(ctx.api_endpoints)} + Technologies detected: {len(ctx.technologies)} + Vulnerabilities found: {len(ctx.vulnerabilities)} + + WAF Detected: {waf_result.get('waf_name', 'None')} + + Context saved to: {saved['json']} + ───────────────────────────────────────────── +""") return { "context": saved["context"], "context_file": str(saved["json"]), "context_text_file": str(saved["txt"]), - "context_text": ctx.get_llm_prompt_context() + "context_text": ctx.get_llm_prompt_context(), + "summary": { + "subdomains": len(ctx.subdomains), + "live_hosts": len(ctx.live_hosts), + "open_ports": len(ctx.open_ports), + "urls": len(ctx.urls), + "vulnerabilities": len(ctx.vulnerabilities), + "waf": waf_result.get('waf_name') + } } + + +# ============================================================================= +# LEGACY CLASSES (Backwards Compatibility) +# ============================================================================= + +class NetworkScanner(PortScanner): + """Legacy NetworkScanner - now uses PortScanner.""" + pass + + +class WebRecon: + """Legacy web reconnaissance - now uses multiple specialized classes.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + self.fingerprinter = TechFingerprinter(config) + self.waf_detector = WAFDetector(config) + + def analyze(self, url: str) -> Dict: + """Analyze web application.""" + results = { + "url": url, + "technologies": [], + "headers": {}, + "security_headers": {}, + "endpoints": [], + "forms": [], + "vulnerabilities": [], + "waf": None + } + + # Technology fingerprinting + tech_results = self.fingerprinter.fingerprint(url) + results["technologies"] = tech_results.get("technologies", []) + results["headers"] = tech_results.get("headers", {}) + + # WAF detection + waf_results = self.waf_detector.detect(url) + results["waf"] = waf_results.get("waf_name") + + # Security headers check + security_headers = ['X-Frame-Options', 'X-Content-Type-Options', 'Strict-Transport-Security', + 'Content-Security-Policy', 'X-XSS-Protection', 'Referrer-Policy'] + + for header in security_headers: + if header in results["headers"]: + results["security_headers"][header] = results["headers"][header] + else: + results["security_headers"][header] = "Missing" + + return results + + +class OSINTCollector: + """OSINT collection.""" + + def __init__(self, config: Dict = None): + self.config = config or {} + + def collect(self, target: str) -> Dict: + """Collect OSINT data.""" + return { + "target": target, + "emails": [], + "social_media": {}, + "data_breaches": [], + "metadata": {} + } + + +class SubdomainFinder(AdvancedSubdomainEnum): + """Legacy SubdomainFinder - now uses AdvancedSubdomainEnum.""" + + def find(self, domain: str) -> List[str]: + """Find subdomains.""" + results = self.enumerate(domain, depth="quick") + return results.get("subdomains", [])