v0.1.1: Major AI improvements, new security modules, and documentation fixes

## AI & CVE Improvements
- Fix AI report to display actual subdomain names instead of generic placeholders
- Add 10-year CVE filter to reduce false positives from outdated vulnerabilities
- Integrate CISA KEV (Known Exploited Vulnerabilities) database support
- Improve AI analysis prompt for more accurate security findings

## New Security Modules
- Add wildcard DNS detection with multi-phase validation (DNS + HTTP)
- Add TLS certificate analyzer for certificate chain inspection
- Add comprehensive rate limiting module for API requests
- Add retry mechanism with exponential backoff
- Add stealth mode for reduced detection during scans
- Add progress tracking module for better UX

## Code Refactoring
- Extract scanner output logic to dedicated module
- Add base source interface for consistent passive source implementation
- Reduce admin panel paths to common generic patterns only
- Improve HTTP client with connection pooling
- Add JSON output formatter

## Documentation Updates
- Correct passive source count to 20 (was incorrectly stated as 34)
- Fix AI model names: deepseek-r1:1.5b (fast) + qwen2.5-coder:7b (deep)
- Update all markdown files for consistency
- Relocate demo GIFs to assets/ directory
- Add benchmark disclaimer for test variability

## Files Changed
- 4 documentation files updated (README, AI_SETUP, BENCHMARK, EXAMPLES)
- 11 new source files added
- 12 existing files modified
This commit is contained in:
Vyntral
2025-11-21 12:00:58 +01:00
parent 163b51f141
commit b1bf119c82
28 changed files with 4650 additions and 882 deletions

View File

@@ -22,8 +22,8 @@ ollama --version
### 2. Pull Recommended Models ### 2. Pull Recommended Models
```bash ```bash
# Fast triage model (3GB) - REQUIRED # Fast triage model (1.1GB) - REQUIRED
ollama pull phi3.5:3.8b ollama pull deepseek-r1:1.5b
# Deep analysis model (6GB) - REQUIRED # Deep analysis model (6GB) - REQUIRED
ollama pull qwen2.5-coder:7b ollama pull qwen2.5-coder:7b
@@ -66,7 +66,7 @@ Leave this running in a terminal. Ollama will run on `http://localhost:11434`
┌──────────────────────────────────────────────┐ ┌──────────────────────────────────────────────┐
│ TIER 1: FAST TRIAGE (Phi-3.5:3.8b) │ TIER 1: FAST TRIAGE (DeepSeek-R1:1.5b)
│ • Quick classification: relevant vs skip │ │ • Quick classification: relevant vs skip │
│ • Completes in ~2-5 seconds │ │ • Completes in ~2-5 seconds │
│ • Filters false positives │ │ • Filters false positives │
@@ -179,7 +179,7 @@ Function calling requires models that support tool use:
-**qwen2.5-coder:7b** (default deep model) - Full support -**qwen2.5-coder:7b** (default deep model) - Full support
-**llama3.1:8b** - Excellent function calling -**llama3.1:8b** - Excellent function calling
-**llama3.2:3b** - Basic support -**llama3.2:3b** - Basic support
- ⚠️ **phi3.5:3.8b** (fast model) - No function calling (triage only) - **deepseek-r1:1.5b** (fast model) - Excellent reasoning for size
### Rate Limits ### Rate Limits
@@ -225,7 +225,7 @@ God's Eye automatically handles rate limiting and caches results.
```bash ```bash
# Use different models # Use different models
./god-eye -d target.com --enable-ai \ ./god-eye -d target.com --enable-ai \
--ai-fast-model phi3.5:3.8b \ --ai-fast-model deepseek-r1:1.5b \
--ai-deep-model deepseek-coder-v2:16b --ai-deep-model deepseek-coder-v2:16b
# Disable cascade (deep analysis only) # Disable cascade (deep analysis only)
@@ -250,7 +250,7 @@ God's Eye automatically handles rate limiting and caches results.
|------|---------|-------------| |------|---------|-------------|
| `--enable-ai` | `false` | Enable AI analysis | | `--enable-ai` | `false` | Enable AI analysis |
| `--ai-url` | `http://localhost:11434` | Ollama API URL | | `--ai-url` | `http://localhost:11434` | Ollama API URL |
| `--ai-fast-model` | `phi3.5:3.8b` | Fast triage model | | `--ai-fast-model` | `deepseek-r1:1.5b` | Fast triage model |
| `--ai-deep-model` | `qwen2.5-coder:7b` | Deep analysis model | | `--ai-deep-model` | `qwen2.5-coder:7b` | Deep analysis model |
| `--ai-cascade` | `true` | Use cascade mode | | `--ai-cascade` | `true` | Use cascade mode |
| `--ai-deep` | `false` | Deep analysis on all findings | | `--ai-deep` | `false` | Deep analysis on all findings |
@@ -282,7 +282,7 @@ ollama list
**Solution:** **Solution:**
```bash ```bash
# Pull missing model # Pull missing model
ollama pull phi3.5:3.8b ollama pull deepseek-r1:1.5b
ollama pull qwen2.5-coder:7b ollama pull qwen2.5-coder:7b
# Verify # Verify
@@ -320,7 +320,7 @@ ollama list
**Solutions:** **Solutions:**
- **Option 1:** Use smaller models - **Option 1:** Use smaller models
```bash ```bash
ollama pull phi3.5:3.8b # 3GB instead of 7GB ollama pull deepseek-r1:1.5b # 3GB instead of 7GB
``` ```
- **Option 2:** Disable cascade - **Option 2:** Disable cascade
@@ -350,7 +350,7 @@ ollama list
| **AI Analysis Time** | ~30-40 seconds | | **AI Analysis Time** | ~30-40 seconds |
| **AI Overhead** | ~20% of total scan time | | **AI Overhead** | ~20% of total scan time |
| **Memory Usage** | ~7GB (both models loaded) | | **Memory Usage** | ~7GB (both models loaded) |
| **Models Used** | phi3.5:3.8b + qwen2.5-coder:7b | | **Models Used** | deepseek-r1:1.5b + qwen2.5-coder:7b |
| **Cascade Mode** | Enabled (default) | | **Cascade Mode** | Enabled (default) |
**Sample AI Findings:** **Sample AI Findings:**
@@ -377,7 +377,7 @@ ollama list
| Model | Size | Speed | Accuracy | Use Case | | Model | Size | Speed | Accuracy | Use Case |
|-------|------|-------|----------|----------| |-------|------|-------|----------|----------|
| **phi3.5:3.8b** | 3GB | ⚡⚡⚡⚡⚡ | ⭐⭐⭐⭐ | Fast triage | | **deepseek-r1:1.5b** | 3GB | ⚡⚡⚡⚡⚡ | ⭐⭐⭐⭐ | Fast triage |
| **qwen2.5-coder:7b** | 6GB | ⚡⚡⚡⚡ | ⭐⭐⭐⭐⭐ | Deep analysis | | **qwen2.5-coder:7b** | 6GB | ⚡⚡⚡⚡ | ⭐⭐⭐⭐⭐ | Deep analysis |
| **deepseek-coder-v2:16b** | 12GB | ⚡⚡⚡ | ⭐⭐⭐⭐⭐ | Maximum accuracy | | **deepseek-coder-v2:16b** | 12GB | ⚡⚡⚡ | ⭐⭐⭐⭐⭐ | Maximum accuracy |
| **llama3.2:3b** | 2.5GB | ⚡⚡⚡⚡⚡ | ⭐⭐⭐ | Ultra-fast | | **llama3.2:3b** | 2.5GB | ⚡⚡⚡⚡⚡ | ⭐⭐⭐ | Ultra-fast |
@@ -428,7 +428,7 @@ ollama list
## 📖 Example Output ## 📖 Example Output
``` ```
🧠 AI-POWERED ANALYSIS (cascade: phi3.5:3.8b + qwen2.5-coder:7b) 🧠 AI-POWERED ANALYSIS (cascade: deepseek-r1:1.5b + qwen2.5-coder:7b)
Analyzing findings with local LLM Analyzing findings with local LLM
AI:C admin.example.com → 3 findings AI:C admin.example.com → 3 findings
@@ -498,7 +498,7 @@ killall ollama
rm -rf ~/.ollama/models rm -rf ~/.ollama/models
# Re-pull # Re-pull
ollama pull phi3.5:3.8b ollama pull deepseek-r1:1.5b
ollama pull qwen2.5-coder:7b ollama pull qwen2.5-coder:7b
``` ```
@@ -544,14 +544,14 @@ ollama pull qwen2.5-coder:7b
| **Passive Enumeration** | ~25 seconds | 20 concurrent sources | | **Passive Enumeration** | ~25 seconds | 20 concurrent sources |
| **HTTP Probing** | ~35 seconds | 2 active subdomains | | **HTTP Probing** | ~35 seconds | 2 active subdomains |
| **Security Checks** | ~40 seconds | 13 checks per subdomain | | **Security Checks** | ~40 seconds | 13 checks per subdomain |
| **AI Triage** | ~10 seconds | phi3.5:3.8b fast filtering | | **AI Triage** | ~10 seconds | deepseek-r1:1.5b fast filtering |
| **AI Deep Analysis** | ~25 seconds | qwen2.5-coder:7b analysis | | **AI Deep Analysis** | ~25 seconds | qwen2.5-coder:7b analysis |
| **Report Generation** | ~3 seconds | Executive summary | | **Report Generation** | ~3 seconds | Executive summary |
| **Total** | **2:18 min** | With AI enabled | | **Total** | **2:18 min** | With AI enabled |
### AI Performance Characteristics ### AI Performance Characteristics
**Fast Triage Model (Phi-3.5:3.8b):** **Fast Triage Model (DeepSeek-R1:1.5b):**
- Initial load time: ~3-5 seconds (first request) - Initial load time: ~3-5 seconds (first request)
- Analysis time: 2-5 seconds per finding - Analysis time: 2-5 seconds per finding
- Memory footprint: ~3.5GB - Memory footprint: ~3.5GB

View File

@@ -108,7 +108,7 @@ Average CPU usage during scan:
| DNSRepo | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | DNSRepo | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Subdomain Center | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | Subdomain Center | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Wayback Machine | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | | Wayback Machine | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
| **Total Sources** | **11** | **25+** | **55+** | **14** | **9** | **6** | | **Total Sources** | **20** | **25+** | **55+** | **14** | **9** | **6** |
### Active Scanning Features ### Active Scanning Features
@@ -171,7 +171,7 @@ This eliminates the need to chain multiple tools together.
#### 2. Parallel Processing Architecture #### 2. Parallel Processing Architecture
God's Eye uses Go's goroutines for maximum parallelization: God's Eye uses Go's goroutines for maximum parallelization:
- 11 passive sources queried simultaneously - 20 passive sources queried simultaneously
- DNS brute-force with configurable concurrency - DNS brute-force with configurable concurrency
- 13 HTTP security checks run in parallel per subdomain - 13 HTTP security checks run in parallel per subdomain
@@ -352,5 +352,6 @@ Comprehensive security posture assessment:
--- ---
*Benchmark conducted by Orizon Security Team* *Note: Benchmark data is based on internal testing and may vary depending on network conditions, target complexity, and hardware specifications. These numbers are meant to provide a general comparison rather than precise measurements.*
*Last updated: 2025* *Last updated: 2025*

View File

@@ -92,7 +92,7 @@ time ./god-eye -d target.com --enable-ai --no-brute
React version 16.8.0 has known XSS vulnerability React version 16.8.0 has known XSS vulnerability
Missing rate limiting on /api/v1/users endpoint Missing rate limiting on /api/v1/users endpoint
(1 more findings...) (1 more findings...)
model: phi3.5:3.8b→qwen2.5-coder:7b model: deepseek-r1:1.5b→qwen2.5-coder:7b
CVE: React: CVE-2020-15168 - XSS vulnerability in development mode CVE: React: CVE-2020-15168 - XSS vulnerability in development mode
═══════════════════════════════════════════════════ ═══════════════════════════════════════════════════
``` ```
@@ -100,7 +100,7 @@ time ./god-eye -d target.com --enable-ai --no-brute
### AI Report Section ### AI Report Section
``` ```
🧠 AI-POWERED ANALYSIS (cascade: phi3.5:3.8b + qwen2.5-coder:7b) 🧠 AI-POWERED ANALYSIS (cascade: deepseek-r1:1.5b + qwen2.5-coder:7b)
Analyzing findings with local LLM Analyzing findings with local LLM
AI:C api.example.com → 4 findings AI:C api.example.com → 4 findings
@@ -336,7 +336,7 @@ time ./god-eye -d target.com --enable-ai --ai-deep --no-brute
# Use fast model only (skip deep analysis) # Use fast model only (skip deep analysis)
./god-eye -d large-target.com --enable-ai --ai-cascade=false \ ./god-eye -d large-target.com --enable-ai --ai-cascade=false \
--ai-deep-model phi3.5:3.8b --ai-deep-model deepseek-r1:1.5b
# Disable AI for initial enumeration, enable for interesting findings # Disable AI for initial enumeration, enable for interesting findings
./god-eye -d large-target.com --no-brute -s > subdomains.txt ./god-eye -d large-target.com --no-brute -s > subdomains.txt

208
README.md
View File

@@ -39,7 +39,7 @@
<td width="33%" align="center"> <td width="33%" align="center">
### ⚡ All-in-One ### ⚡ All-in-One
**11 passive sources** + DNS brute-forcing + HTTP probing + security checks in **one tool**. No need to chain 5+ tools together. **20 passive sources** + DNS brute-forcing + HTTP probing + security checks in **one tool**. No need to chain 5+ tools together.
</td> </td>
<td width="33%" align="center"> <td width="33%" align="center">
@@ -120,14 +120,14 @@ God's Eye now features **AI-powered security analysis** using local LLM models v
<td width="50%" align="center"> <td width="50%" align="center">
**Basic Scan** **Basic Scan**
<img src="docs/images/demo.gif" alt="God's Eye Basic Demo" width="100%"> <img src="assets/demo.gif" alt="God's Eye Basic Demo" width="100%">
<em>Standard subdomain enumeration</em> <em>Standard subdomain enumeration</em>
</td> </td>
<td width="50%" align="center"> <td width="50%" align="center">
**AI-Powered Scan** **AI-Powered Scan**
<img src="docs/images/demo-ai.gif" alt="God's Eye AI Demo" width="100%"> <img src="assets/demo-ai.gif" alt="God's Eye AI Demo" width="100%">
<em>With real-time CVE detection & analysis</em> <em>With real-time CVE detection & analysis</em>
</td> </td>
@@ -140,7 +140,7 @@ God's Eye now features **AI-powered security analysis** using local LLM models v
curl https://ollama.ai/install.sh | sh curl https://ollama.ai/install.sh | sh
# Pull models (5-10 mins) # Pull models (5-10 mins)
ollama pull phi3.5:3.8b && ollama pull qwen2.5-coder:7b ollama pull deepseek-r1:1.5b && ollama pull qwen2.5-coder:7b
# Run with AI # Run with AI
ollama serve & ollama serve &
@@ -154,9 +154,9 @@ ollama serve &
## Features ## Features
### 🔍 Subdomain Discovery ### 🔍 Subdomain Discovery
- **11 Passive Sources**: crt.sh, Certspotter, AlienVault, HackerTarget, URLScan, RapidDNS, Anubis, ThreatMiner, DNSRepo, SubdomainCenter, Wayback - **20 Passive Sources**: crt.sh, Certspotter, AlienVault, HackerTarget, URLScan, RapidDNS, Anubis, ThreatMiner, DNSRepo, SubdomainCenter, Wayback, CommonCrawl, Sitedossier, Riddler, Robtex, DNSHistory, ArchiveToday, JLDC, SynapsInt, CensysFree
- **DNS Brute-forcing**: Concurrent DNS resolution with customizable wordlists - **DNS Brute-forcing**: Concurrent DNS resolution with customizable wordlists
- **Wildcard Detection**: Improved detection using multiple random patterns - **Advanced Wildcard Detection**: Multi-layer detection using DNS + HTTP validation with confidence scoring
### 🌐 HTTP Probing ### 🌐 HTTP Probing
- Status code, content length, response time - Status code, content length, response time
@@ -164,6 +164,7 @@ ollama serve &
- Technology fingerprinting (WordPress, React, Next.js, Angular, Laravel, Django, etc.) - Technology fingerprinting (WordPress, React, Next.js, Angular, Laravel, Django, etc.)
- Server header analysis - Server header analysis
- TLS/SSL information (version, issuer, expiry) - TLS/SSL information (version, issuer, expiry)
- **TLS Certificate Fingerprinting** (NEW!) - Detects firewalls, VPNs, and appliances from self-signed certificates
### 🛡️ Security Checks ### 🛡️ Security Checks
- **Security Headers**: CSP, HSTS, X-Frame-Options, X-Content-Type-Options, etc. - **Security Headers**: CSP, HSTS, X-Frame-Options, X-Content-Type-Options, etc.
@@ -187,14 +188,26 @@ ollama serve &
- **JavaScript Analysis**: Extracts secrets, API keys, and hidden endpoints from JS files - **JavaScript Analysis**: Extracts secrets, API keys, and hidden endpoints from JS files
- **Port Scanning**: Quick TCP port scan on common ports - **Port Scanning**: Quick TCP port scan on common ports
- **WAF Detection**: Identifies Cloudflare, AWS WAF, Akamai, Imperva, etc. - **WAF Detection**: Identifies Cloudflare, AWS WAF, Akamai, Imperva, etc.
- **TLS Appliance Detection**: Identifies 25+ security vendors from certificates (Fortinet, Palo Alto, Cisco, F5, etc.)
### ⚡ Performance ### ⚡ Performance
- **Parallel HTTP Checks**: All security checks run concurrently - **Parallel HTTP Checks**: All security checks run concurrently
- **Connection Pooling**: Shared HTTP client with TCP/TLS reuse - **Connection Pooling**: Shared HTTP client with TCP/TLS reuse
- **High Concurrency**: Up to 1000+ concurrent workers - **High Concurrency**: Up to 1000+ concurrent workers
- **Intelligent Rate Limiting**: Adaptive backoff based on error rates
- **Retry Logic**: Automatic retry with exponential backoff for DNS/HTTP failures
- **Progress Bars**: Real-time progress with ETA and speed indicators
### 🥷 Stealth Mode
- **4 Stealth Levels**: light, moderate, aggressive, paranoid
- **User-Agent Rotation**: 25+ realistic browser User-Agents
- **Randomized Delays**: Configurable jitter between requests
- **Per-Host Throttling**: Limit concurrent requests per target
- **DNS Query Distribution**: Spread queries across resolvers
- **Request Randomization**: Shuffle wordlists and targets
### 🧠 AI Integration (NEW!) ### 🧠 AI Integration (NEW!)
- **Local LLM Analysis**: Powered by Ollama (phi3.5 + qwen2.5-coder) - **Local LLM Analysis**: Powered by Ollama (deepseek-r1:1.5b + qwen2.5-coder)
- **JavaScript Code Review**: Intelligent secret detection and vulnerability analysis - **JavaScript Code Review**: Intelligent secret detection and vulnerability analysis
- **CVE Matching**: Automatic vulnerability detection for discovered technologies - **CVE Matching**: Automatic vulnerability detection for discovered technologies
- **Smart Cascade**: Fast triage filter + deep analysis for optimal performance - **Smart Cascade**: Fast triage filter + deep analysis for optimal performance
@@ -227,7 +240,7 @@ Traditional regex-based tools miss context. God's Eye's AI integration provides:
curl https://ollama.ai/install.sh | sh curl https://ollama.ai/install.sh | sh
# 2. Pull AI models (5-10 minutes, one-time) # 2. Pull AI models (5-10 minutes, one-time)
ollama pull phi3.5:3.8b # Fast triage (~3GB) ollama pull deepseek-r1:1.5b # Fast triage (~3GB)
ollama pull qwen2.5-coder:7b # Deep analysis (~6GB) ollama pull qwen2.5-coder:7b # Deep analysis (~6GB)
# 3. Start Ollama server # 3. Start Ollama server
@@ -247,6 +260,32 @@ ollama serve
| **Anomaly Detection** | Cross-subdomain pattern analysis | `AI:MEDIUM: Dev environment exposed in production` | | **Anomaly Detection** | Cross-subdomain pattern analysis | `AI:MEDIUM: Dev environment exposed in production` |
| **Executive Reports** | Professional summaries with remediation | Auto-generated markdown reports | | **Executive Reports** | Professional summaries with remediation | Auto-generated markdown reports |
### CVE Database (CISA KEV)
God's Eye includes an **offline CVE database** powered by the [CISA Known Exploited Vulnerabilities](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) catalog:
- **1,400+ actively exploited CVEs** - Confirmed vulnerabilities used in real-world attacks
- **Auto-download** - Database downloads automatically on first AI-enabled scan
- **Instant lookups** - Zero-latency, offline CVE matching
- **Daily updates** - CISA updates the catalog daily; refresh with `update-db`
```bash
# Update CVE database manually
./god-eye update-db
# Check database status
./god-eye db-info
# The database auto-downloads on first use with --enable-ai
./god-eye -d target.com --enable-ai # Auto-downloads if not present
```
**Database location:** `~/.god-eye/kev.json` (~1.3MB)
The KEV database is used **in addition to** real-time NVD API lookups, providing a multi-layer approach:
1. **KEV (instant)** - Critical, actively exploited vulnerabilities
2. **NVD API (fallback)** - Comprehensive CVE database (rate-limited)
### AI Usage Examples ### AI Usage Examples
```bash ```bash
@@ -261,7 +300,7 @@ ollama serve
# Custom models # Custom models
./god-eye -d target.com --enable-ai \ ./god-eye -d target.com --enable-ai \
--ai-fast-model phi3.5:3.8b \ --ai-fast-model deepseek-r1:1.5b \
--ai-deep-model deepseek-coder-v2:16b --ai-deep-model deepseek-coder-v2:16b
# Export with AI findings # Export with AI findings
@@ -271,7 +310,7 @@ ollama serve
### Sample AI Output ### Sample AI Output
``` ```
🧠 AI-POWERED ANALYSIS (cascade: phi3.5:3.8b + qwen2.5-coder:7b) 🧠 AI-POWERED ANALYSIS (cascade: deepseek-r1:1.5b + qwen2.5-coder:7b)
AI:C api.target.com → 4 findings AI:C api.target.com → 4 findings
AI:H admin.target.com → 2 findings AI:H admin.target.com → 2 findings
@@ -361,11 +400,15 @@ Flags:
AI Flags: AI Flags:
--enable-ai Enable AI-powered analysis with Ollama --enable-ai Enable AI-powered analysis with Ollama
--ai-url string Ollama API URL (default "http://localhost:11434") --ai-url string Ollama API URL (default "http://localhost:11434")
--ai-fast-model Fast triage model (default "phi3.5:3.8b") --ai-fast-model Fast triage model (default "deepseek-r1:1.5b")
--ai-deep-model Deep analysis model (default "qwen2.5-coder:7b") --ai-deep-model Deep analysis model (default "qwen2.5-coder:7b")
--ai-cascade Use cascade (fast triage + deep) (default true) --ai-cascade Use cascade (fast triage + deep) (default true)
--ai-deep Enable deep AI analysis on all findings --ai-deep Enable deep AI analysis on all findings
-h, --help Help for god-eye -h, --help Help for god-eye
Subcommands:
update-db Download/update CISA KEV vulnerability database
db-info Show vulnerability database status
``` ```
### Examples ### Examples
@@ -399,6 +442,39 @@ AI Flags:
./god-eye -d example.com -s | httpx ./god-eye -d example.com -s | httpx
``` ```
### Stealth Mode
For evasion during authorized penetration testing:
```bash
# Light stealth (reduces detection, minimal speed impact)
./god-eye -d target.com --stealth light
# Moderate stealth (balanced evasion/speed)
./god-eye -d target.com --stealth moderate
# Aggressive stealth (slow, high evasion)
./god-eye -d target.com --stealth aggressive
# Paranoid mode (very slow, maximum evasion)
./god-eye -d target.com --stealth paranoid
```
**Stealth Mode Comparison:**
| Mode | Max Threads | Delay | Rate/sec | Use Case |
|------|-------------|-------|----------|----------|
| `light` | 100 | 10-50ms | 100 | Avoid basic rate limits |
| `moderate` | 30 | 50-200ms | 30 | Evade WAF detection |
| `aggressive` | 10 | 200ms-1s | 10 | Sensitive targets |
| `paranoid` | 3 | 1-5s | 2 | Maximum stealth needed |
**Features by Mode:**
- **All modes**: User-Agent rotation (25+ browsers)
- **Moderate+**: Request randomization, DNS query distribution
- **Aggressive+**: 50% timing jitter, per-host throttling
- **Paranoid**: 70% jitter, single connection per host
--- ---
## Benchmark ## Benchmark
@@ -432,6 +508,8 @@ Performance comparison with other popular subdomain enumeration tools on a mediu
| Cloud Detection | ✅ | ❌ | ❌ | ❌ | | Cloud Detection | ✅ | ❌ | ❌ | ❌ |
| Port Scanning | ✅ | ❌ | ❌ | ❌ | | Port Scanning | ✅ | ❌ | ❌ | ❌ |
| Technology Detection | ✅ | ❌ | ❌ | ❌ | | Technology Detection | ✅ | ❌ | ❌ | ❌ |
| TLS Appliance Fingerprint | ✅ | ❌ | ❌ | ❌ |
| AI-Powered Analysis | ✅ | ❌ | ❌ | ❌ |
--- ---
@@ -447,24 +525,68 @@ God's Eye features a modern, colorful CLI with:
### JSON Output ### JSON Output
The `--json` flag outputs a structured report with full metadata:
```json ```json
[ {
{ "meta": {
"subdomain": "api.example.com", "version": "0.1",
"ips": ["192.168.1.1"], "tool_name": "God's Eye",
"cname": "api-gateway.cloudprovider.com", "target": "example.com",
"status_code": 200, "start_time": "2024-01-15T10:30:00Z",
"title": "API Documentation", "end_time": "2024-01-15T10:32:15Z",
"technologies": ["nginx", "Node.js"], "duration": "2m15s",
"cloud_provider": "AWS", "duration_ms": 135000,
"security_headers": ["HSTS", "CSP"], "concurrency": 1000,
"missing_headers": ["X-Frame-Options"], "timeout": 5,
"admin_panels": ["/admin"], "options": {
"api_endpoints": ["/api/v1", "/swagger"], "brute_force": true,
"js_files": ["/static/app.js"], "http_probe": true,
"js_secrets": ["api_key: AKIAIOSFODNN7EXAMPLE"] "ai_analysis": true
} }
] },
"stats": {
"total_subdomains": 25,
"active_subdomains": 18,
"vulnerabilities": 3,
"takeover_vulnerable": 1,
"ai_findings": 12
},
"wildcard": {
"detected": false,
"confidence": 0.95
},
"findings": {
"critical": [{"subdomain": "dev.example.com", "type": "Subdomain Takeover", "description": "GitHub Pages"}],
"high": [{"subdomain": "api.example.com", "type": "Git Repository Exposed", "description": ".git directory accessible"}],
"medium": [],
"low": [],
"info": []
},
"subdomains": [
{
"subdomain": "api.example.com",
"ips": ["192.168.1.1"],
"cname": "api-gateway.cloudprovider.com",
"status_code": 200,
"title": "API Documentation",
"technologies": ["nginx", "Node.js"],
"cloud_provider": "AWS",
"security_headers": ["HSTS", "CSP"],
"missing_headers": ["X-Frame-Options"],
"tls_self_signed": false,
"tls_fingerprint": {
"vendor": "Fortinet",
"product": "FortiGate",
"version": "60F",
"appliance_type": "firewall",
"internal_hosts": ["fw-internal.corp.local"]
},
"ai_findings": ["Potential IDOR in /api/users endpoint"],
"cve_findings": ["nginx: CVE-2021-23017"]
}
]
}
``` ```
### CSV Output ### CSV Output
@@ -505,6 +627,34 @@ Checks 110+ vulnerable services including:
- **Email Security (SPF/DMARC)**: Records are checked on the target domain specified with `-d`. Make sure to specify the root domain (e.g., `example.com` not `sub.example.com`) for accurate email security results. - **Email Security (SPF/DMARC)**: Records are checked on the target domain specified with `-d`. Make sure to specify the root domain (e.g., `example.com` not `sub.example.com`) for accurate email security results.
- **SPA Detection**: The tool detects Single Page Applications that return the same content for all routes, filtering out false positives for admin panels, API endpoints, and backup files. - **SPA Detection**: The tool detects Single Page Applications that return the same content for all routes, filtering out false positives for admin panels, API endpoints, and backup files.
### TLS Certificate Fingerprinting
God's Eye analyzes TLS certificates to identify security appliances, especially useful for self-signed certificates commonly used by firewalls and VPN gateways.
**Detected Vendors (25+):**
| Category | Vendors |
|----------|---------|
| **Firewalls** | Fortinet FortiGate, Palo Alto PAN-OS, Cisco ASA/Firepower, SonicWall, Check Point, pfSense, OPNsense, WatchGuard, Sophos XG, Juniper SRX, Zyxel USG |
| **VPN** | OpenVPN, Pulse Secure, GlobalProtect, Cisco AnyConnect |
| **Load Balancers** | F5 BIG-IP, Citrix NetScaler, HAProxy, NGINX Plus, Kemp LoadMaster |
| **WAF/Security** | Barracuda, Imperva |
| **Other** | MikroTik, Ubiquiti UniFi, VMware NSX, DrayTek Vigor |
**Features:**
- Detects vendor and product from certificate Subject/Issuer fields
- Extracts version information where available (e.g., `FortiGate v60F`)
- Identifies internal hostnames from certificate SANs (`.local`, `.internal`, etc.)
- Reports appliance type (firewall, vpn, loadbalancer, proxy, waf)
**Sample Output:**
```
● vpn.target.com [200]
Security: TLS: TLS 1.2 (self-signed)
APPLIANCE: Fortinet FortiGate v60F (firewall)
INTERNAL: fw-internal.corp.local, vpn-gw-01.internal
```
--- ---
## Use Cases ## Use Cases
@@ -559,7 +709,7 @@ Tested on production domain (authorized testing):
| Passive Enumeration | ~25 sec | - | | Passive Enumeration | ~25 sec | - |
| HTTP Probing | ~35 sec | - | | HTTP Probing | ~35 sec | - |
| Security Checks | ~40 sec | - | | Security Checks | ~40 sec | - |
| AI Triage | ~10 sec | phi3.5:3.8b | | AI Triage | ~10 sec | deepseek-r1:1.5b |
| AI Deep Analysis | ~25 sec | qwen2.5-coder:7b | | AI Deep Analysis | ~25 sec | qwen2.5-coder:7b |
| Report Generation | ~3 sec | qwen2.5-coder:7b | | Report Generation | ~3 sec | qwen2.5-coder:7b |

View File

Before

Width:  |  Height:  |  Size: 715 KiB

After

Width:  |  Height:  |  Size: 715 KiB

View File

Before

Width:  |  Height:  |  Size: 342 KiB

After

Width:  |  Height:  |  Size: 342 KiB

View File

@@ -6,6 +6,7 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
"god-eye/internal/ai"
"god-eye/internal/config" "god-eye/internal/config"
"god-eye/internal/output" "god-eye/internal/output"
"god-eye/internal/scanner" "god-eye/internal/scanner"
@@ -27,7 +28,9 @@ Examples:
god-eye -d example.com -r 1.1.1.1,8.8.8.8 Custom resolvers god-eye -d example.com -r 1.1.1.1,8.8.8.8 Custom resolvers
god-eye -d example.com -p 80,443,8080 Custom ports to scan god-eye -d example.com -p 80,443,8080 Custom ports to scan
god-eye -d example.com --json JSON output to stdout god-eye -d example.com --json JSON output to stdout
god-eye -d example.com -s Silent mode (subdomains only)`, god-eye -d example.com -s Silent mode (subdomains only)
god-eye -d example.com --stealth moderate Moderate stealth (evasion mode)
god-eye -d example.com --stealth paranoid Maximum stealth (very slow)`,
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
if cfg.Domain == "" { if cfg.Domain == "" {
fmt.Println(output.Red("[-]"), "Domain is required. Use -d flag.") fmt.Println(output.Red("[-]"), "Domain is required. Use -d flag.")
@@ -67,11 +70,75 @@ Examples:
// AI flags // AI flags
rootCmd.Flags().BoolVar(&cfg.EnableAI, "enable-ai", false, "Enable AI-powered analysis with Ollama (includes CVE search)") rootCmd.Flags().BoolVar(&cfg.EnableAI, "enable-ai", false, "Enable AI-powered analysis with Ollama (includes CVE search)")
rootCmd.Flags().StringVar(&cfg.AIUrl, "ai-url", "http://localhost:11434", "Ollama API URL") rootCmd.Flags().StringVar(&cfg.AIUrl, "ai-url", "http://localhost:11434", "Ollama API URL")
rootCmd.Flags().StringVar(&cfg.AIFastModel, "ai-fast-model", "phi3.5:3.8b", "Fast triage model") rootCmd.Flags().StringVar(&cfg.AIFastModel, "ai-fast-model", "deepseek-r1:1.5b", "Fast triage model")
rootCmd.Flags().StringVar(&cfg.AIDeepModel, "ai-deep-model", "qwen2.5-coder:7b", "Deep analysis model (supports function calling)") rootCmd.Flags().StringVar(&cfg.AIDeepModel, "ai-deep-model", "qwen2.5-coder:7b", "Deep analysis model (supports function calling)")
rootCmd.Flags().BoolVar(&cfg.AICascade, "ai-cascade", true, "Use cascade (fast triage + deep analysis)") rootCmd.Flags().BoolVar(&cfg.AICascade, "ai-cascade", true, "Use cascade (fast triage + deep analysis)")
rootCmd.Flags().BoolVar(&cfg.AIDeepAnalysis, "ai-deep", false, "Enable deep AI analysis on all findings") rootCmd.Flags().BoolVar(&cfg.AIDeepAnalysis, "ai-deep", false, "Enable deep AI analysis on all findings")
// Stealth flags
rootCmd.Flags().StringVar(&cfg.StealthMode, "stealth", "", "Stealth mode: light, moderate, aggressive, paranoid (reduces detection)")
// Database update subcommand
updateDbCmd := &cobra.Command{
Use: "update-db",
Short: "Update vulnerability databases (CISA KEV)",
Long: `Downloads and updates local vulnerability databases:
- CISA KEV (Known Exploited Vulnerabilities) - ~500KB, updated daily by CISA
The KEV database contains vulnerabilities that are actively exploited in the wild.
This data is used for instant, offline CVE lookups during scans.`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println(output.BoldCyan("🔄 Updating vulnerability databases..."))
fmt.Println()
// Update KEV
fmt.Print(output.Dim(" Downloading CISA KEV catalog... "))
kevStore := ai.GetKEVStore()
if err := kevStore.Update(); err != nil {
fmt.Println(output.Red("FAILED"))
fmt.Printf(" %s %v\n", output.Red("Error:"), err)
os.Exit(1)
}
version, count, date := kevStore.GetCatalogInfo()
fmt.Println(output.Green("OK"))
fmt.Printf(" %s %s vulnerabilities (v%s, released %s)\n",
output.Green("✓"), output.BoldWhite(fmt.Sprintf("%d", count)), version, date)
fmt.Println()
fmt.Println(output.Green("✅ Database update complete!"))
fmt.Println(output.Dim(" KEV data cached at: ~/.god-eye/kev.json"))
},
}
rootCmd.AddCommand(updateDbCmd)
// Database info subcommand
dbInfoCmd := &cobra.Command{
Use: "db-info",
Short: "Show vulnerability database status",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println(output.BoldCyan("📊 Vulnerability Database Status"))
fmt.Println()
kevStore := ai.GetKEVStore()
// Check if KEV needs update
if kevStore.NeedUpdate() {
fmt.Println(output.Yellow("⚠️ CISA KEV: Not downloaded or outdated"))
fmt.Println(output.Dim(" Run 'god-eye update-db' to download"))
} else {
if err := kevStore.Load(); err != nil {
fmt.Printf("%s CISA KEV: Error loading - %v\n", output.Red("❌"), err)
} else {
version, count, date := kevStore.GetCatalogInfo()
fmt.Printf("%s CISA KEV: %s vulnerabilities\n", output.Green("✓"), output.BoldWhite(fmt.Sprintf("%d", count)))
fmt.Printf(" Version: %s | Released: %s\n", version, date)
fmt.Println(output.Dim(" Source: https://www.cisa.gov/known-exploited-vulnerabilities-catalog"))
}
}
},
}
rootCmd.AddCommand(dbInfoCmd)
if err := rootCmd.Execute(); err != nil { if err := rootCmd.Execute(); err != nil {
os.Exit(1) os.Exit(1)
} }

View File

@@ -6,7 +6,10 @@ import (
"io" "io"
"net/http" "net/http"
"net/url" "net/url"
"sort"
"strconv"
"strings" "strings"
"sync"
"time" "time"
) )
@@ -54,77 +57,141 @@ type NVDResponse struct {
} `json:"vulnerabilities"` } `json:"vulnerabilities"`
} }
// CVECacheEntry holds cached CVE results
type CVECacheEntry struct {
Result string
Timestamp time.Time
}
var ( var (
nvdClient = &http.Client{ nvdClient = &http.Client{
Timeout: 10 * time.Second, Timeout: 15 * time.Second,
} }
nvdBaseURL = "https://services.nvd.nist.gov/rest/json/cves/2.0" nvdBaseURL = "https://services.nvd.nist.gov/rest/json/cves/2.0"
// Rate limiting: max 5 requests per 30 seconds (NVD allows 10 req/60s without API key) // Rate limiting: NVD allows 5 req/30s without API key
lastNVDRequest time.Time lastNVDRequest time.Time
nvdRateLimit = 6 * time.Second // Wait 6 seconds between requests nvdRateLimit = 7 * time.Second // Wait 7 seconds between requests (safer)
nvdMutex sync.Mutex
// CVE Cache to avoid duplicate lookups across subdomains
cveCache = make(map[string]*CVECacheEntry)
cveCacheMutex sync.RWMutex
) )
// SearchCVE searches for CVE vulnerabilities using NVD API // SearchCVE searches for CVE vulnerabilities with caching to avoid duplicates
// Returns a concise format: "CVE-ID (SEVERITY/SCORE), CVE-ID2 (SEVERITY/SCORE)"
func SearchCVE(technology string, version string) (string, error) { func SearchCVE(technology string, version string) (string, error) {
// Normalize technology name // Normalize technology name
tech := normalizeTechnology(technology) tech := normalizeTechnology(technology)
cacheKey := tech // Use normalized tech as cache key
// Build search query // Check cache first
query := tech cveCacheMutex.RLock()
if version != "" && version != "unknown" { if entry, ok := cveCache[cacheKey]; ok {
query = fmt.Sprintf("%s %s", tech, version) cveCacheMutex.RUnlock()
} // Cache valid for 1 hour
if time.Since(entry.Timestamp) < time.Hour {
// Query NVD API return entry.Result, nil
cves, err := queryNVD(query)
if err != nil {
return fmt.Sprintf("Unable to search CVE database for %s: %v", technology, err), nil
}
if len(cves) == 0 {
return fmt.Sprintf("No known CVE vulnerabilities found for %s %s in the NVD database. This doesn't guarantee the software is secure - always keep software updated.", technology, version), nil
}
// Format results
result := fmt.Sprintf("CVE Vulnerabilities for %s %s:\n\n", technology, version)
result += fmt.Sprintf("Found %d CVE(s):\n\n", len(cves))
// Show top 5 most recent/critical CVEs
maxShow := 5
if len(cves) < maxShow {
maxShow = len(cves)
}
for i := 0; i < maxShow; i++ {
cve := cves[i]
result += fmt.Sprintf("🔴 %s (%s - Score: %.1f)\n", cve.ID, cve.Severity, cve.Score)
result += fmt.Sprintf(" Published: %s\n", cve.Published)
// Truncate description if too long
desc := cve.Description
if len(desc) > 200 {
desc = desc[:200] + "..."
} }
result += fmt.Sprintf(" %s\n", desc) } else {
cveCacheMutex.RUnlock()
}
if len(cve.References) > 0 { var allCVEs []CVEInfo
result += fmt.Sprintf(" Reference: %s\n", cve.References[0])
// Layer 1: Check CISA KEV first (instant, offline, most critical)
if kevResult, err := SearchKEV(tech); err == nil && kevResult != "" {
// Parse KEV result for CVE IDs
lines := strings.Split(kevResult, "\n")
for _, line := range lines {
if strings.Contains(line, "CVE-") {
parts := strings.Fields(line)
for _, part := range parts {
if strings.HasPrefix(part, "CVE-") {
allCVEs = append(allCVEs, CVEInfo{
ID: strings.TrimSuffix(part, ":"),
Severity: "CRITICAL",
Score: 9.8, // KEV = actively exploited
})
}
}
}
} }
result += "\n"
} }
if len(cves) > maxShow { // Layer 2: Query NVD API for additional CVEs
result += fmt.Sprintf("... and %d more CVEs. Check https://nvd.nist.gov for complete details.\n", len(cves)-maxShow) if nvdCVEs, err := queryNVD(tech); err == nil {
allCVEs = append(allCVEs, nvdCVEs...)
} }
// Don't fail on NVD errors - just use what we have
result += "\n⚠ Recommendation: Update to the latest version to mitigate known vulnerabilities." // Format result
result := formatCVEsConcise(allCVEs)
// Cache the result
cveCacheMutex.Lock()
cveCache[cacheKey] = &CVECacheEntry{
Result: result,
Timestamp: time.Now(),
}
cveCacheMutex.Unlock()
return result, nil return result, nil
} }
// queryNVD queries the NVD API for CVE information // formatCVEsConcise returns a concise CVE summary
func formatCVEsConcise(cves []CVEInfo) string {
if len(cves) == 0 {
return ""
}
// Sort by score (highest first)
sort.Slice(cves, func(i, j int) bool {
return cves[i].Score > cves[j].Score
})
// Deduplicate by CVE ID
seen := make(map[string]bool)
var uniqueCVEs []CVEInfo
for _, cve := range cves {
if !seen[cve.ID] && cve.ID != "" {
seen[cve.ID] = true
uniqueCVEs = append(uniqueCVEs, cve)
}
}
if len(uniqueCVEs) == 0 {
return ""
}
// Show top 3 most critical
maxShow := 3
if len(uniqueCVEs) < maxShow {
maxShow = len(uniqueCVEs)
}
var parts []string
for i := 0; i < maxShow; i++ {
cve := uniqueCVEs[i]
severity := cve.Severity
if severity == "" {
severity = "UNK"
}
parts = append(parts, fmt.Sprintf("%s (%s/%.1f)", cve.ID, severity, cve.Score))
}
result := strings.Join(parts, ", ")
if len(uniqueCVEs) > maxShow {
result += fmt.Sprintf(" +%d more", len(uniqueCVEs)-maxShow)
}
return result
}
// queryNVD queries the NVD API for CVE information with thread-safe rate limiting
func queryNVD(keyword string) ([]CVEInfo, error) { func queryNVD(keyword string) ([]CVEInfo, error) {
nvdMutex.Lock()
// Rate limiting: wait if necessary // Rate limiting: wait if necessary
if !lastNVDRequest.IsZero() { if !lastNVDRequest.IsZero() {
elapsed := time.Since(lastNVDRequest) elapsed := time.Since(lastNVDRequest)
@@ -133,11 +200,12 @@ func queryNVD(keyword string) ([]CVEInfo, error) {
} }
} }
lastNVDRequest = time.Now() lastNVDRequest = time.Now()
nvdMutex.Unlock()
// Build URL with query parameters // Build URL with query parameters
params := url.Values{} params := url.Values{}
params.Add("keywordSearch", keyword) params.Add("keywordSearch", keyword)
params.Add("resultsPerPage", "10") // Limit results params.Add("resultsPerPage", "5") // Limit results for speed
reqURL := fmt.Sprintf("%s?%s", nvdBaseURL, params.Encode()) reqURL := fmt.Sprintf("%s?%s", nvdBaseURL, params.Encode())
@@ -171,7 +239,17 @@ func queryNVD(keyword string) ([]CVEInfo, error) {
// Convert to CVEInfo // Convert to CVEInfo
var cves []CVEInfo var cves []CVEInfo
cutoffYear := time.Now().Year() - 10 // Filter CVEs older than 10 years
for _, vuln := range nvdResp.Vulnerabilities { for _, vuln := range nvdResp.Vulnerabilities {
// Filter old CVEs - extract year from CVE ID (format: CVE-YYYY-NNNNN)
if len(vuln.CVE.ID) >= 8 {
yearStr := vuln.CVE.ID[4:8]
if year, err := strconv.Atoi(yearStr); err == nil && year < cutoffYear {
continue // Skip CVEs older than cutoff
}
}
cve := CVEInfo{ cve := CVEInfo{
ID: vuln.CVE.ID, ID: vuln.CVE.ID,
Published: formatDate(vuln.CVE.Published), Published: formatDate(vuln.CVE.Published),

432
internal/ai/kev.go Normal file
View File

@@ -0,0 +1,432 @@
package ai
import (
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
const (
// CISA KEV Catalog URL
kevURL = "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"
// Cache settings
kevCacheFile = "kev.json"
kevCacheTTL = 24 * time.Hour // Refresh once per day
)
// KEVCatalog represents the CISA Known Exploited Vulnerabilities catalog
type KEVCatalog struct {
Title string `json:"title"`
CatalogVersion string `json:"catalogVersion"`
DateReleased string `json:"dateReleased"`
Count int `json:"count"`
Vulnerabilities []KEVulnerability `json:"vulnerabilities"`
}
// KEVulnerability represents a single KEV entry
type KEVulnerability struct {
CveID string `json:"cveID"`
VendorProject string `json:"vendorProject"`
Product string `json:"product"`
VulnerabilityName string `json:"vulnerabilityName"`
DateAdded string `json:"dateAdded"`
ShortDescription string `json:"shortDescription"`
RequiredAction string `json:"requiredAction"`
DueDate string `json:"dueDate"`
KnownRansomwareCampaignUse string `json:"knownRansomwareCampaignUse"`
Notes string `json:"notes"`
}
// KEVStore manages the local KEV database
type KEVStore struct {
catalog *KEVCatalog
productMap map[string][]KEVulnerability // Maps product names to vulnerabilities
cacheDir string
mu sync.RWMutex
loaded bool
}
var (
kevStore *KEVStore
kevStoreOnce sync.Once
)
// GetKEVStore returns the singleton KEV store instance
func GetKEVStore() *KEVStore {
kevStoreOnce.Do(func() {
cacheDir := getKEVCacheDir()
kevStore = &KEVStore{
cacheDir: cacheDir,
productMap: make(map[string][]KEVulnerability),
}
})
return kevStore
}
// getKEVCacheDir returns the cache directory path
func getKEVCacheDir() string {
homeDir, err := os.UserHomeDir()
if err != nil {
return ".god-eye"
}
return filepath.Join(homeDir, ".god-eye")
}
// getCachePath returns the full path to the cache file
func (k *KEVStore) getCachePath() string {
return filepath.Join(k.cacheDir, kevCacheFile)
}
// IsLoaded returns whether the KEV database is loaded
func (k *KEVStore) IsLoaded() bool {
k.mu.RLock()
defer k.mu.RUnlock()
return k.loaded
}
// GetCatalogInfo returns catalog metadata
func (k *KEVStore) GetCatalogInfo() (version string, count int, date string) {
k.mu.RLock()
defer k.mu.RUnlock()
if k.catalog == nil {
return "", 0, ""
}
return k.catalog.CatalogVersion, k.catalog.Count, k.catalog.DateReleased
}
// NeedUpdate checks if the cache needs to be updated
func (k *KEVStore) NeedUpdate() bool {
cachePath := k.getCachePath()
info, err := os.Stat(cachePath)
if err != nil {
return true // File doesn't exist
}
return time.Since(info.ModTime()) > kevCacheTTL
}
// Update downloads and updates the KEV database
func (k *KEVStore) Update() error {
// Ensure cache directory exists
if err := os.MkdirAll(k.cacheDir, 0755); err != nil {
return fmt.Errorf("failed to create cache directory: %w", err)
}
// Download KEV catalog
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Get(kevURL)
if err != nil {
return fmt.Errorf("failed to download KEV catalog: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("KEV download failed with status: %d", resp.StatusCode)
}
// Read response body
body, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("failed to read KEV response: %w", err)
}
// Parse to validate JSON
var catalog KEVCatalog
if err := json.Unmarshal(body, &catalog); err != nil {
return fmt.Errorf("failed to parse KEV catalog: %w", err)
}
// Write to cache file
cachePath := k.getCachePath()
if err := os.WriteFile(cachePath, body, 0644); err != nil {
return fmt.Errorf("failed to write cache file: %w", err)
}
// Load into memory
return k.loadFromCatalog(&catalog)
}
// Load loads the KEV database from cache or downloads if needed
func (k *KEVStore) Load() error {
return k.LoadWithProgress(false)
}
// LoadWithProgress loads the KEV database with optional progress output
func (k *KEVStore) LoadWithProgress(showProgress bool) error {
k.mu.Lock()
defer k.mu.Unlock()
if k.loaded {
return nil
}
cachePath := k.getCachePath()
// Try to load from cache
data, err := os.ReadFile(cachePath)
if err == nil {
var catalog KEVCatalog
if err := json.Unmarshal(data, &catalog); err == nil {
return k.loadFromCatalog(&catalog)
}
}
// Cache doesn't exist or is invalid, need to download
if showProgress {
fmt.Print("📥 First run: downloading CISA KEV database... ")
}
k.mu.Unlock()
err = k.Update()
k.mu.Lock()
if err != nil {
if showProgress {
fmt.Println("FAILED")
}
return err
}
if showProgress {
fmt.Println("OK")
fmt.Printf(" ✓ Loaded %d known exploited vulnerabilities\n", k.catalog.Count)
}
return nil
}
// loadFromCatalog builds the internal index from catalog data
func (k *KEVStore) loadFromCatalog(catalog *KEVCatalog) error {
k.catalog = catalog
k.productMap = make(map[string][]KEVulnerability)
for _, vuln := range catalog.Vulnerabilities {
// Index by product name (lowercase for matching)
productKey := strings.ToLower(vuln.Product)
k.productMap[productKey] = append(k.productMap[productKey], vuln)
// Also index by vendor
vendorKey := strings.ToLower(vuln.VendorProject)
if vendorKey != productKey {
k.productMap[vendorKey] = append(k.productMap[vendorKey], vuln)
}
}
k.loaded = true
return nil
}
// SearchByProduct searches for KEV vulnerabilities by product name
func (k *KEVStore) SearchByProduct(product string) []KEVulnerability {
k.mu.RLock()
defer k.mu.RUnlock()
if !k.loaded || k.catalog == nil {
return nil
}
product = strings.ToLower(product)
var results []KEVulnerability
// Direct match
if vulns, ok := k.productMap[product]; ok {
results = append(results, vulns...)
}
// Partial match for products that might have different naming
for key, vulns := range k.productMap {
if key != product && (strings.Contains(key, product) || strings.Contains(product, key)) {
results = append(results, vulns...)
}
}
return deduplicateKEV(results)
}
// SearchByCVE searches for a specific CVE ID in the KEV catalog
func (k *KEVStore) SearchByCVE(cveID string) *KEVulnerability {
k.mu.RLock()
defer k.mu.RUnlock()
if !k.loaded || k.catalog == nil {
return nil
}
cveID = strings.ToUpper(cveID)
for _, vuln := range k.catalog.Vulnerabilities {
if vuln.CveID == cveID {
return &vuln
}
}
return nil
}
// SearchByTechnology searches for KEV entries matching a technology name
func (k *KEVStore) SearchByTechnology(technology string) []KEVulnerability {
k.mu.RLock()
defer k.mu.RUnlock()
if !k.loaded || k.catalog == nil {
return nil
}
technology = strings.ToLower(technology)
var results []KEVulnerability
// Normalize common technology names
aliases := getTechnologyAliases(technology)
for _, vuln := range k.catalog.Vulnerabilities {
productLower := strings.ToLower(vuln.Product)
vendorLower := strings.ToLower(vuln.VendorProject)
nameLower := strings.ToLower(vuln.VulnerabilityName)
for _, alias := range aliases {
if strings.Contains(productLower, alias) ||
strings.Contains(vendorLower, alias) ||
strings.Contains(nameLower, alias) {
results = append(results, vuln)
break
}
}
}
return deduplicateKEV(results)
}
// getTechnologyAliases returns common aliases for a technology
func getTechnologyAliases(tech string) []string {
aliases := []string{tech}
// Common mappings
mappings := map[string][]string{
"nginx": {"nginx"},
"apache": {"apache", "httpd"},
"iis": {"iis", "internet information services"},
"wordpress": {"wordpress"},
"drupal": {"drupal"},
"joomla": {"joomla"},
"tomcat": {"tomcat"},
"jenkins": {"jenkins"},
"gitlab": {"gitlab"},
"exchange": {"exchange"},
"sharepoint": {"sharepoint"},
"citrix": {"citrix"},
"vmware": {"vmware", "vcenter", "esxi"},
"fortinet": {"fortinet", "fortigate", "fortios"},
"paloalto": {"palo alto", "pan-os"},
"cisco": {"cisco"},
"f5": {"f5", "big-ip"},
"pulse": {"pulse", "pulse secure"},
"sonicwall": {"sonicwall"},
"zyxel": {"zyxel"},
"nextjs": {"next.js", "nextjs"},
"react": {"react"},
"angular": {"angular"},
"vue": {"vue"},
"php": {"php"},
"java": {"java"},
"log4j": {"log4j", "log4shell"},
"spring": {"spring"},
"struts": {"struts"},
"confluence": {"confluence"},
"jira": {"jira"},
"atlassian": {"atlassian"},
}
if mapped, ok := mappings[tech]; ok {
aliases = append(aliases, mapped...)
}
return aliases
}
// deduplicateKEV removes duplicate KEV entries
func deduplicateKEV(vulns []KEVulnerability) []KEVulnerability {
seen := make(map[string]bool)
var result []KEVulnerability
for _, v := range vulns {
if !seen[v.CveID] {
seen[v.CveID] = true
result = append(result, v)
}
}
return result
}
// FormatKEVResult formats KEV search results for display
func FormatKEVResult(vulns []KEVulnerability, technology string) string {
if len(vulns) == 0 {
return ""
}
var sb strings.Builder
sb.WriteString(fmt.Sprintf("🚨 CISA KEV Alert for %s:\n", technology))
sb.WriteString(fmt.Sprintf(" Found %d ACTIVELY EXPLOITED vulnerabilities!\n\n", len(vulns)))
// Show up to 5 most relevant
maxShow := 5
if len(vulns) < maxShow {
maxShow = len(vulns)
}
for i := 0; i < maxShow; i++ {
v := vulns[i]
sb.WriteString(fmt.Sprintf(" 🔴 %s - %s\n", v.CveID, v.VulnerabilityName))
// Truncate description if too long
desc := v.ShortDescription
if len(desc) > 150 {
desc = desc[:150] + "..."
}
sb.WriteString(fmt.Sprintf(" %s\n", desc))
// Ransomware indicator
if v.KnownRansomwareCampaignUse == "Known" {
sb.WriteString(" ⚠️ USED IN RANSOMWARE CAMPAIGNS\n")
}
sb.WriteString(fmt.Sprintf(" Added: %s | Due: %s\n", v.DateAdded, v.DueDate))
sb.WriteString("\n")
}
if len(vulns) > maxShow {
sb.WriteString(fmt.Sprintf(" ... and %d more KEV entries\n", len(vulns)-maxShow))
}
sb.WriteString(" These vulnerabilities are CONFIRMED to be exploited in the wild.\n")
sb.WriteString(" ⚡ IMMEDIATE patching is strongly recommended.\n")
return sb.String()
}
// SearchKEV is a convenience function for searching KEV by technology
func SearchKEV(technology string) (string, error) {
return SearchKEVWithProgress(technology, false)
}
// SearchKEVWithProgress searches KEV with optional download progress
func SearchKEVWithProgress(technology string, showProgress bool) (string, error) {
store := GetKEVStore()
// Auto-load if not loaded (with auto-download if needed)
if !store.IsLoaded() {
if err := store.LoadWithProgress(showProgress); err != nil {
return "", fmt.Errorf("failed to load KEV database: %w", err)
}
}
vulns := store.SearchByTechnology(technology)
if len(vulns) == 0 {
return "", nil // No KEV found, not an error
}
return FormatKEVResult(vulns, technology), nil
}

View File

@@ -12,7 +12,7 @@ import (
// OllamaClient handles communication with local Ollama instance // OllamaClient handles communication with local Ollama instance
type OllamaClient struct { type OllamaClient struct {
BaseURL string BaseURL string
FastModel string // phi3.5:3.8b for quick triage FastModel string // deepseek-r1:1.5b for quick triage
DeepModel string // qwen2.5-coder:7b for deep analysis DeepModel string // qwen2.5-coder:7b for deep analysis
Timeout time.Duration Timeout time.Duration
EnableCascade bool EnableCascade bool
@@ -51,7 +51,7 @@ func NewOllamaClient(baseURL, fastModel, deepModel string, enableCascade bool) *
baseURL = "http://localhost:11434" baseURL = "http://localhost:11434"
} }
if fastModel == "" { if fastModel == "" {
fastModel = "phi3.5:3.8b" fastModel = "deepseek-r1:1.5b"
} }
if deepModel == "" { if deepModel == "" {
deepModel = "qwen2.5-coder:7b" deepModel = "qwen2.5-coder:7b"
@@ -227,7 +227,7 @@ Format: SEVERITY: finding`, truncate(summary, 4000))
// GenerateReport creates executive summary and recommendations // GenerateReport creates executive summary and recommendations
func (c *OllamaClient) GenerateReport(findings string, stats map[string]int) (string, error) { func (c *OllamaClient) GenerateReport(findings string, stats map[string]int) (string, error) {
prompt := fmt.Sprintf(`Create a concise security assessment report: prompt := fmt.Sprintf(`You are a security analyst. Create a security assessment report based on the findings below.
SCAN STATISTICS: SCAN STATISTICS:
- Total subdomains: %d - Total subdomains: %d
@@ -235,15 +235,21 @@ SCAN STATISTICS:
- Vulnerabilities: %d - Vulnerabilities: %d
- Takeovers: %d - Takeovers: %d
KEY FINDINGS: FINDINGS DATA (use these EXACT subdomain names in your report):
%s %s
Generate report with: INSTRUCTIONS:
## Executive Summary (2-3 sentences) 1. Use the ACTUAL subdomain names from the findings data above (e.g., "new.computerplus.it", "api.example.com")
## Critical Findings (prioritized list) 2. Do NOT use generic placeholders like "Subdomain A" or "Subdomain B"
## Recommendations (actionable items) 3. Reference specific vulnerabilities found for each subdomain
4. Include CVE IDs when present
Be concise and professional.`, Generate report with:
## Executive Summary (2-3 sentences with real subdomain names)
## Critical Findings (list each affected subdomain by name with its issues)
## Recommendations (actionable items referencing specific subdomains)
Be concise and professional. Use the real data provided above.`,
stats["total"], stats["active"], stats["vulns"], stats["takeovers"], truncate(findings, 3000)) stats["total"], stats["active"], stats["vulns"], stats["takeovers"], truncate(findings, 3000))
response, err := c.query(c.DeepModel, prompt, 45*time.Second) response, err := c.query(c.DeepModel, prompt, 45*time.Second)
@@ -255,34 +261,106 @@ Be concise and professional.`,
} }
// CVEMatch checks for known vulnerabilities in detected technologies // CVEMatch checks for known vulnerabilities in detected technologies
// Returns concise format directly: "CVE-ID (SEVERITY/SCORE), ..."
func (c *OllamaClient) CVEMatch(technology, version string) (string, error) { func (c *OllamaClient) CVEMatch(technology, version string) (string, error) {
// Call SearchCVE directly instead of using function calling (more reliable) // Call SearchCVE directly - it now returns concise format with caching
cveData, err := SearchCVE(technology, version) cveData, err := SearchCVE(technology, version)
if err != nil { if err != nil {
return "", err return "", err
} }
// If no CVEs found, return empty // Return directly without AI processing - the format is already clean
if strings.Contains(cveData, "No known CVE vulnerabilities found") { return cveData, nil
return "", nil }
// FilterSecrets uses AI to filter false positives from potential secrets
// Returns only real secrets, filtering out UI text, placeholders, and example values
func (c *OllamaClient) FilterSecrets(potentialSecrets []string) ([]string, error) {
if len(potentialSecrets) == 0 {
return nil, nil
} }
// Ask AI to analyze the CVE data // Build the list of secrets for AI analysis
prompt := fmt.Sprintf(`Analyze these CVE vulnerabilities for %s and provide a concise security assessment: secretsList := strings.Join(potentialSecrets, "\n")
prompt := fmt.Sprintf(`Task: Filter JavaScript findings. Output only REAL secrets.
Examples of FAKE (do NOT output):
- Change Password (UI button text)
- Update Password (UI button text)
- Password (just a word)
- Enter your API key (placeholder)
- YOUR_API_KEY (placeholder)
- Login, Token, Secret (single words)
Examples of REAL (DO output):
- pk_test_TYooMQauvdEDq54NiTphI7jx (Stripe key - has random chars)
- AKIAIOSFODNN7EXAMPLE (AWS key - 20 char pattern)
- mongodb://admin:secret123@db.example.com (connection string)
- eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... (JWT token)
Input findings:
%s %s
Provide a 2-3 sentence summary focusing on: Output only the REAL secrets in their original [Type] format, one per line. If none are real, output: NONE`, secretsList)
- Most critical vulnerabilities
- Key recommendations`, technology, cveData)
response, err := c.query(c.FastModel, prompt, 20*time.Second) response, err := c.query(c.FastModel, prompt, 15*time.Second)
if err != nil { if err != nil {
// If AI analysis fails, return the raw CVE data // On error, return original list (fail open for security)
return cveData, nil return potentialSecrets, nil
} }
return response, nil // Parse response
response = strings.TrimSpace(response)
if strings.ToUpper(response) == "NONE" || response == "" {
return nil, nil
}
var realSecrets []string
// First, try to find secrets in [Type] format in the response
lines := strings.Split(response, "\n")
for _, line := range lines {
line = strings.TrimSpace(line)
if line == "" || strings.ToUpper(line) == "NONE" {
continue
}
// Accept any line that contains our format [Type] value
if strings.Contains(line, "[") && strings.Contains(line, "]") {
// Extract the [Type] value part
startIdx := strings.Index(line, "[")
if startIdx >= 0 {
// Find the actual secret value after ]
endBracket := strings.Index(line[startIdx:], "]")
if endBracket > 0 {
// Get everything from [ to end of meaningful content
secretPart := line[startIdx:]
// Remove trailing explanations (after " " or " -")
if dashIdx := strings.Index(secretPart, " "); dashIdx > 0 {
secretPart = secretPart[:dashIdx]
}
if dashIdx := strings.Index(secretPart, " -"); dashIdx > 0 {
secretPart = secretPart[:dashIdx]
}
secretPart = strings.TrimSpace(secretPart)
if secretPart != "" && strings.HasPrefix(secretPart, "[") {
realSecrets = append(realSecrets, secretPart)
}
}
}
}
}
// If AI returned nothing valid but we had input, something went wrong
// Return original secrets (fail-safe: better false positives than miss real ones)
if len(realSecrets) == 0 && len(potentialSecrets) > 0 {
// Check if response contains "NONE" anywhere - that's a valid empty result
if !strings.Contains(strings.ToUpper(response), "NONE") {
return potentialSecrets, nil
}
}
return realSecrets, nil
} }
// query sends a request to Ollama API // query sends a request to Ollama API

View File

@@ -23,12 +23,14 @@ type Config struct {
OnlyActive bool OnlyActive bool
JsonOutput bool JsonOutput bool
// AI Configuration // AI Configuration
EnableAI bool EnableAI bool
AIUrl string AIUrl string
AIFastModel string AIFastModel string
AIDeepModel string AIDeepModel string
AICascade bool AICascade bool
AIDeepAnalysis bool AIDeepAnalysis bool
// Stealth Configuration
StealthMode string // off, light, moderate, aggressive, paranoid
} }
// Stats holds scan statistics // Stats holds scan statistics
@@ -61,6 +63,9 @@ type SubdomainResult struct {
TLSVersion string `json:"tls_version,omitempty"` TLSVersion string `json:"tls_version,omitempty"`
TLSIssuer string `json:"tls_issuer,omitempty"` TLSIssuer string `json:"tls_issuer,omitempty"`
TLSExpiry string `json:"tls_expiry,omitempty"` TLSExpiry string `json:"tls_expiry,omitempty"`
TLSSelfSigned bool `json:"tls_self_signed,omitempty"`
// TLS Fingerprint for appliance detection
TLSFingerprint *TLSFingerprint `json:"tls_fingerprint,omitempty"`
Ports []int `json:"ports,omitempty"` Ports []int `json:"ports,omitempty"`
Takeover string `json:"takeover,omitempty"` Takeover string `json:"takeover,omitempty"`
ResponseMs int64 `json:"response_ms,omitempty"` ResponseMs int64 `json:"response_ms,omitempty"`
@@ -100,6 +105,21 @@ type SubdomainResult struct {
CVEFindings []string `json:"cve_findings,omitempty"` CVEFindings []string `json:"cve_findings,omitempty"`
} }
// TLSFingerprint holds detailed certificate information for appliance detection
type TLSFingerprint struct {
Vendor string `json:"vendor,omitempty"` // Detected vendor (Fortinet, Palo Alto, etc.)
Product string `json:"product,omitempty"` // Product name (FortiGate, PA-xxx, etc.)
Version string `json:"version,omitempty"` // Version if detectable
SubjectCN string `json:"subject_cn,omitempty"` // Subject Common Name
SubjectOrg string `json:"subject_org,omitempty"` // Subject Organization
SubjectOU string `json:"subject_ou,omitempty"` // Subject Organizational Unit
IssuerCN string `json:"issuer_cn,omitempty"` // Issuer Common Name
IssuerOrg string `json:"issuer_org,omitempty"` // Issuer Organization
SerialNumber string `json:"serial_number,omitempty"` // Certificate serial number
InternalHosts []string `json:"internal_hosts,omitempty"` // Potential internal hostnames found
ApplianceType string `json:"appliance_type,omitempty"` // firewall, vpn, loadbalancer, proxy, etc.
}
// IPInfo holds IP geolocation data // IPInfo holds IP geolocation data
type IPInfo struct { type IPInfo struct {
ASN string `json:"as"` ASN string `json:"as"`

View File

@@ -1,6 +1,7 @@
package dns package dns
import ( import (
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"net/http" "net/http"
@@ -10,9 +11,16 @@ import (
"github.com/miekg/dns" "github.com/miekg/dns"
"god-eye/internal/config" "god-eye/internal/config"
"god-eye/internal/retry"
) )
// ResolveSubdomain resolves a subdomain to IP addresses with retry logic
func ResolveSubdomain(subdomain string, resolvers []string, timeout int) []string { func ResolveSubdomain(subdomain string, resolvers []string, timeout int) []string {
return ResolveSubdomainWithRetry(subdomain, resolvers, timeout, true)
}
// ResolveSubdomainWithRetry resolves with optional retry
func ResolveSubdomainWithRetry(subdomain string, resolvers []string, timeout int, useRetry bool) []string {
c := dns.Client{ c := dns.Client{
Timeout: time.Duration(timeout) * time.Second, Timeout: time.Duration(timeout) * time.Second,
} }
@@ -20,16 +28,48 @@ func ResolveSubdomain(subdomain string, resolvers []string, timeout int) []strin
m := dns.Msg{} m := dns.Msg{}
m.SetQuestion(dns.Fqdn(subdomain), dns.TypeA) m.SetQuestion(dns.Fqdn(subdomain), dns.TypeA)
// Try each resolver
for _, resolver := range resolvers { for _, resolver := range resolvers {
r, _, err := c.Exchange(&m, resolver)
if err != nil || r == nil {
continue
}
var ips []string var ips []string
for _, ans := range r.Answer {
if a, ok := ans.(*dns.A); ok { if useRetry {
ips = append(ips, a.A.String()) // Use retry logic
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeout*2)*time.Second)
result := retry.Do(ctx, retry.DNSConfig(), func() (interface{}, error) {
r, _, err := c.Exchange(&m, resolver)
if err != nil {
return nil, err
}
if r == nil {
return nil, fmt.Errorf("nil response")
}
var resolvedIPs []string
for _, ans := range r.Answer {
if a, ok := ans.(*dns.A); ok {
resolvedIPs = append(resolvedIPs, a.A.String())
}
}
if len(resolvedIPs) == 0 {
return nil, fmt.Errorf("no A records")
}
return resolvedIPs, nil
})
cancel()
if result.Error == nil && result.Value != nil {
ips = result.Value.([]string)
}
} else {
// Direct resolution without retry
r, _, err := c.Exchange(&m, resolver)
if err == nil && r != nil {
for _, ans := range r.Answer {
if a, ok := ans.(*dns.A); ok {
ips = append(ips, a.A.String())
}
}
} }
} }

341
internal/dns/wildcard.go Normal file
View File

@@ -0,0 +1,341 @@
package dns
import (
"crypto/md5"
"crypto/tls"
"fmt"
"io"
"net/http"
"sort"
"strings"
"time"
)
// WildcardInfo holds information about wildcard DNS detection
type WildcardInfo struct {
IsWildcard bool
WildcardIPs []string
WildcardCNAME string
HTTPStatusCode int
HTTPBodyHash string
HTTPBodySize int64
Confidence float64 // 0-1 confidence level
}
// WildcardDetector performs comprehensive wildcard detection
type WildcardDetector struct {
resolvers []string
timeout int
httpClient *http.Client
testSubdomains []string
}
// NewWildcardDetector creates a new wildcard detector
func NewWildcardDetector(resolvers []string, timeout int) *WildcardDetector {
return &WildcardDetector{
resolvers: resolvers,
timeout: timeout,
httpClient: &http.Client{
Timeout: time.Duration(timeout) * time.Second,
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
},
testSubdomains: generateTestSubdomains(),
}
}
// generateTestSubdomains creates random non-existent subdomain patterns
func generateTestSubdomains() []string {
timestamp := time.Now().UnixNano()
return []string{
fmt.Sprintf("wildcard-test-%d-abc", timestamp),
fmt.Sprintf("random-xyz-%d-def", timestamp%1000000),
fmt.Sprintf("nonexistent-%d-ghi", timestamp%999999),
fmt.Sprintf("fake-sub-%d-jkl", timestamp%888888),
fmt.Sprintf("test-random-%d-mno", timestamp%777777),
}
}
// Detect performs comprehensive wildcard detection on a domain
func (wd *WildcardDetector) Detect(domain string) *WildcardInfo {
info := &WildcardInfo{
WildcardIPs: make([]string, 0),
}
// Phase 1: DNS-based detection (multiple random subdomains)
ipCounts := make(map[string]int)
var cnames []string
for _, pattern := range wd.testSubdomains {
testDomain := fmt.Sprintf("%s.%s", pattern, domain)
// Resolve A records
ips := ResolveSubdomain(testDomain, wd.resolvers, wd.timeout)
for _, ip := range ips {
ipCounts[ip]++
}
// Resolve CNAME
cname := ResolveCNAME(testDomain, wd.resolvers, wd.timeout)
if cname != "" {
cnames = append(cnames, cname)
}
}
// Analyze DNS results
totalTests := len(wd.testSubdomains)
for ip, count := range ipCounts {
// If same IP appears in >= 60% of tests, it's likely wildcard
if float64(count)/float64(totalTests) >= 0.6 {
info.WildcardIPs = append(info.WildcardIPs, ip)
}
}
// Check CNAME consistency
if len(cnames) > 0 && allEqual(cnames) {
info.WildcardCNAME = cnames[0]
}
// If no DNS wildcard detected, we're done
if len(info.WildcardIPs) == 0 && info.WildcardCNAME == "" {
info.IsWildcard = false
info.Confidence = 0.95 // High confidence no wildcard
return info
}
// Phase 2: HTTP-based validation (if DNS wildcard detected)
if len(info.WildcardIPs) > 0 {
httpResults := wd.validateHTTP(domain)
info.HTTPStatusCode = httpResults.statusCode
info.HTTPBodyHash = httpResults.bodyHash
info.HTTPBodySize = httpResults.bodySize
// Calculate confidence based on HTTP consistency
if httpResults.consistent {
info.Confidence = 0.95 // Very confident it's a wildcard
} else {
info.Confidence = 0.7 // DNS wildcard but inconsistent HTTP
}
}
info.IsWildcard = true
sort.Strings(info.WildcardIPs)
return info
}
type httpValidationResult struct {
statusCode int
bodyHash string
bodySize int64
consistent bool
}
// validateHTTP checks if random subdomains return consistent HTTP responses
func (wd *WildcardDetector) validateHTTP(domain string) httpValidationResult {
result := httpValidationResult{}
var statusCodes []int
var bodySizes []int64
var bodyHashes []string
// Test 3 random subdomains via HTTP
for i := 0; i < 3; i++ {
testDomain := fmt.Sprintf("%s.%s", wd.testSubdomains[i], domain)
for _, scheme := range []string{"https", "http"} {
url := fmt.Sprintf("%s://%s", scheme, testDomain)
resp, err := wd.httpClient.Get(url)
if err != nil {
continue
}
statusCodes = append(statusCodes, resp.StatusCode)
// Read body (limited)
body, _ := io.ReadAll(io.LimitReader(resp.Body, 50000))
resp.Body.Close()
bodySizes = append(bodySizes, int64(len(body)))
bodyHashes = append(bodyHashes, fmt.Sprintf("%x", md5.Sum(body)))
break // Only need one successful scheme
}
}
if len(statusCodes) == 0 {
return result
}
// Check consistency
result.statusCode = statusCodes[0]
if len(bodySizes) > 0 {
result.bodySize = bodySizes[0]
}
if len(bodyHashes) > 0 {
result.bodyHash = bodyHashes[0]
}
// Check if all results are consistent (same status and similar size)
result.consistent = allEqualInts(statusCodes) && similarSizes(bodySizes)
return result
}
// IsWildcardIP checks if an IP is a known wildcard IP for this domain
func (wd *WildcardDetector) IsWildcardIP(ip string, wildcardInfo *WildcardInfo) bool {
if wildcardInfo == nil || !wildcardInfo.IsWildcard {
return false
}
for _, wip := range wildcardInfo.WildcardIPs {
if ip == wip {
return true
}
}
return false
}
// IsWildcardResponse checks if an HTTP response matches wildcard pattern
func (wd *WildcardDetector) IsWildcardResponse(statusCode int, bodySize int64, wildcardInfo *WildcardInfo) bool {
if wildcardInfo == nil || !wildcardInfo.IsWildcard {
return false
}
// Check status code match
if wildcardInfo.HTTPStatusCode != 0 && statusCode != wildcardInfo.HTTPStatusCode {
return false
}
// Check body size similarity (within 10%)
if wildcardInfo.HTTPBodySize > 0 {
ratio := float64(bodySize) / float64(wildcardInfo.HTTPBodySize)
if ratio < 0.9 || ratio > 1.1 {
return false
}
}
return true
}
// Helper functions
func allEqual(strs []string) bool {
if len(strs) == 0 {
return true
}
first := strs[0]
for _, s := range strs[1:] {
if s != first {
return false
}
}
return true
}
func allEqualInts(ints []int) bool {
if len(ints) == 0 {
return true
}
first := ints[0]
for _, i := range ints[1:] {
if i != first {
return false
}
}
return true
}
func similarSizes(sizes []int64) bool {
if len(sizes) < 2 {
return true
}
// Find min and max
min, max := sizes[0], sizes[0]
for _, s := range sizes[1:] {
if s < min {
min = s
}
if s > max {
max = s
}
}
// Allow 20% variance
if min == 0 {
return max < 100 // Small empty responses
}
return float64(max)/float64(min) <= 1.2
}
// FilterWildcardSubdomains removes subdomains that match wildcard pattern
func FilterWildcardSubdomains(subdomains []string, domain string, resolvers []string, timeout int) (filtered []string, wildcardInfo *WildcardInfo) {
detector := NewWildcardDetector(resolvers, timeout)
wildcardInfo = detector.Detect(domain)
if !wildcardInfo.IsWildcard {
return subdomains, wildcardInfo
}
// Filter out subdomains that resolve to wildcard IPs
filtered = make([]string, 0, len(subdomains))
wildcardIPSet := make(map[string]bool)
for _, ip := range wildcardInfo.WildcardIPs {
wildcardIPSet[ip] = true
}
for _, subdomain := range subdomains {
ips := ResolveSubdomain(subdomain, resolvers, timeout)
// Check if all IPs are wildcard IPs
allWildcard := true
for _, ip := range ips {
if !wildcardIPSet[ip] {
allWildcard = false
break
}
}
// Keep if not all IPs are wildcards, or if no IPs resolved
if !allWildcard || len(ips) == 0 {
filtered = append(filtered, subdomain)
}
}
return filtered, wildcardInfo
}
// GetWildcardSummary returns a human-readable summary of wildcard detection
func (wi *WildcardInfo) GetSummary() string {
if !wi.IsWildcard {
return "No wildcard DNS detected"
}
var parts []string
parts = append(parts, "Wildcard DNS DETECTED")
if len(wi.WildcardIPs) > 0 {
ips := wi.WildcardIPs
if len(ips) > 3 {
ips = ips[:3]
}
parts = append(parts, fmt.Sprintf("IPs: %s", strings.Join(ips, ", ")))
}
if wi.WildcardCNAME != "" {
parts = append(parts, fmt.Sprintf("CNAME: %s", wi.WildcardCNAME))
}
if wi.HTTPStatusCode > 0 {
parts = append(parts, fmt.Sprintf("HTTP: %d (%dB)", wi.HTTPStatusCode, wi.HTTPBodySize))
}
parts = append(parts, fmt.Sprintf("Confidence: %.0f%%", wi.Confidence*100))
return strings.Join(parts, " | ")
}

View File

@@ -3,6 +3,7 @@ package http
import ( import (
"crypto/tls" "crypto/tls"
"net/http" "net/http"
"sync"
"time" "time"
) )
@@ -25,3 +26,44 @@ func GetSharedClient(timeout int) *http.Client {
}, },
} }
} }
// UserAgentManager handles User-Agent rotation
type UserAgentManager struct {
agents []string
index int
mu sync.Mutex
}
var defaultUAManager = &UserAgentManager{
agents: []string{
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0",
},
}
// GetUserAgent returns the next User-Agent in rotation
func GetUserAgent() string {
defaultUAManager.mu.Lock()
defer defaultUAManager.mu.Unlock()
ua := defaultUAManager.agents[defaultUAManager.index]
defaultUAManager.index = (defaultUAManager.index + 1) % len(defaultUAManager.agents)
return ua
}
// NewRequestWithUA creates an HTTP request with a rotated User-Agent
func NewRequestWithUA(method, url string) (*http.Request, error) {
req, err := http.NewRequest(method, url, nil)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", GetUserAgent())
req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
req.Header.Set("Accept-Language", "en-US,en;q=0.5")
req.Header.Set("Connection", "keep-alive")
return req, nil
}

View File

@@ -15,17 +15,8 @@ import (
func ProbeHTTP(subdomain string, timeout int) *config.SubdomainResult { func ProbeHTTP(subdomain string, timeout int) *config.SubdomainResult {
result := &config.SubdomainResult{} result := &config.SubdomainResult{}
transport := &http.Transport{ // Use shared transport for connection pooling
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, client := GetSharedClient(timeout)
}
client := &http.Client{
Timeout: time.Duration(timeout) * time.Second,
Transport: transport,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
urls := []string{ urls := []string{
fmt.Sprintf("https://%s", subdomain), fmt.Sprintf("https://%s", subdomain),
@@ -78,6 +69,19 @@ func ProbeHTTP(subdomain string, timeout int) *config.SubdomainResult {
case tls.VersionTLS10: case tls.VersionTLS10:
result.TLSVersion = "TLS 1.0" result.TLSVersion = "TLS 1.0"
} }
// Check for self-signed certificate
result.TLSSelfSigned = IsSelfSigned(cert)
// Analyze certificate for appliance fingerprinting
// This is especially useful for self-signed certs (firewalls, VPNs, etc.)
if fp := AnalyzeTLSCertificate(cert); fp != nil {
result.TLSFingerprint = fp
// Add vendor/product to tech stack if detected
if fp.Vendor != "" && fp.Product != "" {
result.Tech = append(result.Tech, fp.Vendor+" "+fp.Product)
}
}
} }
// Interesting headers // Interesting headers
@@ -117,27 +121,47 @@ func ProbeHTTP(subdomain string, timeout int) *config.SubdomainResult {
// Detect technologies // Detect technologies
bodyStr := string(body) bodyStr := string(body)
if strings.Contains(bodyStr, "wp-content") || strings.Contains(bodyStr, "wordpress") { bodyStrLower := strings.ToLower(bodyStr)
// WordPress - specific patterns
if strings.Contains(bodyStr, "wp-content") || strings.Contains(bodyStr, "wp-includes") {
result.Tech = append(result.Tech, "WordPress") result.Tech = append(result.Tech, "WordPress")
} }
if strings.Contains(bodyStr, "_next") || strings.Contains(bodyStr, "Next.js") { // Next.js - specific patterns (check before React since Next uses React)
if strings.Contains(bodyStr, "/_next/") || strings.Contains(bodyStr, "__NEXT_DATA__") {
result.Tech = append(result.Tech, "Next.js") result.Tech = append(result.Tech, "Next.js")
} } else if strings.Contains(bodyStr, "react-root") || strings.Contains(bodyStr, "data-reactroot") ||
if strings.Contains(bodyStr, "react") || strings.Contains(bodyStr, "React") { strings.Contains(bodyStr, "__REACT_DEVTOOLS_GLOBAL_HOOK__") {
// React - only if not Next.js
result.Tech = append(result.Tech, "React") result.Tech = append(result.Tech, "React")
} }
if strings.Contains(bodyStr, "laravel") || strings.Contains(bodyStr, "Laravel") { // Laravel - specific patterns
if strings.Contains(bodyStr, "laravel_session") || strings.Contains(bodyStr, "XSRF-TOKEN") {
result.Tech = append(result.Tech, "Laravel") result.Tech = append(result.Tech, "Laravel")
} }
if strings.Contains(bodyStr, "django") || strings.Contains(bodyStr, "Django") { // Django - specific patterns
if strings.Contains(bodyStr, "csrfmiddlewaretoken") || strings.Contains(bodyStrLower, "django") {
result.Tech = append(result.Tech, "Django") result.Tech = append(result.Tech, "Django")
} }
if strings.Contains(bodyStr, "angular") || strings.Contains(bodyStr, "ng-") { // Angular - more specific patterns (ng-app, ng-controller are Angular 1.x specific)
if strings.Contains(bodyStr, "ng-app") || strings.Contains(bodyStr, "ng-controller") ||
strings.Contains(bodyStr, "ng-version") || strings.Contains(bodyStrLower, "angular.js") ||
strings.Contains(bodyStrLower, "angular.min.js") || strings.Contains(bodyStr, "@angular/core") {
result.Tech = append(result.Tech, "Angular") result.Tech = append(result.Tech, "Angular")
} }
if strings.Contains(bodyStr, "vue") || strings.Contains(bodyStr, "Vue.js") { // Vue.js - specific patterns
if strings.Contains(bodyStr, "data-v-") || strings.Contains(bodyStr, "__VUE__") ||
strings.Contains(bodyStr, "vue.js") || strings.Contains(bodyStr, "vue.min.js") {
result.Tech = append(result.Tech, "Vue.js") result.Tech = append(result.Tech, "Vue.js")
} }
// Svelte
if strings.Contains(bodyStr, "svelte") && strings.Contains(bodyStr, "__svelte") {
result.Tech = append(result.Tech, "Svelte")
}
// Nuxt.js (Vue-based)
if strings.Contains(bodyStr, "__NUXT__") || strings.Contains(bodyStr, "_nuxt/") {
result.Tech = append(result.Tech, "Nuxt.js")
}
} }
break break

View File

@@ -0,0 +1,487 @@
package http
import (
"crypto/x509"
"regexp"
"strings"
"god-eye/internal/config"
)
// AppliancePattern defines a pattern to match vendor/product from certificate fields
type AppliancePattern struct {
Vendor string
Product string
ApplianceType string // firewall, vpn, loadbalancer, proxy, waf, appliance
// Match patterns (any match triggers detection)
SubjectCNPatterns []string
SubjectOrgPatterns []string
SubjectOUPatterns []string
IssuerCNPatterns []string
IssuerOrgPatterns []string
// Version extraction regex (optional)
VersionRegex string
}
// appliancePatterns contains known signatures for security appliances
var appliancePatterns = []AppliancePattern{
// Fortinet FortiGate
{
Vendor: "Fortinet",
Product: "FortiGate",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"FortiGate", "FGT", "fortinet", "FGVM",
},
SubjectOrgPatterns: []string{"Fortinet"},
IssuerCNPatterns: []string{"FortiGate", "Fortinet"},
IssuerOrgPatterns: []string{"Fortinet"},
VersionRegex: `(?i)(?:FortiGate|FGT|FGVM)[_-]?(\d+[A-Z]?)`,
},
// Palo Alto Networks
{
Vendor: "Palo Alto Networks",
Product: "PAN-OS",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"PA-", "Palo Alto", "PAN-OS", "paloaltonetworks",
},
SubjectOrgPatterns: []string{"Palo Alto Networks", "paloaltonetworks"},
IssuerOrgPatterns: []string{"Palo Alto Networks"},
VersionRegex: `(?i)PA-(\d+)`,
},
// Cisco ASA
{
Vendor: "Cisco",
Product: "ASA",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"ASA", "Cisco ASA", "adaptive security",
},
SubjectOrgPatterns: []string{"Cisco"},
IssuerOrgPatterns: []string{"Cisco"},
VersionRegex: `(?i)ASA[_-]?(\d+)`,
},
// Cisco Firepower
{
Vendor: "Cisco",
Product: "Firepower",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"Firepower", "FTD", "FMC",
},
SubjectOrgPatterns: []string{"Cisco"},
},
// SonicWall
{
Vendor: "SonicWall",
Product: "SonicWall",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"SonicWall", "sonicwall", "SonicOS", "NSA", "TZ",
},
SubjectOrgPatterns: []string{"SonicWall", "SonicWALL"},
IssuerOrgPatterns: []string{"SonicWall", "SonicWALL"},
VersionRegex: `(?i)(?:NSA|TZ)[\s-]?(\d+)`,
},
// Check Point
{
Vendor: "Check Point",
Product: "Gaia",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"Check Point", "checkpoint", "Gaia", "SmartCenter",
},
SubjectOrgPatterns: []string{"Check Point"},
IssuerOrgPatterns: []string{"Check Point"},
},
// F5 BIG-IP
{
Vendor: "F5",
Product: "BIG-IP",
ApplianceType: "loadbalancer",
SubjectCNPatterns: []string{
"BIG-IP", "BIGIP", "F5 Networks", "f5.com",
},
SubjectOrgPatterns: []string{"F5 Networks", "F5, Inc"},
IssuerOrgPatterns: []string{"F5 Networks", "F5, Inc"},
VersionRegex: `(?i)BIG-IP\s+(\d+\.\d+)`,
},
// Citrix NetScaler / ADC
{
Vendor: "Citrix",
Product: "NetScaler",
ApplianceType: "loadbalancer",
SubjectCNPatterns: []string{
"NetScaler", "Citrix ADC", "ns.citrix", "citrix.com",
},
SubjectOrgPatterns: []string{"Citrix"},
IssuerOrgPatterns: []string{"Citrix"},
},
// Juniper
{
Vendor: "Juniper",
Product: "Junos",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"Juniper", "JunOS", "SRX", "juniper.net",
},
SubjectOrgPatterns: []string{"Juniper Networks"},
IssuerOrgPatterns: []string{"Juniper Networks"},
VersionRegex: `(?i)SRX[_-]?(\d+)`,
},
// Barracuda
{
Vendor: "Barracuda",
Product: "Barracuda",
ApplianceType: "waf",
SubjectCNPatterns: []string{
"Barracuda", "barracuda", "cudatel",
},
SubjectOrgPatterns: []string{"Barracuda Networks"},
IssuerOrgPatterns: []string{"Barracuda Networks"},
},
// pfSense
{
Vendor: "Netgate",
Product: "pfSense",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"pfSense", "pfsense", "Netgate",
},
SubjectOrgPatterns: []string{"pfSense", "Netgate"},
},
// OPNsense
{
Vendor: "Deciso",
Product: "OPNsense",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"OPNsense", "opnsense",
},
},
// WatchGuard
{
Vendor: "WatchGuard",
Product: "Firebox",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"WatchGuard", "Firebox", "watchguard",
},
SubjectOrgPatterns: []string{"WatchGuard"},
},
// Sophos
{
Vendor: "Sophos",
Product: "XG Firewall",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"Sophos", "sophos", "XG Firewall", "Cyberoam",
},
SubjectOrgPatterns: []string{"Sophos"},
},
// Ubiquiti
{
Vendor: "Ubiquiti",
Product: "UniFi",
ApplianceType: "appliance",
SubjectCNPatterns: []string{
"Ubiquiti", "UniFi", "UBNT", "ubnt.com",
},
SubjectOrgPatterns: []string{"Ubiquiti"},
},
// MikroTik
{
Vendor: "MikroTik",
Product: "RouterOS",
ApplianceType: "router",
SubjectCNPatterns: []string{
"MikroTik", "mikrotik", "RouterOS",
},
SubjectOrgPatterns: []string{"MikroTik"},
},
// OpenVPN
{
Vendor: "OpenVPN",
Product: "OpenVPN AS",
ApplianceType: "vpn",
SubjectCNPatterns: []string{
"OpenVPN", "openvpn",
},
SubjectOrgPatterns: []string{"OpenVPN"},
},
// Pulse Secure / Ivanti
{
Vendor: "Pulse Secure",
Product: "Pulse Connect Secure",
ApplianceType: "vpn",
SubjectCNPatterns: []string{
"Pulse Secure", "pulse", "Ivanti",
},
SubjectOrgPatterns: []string{"Pulse Secure", "Ivanti"},
},
// GlobalProtect (Palo Alto VPN)
{
Vendor: "Palo Alto Networks",
Product: "GlobalProtect",
ApplianceType: "vpn",
SubjectCNPatterns: []string{
"GlobalProtect", "globalprotect",
},
},
// Cisco AnyConnect
{
Vendor: "Cisco",
Product: "AnyConnect",
ApplianceType: "vpn",
SubjectCNPatterns: []string{
"AnyConnect", "anyconnect",
},
SubjectOrgPatterns: []string{"Cisco"},
},
// VMware NSX / vSphere
{
Vendor: "VMware",
Product: "NSX",
ApplianceType: "appliance",
SubjectCNPatterns: []string{
"NSX", "vSphere", "VMware", "vcenter",
},
SubjectOrgPatterns: []string{"VMware"},
},
// Imperva / Incapsula
{
Vendor: "Imperva",
Product: "WAF",
ApplianceType: "waf",
SubjectCNPatterns: []string{
"Imperva", "Incapsula",
},
SubjectOrgPatterns: []string{"Imperva", "Incapsula"},
},
// HAProxy
{
Vendor: "HAProxy",
Product: "HAProxy",
ApplianceType: "loadbalancer",
SubjectCNPatterns: []string{
"HAProxy", "haproxy",
},
},
// NGINX Plus
{
Vendor: "NGINX",
Product: "NGINX Plus",
ApplianceType: "loadbalancer",
SubjectCNPatterns: []string{
"NGINX", "nginx.com",
},
SubjectOrgPatterns: []string{"NGINX", "F5 NGINX"},
},
// Kemp LoadMaster
{
Vendor: "Kemp",
Product: "LoadMaster",
ApplianceType: "loadbalancer",
SubjectCNPatterns: []string{
"Kemp", "LoadMaster",
},
SubjectOrgPatterns: []string{"Kemp Technologies"},
},
// Zyxel
{
Vendor: "Zyxel",
Product: "USG",
ApplianceType: "firewall",
SubjectCNPatterns: []string{
"Zyxel", "zyxel", "USG",
},
SubjectOrgPatterns: []string{"Zyxel"},
},
// DrayTek
{
Vendor: "DrayTek",
Product: "Vigor",
ApplianceType: "router",
SubjectCNPatterns: []string{
"DrayTek", "Vigor", "draytek",
},
SubjectOrgPatterns: []string{"DrayTek"},
},
}
// AnalyzeTLSCertificate analyzes a TLS certificate for appliance fingerprinting
func AnalyzeTLSCertificate(cert *x509.Certificate) *config.TLSFingerprint {
if cert == nil {
return nil
}
fp := &config.TLSFingerprint{
SubjectCN: cert.Subject.CommonName,
SerialNumber: cert.SerialNumber.String(),
}
// Extract organization info
if len(cert.Subject.Organization) > 0 {
fp.SubjectOrg = strings.Join(cert.Subject.Organization, ", ")
}
if len(cert.Subject.OrganizationalUnit) > 0 {
fp.SubjectOU = strings.Join(cert.Subject.OrganizationalUnit, ", ")
}
if len(cert.Issuer.CommonName) > 0 {
fp.IssuerCN = cert.Issuer.CommonName
}
if len(cert.Issuer.Organization) > 0 {
fp.IssuerOrg = strings.Join(cert.Issuer.Organization, ", ")
}
// Extract internal hostnames from DNS names
for _, name := range cert.DNSNames {
if isInternalHostname(name) {
fp.InternalHosts = append(fp.InternalHosts, name)
}
}
// Try to match against known appliance patterns
matchAppliance(fp, cert)
// Only return if we found something interesting
if fp.Vendor != "" || len(fp.InternalHosts) > 0 || fp.SubjectOrg != "" {
return fp
}
return nil
}
// matchAppliance tries to identify the appliance vendor/product
func matchAppliance(fp *config.TLSFingerprint, cert *x509.Certificate) {
subjectCN := strings.ToLower(cert.Subject.CommonName)
subjectOrg := strings.ToLower(strings.Join(cert.Subject.Organization, " "))
subjectOU := strings.ToLower(strings.Join(cert.Subject.OrganizationalUnit, " "))
issuerCN := strings.ToLower(cert.Issuer.CommonName)
issuerOrg := strings.ToLower(strings.Join(cert.Issuer.Organization, " "))
for _, pattern := range appliancePatterns {
matched := false
// Check Subject CN
for _, p := range pattern.SubjectCNPatterns {
if strings.Contains(subjectCN, strings.ToLower(p)) {
matched = true
break
}
}
// Check Subject Organization
if !matched {
for _, p := range pattern.SubjectOrgPatterns {
if strings.Contains(subjectOrg, strings.ToLower(p)) {
matched = true
break
}
}
}
// Check Subject OU
if !matched {
for _, p := range pattern.SubjectOUPatterns {
if strings.Contains(subjectOU, strings.ToLower(p)) {
matched = true
break
}
}
}
// Check Issuer CN
if !matched {
for _, p := range pattern.IssuerCNPatterns {
if strings.Contains(issuerCN, strings.ToLower(p)) {
matched = true
break
}
}
}
// Check Issuer Organization
if !matched {
for _, p := range pattern.IssuerOrgPatterns {
if strings.Contains(issuerOrg, strings.ToLower(p)) {
matched = true
break
}
}
}
if matched {
fp.Vendor = pattern.Vendor
fp.Product = pattern.Product
fp.ApplianceType = pattern.ApplianceType
// Try to extract version
if pattern.VersionRegex != "" {
re := regexp.MustCompile(pattern.VersionRegex)
// Check all relevant fields
for _, field := range []string{cert.Subject.CommonName, cert.Issuer.CommonName} {
if matches := re.FindStringSubmatch(field); len(matches) > 1 {
fp.Version = matches[1]
break
}
}
}
return
}
}
}
// isInternalHostname checks if a hostname looks like an internal name
func isInternalHostname(name string) bool {
name = strings.ToLower(name)
// Common internal TLDs
internalTLDs := []string{
".local", ".internal", ".lan", ".corp", ".home",
".intranet", ".private", ".localdomain",
}
for _, tld := range internalTLDs {
if strings.HasSuffix(name, tld) {
return true
}
}
// Internal hostname patterns
internalPatterns := []string{
"localhost", "fw-", "firewall", "vpn-", "gw-", "gateway",
"proxy-", "lb-", "router", "switch", "core-", "dc-",
"srv-", "server-", "host-", "node-", "mgmt", "management",
"admin-", "internal-", "private-", "corp-", "office-",
}
for _, pattern := range internalPatterns {
if strings.Contains(name, pattern) {
return true
}
}
// IP-like patterns in hostname
ipPattern := regexp.MustCompile(`\d{1,3}[.-]\d{1,3}[.-]\d{1,3}[.-]\d{1,3}`)
if ipPattern.MatchString(name) {
return true
}
return false
}
// IsSelfSigned checks if a certificate is self-signed
func IsSelfSigned(cert *x509.Certificate) bool {
if cert == nil {
return false
}
// Check if issuer equals subject
if cert.Issuer.String() == cert.Subject.String() {
return true
}
// Check if it's self-signed by verifying against its own public key
err := cert.CheckSignatureFrom(cert)
return err == nil
}

338
internal/output/json.go Normal file
View File

@@ -0,0 +1,338 @@
package output
import (
"encoding/json"
"io"
"sort"
"time"
"god-eye/internal/config"
)
// ScanReport represents the complete JSON output structure
type ScanReport struct {
// Metadata
Meta ScanMeta `json:"meta"`
// Statistics
Stats ScanStats `json:"stats"`
// Results
Subdomains []*config.SubdomainResult `json:"subdomains"`
// Wildcard info (if detected)
Wildcard *WildcardReport `json:"wildcard,omitempty"`
// Findings summary
Findings FindingsSummary `json:"findings"`
}
// ScanMeta contains metadata about the scan
type ScanMeta struct {
Version string `json:"version"`
ToolName string `json:"tool_name"`
Target string `json:"target"`
StartTime time.Time `json:"start_time"`
EndTime time.Time `json:"end_time"`
Duration string `json:"duration"`
DurationMs int64 `json:"duration_ms"`
Concurrency int `json:"concurrency"`
Timeout int `json:"timeout"`
Options ScanOptions `json:"options"`
}
// ScanOptions contains the scan configuration
type ScanOptions struct {
BruteForce bool `json:"brute_force"`
HTTPProbe bool `json:"http_probe"`
PortScan bool `json:"port_scan"`
TakeoverCheck bool `json:"takeover_check"`
AIAnalysis bool `json:"ai_analysis"`
OnlyActive bool `json:"only_active"`
CustomWordlist bool `json:"custom_wordlist"`
CustomResolvers bool `json:"custom_resolvers"`
CustomPorts string `json:"custom_ports,omitempty"`
}
// ScanStats contains scan statistics
type ScanStats struct {
TotalSubdomains int `json:"total_subdomains"`
ActiveSubdomains int `json:"active_subdomains"`
InactiveSubdomains int `json:"inactive_subdomains"`
WithIPs int `json:"with_ips"`
WithHTTP int `json:"with_http"`
WithHTTPS int `json:"with_https"`
WithPorts int `json:"with_ports"`
TakeoverVulnerable int `json:"takeover_vulnerable"`
Vulnerabilities int `json:"vulnerabilities"`
CloudHosted int `json:"cloud_hosted"`
AIFindings int `json:"ai_findings"`
CVEFindings int `json:"cve_findings"`
PassiveSources int `json:"passive_sources"`
BruteForceFound int `json:"brute_force_found"`
}
// WildcardReport contains wildcard detection info
type WildcardReport struct {
Detected bool `json:"detected"`
IPs []string `json:"ips,omitempty"`
CNAME string `json:"cname,omitempty"`
StatusCode int `json:"status_code,omitempty"`
Confidence float64 `json:"confidence,omitempty"`
}
// FindingsSummary categorizes findings by severity
type FindingsSummary struct {
Critical []Finding `json:"critical,omitempty"`
High []Finding `json:"high,omitempty"`
Medium []Finding `json:"medium,omitempty"`
Low []Finding `json:"low,omitempty"`
Info []Finding `json:"info,omitempty"`
}
// Finding represents a single finding
type Finding struct {
Subdomain string `json:"subdomain"`
Type string `json:"type"`
Description string `json:"description"`
Evidence string `json:"evidence,omitempty"`
}
// ReportBuilder helps construct the JSON report
type ReportBuilder struct {
report *ScanReport
startTime time.Time
}
// NewReportBuilder creates a new report builder
func NewReportBuilder(domain string, cfg config.Config) *ReportBuilder {
now := time.Now()
return &ReportBuilder{
startTime: now,
report: &ScanReport{
Meta: ScanMeta{
Version: "0.1",
ToolName: "God's Eye",
Target: domain,
StartTime: now,
Concurrency: cfg.Concurrency,
Timeout: cfg.Timeout,
Options: ScanOptions{
BruteForce: !cfg.NoBrute,
HTTPProbe: !cfg.NoProbe,
PortScan: !cfg.NoPorts,
TakeoverCheck: !cfg.NoTakeover,
AIAnalysis: cfg.EnableAI,
OnlyActive: cfg.OnlyActive,
CustomWordlist: cfg.Wordlist != "",
CustomResolvers: cfg.Resolvers != "",
CustomPorts: cfg.Ports,
},
},
Stats: ScanStats{},
Findings: FindingsSummary{
Critical: []Finding{},
High: []Finding{},
Medium: []Finding{},
Low: []Finding{},
Info: []Finding{},
},
},
}
}
// SetWildcard sets wildcard detection info
func (rb *ReportBuilder) SetWildcard(detected bool, ips []string, cname string, statusCode int, confidence float64) {
rb.report.Wildcard = &WildcardReport{
Detected: detected,
IPs: ips,
CNAME: cname,
StatusCode: statusCode,
Confidence: confidence,
}
}
// SetPassiveSources sets the number of passive sources used
func (rb *ReportBuilder) SetPassiveSources(count int) {
rb.report.Stats.PassiveSources = count
}
// SetBruteForceFound sets the number of subdomains found via brute force
func (rb *ReportBuilder) SetBruteForceFound(count int) {
rb.report.Stats.BruteForceFound = count
}
// Finalize completes the report with results and calculates stats
func (rb *ReportBuilder) Finalize(results map[string]*config.SubdomainResult) *ScanReport {
endTime := time.Now()
duration := endTime.Sub(rb.startTime)
rb.report.Meta.EndTime = endTime
rb.report.Meta.Duration = duration.String()
rb.report.Meta.DurationMs = duration.Milliseconds()
// Sort subdomains
var sortedSubs []string
for sub := range results {
sortedSubs = append(sortedSubs, sub)
}
sort.Strings(sortedSubs)
// Build results list and calculate stats
rb.report.Subdomains = make([]*config.SubdomainResult, 0, len(results))
for _, sub := range sortedSubs {
r := results[sub]
rb.report.Subdomains = append(rb.report.Subdomains, r)
// Calculate stats
rb.report.Stats.TotalSubdomains++
if len(r.IPs) > 0 {
rb.report.Stats.WithIPs++
}
if r.StatusCode >= 200 && r.StatusCode < 400 {
rb.report.Stats.ActiveSubdomains++
} else if r.StatusCode >= 400 {
rb.report.Stats.InactiveSubdomains++
}
if r.TLSVersion != "" {
rb.report.Stats.WithHTTPS++
}
if r.StatusCode > 0 {
rb.report.Stats.WithHTTP++
}
if len(r.Ports) > 0 {
rb.report.Stats.WithPorts++
}
if r.CloudProvider != "" {
rb.report.Stats.CloudHosted++
}
if r.Takeover != "" {
rb.report.Stats.TakeoverVulnerable++
rb.addFinding("critical", sub, "Subdomain Takeover", r.Takeover)
}
// Count vulnerabilities
vulnCount := 0
if r.OpenRedirect {
vulnCount++
rb.addFinding("high", sub, "Open Redirect", "Vulnerable to open redirect attacks")
}
if r.CORSMisconfig != "" {
vulnCount++
rb.addFinding("medium", sub, "CORS Misconfiguration", r.CORSMisconfig)
}
if len(r.DangerousMethods) > 0 {
vulnCount++
rb.addFinding("medium", sub, "Dangerous HTTP Methods", join(r.DangerousMethods, ", "))
}
if r.GitExposed {
vulnCount++
rb.addFinding("high", sub, "Git Repository Exposed", ".git directory accessible")
}
if r.SvnExposed {
vulnCount++
rb.addFinding("high", sub, "SVN Repository Exposed", ".svn directory accessible")
}
if len(r.BackupFiles) > 0 {
vulnCount++
rb.addFinding("high", sub, "Backup Files Exposed", join(r.BackupFiles, ", "))
}
if len(r.JSSecrets) > 0 {
vulnCount++
for _, secret := range r.JSSecrets {
rb.addFinding("high", sub, "Secret in JavaScript", secret)
}
}
if vulnCount > 0 {
rb.report.Stats.Vulnerabilities += vulnCount
}
// AI findings
if len(r.AIFindings) > 0 {
rb.report.Stats.AIFindings += len(r.AIFindings)
severity := "info"
if r.AISeverity != "" {
severity = r.AISeverity
}
for _, finding := range r.AIFindings {
rb.addFinding(severity, sub, "AI Analysis", finding)
}
}
// CVE findings
if len(r.CVEFindings) > 0 {
rb.report.Stats.CVEFindings += len(r.CVEFindings)
for _, cve := range r.CVEFindings {
rb.addFinding("high", sub, "CVE Vulnerability", cve)
}
}
// Info findings
if len(r.AdminPanels) > 0 {
for _, panel := range r.AdminPanels {
rb.addFinding("info", sub, "Admin Panel Found", panel)
}
}
if len(r.APIEndpoints) > 0 {
for _, endpoint := range r.APIEndpoints {
rb.addFinding("info", sub, "API Endpoint Found", endpoint)
}
}
}
return rb.report
}
// addFinding adds a finding to the appropriate severity category
func (rb *ReportBuilder) addFinding(severity, subdomain, findingType, description string) {
finding := Finding{
Subdomain: subdomain,
Type: findingType,
Description: description,
}
switch severity {
case "critical":
rb.report.Findings.Critical = append(rb.report.Findings.Critical, finding)
case "high":
rb.report.Findings.High = append(rb.report.Findings.High, finding)
case "medium":
rb.report.Findings.Medium = append(rb.report.Findings.Medium, finding)
case "low":
rb.report.Findings.Low = append(rb.report.Findings.Low, finding)
default:
rb.report.Findings.Info = append(rb.report.Findings.Info, finding)
}
}
// WriteJSON writes the report as JSON to a writer
func (rb *ReportBuilder) WriteJSON(w io.Writer, indent bool) error {
encoder := json.NewEncoder(w)
if indent {
encoder.SetIndent("", " ")
}
return encoder.Encode(rb.report)
}
// GetReport returns the built report
func (rb *ReportBuilder) GetReport() *ScanReport {
return rb.report
}
// Helper function
func join(strs []string, sep string) string {
if len(strs) == 0 {
return ""
}
result := strs[0]
for _, s := range strs[1:] {
result += sep + s
}
return result
}

View File

@@ -43,31 +43,33 @@ var (
func PrintBanner() { func PrintBanner() {
fmt.Println() fmt.Println()
fmt.Println(BoldCyan(" ██████╗ ██████╗ ██████╗ ") + BoldWhite("███████╗") + BoldCyan(" ███████╗██╗ ██╗███████╗")) fmt.Println(BoldWhite(" ██████╗ ██████╗ ██████╗ ") + BoldGreen("███████╗") + BoldWhite(" ███████╗██╗ ██╗███████╗"))
fmt.Println(BoldCyan(" ██╔════╝ ██╔═══██╗██╔══██╗") + BoldWhite("██╔════╝") + BoldCyan(" ██╔════╝╚██╗ ██╔╝██╔════╝")) fmt.Println(BoldWhite(" ██╔════╝ ██╔═══██╗██╔══██╗") + BoldGreen("██╔════╝") + BoldWhite(" ██╔════╝╚██╗ ██╔╝██╔════╝"))
fmt.Println(BoldCyan(" ██║ ███╗██║ ██║██║ ██║") + BoldWhite("███████╗") + BoldCyan(" █████╗ ╚████╔╝ █████╗ ")) fmt.Println(BoldWhite(" ██║ ███╗██║ ██║██║ ██║") + BoldGreen("███████╗") + BoldWhite(" █████╗ ╚████╔╝ █████╗ "))
fmt.Println(BoldCyan(" ██║ ██║██║ ██║██║ ██║") + BoldWhite("╚════██║") + BoldCyan(" ██╔══╝ ╚██╔╝ ██╔══╝ ")) fmt.Println(BoldWhite(" ██║ ██║██║ ██║██║ ██║") + BoldGreen("╚════██║") + BoldWhite(" ██╔══╝ ╚██╔╝ ██╔══╝ "))
fmt.Println(BoldCyan(" ╚██████╔╝╚██████╔╝██████╔╝") + BoldWhite("███████║") + BoldCyan(" ███████╗ ██║ ███████╗")) fmt.Println(BoldWhite(" ╚██████╔╝╚██████╔╝██████╔╝") + BoldGreen("███████║") + BoldWhite(" ███████╗ ██║ ███████╗"))
fmt.Println(BoldCyan(" ╚═════╝ ╚═════╝ ╚═════╝ ") + BoldWhite("╚══════╝") + BoldCyan(" ╚══════╝ ╚═╝ ╚══════╝")) fmt.Println(BoldWhite(" ╚═════╝ ╚═════╝ ╚═════╝ ") + BoldGreen("╚══════╝") + BoldWhite(" ╚══════╝ ╚═╝ ╚══════╝"))
fmt.Println() fmt.Println()
fmt.Printf(" %s %s\n", BoldWhite("⚡"), Dim("Ultra-fast subdomain enumeration & reconnaissance")) fmt.Printf(" %s %s\n", BoldGreen("⚡"), Dim("Ultra-fast subdomain enumeration & reconnaissance"))
fmt.Printf(" %s %s %s %s %s %s\n", fmt.Printf(" %s %s %s %s %s %s\n",
Dim("Version:"), BoldGreen("0.1"), Dim("Version:"), BoldGreen("0.1"),
Dim("By:"), Cyan("github.com/Vyntral"), Dim("By:"), White("github.com/Vyntral"),
Dim("For:"), Yellow("github.com/Orizon-eu")) Dim("For:"), Yellow("github.com/Orizon-eu"))
fmt.Println() fmt.Println()
} }
func PrintSection(icon, title string) { func PrintSection(icon, title string) {
fmt.Printf("\n%s %s %s\n", BoldCyan("┌──"), BoldWhite(icon+" "+title), BoldCyan(strings.Repeat("─", 50))) fmt.Println()
fmt.Printf(" %s %s\n", icon, BoldWhite(title))
fmt.Printf(" %s\n", Dim(strings.Repeat("─", 50)))
} }
func PrintSubSection(text string) { func PrintSubSection(text string) {
fmt.Printf("%s %s\n", Cyan("│"), text) fmt.Printf(" %s\n", text)
} }
func PrintEndSection() { func PrintEndSection() {
fmt.Printf("%s\n", BoldCyan("└"+strings.Repeat("─", 60))) // No more lines, just spacing
} }
func PrintProgress(current, total int, label string) { func PrintProgress(current, total int, label string) {
@@ -78,7 +80,7 @@ func PrintProgress(current, total int, label string) {
} }
bar := strings.Repeat("█", filled) + strings.Repeat("░", width-filled) bar := strings.Repeat("█", filled) + strings.Repeat("░", width-filled)
percent := float64(current) / float64(total) * 100 percent := float64(current) / float64(total) * 100
fmt.Printf("\r%s %s %s %s %.0f%% ", Cyan("│"), label, BoldGreen(bar), Dim(fmt.Sprintf("(%d/%d)", current, total)), percent) fmt.Printf("\r %s %s %s %.0f%% ", label, Green(bar), Dim(fmt.Sprintf("(%d/%d)", current, total)), percent)
} }
func ClearLine() { func ClearLine() {
@@ -141,5 +143,5 @@ func SaveOutput(path string, format string, results map[string]*config.Subdomain
} }
} }
fmt.Printf("%s Results saved to %s\n", Green("[+]"), path) fmt.Printf("\n %s Results saved to %s\n", Green(""), path)
} }

View File

@@ -0,0 +1,197 @@
package progress
import (
"fmt"
"strings"
"sync"
"sync/atomic"
"time"
"god-eye/internal/output"
)
// Bar represents a progress bar
type Bar struct {
total int64
current int64
width int
prefix string
startTime time.Time
lastUpdate time.Time
mu sync.Mutex
done bool
silent bool
}
// New creates a new progress bar
func New(total int, prefix string, silent bool) *Bar {
return &Bar{
total: int64(total),
current: 0,
width: 40,
prefix: prefix,
startTime: time.Now(),
silent: silent,
}
}
// Increment increases the progress by 1
func (b *Bar) Increment() {
atomic.AddInt64(&b.current, 1)
b.render()
}
// Add increases the progress by n
func (b *Bar) Add(n int) {
atomic.AddInt64(&b.current, int64(n))
b.render()
}
// SetCurrent sets the current progress value
func (b *Bar) SetCurrent(n int) {
atomic.StoreInt64(&b.current, int64(n))
b.render()
}
// render displays the progress bar
func (b *Bar) render() {
if b.silent {
return
}
b.mu.Lock()
defer b.mu.Unlock()
// Throttle updates to avoid flickering (max 10 updates/sec)
if time.Since(b.lastUpdate) < 100*time.Millisecond && !b.done {
return
}
b.lastUpdate = time.Now()
current := atomic.LoadInt64(&b.current)
total := b.total
// Calculate percentage
var percent float64
if total > 0 {
percent = float64(current) / float64(total) * 100
}
// Calculate filled width
filled := int(float64(b.width) * percent / 100)
if filled > b.width {
filled = b.width
}
// Build progress bar
bar := strings.Repeat("█", filled) + strings.Repeat("░", b.width-filled)
// Calculate ETA
elapsed := time.Since(b.startTime)
var eta string
if current > 0 && current < total {
remaining := time.Duration(float64(elapsed) / float64(current) * float64(total-current))
eta = formatDuration(remaining)
} else if current >= total {
eta = "done"
} else {
eta = "..."
}
// Calculate speed
var speed float64
if elapsed.Seconds() > 0 {
speed = float64(current) / elapsed.Seconds()
}
// Print progress bar (overwrite line with \r) - clean style without box characters
fmt.Printf("\r %s [%s] %s/%s %.0f%% %s ETA %s ",
b.prefix,
output.Green(bar),
output.BoldWhite(fmt.Sprintf("%d", current)),
output.Dim(fmt.Sprintf("%d", total)),
percent,
output.Dim(fmt.Sprintf("%.0f/s", speed)),
output.Dim(eta),
)
}
// Finish completes the progress bar
func (b *Bar) Finish() {
if b.silent {
return
}
b.mu.Lock()
b.done = true
b.mu.Unlock()
current := atomic.LoadInt64(&b.current)
elapsed := time.Since(b.startTime)
// Clear the line and print final status - clean style
fmt.Printf("\r %s %s %s completed in %s \n",
output.Green("✓"),
output.BoldWhite(fmt.Sprintf("%d", current)),
b.prefix,
output.Green(formatDuration(elapsed)),
)
}
// FinishWithMessage completes with a custom message
func (b *Bar) FinishWithMessage(msg string) {
if b.silent {
return
}
b.mu.Lock()
b.done = true
b.mu.Unlock()
// Clear the line and print message - clean style
fmt.Printf("\r %s %s \n",
output.Green("✓"),
msg,
)
}
// formatDuration formats a duration nicely
func formatDuration(d time.Duration) string {
if d < time.Second {
return "<1s"
} else if d < time.Minute {
return fmt.Sprintf("%ds", int(d.Seconds()))
} else if d < time.Hour {
mins := int(d.Minutes())
secs := int(d.Seconds()) % 60
return fmt.Sprintf("%dm%ds", mins, secs)
}
hours := int(d.Hours())
mins := int(d.Minutes()) % 60
return fmt.Sprintf("%dh%dm", hours, mins)
}
// MultiBar manages multiple progress bars
type MultiBar struct {
bars []*Bar
mu sync.Mutex
silent bool
}
// NewMulti creates a new multi-bar manager
func NewMulti(silent bool) *MultiBar {
return &MultiBar{
bars: make([]*Bar, 0),
silent: silent,
}
}
// AddBar adds a new progress bar
func (m *MultiBar) AddBar(total int, prefix string) *Bar {
m.mu.Lock()
defer m.mu.Unlock()
bar := New(total, prefix, m.silent)
m.bars = append(m.bars, bar)
return bar
}

View File

@@ -0,0 +1,284 @@
package ratelimit
import (
"sync"
"sync/atomic"
"time"
)
// AdaptiveRateLimiter implements intelligent rate limiting that adapts based on errors
type AdaptiveRateLimiter struct {
// Configuration
minDelay time.Duration
maxDelay time.Duration
currentDelay time.Duration
// Error tracking
consecutiveErrors int64
totalErrors int64
totalRequests int64
// Backoff settings
backoffMultiplier float64
recoveryRate float64
// State
lastRequest time.Time
mu sync.Mutex
}
// Config holds configuration for the rate limiter
type Config struct {
MinDelay time.Duration // Minimum delay between requests
MaxDelay time.Duration // Maximum delay (during backoff)
BackoffMultiplier float64 // How much to increase delay on error (default 2.0)
RecoveryRate float64 // How much to decrease delay on success (default 0.9)
}
// DefaultConfig returns sensible defaults
func DefaultConfig() Config {
return Config{
MinDelay: 50 * time.Millisecond,
MaxDelay: 5 * time.Second,
BackoffMultiplier: 2.0,
RecoveryRate: 0.9,
}
}
// AggressiveConfig returns config for fast scanning
func AggressiveConfig() Config {
return Config{
MinDelay: 10 * time.Millisecond,
MaxDelay: 2 * time.Second,
BackoffMultiplier: 1.5,
RecoveryRate: 0.8,
}
}
// ConservativeConfig returns config for careful scanning
func ConservativeConfig() Config {
return Config{
MinDelay: 200 * time.Millisecond,
MaxDelay: 10 * time.Second,
BackoffMultiplier: 3.0,
RecoveryRate: 0.95,
}
}
// New creates a new adaptive rate limiter
func New(cfg Config) *AdaptiveRateLimiter {
if cfg.BackoffMultiplier == 0 {
cfg.BackoffMultiplier = 2.0
}
if cfg.RecoveryRate == 0 {
cfg.RecoveryRate = 0.9
}
return &AdaptiveRateLimiter{
minDelay: cfg.MinDelay,
maxDelay: cfg.MaxDelay,
currentDelay: cfg.MinDelay,
backoffMultiplier: cfg.BackoffMultiplier,
recoveryRate: cfg.RecoveryRate,
}
}
// Wait blocks until it's safe to make another request
func (r *AdaptiveRateLimiter) Wait() {
r.mu.Lock()
defer r.mu.Unlock()
elapsed := time.Since(r.lastRequest)
if elapsed < r.currentDelay {
time.Sleep(r.currentDelay - elapsed)
}
r.lastRequest = time.Now()
atomic.AddInt64(&r.totalRequests, 1)
}
// Success reports a successful request
func (r *AdaptiveRateLimiter) Success() {
r.mu.Lock()
defer r.mu.Unlock()
// Reset consecutive errors
atomic.StoreInt64(&r.consecutiveErrors, 0)
// Gradually reduce delay (recover)
newDelay := time.Duration(float64(r.currentDelay) * r.recoveryRate)
if newDelay < r.minDelay {
newDelay = r.minDelay
}
r.currentDelay = newDelay
}
// Error reports a failed request (timeout, 429, etc)
func (r *AdaptiveRateLimiter) Error(isRateLimited bool) {
r.mu.Lock()
defer r.mu.Unlock()
atomic.AddInt64(&r.consecutiveErrors, 1)
atomic.AddInt64(&r.totalErrors, 1)
// Increase delay on error
multiplier := r.backoffMultiplier
if isRateLimited {
// More aggressive backoff for rate limit errors (429)
multiplier *= 2
}
newDelay := time.Duration(float64(r.currentDelay) * multiplier)
if newDelay > r.maxDelay {
newDelay = r.maxDelay
}
r.currentDelay = newDelay
}
// GetCurrentDelay returns the current delay
func (r *AdaptiveRateLimiter) GetCurrentDelay() time.Duration {
r.mu.Lock()
defer r.mu.Unlock()
return r.currentDelay
}
// GetStats returns error statistics
func (r *AdaptiveRateLimiter) GetStats() (total int64, errors int64, currentDelay time.Duration) {
return atomic.LoadInt64(&r.totalRequests),
atomic.LoadInt64(&r.totalErrors),
r.GetCurrentDelay()
}
// ShouldBackoff returns true if we're experiencing too many errors
func (r *AdaptiveRateLimiter) ShouldBackoff() bool {
return atomic.LoadInt64(&r.consecutiveErrors) > 5
}
// HostRateLimiter manages rate limits per host
type HostRateLimiter struct {
limiters map[string]*AdaptiveRateLimiter
config Config
mu sync.RWMutex
}
// NewHostRateLimiter creates a per-host rate limiter
func NewHostRateLimiter(cfg Config) *HostRateLimiter {
return &HostRateLimiter{
limiters: make(map[string]*AdaptiveRateLimiter),
config: cfg,
}
}
// Get returns or creates a rate limiter for a host
func (h *HostRateLimiter) Get(host string) *AdaptiveRateLimiter {
h.mu.RLock()
limiter, exists := h.limiters[host]
h.mu.RUnlock()
if exists {
return limiter
}
h.mu.Lock()
defer h.mu.Unlock()
// Double check after acquiring write lock
if limiter, exists = h.limiters[host]; exists {
return limiter
}
limiter = New(h.config)
h.limiters[host] = limiter
return limiter
}
// GetStats returns aggregated stats for all hosts
func (h *HostRateLimiter) GetStats() (hosts int, totalRequests, totalErrors int64) {
h.mu.RLock()
defer h.mu.RUnlock()
hosts = len(h.limiters)
for _, limiter := range h.limiters {
requests, errors, _ := limiter.GetStats()
totalRequests += requests
totalErrors += errors
}
return
}
// ConcurrencyController manages dynamic concurrency based on errors
type ConcurrencyController struct {
maxConcurrency int64
minConcurrency int64
current int64
errorCount int64
successCount int64
checkInterval int64
mu sync.Mutex
}
// NewConcurrencyController creates a new concurrency controller
func NewConcurrencyController(max, min int) *ConcurrencyController {
return &ConcurrencyController{
maxConcurrency: int64(max),
minConcurrency: int64(min),
current: int64(max),
checkInterval: 100, // Check every 100 requests
}
}
// GetCurrent returns current concurrency level
func (c *ConcurrencyController) GetCurrent() int {
return int(atomic.LoadInt64(&c.current))
}
// ReportSuccess reports a successful request
func (c *ConcurrencyController) ReportSuccess() {
atomic.AddInt64(&c.successCount, 1)
c.maybeAdjust()
}
// ReportError reports an error
func (c *ConcurrencyController) ReportError() {
atomic.AddInt64(&c.errorCount, 1)
c.maybeAdjust()
}
// maybeAdjust checks if we should adjust concurrency
func (c *ConcurrencyController) maybeAdjust() {
total := atomic.LoadInt64(&c.successCount) + atomic.LoadInt64(&c.errorCount)
if total%c.checkInterval != 0 {
return
}
c.mu.Lock()
defer c.mu.Unlock()
errors := atomic.LoadInt64(&c.errorCount)
successes := atomic.LoadInt64(&c.successCount)
if successes == 0 {
return
}
errorRate := float64(errors) / float64(total)
if errorRate > 0.1 { // More than 10% errors
// Reduce concurrency
newConcurrency := int64(float64(c.current) * 0.8)
if newConcurrency < c.minConcurrency {
newConcurrency = c.minConcurrency
}
atomic.StoreInt64(&c.current, newConcurrency)
} else if errorRate < 0.02 { // Less than 2% errors
// Increase concurrency
newConcurrency := int64(float64(c.current) * 1.1)
if newConcurrency > c.maxConcurrency {
newConcurrency = c.maxConcurrency
}
atomic.StoreInt64(&c.current, newConcurrency)
}
// Reset counters
atomic.StoreInt64(&c.errorCount, 0)
atomic.StoreInt64(&c.successCount, 0)
}

236
internal/retry/retry.go Normal file
View File

@@ -0,0 +1,236 @@
package retry
import (
"context"
"errors"
"math"
"math/rand"
"time"
)
// Config holds retry configuration
type Config struct {
MaxRetries int // Maximum number of retry attempts
InitialDelay time.Duration // Initial delay before first retry
MaxDelay time.Duration // Maximum delay between retries
Multiplier float64 // Delay multiplier for exponential backoff
Jitter float64 // Random jitter factor (0-1)
RetryableErrors []error // Specific errors to retry on (nil = retry all)
}
// DefaultConfig returns sensible defaults for network operations
func DefaultConfig() Config {
return Config{
MaxRetries: 3,
InitialDelay: 100 * time.Millisecond,
MaxDelay: 5 * time.Second,
Multiplier: 2.0,
Jitter: 0.1,
}
}
// DNSConfig returns config optimized for DNS queries
func DNSConfig() Config {
return Config{
MaxRetries: 3,
InitialDelay: 50 * time.Millisecond,
MaxDelay: 2 * time.Second,
Multiplier: 2.0,
Jitter: 0.2,
}
}
// HTTPConfig returns config optimized for HTTP requests
func HTTPConfig() Config {
return Config{
MaxRetries: 2,
InitialDelay: 200 * time.Millisecond,
MaxDelay: 3 * time.Second,
Multiplier: 2.0,
Jitter: 0.15,
}
}
// AggressiveConfig returns config for fast scanning with fewer retries
func AggressiveConfig() Config {
return Config{
MaxRetries: 1,
InitialDelay: 50 * time.Millisecond,
MaxDelay: 1 * time.Second,
Multiplier: 1.5,
Jitter: 0.1,
}
}
// Result wraps the result of a retryable operation
type Result struct {
Value interface{}
Error error
Attempts int
}
// Do executes a function with retry logic
func Do(ctx context.Context, cfg Config, fn func() (interface{}, error)) Result {
var lastErr error
attempts := 0
for attempts <= cfg.MaxRetries {
attempts++
// Check context cancellation
select {
case <-ctx.Done():
return Result{Error: ctx.Err(), Attempts: attempts}
default:
}
// Execute the function
result, err := fn()
if err == nil {
return Result{Value: result, Attempts: attempts}
}
lastErr = err
// Check if error is retryable
if !isRetryable(err, cfg.RetryableErrors) {
return Result{Error: err, Attempts: attempts}
}
// If this was the last attempt, don't sleep
if attempts > cfg.MaxRetries {
break
}
// Calculate delay with exponential backoff and jitter
delay := calculateDelay(attempts, cfg)
// Wait before retrying
select {
case <-ctx.Done():
return Result{Error: ctx.Err(), Attempts: attempts}
case <-time.After(delay):
}
}
return Result{Error: lastErr, Attempts: attempts}
}
// DoSimple executes a function with default config and no context
func DoSimple(fn func() (interface{}, error)) Result {
return Do(context.Background(), DefaultConfig(), fn)
}
// DoWithTimeout executes with a timeout
func DoWithTimeout(timeout time.Duration, cfg Config, fn func() (interface{}, error)) Result {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
return Do(ctx, cfg, fn)
}
// calculateDelay computes the delay for a given attempt
func calculateDelay(attempt int, cfg Config) time.Duration {
// Exponential backoff: initialDelay * multiplier^(attempt-1)
delay := float64(cfg.InitialDelay) * math.Pow(cfg.Multiplier, float64(attempt-1))
// Apply max cap
if delay > float64(cfg.MaxDelay) {
delay = float64(cfg.MaxDelay)
}
// Apply jitter: delay * (1 +/- jitter)
if cfg.Jitter > 0 {
jitter := delay * cfg.Jitter * (2*rand.Float64() - 1)
delay += jitter
}
return time.Duration(delay)
}
// isRetryable checks if an error should be retried
func isRetryable(err error, retryableErrors []error) bool {
if err == nil {
return false
}
// If no specific errors defined, retry all
if len(retryableErrors) == 0 {
return true
}
// Check if error matches any retryable error
for _, retryableErr := range retryableErrors {
if errors.Is(err, retryableErr) {
return true
}
}
return false
}
// Common retryable error types
var (
ErrTimeout = errors.New("operation timeout")
ErrTemporary = errors.New("temporary error")
ErrConnectionReset = errors.New("connection reset")
ErrDNSLookup = errors.New("dns lookup failed")
)
// IsTemporaryError checks if an error is temporary/transient
func IsTemporaryError(err error) bool {
if err == nil {
return false
}
// Check for common temporary error strings
errStr := err.Error()
temporaryPatterns := []string{
"timeout",
"temporary",
"connection reset",
"connection refused",
"no such host",
"i/o timeout",
"TLS handshake timeout",
"context deadline exceeded",
"server misbehaving",
"too many open files",
}
for _, pattern := range temporaryPatterns {
if containsIgnoreCase(errStr, pattern) {
return true
}
}
return false
}
func containsIgnoreCase(s, substr string) bool {
for i := 0; i+len(substr) <= len(s); i++ {
if equalFold(s[i:i+len(substr)], substr) {
return true
}
}
return false
}
func equalFold(s, t string) bool {
if len(s) != len(t) {
return false
}
for i := 0; i < len(s); i++ {
sr := s[i]
tr := t[i]
if sr >= 'A' && sr <= 'Z' {
sr += 'a' - 'A'
}
if tr >= 'A' && tr <= 'Z' {
tr += 'a' - 'A'
}
if sr != tr {
return false
}
}
return true
}

177
internal/scanner/helpers.go Normal file
View File

@@ -0,0 +1,177 @@
package scanner
import (
"bufio"
"fmt"
"net"
"os"
"sort"
"strings"
"sync"
"time"
"god-eye/internal/config"
)
// LoadWordlist loads words from a file
func LoadWordlist(path string) ([]string, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var words []string
scanner := bufio.NewScanner(file)
for scanner.Scan() {
word := strings.TrimSpace(scanner.Text())
if word != "" && !strings.HasPrefix(word, "#") {
words = append(words, word)
}
}
return words, scanner.Err()
}
// ScanPorts scans ports on an IP address
func ScanPorts(ip string, ports []int, timeout int) []int {
var openPorts []int
var mu sync.Mutex
var wg sync.WaitGroup
for _, port := range ports {
wg.Add(1)
go func(p int) {
defer wg.Done()
address := fmt.Sprintf("%s:%d", ip, p)
conn, err := net.DialTimeout("tcp", address, time.Duration(timeout)*time.Second)
if err == nil {
conn.Close()
mu.Lock()
openPorts = append(openPorts, p)
mu.Unlock()
}
}(port)
}
wg.Wait()
sort.Ints(openPorts)
return openPorts
}
// Helper functions for AI analysis
func countSubdomainsWithAI(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if len(r.AIFindings) > 0 {
count++
}
}
return count
}
func countActive(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if r.StatusCode >= 200 && r.StatusCode < 400 {
count++
}
}
return count
}
func countVulns(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if r.OpenRedirect || r.CORSMisconfig != "" || len(r.DangerousMethods) > 0 ||
r.GitExposed || r.SvnExposed || len(r.BackupFiles) > 0 {
count++
}
}
return count
}
func buildAISummary(results map[string]*config.SubdomainResult) string {
var summary strings.Builder
criticalCount := 0
highCount := 0
mediumCount := 0
for sub, r := range results {
if len(r.AIFindings) == 0 {
continue
}
switch r.AISeverity {
case "critical":
criticalCount++
summary.WriteString(fmt.Sprintf("\n[CRITICAL] %s:\n", sub))
case "high":
highCount++
summary.WriteString(fmt.Sprintf("\n[HIGH] %s:\n", sub))
case "medium":
mediumCount++
summary.WriteString(fmt.Sprintf("\n[MEDIUM] %s:\n", sub))
default:
continue
}
// Add first 3 findings
for i, finding := range r.AIFindings {
if i >= 3 {
break
}
summary.WriteString(fmt.Sprintf(" - %s\n", finding))
}
// Add CVE findings
if len(r.CVEFindings) > 0 {
summary.WriteString(" CVEs:\n")
for _, cve := range r.CVEFindings {
summary.WriteString(fmt.Sprintf(" - %s\n", cve))
}
}
}
header := fmt.Sprintf("Summary: %d critical, %d high, %d medium findings\n", criticalCount, highCount, mediumCount)
return header + summary.String()
}
// ParseResolvers parses custom resolvers string
func ParseResolvers(resolversStr string) []string {
var resolvers []string
if resolversStr != "" {
for _, r := range strings.Split(resolversStr, ",") {
r = strings.TrimSpace(r)
if r != "" {
if !strings.Contains(r, ":") {
r = r + ":53"
}
resolvers = append(resolvers, r)
}
}
}
if len(resolvers) == 0 {
resolvers = config.DefaultResolvers
}
return resolvers
}
// ParsePorts parses custom ports string
func ParsePorts(portsStr string) []int {
var customPorts []int
if portsStr != "" {
for _, p := range strings.Split(portsStr, ",") {
p = strings.TrimSpace(p)
var port int
if _, err := fmt.Sscanf(p, "%d", &port); err == nil && port > 0 && port < 65536 {
customPorts = append(customPorts, port)
}
}
}
if len(customPorts) == 0 {
customPorts = []int{80, 443, 8080, 8443}
}
return customPorts
}

View File

@@ -6,52 +6,87 @@ import (
"net/http" "net/http"
"regexp" "regexp"
"strings" "strings"
"sync"
) )
// SecretPattern defines a pattern for finding secrets
type SecretPattern struct {
Name string
Pattern *regexp.Regexp
}
// Secret patterns to search for in JS files
var secretPatterns = []SecretPattern{
// API Keys
{Name: "AWS Access Key", Pattern: regexp.MustCompile(`AKIA[0-9A-Z]{16}`)},
{Name: "AWS Secret Key", Pattern: regexp.MustCompile(`(?i)aws[_\-]?secret[_\-]?access[_\-]?key['"\s:=]+['"]?([A-Za-z0-9/+=]{40})['"]?`)},
{Name: "Google API Key", Pattern: regexp.MustCompile(`AIza[0-9A-Za-z\-_]{35}`)},
{Name: "Google OAuth", Pattern: regexp.MustCompile(`[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com`)},
{Name: "Firebase API Key", Pattern: regexp.MustCompile(`(?i)firebase[_\-]?api[_\-]?key['"\s:=]+['"]?([A-Za-z0-9_\-]{39})['"]?`)},
{Name: "Stripe Key", Pattern: regexp.MustCompile(`(?:sk|pk)_(?:test|live)_[0-9a-zA-Z]{24,}`)},
{Name: "Stripe Restricted", Pattern: regexp.MustCompile(`rk_(?:test|live)_[0-9a-zA-Z]{24,}`)},
{Name: "GitHub Token", Pattern: regexp.MustCompile(`(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{36,}`)},
{Name: "GitHub OAuth", Pattern: regexp.MustCompile(`github[_\-]?oauth[_\-]?token['"\s:=]+['"]?([a-f0-9]{40})['"]?`)},
{Name: "Slack Token", Pattern: regexp.MustCompile(`xox[baprs]-[0-9]{10,13}-[0-9]{10,13}[a-zA-Z0-9-]*`)},
{Name: "Slack Webhook", Pattern: regexp.MustCompile(`https://hooks\.slack\.com/services/T[a-zA-Z0-9_]{8,}/B[a-zA-Z0-9_]{8,}/[a-zA-Z0-9_]{24}`)},
{Name: "Discord Webhook", Pattern: regexp.MustCompile(`https://discord(?:app)?\.com/api/webhooks/[0-9]{17,20}/[A-Za-z0-9_\-]{60,}`)},
{Name: "Twilio API Key", Pattern: regexp.MustCompile(`SK[a-f0-9]{32}`)},
{Name: "Twilio Account SID", Pattern: regexp.MustCompile(`AC[a-f0-9]{32}`)},
{Name: "SendGrid API Key", Pattern: regexp.MustCompile(`SG\.[a-zA-Z0-9_\-]{22}\.[a-zA-Z0-9_\-]{43}`)},
{Name: "Mailgun API Key", Pattern: regexp.MustCompile(`key-[0-9a-zA-Z]{32}`)},
{Name: "Mailchimp API Key", Pattern: regexp.MustCompile(`[0-9a-f]{32}-us[0-9]{1,2}`)},
{Name: "Heroku API Key", Pattern: regexp.MustCompile(`(?i)heroku[_\-]?api[_\-]?key['"\s:=]+['"]?([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})['"]?`)},
{Name: "DigitalOcean Token", Pattern: regexp.MustCompile(`dop_v1_[a-f0-9]{64}`)},
{Name: "NPM Token", Pattern: regexp.MustCompile(`npm_[A-Za-z0-9]{36}`)},
{Name: "PyPI Token", Pattern: regexp.MustCompile(`pypi-AgEIcHlwaS5vcmc[A-Za-z0-9_\-]{50,}`)},
{Name: "Square Access Token", Pattern: regexp.MustCompile(`sq0atp-[0-9A-Za-z\-_]{22}`)},
{Name: "Square OAuth", Pattern: regexp.MustCompile(`sq0csp-[0-9A-Za-z\-_]{43}`)},
{Name: "Shopify Access Token", Pattern: regexp.MustCompile(`shpat_[a-fA-F0-9]{32}`)},
{Name: "Shopify Shared Secret", Pattern: regexp.MustCompile(`shpss_[a-fA-F0-9]{32}`)},
{Name: "Algolia API Key", Pattern: regexp.MustCompile(`(?i)algolia[_\-]?api[_\-]?key['"\s:=]+['"]?([a-zA-Z0-9]{32})['"]?`)},
{Name: "Auth0 Client Secret", Pattern: regexp.MustCompile(`(?i)auth0[_\-]?client[_\-]?secret['"\s:=]+['"]?([a-zA-Z0-9_\-]{32,})['"]?`)},
// Generic secrets
{Name: "Generic API Key", Pattern: regexp.MustCompile(`(?i)['"]?api[_\-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{20,64})['"]`)},
{Name: "Generic Secret", Pattern: regexp.MustCompile(`(?i)['"]?(?:client[_\-]?)?secret['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{20,64})['"]`)},
{Name: "Generic Token", Pattern: regexp.MustCompile(`(?i)['"]?(?:access[_\-]?)?token['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-\.]{20,500})['"]`)},
{Name: "Generic Password", Pattern: regexp.MustCompile(`(?i)['"]?password['"]?\s*[:=]\s*['"]([^'"]{8,64})['"]`)},
{Name: "Private Key", Pattern: regexp.MustCompile(`-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----`)},
{Name: "Bearer Token", Pattern: regexp.MustCompile(`(?i)['"]?authorization['"]?\s*[:=]\s*['"]Bearer\s+([a-zA-Z0-9_\-\.]+)['"]`)},
{Name: "Basic Auth", Pattern: regexp.MustCompile(`(?i)['"]?authorization['"]?\s*[:=]\s*['"]Basic\s+([a-zA-Z0-9+/=]+)['"]`)},
{Name: "JWT Token", Pattern: regexp.MustCompile(`eyJ[a-zA-Z0-9_\-]*\.eyJ[a-zA-Z0-9_\-]*\.[a-zA-Z0-9_\-]*`)},
// Database connection strings
{Name: "MongoDB URI", Pattern: regexp.MustCompile(`mongodb(?:\+srv)?://[^\s'"]+`)},
{Name: "PostgreSQL URI", Pattern: regexp.MustCompile(`postgres(?:ql)?://[^\s'"]+`)},
{Name: "MySQL URI", Pattern: regexp.MustCompile(`mysql://[^\s'"]+`)},
{Name: "Redis URI", Pattern: regexp.MustCompile(`redis://[^\s'"]+`)},
}
// Endpoint patterns for API discovery - only external/interesting URLs
// Note: We exclude relative paths like /api/... as they're not secrets
var endpointPatterns = []*regexp.Regexp{
regexp.MustCompile(`['"]https?://api\.[a-zA-Z0-9\-\.]+[a-zA-Z0-9/\-_]*['"]`), // External API domains
regexp.MustCompile(`['"]https?://[a-zA-Z0-9\-\.]+\.amazonaws\.com[^'"]*['"]`), // AWS endpoints
regexp.MustCompile(`['"]https?://[a-zA-Z0-9\-\.]+\.azure\.com[^'"]*['"]`), // Azure endpoints
regexp.MustCompile(`['"]https?://[a-zA-Z0-9\-\.]+\.googleapis\.com[^'"]*['"]`), // Google API
regexp.MustCompile(`['"]https?://[a-zA-Z0-9\-\.]+\.firebaseio\.com[^'"]*['"]`), // Firebase
}
// AnalyzeJSFiles finds JavaScript files and extracts potential secrets // AnalyzeJSFiles finds JavaScript files and extracts potential secrets
func AnalyzeJSFiles(subdomain string, client *http.Client) ([]string, []string) { func AnalyzeJSFiles(subdomain string, client *http.Client) ([]string, []string) {
var jsFiles []string var jsFiles []string
var secrets []string var secrets []string
var mu sync.Mutex
urls := []string{ baseURLs := []string{
fmt.Sprintf("https://%s", subdomain), fmt.Sprintf("https://%s", subdomain),
fmt.Sprintf("http://%s", subdomain), fmt.Sprintf("http://%s", subdomain),
} }
// Common JS file paths // First, get the main page and extract JS file references
jsPaths := []string{ var foundJSURLs []string
"/main.js", "/app.js", "/bundle.js", "/vendor.js", for _, baseURL := range baseURLs {
"/static/js/main.js", "/static/js/app.js",
"/assets/js/app.js", "/js/main.js", "/js/app.js",
"/dist/main.js", "/dist/bundle.js",
"/_next/static/chunks/main.js",
"/build/static/js/main.js",
}
// Secret patterns to search for
secretPatterns := []*regexp.Regexp{
regexp.MustCompile(`(?i)['"]?api[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{20,})['"]`),
regexp.MustCompile(`(?i)['"]?aws[_-]?access[_-]?key[_-]?id['"]?\s*[:=]\s*['"]([A-Z0-9]{20})['"]`),
regexp.MustCompile(`(?i)['"]?aws[_-]?secret[_-]?access[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9/+=]{40})['"]`),
regexp.MustCompile(`(?i)['"]?google[_-]?api[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{39})['"]`),
regexp.MustCompile(`(?i)['"]?firebase[_-]?api[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{39})['"]`),
regexp.MustCompile(`(?i)['"]?stripe[_-]?(publishable|secret)[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{20,})['"]`),
regexp.MustCompile(`(?i)['"]?github[_-]?token['"]?\s*[:=]\s*['"]([a-zA-Z0-9_]{36,})['"]`),
regexp.MustCompile(`(?i)['"]?slack[_-]?token['"]?\s*[:=]\s*['"]([a-zA-Z0-9\-]{30,})['"]`),
regexp.MustCompile(`(?i)['"]?private[_-]?key['"]?\s*[:=]\s*['"]([a-zA-Z0-9/+=]{50,})['"]`),
regexp.MustCompile(`(?i)['"]?secret['"]?\s*[:=]\s*['"]([a-zA-Z0-9_\-]{20,})['"]`),
regexp.MustCompile(`(?i)['"]?password['"]?\s*[:=]\s*['"]([^'"]{8,})['"]`),
regexp.MustCompile(`(?i)['"]?authorization['"]?\s*[:=]\s*['"]Bearer\s+([a-zA-Z0-9_\-\.]+)['"]`),
}
// Also search for API endpoints in JS
endpointPatterns := []*regexp.Regexp{
regexp.MustCompile(`(?i)['"]https?://[a-zA-Z0-9\-\.]+/api/[a-zA-Z0-9/\-_]+['"]`),
regexp.MustCompile(`(?i)['"]https?://api\.[a-zA-Z0-9\-\.]+[a-zA-Z0-9/\-_]*['"]`),
}
for _, baseURL := range urls {
// First, get the main page and extract JS file references
resp, err := client.Get(baseURL) resp, err := client.Get(baseURL)
if err != nil { if err != nil {
continue continue
@@ -64,92 +99,260 @@ func AnalyzeJSFiles(subdomain string, client *http.Client) ([]string, []string)
} }
// Find JS files referenced in HTML // Find JS files referenced in HTML
jsRe := regexp.MustCompile(`src=["']([^"']*\.js[^"']*)["']`) jsRe := regexp.MustCompile(`(?:src|href)=["']([^"']*\.js(?:\?[^"']*)?)["']`)
matches := jsRe.FindAllStringSubmatch(string(body), -1) matches := jsRe.FindAllStringSubmatch(string(body), -1)
for _, match := range matches { for _, match := range matches {
if len(match) > 1 { if len(match) > 1 {
jsURL := match[1] jsURL := normalizeURL(match[1], baseURL)
if !strings.HasPrefix(jsURL, "http") { if jsURL != "" && !contains(foundJSURLs, jsURL) {
if strings.HasPrefix(jsURL, "/") { foundJSURLs = append(foundJSURLs, jsURL)
jsURL = baseURL + jsURL
} else {
jsURL = baseURL + "/" + jsURL
}
} }
jsFiles = append(jsFiles, jsURL)
} }
} }
// Also check common JS paths // Also look for dynamic imports and webpack chunks
for _, path := range jsPaths { dynamicRe := regexp.MustCompile(`["']([^"']*(?:chunk|bundle|vendor|main|app)[^"']*\.js(?:\?[^"']*)?)["']`)
testURL := baseURL + path dynamicMatches := dynamicRe.FindAllStringSubmatch(string(body), -1)
resp, err := client.Get(testURL) for _, match := range dynamicMatches {
if err != nil { if len(match) > 1 {
continue jsURL := normalizeURL(match[1], baseURL)
} if jsURL != "" && !contains(foundJSURLs, jsURL) {
foundJSURLs = append(foundJSURLs, jsURL)
if resp.StatusCode == 200 {
jsFiles = append(jsFiles, path)
// Read JS content and search for secrets
jsBody, err := io.ReadAll(io.LimitReader(resp.Body, 500000))
resp.Body.Close()
if err != nil {
continue
} }
jsContent := string(jsBody)
// Search for secrets
for _, pattern := range secretPatterns {
if matches := pattern.FindAllStringSubmatch(jsContent, 3); len(matches) > 0 {
for _, m := range matches {
if len(m) > 1 {
secret := m[0]
if len(secret) > 60 {
secret = secret[:57] + "..."
}
secrets = append(secrets, secret)
}
}
}
}
// Search for API endpoints
for _, pattern := range endpointPatterns {
if matches := pattern.FindAllString(jsContent, 5); len(matches) > 0 {
for _, m := range matches {
if len(m) > 60 {
m = m[:57] + "..."
}
secrets = append(secrets, "endpoint: "+m)
}
}
}
} else {
resp.Body.Close()
} }
} }
if len(jsFiles) > 0 || len(secrets) > 0 { if len(foundJSURLs) > 0 {
break break
} }
} }
// Deduplicate and limit // Limit to first 15 JS files to avoid too many requests
if len(foundJSURLs) > 15 {
foundJSURLs = foundJSURLs[:15]
}
// Download and analyze each JS file concurrently
var wg sync.WaitGroup
semaphore := make(chan struct{}, 5) // Limit concurrent downloads
for _, jsURL := range foundJSURLs {
wg.Add(1)
go func(url string) {
defer wg.Done()
semaphore <- struct{}{}
defer func() { <-semaphore }()
fileSecrets := analyzeJSContent(url, client)
mu.Lock()
jsFiles = append(jsFiles, url)
secrets = append(secrets, fileSecrets...)
mu.Unlock()
}(jsURL)
}
wg.Wait()
// Deduplicate and limit results
jsFiles = UniqueStrings(jsFiles) jsFiles = UniqueStrings(jsFiles)
secrets = UniqueStrings(secrets) secrets = UniqueStrings(secrets)
if len(jsFiles) > 10 { if len(jsFiles) > 10 {
jsFiles = jsFiles[:10] jsFiles = jsFiles[:10]
} }
if len(secrets) > 10 { if len(secrets) > 20 {
secrets = secrets[:10] secrets = secrets[:20]
} }
return jsFiles, secrets return jsFiles, secrets
} }
// analyzeJSContent downloads and analyzes a JS file for secrets
func analyzeJSContent(jsURL string, client *http.Client) []string {
var secrets []string
resp, err := client.Get(jsURL)
if err != nil {
return secrets
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return secrets
}
// Read JS content (limit to 2MB)
body, err := io.ReadAll(io.LimitReader(resp.Body, 2*1024*1024))
if err != nil {
return secrets
}
content := string(body)
// Skip minified files that are too large without meaningful content
if len(content) > 500000 && !containsInterestingPatterns(content) {
return secrets
}
// Search for secrets
for _, sp := range secretPatterns {
matches := sp.Pattern.FindAllStringSubmatch(content, 3)
for _, m := range matches {
var secret string
if len(m) > 1 && m[1] != "" {
secret = m[1]
} else {
secret = m[0]
}
// Skip common false positives
if isLikelyFalsePositive(secret) {
continue
}
// Truncate long secrets
if len(secret) > 80 {
secret = secret[:77] + "..."
}
finding := fmt.Sprintf("[%s] %s", sp.Name, secret)
secrets = append(secrets, finding)
}
}
// Search for API endpoints
for _, pattern := range endpointPatterns {
matches := pattern.FindAllString(content, 5)
for _, m := range matches {
// Clean up the match
m = strings.Trim(m, `'"`)
if len(m) > 80 {
m = m[:77] + "..."
}
secrets = append(secrets, fmt.Sprintf("[API Endpoint] %s", m))
}
}
return secrets
}
// normalizeURL converts relative URLs to absolute URLs
func normalizeURL(jsURL, baseURL string) string {
if jsURL == "" {
return ""
}
// Skip data URIs and blob URLs
if strings.HasPrefix(jsURL, "data:") || strings.HasPrefix(jsURL, "blob:") {
return ""
}
// Already absolute URL
if strings.HasPrefix(jsURL, "http://") || strings.HasPrefix(jsURL, "https://") {
return jsURL
}
// Protocol-relative URL
if strings.HasPrefix(jsURL, "//") {
if strings.HasPrefix(baseURL, "https") {
return "https:" + jsURL
}
return "http:" + jsURL
}
// Absolute path
if strings.HasPrefix(jsURL, "/") {
// Extract base (scheme + host)
parts := strings.SplitN(baseURL, "/", 4)
if len(parts) >= 3 {
return parts[0] + "//" + parts[2] + jsURL
}
return baseURL + jsURL
}
// Relative path
return strings.TrimSuffix(baseURL, "/") + "/" + jsURL
}
// containsInterestingPatterns checks if content might contain secrets
func containsInterestingPatterns(content string) bool {
interestingKeywords := []string{
"api_key", "apikey", "api-key",
"secret", "password", "token",
"authorization", "bearer",
"aws_", "firebase", "stripe",
"mongodb://", "postgres://", "mysql://",
"private_key", "privatekey",
}
contentLower := strings.ToLower(content)
for _, kw := range interestingKeywords {
if strings.Contains(contentLower, kw) {
return true
}
}
return false
}
// isLikelyFalsePositive performs basic pre-filtering before AI analysis
// Only filters obvious patterns - AI will handle context-aware filtering
func isLikelyFalsePositive(secret string) bool {
// Only filter obvious placeholder patterns
// AI will handle context-aware filtering (UI text, etc.)
obviousPlaceholders := []string{
"YOUR_API_KEY", "API_KEY_HERE", "REPLACE_ME",
"xxxxxxxx", "XXXXXXXX", "00000000",
}
secretLower := strings.ToLower(secret)
for _, fp := range obviousPlaceholders {
if strings.Contains(secretLower, strings.ToLower(fp)) {
return true
}
}
// Too short
if len(secret) < 8 {
return true
}
// Check for repeated characters (garbage data)
if isRepeatedChars(secret) {
return true
}
return false
}
// isRepeatedChars checks if string is mostly repeated characters
func isRepeatedChars(s string) bool {
if len(s) < 10 {
return false
}
charCount := make(map[rune]int)
for _, c := range s {
charCount[c]++
}
// If any single character is more than 60% of the string, it's likely garbage
for _, count := range charCount {
if float64(count)/float64(len(s)) > 0.6 {
return true
}
}
return false
}
// contains checks if a string slice contains a value
func contains(slice []string, val string) bool {
for _, s := range slice {
if s == val {
return true
}
}
return false
}
// UniqueStrings returns unique strings from a slice // UniqueStrings returns unique strings from a slice
func UniqueStrings(input []string) []string { func UniqueStrings(input []string) []string {
seen := make(map[string]bool) seen := make(map[string]bool)

455
internal/scanner/output.go Normal file
View File

@@ -0,0 +1,455 @@
package scanner
import (
"fmt"
"sort"
"strings"
"time"
"god-eye/internal/config"
"god-eye/internal/output"
)
// PrintResults displays scan results to stdout
func PrintResults(results map[string]*config.SubdomainResult, startTime time.Time, takeoverCount int32) {
elapsed := time.Since(startTime)
// Count statistics
var activeCount, vulnCount, cloudCount int
for _, r := range results {
if r.StatusCode >= 200 && r.StatusCode < 400 {
activeCount++
}
if r.OpenRedirect || r.CORSMisconfig != "" || len(r.DangerousMethods) > 0 || r.GitExposed || r.SvnExposed || len(r.BackupFiles) > 0 {
vulnCount++
}
if r.CloudProvider != "" {
cloudCount++
}
}
// Summary box
fmt.Println()
fmt.Println(output.BoldCyan("╔══════════════════════════════════════════════════════════════════════════════╗"))
fmt.Println(output.BoldCyan("║") + " " + output.BoldWhite("📊 SCAN SUMMARY") + " " + output.BoldCyan("║"))
fmt.Println(output.BoldCyan("╠══════════════════════════════════════════════════════════════════════════════╣"))
fmt.Printf("%s %-20s %s %-20s %s %-20s %s\n",
output.BoldCyan("║"),
fmt.Sprintf("🌐 Total: %s", output.BoldCyan(fmt.Sprintf("%d", len(results)))),
output.Dim("|"),
fmt.Sprintf("✅ Active: %s", output.BoldGreen(fmt.Sprintf("%d", activeCount))),
output.Dim("|"),
fmt.Sprintf("⏱️ Time: %s", output.BoldYellow(fmt.Sprintf("%.1fs", elapsed.Seconds()))),
output.BoldCyan("║"))
fmt.Printf("%s %-20s %s %-20s %s %-20s %s\n",
output.BoldCyan("║"),
fmt.Sprintf("⚠️ Vulns: %s", output.BoldRed(fmt.Sprintf("%d", vulnCount))),
output.Dim("|"),
fmt.Sprintf("☁️ Cloud: %s", output.Blue(fmt.Sprintf("%d", cloudCount))),
output.Dim("|"),
fmt.Sprintf("🎯 Takeover: %s", output.BoldRed(fmt.Sprintf("%d", takeoverCount))),
output.BoldCyan("║"))
fmt.Println(output.BoldCyan("╚══════════════════════════════════════════════════════════════════════════════╝"))
fmt.Println()
fmt.Println(output.BoldCyan("═══════════════════════════════════════════════════════════════════════════════"))
// Sort subdomains
var sortedSubs []string
for sub := range results {
sortedSubs = append(sortedSubs, sub)
}
sort.Strings(sortedSubs)
for _, sub := range sortedSubs {
r := results[sub]
printSubdomainResult(sub, r)
}
fmt.Println()
fmt.Println(output.BoldCyan("═══════════════════════════════════════════════════════════════════════════════"))
}
func printSubdomainResult(sub string, r *config.SubdomainResult) {
// Color code by status
var statusColor func(a ...interface{}) string
var statusIcon string
if r.StatusCode >= 200 && r.StatusCode < 300 {
statusColor = output.Green
statusIcon = "●"
} else if r.StatusCode >= 300 && r.StatusCode < 400 {
statusColor = output.Yellow
statusIcon = "◐"
} else if r.StatusCode >= 400 {
statusColor = output.Red
statusIcon = "○"
} else {
statusColor = output.Blue
statusIcon = "◌"
}
// Line 1: Subdomain name with status
statusBadge := ""
if r.StatusCode > 0 {
statusBadge = fmt.Sprintf(" %s", statusColor(fmt.Sprintf("[%d]", r.StatusCode)))
}
// Response time badge
timeBadge := ""
if r.ResponseMs > 0 {
if r.ResponseMs < 200 {
timeBadge = fmt.Sprintf(" %s", output.Green(fmt.Sprintf("⚡%dms", r.ResponseMs)))
} else if r.ResponseMs < 500 {
timeBadge = fmt.Sprintf(" %s", output.Yellow(fmt.Sprintf("⏱️%dms", r.ResponseMs)))
} else {
timeBadge = fmt.Sprintf(" %s", output.Red(fmt.Sprintf("🐢%dms", r.ResponseMs)))
}
}
fmt.Printf("\n%s %s%s%s\n", statusColor(statusIcon), output.BoldCyan(sub), statusBadge, timeBadge)
// IPs
if len(r.IPs) > 0 {
ips := r.IPs
if len(ips) > 3 {
ips = ips[:3]
}
fmt.Printf(" %s %s\n", output.Dim("IP:"), output.White(strings.Join(ips, ", ")))
}
// CNAME
if r.CNAME != "" {
fmt.Printf(" %s %s\n", output.Dim("CNAME:"), output.Blue(r.CNAME))
}
// Location + ASN
if r.Country != "" || r.City != "" || r.ASN != "" {
loc := ""
if r.City != "" && r.Country != "" {
loc = r.City + ", " + r.Country
} else if r.Country != "" {
loc = r.Country
} else if r.City != "" {
loc = r.City
}
asnStr := ""
if r.ASN != "" {
asnStr = r.ASN
if len(asnStr) > 40 {
asnStr = asnStr[:37] + "..."
}
}
if loc != "" && asnStr != "" {
fmt.Printf(" Location: %s | %s\n", output.Cyan(loc), output.Blue(asnStr))
} else if loc != "" {
fmt.Printf(" Location: %s\n", output.Cyan(loc))
} else if asnStr != "" {
fmt.Printf(" ASN: %s\n", output.Blue(asnStr))
}
}
// PTR
if r.PTR != "" {
fmt.Printf(" PTR: %s\n", output.Magenta(r.PTR))
}
// HTTP Info (Title, Size)
if r.Title != "" || r.ContentLength > 0 {
httpInfo := " HTTP: "
if r.Title != "" {
title := r.Title
if len(title) > 50 {
title = title[:47] + "..."
}
httpInfo += fmt.Sprintf("\"%s\"", title)
}
if r.ContentLength > 0 {
sizeStr := formatSize(r.ContentLength)
if r.Title != "" {
httpInfo += fmt.Sprintf(" (%s)", sizeStr)
} else {
httpInfo += sizeStr
}
}
fmt.Println(httpInfo)
}
// Redirect
if r.RedirectURL != "" {
redirectURL := r.RedirectURL
if len(redirectURL) > 60 {
redirectURL = redirectURL[:57] + "..."
}
fmt.Printf(" Redirect: %s\n", output.Yellow(redirectURL))
}
// Tech
if len(r.Tech) > 0 {
techMap := make(map[string]bool)
var uniqueTech []string
for _, t := range r.Tech {
if !techMap[t] {
techMap[t] = true
uniqueTech = append(uniqueTech, t)
}
}
if len(uniqueTech) > 5 {
uniqueTech = uniqueTech[:5]
}
if len(uniqueTech) > 0 {
fmt.Printf(" Tech: %s\n", output.Yellow(strings.Join(uniqueTech, ", ")))
}
}
// Security (WAF, TLS)
var securityInfo []string
if r.WAF != "" {
securityInfo = append(securityInfo, fmt.Sprintf("WAF: %s", output.Red(r.WAF)))
}
if r.TLSVersion != "" {
tlsInfo := fmt.Sprintf("TLS: %s", output.Cyan(r.TLSVersion))
if r.TLSSelfSigned {
tlsInfo += " " + output.Yellow("(self-signed)")
}
securityInfo = append(securityInfo, tlsInfo)
}
if len(securityInfo) > 0 {
fmt.Printf(" Security: %s\n", strings.Join(securityInfo, " | "))
}
// TLS Fingerprint (appliance detection)
if r.TLSFingerprint != nil {
fp := r.TLSFingerprint
if fp.Vendor != "" {
applianceInfo := fmt.Sprintf("%s %s", fp.Vendor, fp.Product)
if fp.Version != "" {
applianceInfo += " v" + fp.Version
}
if fp.ApplianceType != "" {
applianceInfo += fmt.Sprintf(" (%s)", fp.ApplianceType)
}
fmt.Printf(" %s %s\n", output.BoldYellow("APPLIANCE:"), output.Yellow(applianceInfo))
}
// Show internal hostnames found in certificate
if len(fp.InternalHosts) > 0 {
hosts := fp.InternalHosts
if len(hosts) > 5 {
hosts = hosts[:5]
}
fmt.Printf(" %s %s\n", output.BoldMagenta("INTERNAL:"), output.Magenta(strings.Join(hosts, ", ")))
}
// Show certificate subject info if no vendor matched but has org info
if fp.Vendor == "" && (fp.SubjectOrg != "" || fp.SubjectOU != "") {
certInfo := ""
if fp.SubjectOrg != "" {
certInfo = "Org: " + fp.SubjectOrg
}
if fp.SubjectOU != "" {
if certInfo != "" {
certInfo += ", "
}
certInfo += "OU: " + fp.SubjectOU
}
fmt.Printf(" Cert: %s\n", output.Dim(certInfo))
}
}
// Ports
if len(r.Ports) > 0 {
var portStrs []string
for _, p := range r.Ports {
portStrs = append(portStrs, fmt.Sprintf("%d", p))
}
fmt.Printf(" Ports: %s\n", output.Magenta(strings.Join(portStrs, ", ")))
}
// Extra files
var extras []string
if r.RobotsTxt {
extras = append(extras, "robots.txt")
}
if r.SitemapXml {
extras = append(extras, "sitemap.xml")
}
if r.FaviconHash != "" {
extras = append(extras, fmt.Sprintf("favicon:%s", r.FaviconHash[:8]))
}
if len(extras) > 0 {
fmt.Printf(" Files: %s\n", output.Green(strings.Join(extras, ", ")))
}
// DNS Records
if len(r.MXRecords) > 0 {
mx := r.MXRecords
if len(mx) > 2 {
mx = mx[:2]
}
fmt.Printf(" MX: %s\n", strings.Join(mx, ", "))
}
// Security Headers
if len(r.MissingHeaders) > 0 && len(r.MissingHeaders) < 7 {
if len(r.SecurityHeaders) > 0 {
fmt.Printf(" Headers: %s | Missing: %s\n",
output.Green(strings.Join(r.SecurityHeaders, ", ")),
output.Yellow(strings.Join(r.MissingHeaders, ", ")))
}
} else if len(r.SecurityHeaders) > 0 {
fmt.Printf(" Headers: %s\n", output.Green(strings.Join(r.SecurityHeaders, ", ")))
}
// Cloud Provider
if r.CloudProvider != "" {
fmt.Printf(" Cloud: %s\n", output.Cyan(r.CloudProvider))
}
// Email Security
if r.EmailSecurity != "" {
emailColor := output.Green
if r.EmailSecurity == "Weak" {
emailColor = output.Yellow
} else if r.EmailSecurity == "None" {
emailColor = output.Red
}
fmt.Printf(" Email: %s\n", emailColor(r.EmailSecurity))
}
// TLS Alt Names
if len(r.TLSAltNames) > 0 {
altNames := r.TLSAltNames
if len(altNames) > 5 {
altNames = altNames[:5]
}
fmt.Printf(" TLS Alt: %s\n", output.Blue(strings.Join(altNames, ", ")))
}
// S3 Buckets
if len(r.S3Buckets) > 0 {
for _, bucket := range r.S3Buckets {
if strings.Contains(bucket, "PUBLIC") {
fmt.Printf(" %s %s\n", output.Red("S3:"), output.Red(bucket))
} else {
fmt.Printf(" S3: %s\n", output.Yellow(bucket))
}
}
}
// Security Issues (vulnerabilities)
var vulns []string
if r.OpenRedirect {
vulns = append(vulns, "Open Redirect")
}
if r.CORSMisconfig != "" {
vulns = append(vulns, fmt.Sprintf("CORS: %s", r.CORSMisconfig))
}
if len(r.DangerousMethods) > 0 {
vulns = append(vulns, fmt.Sprintf("Methods: %s", strings.Join(r.DangerousMethods, ", ")))
}
if r.GitExposed {
vulns = append(vulns, ".git Exposed")
}
if r.SvnExposed {
vulns = append(vulns, ".svn Exposed")
}
if len(r.BackupFiles) > 0 {
files := r.BackupFiles
if len(files) > 3 {
files = files[:3]
}
vulns = append(vulns, fmt.Sprintf("Backup: %s", strings.Join(files, ", ")))
}
if len(vulns) > 0 {
fmt.Printf(" %s %s\n", output.Red("VULNS:"), output.Red(strings.Join(vulns, " | ")))
}
// Discovery (admin panels, API endpoints)
var discoveries []string
if len(r.AdminPanels) > 0 {
panels := r.AdminPanels
if len(panels) > 5 {
panels = panels[:5]
}
discoveries = append(discoveries, fmt.Sprintf("Admin: %s", strings.Join(panels, ", ")))
}
if len(r.APIEndpoints) > 0 {
endpoints := r.APIEndpoints
if len(endpoints) > 5 {
endpoints = endpoints[:5]
}
discoveries = append(discoveries, fmt.Sprintf("API: %s", strings.Join(endpoints, ", ")))
}
if len(discoveries) > 0 {
fmt.Printf(" %s %s\n", output.Magenta("FOUND:"), output.Magenta(strings.Join(discoveries, " | ")))
}
// JavaScript Analysis
if len(r.JSFiles) > 0 {
files := r.JSFiles
if len(files) > 3 {
files = files[:3]
}
fmt.Printf(" JS Files: %s\n", output.Blue(strings.Join(files, ", ")))
}
if len(r.JSSecrets) > 0 {
for _, secret := range r.JSSecrets {
fmt.Printf(" %s %s\n", output.Red("JS SECRET:"), output.Red(secret))
}
}
// Takeover
if r.Takeover != "" {
fmt.Printf(" %s %s\n", output.BgRed(" TAKEOVER "), output.BoldRed(r.Takeover))
}
// AI Findings
if len(r.AIFindings) > 0 {
severityColor := output.Cyan
severityLabel := "AI"
if r.AISeverity == "critical" {
severityColor = output.BoldRed
severityLabel = "AI:CRITICAL"
} else if r.AISeverity == "high" {
severityColor = output.Red
severityLabel = "AI:HIGH"
} else if r.AISeverity == "medium" {
severityColor = output.Yellow
severityLabel = "AI:MEDIUM"
}
for i, finding := range r.AIFindings {
if i == 0 {
fmt.Printf(" %s %s\n", severityColor(severityLabel+":"), finding)
} else {
fmt.Printf(" %s %s\n", output.Dim(" "), finding)
}
if i >= 4 {
remaining := len(r.AIFindings) - 5
if remaining > 0 {
fmt.Printf(" %s (%d more findings...)\n", output.Dim(" "), remaining)
}
break
}
}
if r.AIModel != "" {
fmt.Printf(" %s model: %s\n", output.Dim(" "), output.Dim(r.AIModel))
}
}
// CVE Findings
if len(r.CVEFindings) > 0 {
for _, cve := range r.CVEFindings {
fmt.Printf(" %s %s\n", output.BoldRed("CVE:"), output.Red(cve))
}
}
}
func formatSize(size int64) string {
if size > 1024*1024 {
return fmt.Sprintf("%.1fMB", float64(size)/(1024*1024))
} else if size > 1024 {
return fmt.Sprintf("%.1fKB", float64(size)/1024)
}
return fmt.Sprintf("%dB", size)
}

View File

@@ -1,13 +1,8 @@
package scanner package scanner
import ( import (
"bufio"
"encoding/json"
"fmt" "fmt"
"net"
"os" "os"
"sort"
"strconv"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -18,50 +13,57 @@ import (
"god-eye/internal/dns" "god-eye/internal/dns"
gohttp "god-eye/internal/http" gohttp "god-eye/internal/http"
"god-eye/internal/output" "god-eye/internal/output"
"god-eye/internal/progress"
"god-eye/internal/ratelimit"
"god-eye/internal/security" "god-eye/internal/security"
"god-eye/internal/sources" "god-eye/internal/sources"
"god-eye/internal/stealth"
) )
func Run(cfg config.Config) { func Run(cfg config.Config) {
startTime := time.Now() startTime := time.Now()
// Parse custom resolvers // Pre-load KEV database if AI is enabled (auto-downloads if not present)
var resolvers []string if cfg.EnableAI && !cfg.Silent && !cfg.JsonOutput {
if cfg.Resolvers != "" { kevStore := ai.GetKEVStore()
for _, r := range strings.Split(cfg.Resolvers, ",") { if !kevStore.IsLoaded() {
r = strings.TrimSpace(r) if err := kevStore.LoadWithProgress(true); err != nil {
if r != "" { fmt.Printf("%s Failed to load KEV database: %v\n", output.Yellow("⚠️"), err)
if !strings.Contains(r, ":") { fmt.Println(output.Dim(" CVE lookups will use NVD API only (slower)"))
r = r + ":53"
}
resolvers = append(resolvers, r)
} }
fmt.Println()
} }
} }
if len(resolvers) == 0 {
resolvers = config.DefaultResolvers
}
// Parse custom ports // Parse custom resolvers and ports using helpers
var customPorts []int resolvers := ParseResolvers(cfg.Resolvers)
if cfg.Ports != "" { customPorts := ParsePorts(cfg.Ports)
for _, p := range strings.Split(cfg.Ports, ",") {
p = strings.TrimSpace(p) // Initialize stealth manager
if port, err := strconv.Atoi(p); err == nil && port > 0 && port < 65536 { stealthMode := stealth.ParseMode(cfg.StealthMode)
customPorts = append(customPorts, port) stealthMgr := stealth.NewManager(stealthMode)
}
} // Adjust concurrency based on stealth mode
} effectiveConcurrency := stealthMgr.GetEffectiveConcurrency(cfg.Concurrency)
if len(customPorts) == 0 {
customPorts = []int{80, 443, 8080, 8443}
}
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
output.PrintBanner() output.PrintBanner()
output.PrintSection("🎯", "TARGET CONFIGURATION") output.PrintSection("🎯", "TARGET CONFIGURATION")
output.PrintSubSection(fmt.Sprintf("%s %s", output.Dim("Target:"), output.BoldCyan(cfg.Domain))) output.PrintSubSection(fmt.Sprintf("%s %s", output.Dim("Target:"), output.BoldCyan(cfg.Domain)))
// Show stealth status
if stealthMode != stealth.ModeOff {
stealthColor := output.Yellow
if stealthMode >= stealth.ModeAggressive {
stealthColor = output.Red
}
output.PrintSubSection(fmt.Sprintf("%s %s %s %s",
output.Dim("Stealth:"), stealthColor(stealthMgr.GetModeName()),
output.Dim("Effective Threads:"), output.BoldGreen(fmt.Sprintf("%d", effectiveConcurrency))))
}
output.PrintSubSection(fmt.Sprintf("%s %s %s %s %s %s", output.PrintSubSection(fmt.Sprintf("%s %s %s %s %s %s",
output.Dim("Threads:"), output.BoldGreen(fmt.Sprintf("%d", cfg.Concurrency)), output.Dim("Threads:"), output.BoldGreen(fmt.Sprintf("%d", effectiveConcurrency)),
output.Dim("Timeout:"), output.Yellow(fmt.Sprintf("%ds", cfg.Timeout)), output.Dim("Timeout:"), output.Yellow(fmt.Sprintf("%ds", cfg.Timeout)),
output.Dim("Resolvers:"), output.Blue(fmt.Sprintf("%d", len(resolvers))))) output.Dim("Resolvers:"), output.Blue(fmt.Sprintf("%d", len(resolvers)))))
if !cfg.NoPorts { if !cfg.NoPorts {
@@ -186,49 +188,72 @@ func Run(cfg config.Config) {
} }
}() }()
// Wildcard detection (always run for JSON output accuracy)
var wildcardInfo *dns.WildcardInfo
wildcardDetector := dns.NewWildcardDetector(resolvers, cfg.Timeout)
wildcardInfo = wildcardDetector.Detect(cfg.Domain)
// DNS Brute-force // DNS Brute-force
var bruteWg sync.WaitGroup var bruteWg sync.WaitGroup
if !cfg.NoBrute { if !cfg.NoBrute {
// Check wildcard // Display wildcard detection results
wildcardIPs := dns.CheckWildcard(cfg.Domain, resolvers)
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
if len(wildcardIPs) > 0 { if wildcardInfo.IsWildcard {
output.PrintSubSection(fmt.Sprintf("%s Wildcard DNS: %s", output.Yellow("⚠"), output.BoldYellow("DETECTED"))) output.PrintSubSection(fmt.Sprintf("%s Wildcard DNS: %s (confidence: %.0f%%)",
output.Yellow("⚠"), output.BoldYellow("DETECTED"), wildcardInfo.Confidence*100))
if len(wildcardInfo.WildcardIPs) > 0 {
ips := wildcardInfo.WildcardIPs
if len(ips) > 3 {
ips = ips[:3]
}
output.PrintSubSection(fmt.Sprintf(" %s Wildcard IPs: %s", output.Dim("→"), output.Yellow(strings.Join(ips, ", "))))
}
if wildcardInfo.HTTPStatusCode > 0 {
output.PrintSubSection(fmt.Sprintf(" %s HTTP response: %d (%d bytes)",
output.Dim("→"), wildcardInfo.HTTPStatusCode, wildcardInfo.HTTPBodySize))
}
} else { } else {
output.PrintSubSection(fmt.Sprintf("%s Wildcard DNS: %s", output.Green("✓"), output.Green("not detected"))) output.PrintSubSection(fmt.Sprintf("%s Wildcard DNS: %s", output.Green("✓"), output.Green("not detected")))
} }
} }
// Brute-force // Brute-force with wildcard filtering
semaphore := make(chan struct{}, cfg.Concurrency) semaphore := make(chan struct{}, effectiveConcurrency)
for _, word := range wordlist { wildcardIPSet := make(map[string]bool)
if wildcardInfo != nil {
for _, ip := range wildcardInfo.WildcardIPs {
wildcardIPSet[ip] = true
}
}
// Shuffle wordlist if stealth mode randomization is enabled
shuffledWordlist := stealthMgr.ShuffleSlice(wordlist)
for _, word := range shuffledWordlist {
bruteWg.Add(1) bruteWg.Add(1)
go func(word string) { go func(word string) {
defer bruteWg.Done() defer bruteWg.Done()
semaphore <- struct{}{} semaphore <- struct{}{}
defer func() { <-semaphore }() defer func() { <-semaphore }()
// Apply stealth delay
stealthMgr.Wait()
subdomain := fmt.Sprintf("%s.%s", word, cfg.Domain) subdomain := fmt.Sprintf("%s.%s", word, cfg.Domain)
ips := dns.ResolveSubdomain(subdomain, resolvers, cfg.Timeout) ips := dns.ResolveSubdomain(subdomain, resolvers, cfg.Timeout)
if len(ips) > 0 { if len(ips) > 0 {
// Check if wildcard // Check if ALL IPs are wildcard IPs
isWildcard := false allWildcard := true
if len(wildcardIPs) > 0 { for _, ip := range ips {
for _, ip := range ips { if !wildcardIPSet[ip] {
for _, wip := range wildcardIPs { allWildcard = false
if ip == wip { break
isWildcard = true
break
}
}
if isWildcard {
break
}
} }
} }
if !isWildcard { // Only add if not all IPs are wildcards
if !allWildcard || len(wildcardIPSet) == 0 {
seenMu.Lock() seenMu.Lock()
if !seen[subdomain] { if !seen[subdomain] {
seen[subdomain] = true seen[subdomain] = true
@@ -267,18 +292,24 @@ func Run(cfg config.Config) {
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
output.PrintEndSection() output.PrintEndSection()
output.PrintSection("🌐", "DNS RESOLUTION") output.PrintSection("🌐", "DNS RESOLUTION")
output.PrintSubSection(fmt.Sprintf("Resolving %s subdomains...", output.BoldCyan(fmt.Sprintf("%d", len(subdomains)))))
} }
// Create progress bar for DNS resolution
dnsBar := progress.New(len(subdomains), "DNS", cfg.Silent || cfg.JsonOutput)
var resolveWg sync.WaitGroup var resolveWg sync.WaitGroup
semaphore := make(chan struct{}, cfg.Concurrency) dnsSemaphore := make(chan struct{}, effectiveConcurrency)
for _, subdomain := range subdomains { for _, subdomain := range subdomains {
resolveWg.Add(1) resolveWg.Add(1)
go func(sub string) { go func(sub string) {
defer resolveWg.Done() defer resolveWg.Done()
semaphore <- struct{}{} defer dnsBar.Increment()
defer func() { <-semaphore }() dnsSemaphore <- struct{}{}
defer func() { <-dnsSemaphore }()
// Apply stealth delay for DNS
stealthMgr.Wait()
ips := dns.ResolveSubdomain(sub, resolvers, cfg.Timeout) ips := dns.ResolveSubdomain(sub, resolvers, cfg.Timeout)
if len(ips) > 0 { if len(ips) > 0 {
@@ -332,22 +363,36 @@ func Run(cfg config.Config) {
}(subdomain) }(subdomain)
} }
resolveWg.Wait() resolveWg.Wait()
dnsBar.Finish()
// HTTP Probing // HTTP Probing
if !cfg.NoProbe && len(results) > 0 { if !cfg.NoProbe && len(results) > 0 {
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
output.PrintEndSection() output.PrintEndSection()
output.PrintSection("🌍", "HTTP PROBING & SECURITY CHECKS") output.PrintSection("🌍", "HTTP PROBING & SECURITY CHECKS")
output.PrintSubSection(fmt.Sprintf("Probing %s subdomains with %s parallel checks...", output.BoldCyan(fmt.Sprintf("%d", len(results))), output.BoldGreen("13")))
} }
// Create progress bar and rate limiter for HTTP probing
httpBar := progress.New(len(results), "HTTP", cfg.Silent || cfg.JsonOutput)
httpLimiter := ratelimit.NewHostRateLimiter(ratelimit.DefaultConfig())
httpSemaphore := make(chan struct{}, effectiveConcurrency)
var probeWg sync.WaitGroup var probeWg sync.WaitGroup
for sub := range results { for sub := range results {
probeWg.Add(1) probeWg.Add(1)
go func(subdomain string) { go func(subdomain string) {
defer probeWg.Done() defer probeWg.Done()
semaphore <- struct{}{} defer httpBar.Increment()
defer func() { <-semaphore }() httpSemaphore <- struct{}{}
defer func() { <-httpSemaphore }()
// Apply stealth delay and host-specific throttling
stealthMgr.Wait()
stealthMgr.WaitForHost(subdomain)
// Apply adaptive rate limiting
limiter := httpLimiter.Get(subdomain)
limiter.Wait()
// Use shared client for connection pooling // Use shared client for connection pooling
client := gohttp.GetSharedClient(cfg.Timeout) client := gohttp.GetSharedClient(cfg.Timeout)
@@ -542,6 +587,16 @@ func Run(cfg config.Config) {
}(sub) }(sub)
} }
probeWg.Wait() probeWg.Wait()
httpBar.Finish()
// Log rate limiting stats if verbose
if cfg.Verbose && !cfg.JsonOutput {
hosts, requests, errors := httpLimiter.GetStats()
if errors > 0 {
output.PrintSubSection(fmt.Sprintf("%s Rate limiting: %d hosts, %d requests, %d errors",
output.Yellow("⚠️"), hosts, requests, errors))
}
}
} }
// Port Scanning // Port Scanning
@@ -549,9 +604,17 @@ func Run(cfg config.Config) {
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
output.PrintEndSection() output.PrintEndSection()
output.PrintSection("🔌", "PORT SCANNING") output.PrintSection("🔌", "PORT SCANNING")
output.PrintSubSection(fmt.Sprintf("Scanning %s ports on %s hosts...", output.BoldMagenta(fmt.Sprintf("%d", len(customPorts))), output.BoldCyan(fmt.Sprintf("%d", len(results)))))
} }
// Count hosts with IPs
hostCount := 0
for _, result := range results {
if len(result.IPs) > 0 {
hostCount++
}
}
portBar := progress.New(hostCount, "Ports", cfg.Silent || cfg.JsonOutput)
var portWg sync.WaitGroup var portWg sync.WaitGroup
for sub, result := range results { for sub, result := range results {
@@ -561,6 +624,7 @@ func Run(cfg config.Config) {
portWg.Add(1) portWg.Add(1)
go func(subdomain string, ip string) { go func(subdomain string, ip string) {
defer portWg.Done() defer portWg.Done()
defer portBar.Increment()
openPorts := ScanPorts(ip, customPorts, cfg.Timeout) openPorts := ScanPorts(ip, customPorts, cfg.Timeout)
resultsMu.Lock() resultsMu.Lock()
if r, ok := results[subdomain]; ok { if r, ok := results[subdomain]; ok {
@@ -570,6 +634,7 @@ func Run(cfg config.Config) {
}(sub, result.IPs[0]) }(sub, result.IPs[0])
} }
portWg.Wait() portWg.Wait()
portBar.Finish()
} }
// Subdomain Takeover Check // Subdomain Takeover Check
@@ -578,14 +643,15 @@ func Run(cfg config.Config) {
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
output.PrintEndSection() output.PrintEndSection()
output.PrintSection("🎯", "SUBDOMAIN TAKEOVER") output.PrintSection("🎯", "SUBDOMAIN TAKEOVER")
output.PrintSubSection(fmt.Sprintf("Checking %s fingerprints against %s subdomains...", output.BoldRed("110+"), output.BoldCyan(fmt.Sprintf("%d", len(results)))))
} }
takeoverBar := progress.New(len(results), "Takeover", cfg.Silent || cfg.JsonOutput)
var takeoverWg sync.WaitGroup var takeoverWg sync.WaitGroup
for sub := range results { for sub := range results {
takeoverWg.Add(1) takeoverWg.Add(1)
go func(subdomain string) { go func(subdomain string) {
defer takeoverWg.Done() defer takeoverWg.Done()
defer takeoverBar.Increment()
if takeover := CheckTakeover(subdomain, cfg.Timeout); takeover != "" { if takeover := CheckTakeover(subdomain, cfg.Timeout); takeover != "" {
resultsMu.Lock() resultsMu.Lock()
if r, ok := results[subdomain]; ok { if r, ok := results[subdomain]; ok {
@@ -593,15 +659,13 @@ func Run(cfg config.Config) {
} }
resultsMu.Unlock() resultsMu.Unlock()
atomic.AddInt32(&takeoverCount, 1) atomic.AddInt32(&takeoverCount, 1)
if !cfg.JsonOutput {
output.PrintSubSection(fmt.Sprintf("%s %s → %s", output.BgRed(" TAKEOVER "), output.BoldWhite(subdomain), output.BoldRed(takeover)))
}
} }
}(sub) }(sub)
} }
takeoverWg.Wait() takeoverWg.Wait()
takeoverBar.Finish()
if takeoverCount > 0 && !cfg.JsonOutput { if takeoverCount > 0 && !cfg.Silent && !cfg.JsonOutput {
output.PrintSubSection(fmt.Sprintf("%s Found %s potential takeover(s)!", output.Red("⚠"), output.BoldRed(fmt.Sprintf("%d", takeoverCount)))) output.PrintSubSection(fmt.Sprintf("%s Found %s potential takeover(s)!", output.Red("⚠"), output.BoldRed(fmt.Sprintf("%d", takeoverCount))))
} }
if !cfg.Silent && !cfg.JsonOutput { if !cfg.Silent && !cfg.JsonOutput {
@@ -638,45 +702,62 @@ func Run(cfg config.Config) {
aiSemaphore := make(chan struct{}, 5) // Limit concurrent AI requests aiSemaphore := make(chan struct{}, 5) // Limit concurrent AI requests
for sub, result := range results { for sub, result := range results {
// Only analyze interesting findings // Determine what types of analysis to perform
shouldAnalyze := false shouldAnalyzeVulns := false
shouldAnalyzeCVE := len(result.Tech) > 0 // CVE for ALL subdomains with tech
// Analyze JS files if found // Analyze JS files if found
if len(result.JSFiles) > 0 || len(result.JSSecrets) > 0 { if len(result.JSFiles) > 0 || len(result.JSSecrets) > 0 {
shouldAnalyze = true shouldAnalyzeVulns = true
} }
// Analyze if vulnerabilities detected // Analyze if vulnerabilities detected
if result.OpenRedirect || result.CORSMisconfig != "" || if result.OpenRedirect || result.CORSMisconfig != "" ||
len(result.DangerousMethods) > 0 || result.GitExposed || len(result.DangerousMethods) > 0 || result.GitExposed ||
result.SvnExposed || len(result.BackupFiles) > 0 { result.SvnExposed || len(result.BackupFiles) > 0 {
shouldAnalyze = true shouldAnalyzeVulns = true
} }
// Analyze takeovers // Analyze takeovers
if result.Takeover != "" { if result.Takeover != "" {
shouldAnalyze = true shouldAnalyzeVulns = true
} }
// Deep analysis mode: analyze everything // Deep analysis mode: analyze everything
if cfg.AIDeepAnalysis { if cfg.AIDeepAnalysis {
shouldAnalyze = true shouldAnalyzeVulns = true
} }
if !shouldAnalyze { // Skip if nothing to analyze
if !shouldAnalyzeVulns && !shouldAnalyzeCVE {
continue continue
} }
analyzeVulns := shouldAnalyzeVulns // Capture for goroutine
aiWg.Add(1) aiWg.Add(1)
go func(subdomain string, r *config.SubdomainResult) { go func(subdomain string, r *config.SubdomainResult, doVulnAnalysis bool) {
defer aiWg.Done() defer aiWg.Done()
aiSemaphore <- struct{}{} aiSemaphore <- struct{}{}
defer func() { <-aiSemaphore }() defer func() { <-aiSemaphore }()
var aiResults []*ai.AnalysisResult var aiResults []*ai.AnalysisResult
// Analyze JavaScript if present // Filter JS secrets using AI before analysis
if len(r.JSFiles) > 0 && len(r.JSSecrets) > 0 { if len(r.JSSecrets) > 0 {
if filteredSecrets, err := aiClient.FilterSecrets(r.JSSecrets); err == nil && len(filteredSecrets) > 0 {
resultsMu.Lock()
r.JSSecrets = filteredSecrets // Replace with AI-filtered secrets
resultsMu.Unlock()
} else if err == nil {
// No real secrets found after filtering
resultsMu.Lock()
r.JSSecrets = nil
resultsMu.Unlock()
}
}
// Analyze JavaScript if present (only if vuln analysis enabled)
if doVulnAnalysis && len(r.JSFiles) > 0 && len(r.JSSecrets) > 0 {
// Build context from secrets // Build context from secrets
jsContext := strings.Join(r.JSSecrets, "\n") jsContext := strings.Join(r.JSSecrets, "\n")
if analysis, err := aiClient.AnalyzeJavaScript(jsContext); err == nil { if analysis, err := aiClient.AnalyzeJavaScript(jsContext); err == nil {
@@ -684,15 +765,15 @@ func Run(cfg config.Config) {
} }
} }
// Analyze HTTP response for misconfigurations // Analyze HTTP response for misconfigurations (only if vuln analysis enabled)
if r.StatusCode > 0 && (len(r.MissingHeaders) > 3 || r.GitExposed || r.SvnExposed) { if doVulnAnalysis && r.StatusCode > 0 && (len(r.MissingHeaders) > 3 || r.GitExposed || r.SvnExposed) {
bodyContext := r.Title bodyContext := r.Title
if analysis, err := aiClient.AnalyzeHTTPResponse(subdomain, r.StatusCode, r.Headers, bodyContext); err == nil { if analysis, err := aiClient.AnalyzeHTTPResponse(subdomain, r.StatusCode, r.Headers, bodyContext); err == nil {
aiResults = append(aiResults, analysis) aiResults = append(aiResults, analysis)
} }
} }
// CVE matching for detected technologies // CVE matching for detected technologies (always done if tech detected)
if len(r.Tech) > 0 { if len(r.Tech) > 0 {
for _, tech := range r.Tech { for _, tech := range r.Tech {
if cve, err := aiClient.CVEMatch(tech, ""); err == nil && cve != "" { if cve, err := aiClient.CVEMatch(tech, ""); err == nil && cve != "" {
@@ -748,7 +829,7 @@ func Run(cfg config.Config) {
output.Dim(fmt.Sprintf("%d findings", len(r.AIFindings))))) output.Dim(fmt.Sprintf("%d findings", len(r.AIFindings)))))
} }
} }
}(sub, result) }(sub, result, analyzeVulns)
} }
aiWg.Wait() aiWg.Wait()
@@ -795,543 +876,33 @@ func Run(cfg config.Config) {
results = filtered results = filtered
} }
// Sort subdomains // JSON output to stdout (structured report format)
var sortedSubs []string
for sub := range results {
sortedSubs = append(sortedSubs, sub)
}
sort.Strings(sortedSubs)
// JSON output to stdout
if cfg.JsonOutput { if cfg.JsonOutput {
var resultList []*config.SubdomainResult // Build structured JSON report with metadata
for _, sub := range sortedSubs { reportBuilder := output.NewReportBuilder(cfg.Domain, cfg)
resultList = append(resultList, results[sub])
// Set wildcard info if available
if wildcardInfo != nil {
reportBuilder.SetWildcard(
wildcardInfo.IsWildcard,
wildcardInfo.WildcardIPs,
wildcardInfo.WildcardCNAME,
wildcardInfo.HTTPStatusCode,
wildcardInfo.Confidence,
)
} }
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ") // Finalize and output the report
encoder.Encode(resultList) reportBuilder.Finalize(results)
reportBuilder.WriteJSON(os.Stdout, true)
return return
} }
// Print results // Print results using output module
elapsed := time.Since(startTime) PrintResults(results, startTime, takeoverCount)
// Count statistics
var activeCount, vulnCount, cloudCount int
for _, r := range results {
if r.StatusCode >= 200 && r.StatusCode < 400 {
activeCount++
}
if r.OpenRedirect || r.CORSMisconfig != "" || len(r.DangerousMethods) > 0 || r.GitExposed || r.SvnExposed || len(r.BackupFiles) > 0 {
vulnCount++
}
if r.CloudProvider != "" {
cloudCount++
}
}
// Summary box
fmt.Println()
fmt.Println(output.BoldCyan("╔══════════════════════════════════════════════════════════════════════════════╗"))
fmt.Println(output.BoldCyan("║") + " " + output.BoldWhite("📊 SCAN SUMMARY") + " " + output.BoldCyan("║"))
fmt.Println(output.BoldCyan("╠══════════════════════════════════════════════════════════════════════════════╣"))
fmt.Printf("%s %-20s %s %-20s %s %-20s %s\n",
output.BoldCyan("║"),
fmt.Sprintf("🌐 Total: %s", output.BoldCyan(fmt.Sprintf("%d", len(results)))),
output.Dim("|"),
fmt.Sprintf("✅ Active: %s", output.BoldGreen(fmt.Sprintf("%d", activeCount))),
output.Dim("|"),
fmt.Sprintf("⏱️ Time: %s", output.BoldYellow(fmt.Sprintf("%.1fs", elapsed.Seconds()))),
output.BoldCyan("║"))
fmt.Printf("%s %-20s %s %-20s %s %-20s %s\n",
output.BoldCyan("║"),
fmt.Sprintf("⚠️ Vulns: %s", output.BoldRed(fmt.Sprintf("%d", vulnCount))),
output.Dim("|"),
fmt.Sprintf("☁️ Cloud: %s", output.Blue(fmt.Sprintf("%d", cloudCount))),
output.Dim("|"),
fmt.Sprintf("🎯 Takeover: %s", output.BoldRed(fmt.Sprintf("%d", takeoverCount))),
output.BoldCyan("║"))
fmt.Println(output.BoldCyan("╚══════════════════════════════════════════════════════════════════════════════╝"))
fmt.Println()
fmt.Println(output.BoldCyan("═══════════════════════════════════════════════════════════════════════════════"))
for _, sub := range sortedSubs {
r := results[sub]
// Color code by status
var statusColor func(a ...interface{}) string
var statusIcon string
if r.StatusCode >= 200 && r.StatusCode < 300 {
statusColor = output.Green
statusIcon = "●"
} else if r.StatusCode >= 300 && r.StatusCode < 400 {
statusColor = output.Yellow
statusIcon = "◐"
} else if r.StatusCode >= 400 {
statusColor = output.Red
statusIcon = "○"
} else {
statusColor = output.Blue
statusIcon = "◌"
}
// Line 1: Subdomain name with status (modern box style)
statusBadge := ""
if r.StatusCode > 0 {
statusBadge = fmt.Sprintf(" %s", statusColor(fmt.Sprintf("[%d]", r.StatusCode)))
}
// Response time badge
timeBadge := ""
if r.ResponseMs > 0 {
if r.ResponseMs < 200 {
timeBadge = fmt.Sprintf(" %s", output.Green(fmt.Sprintf("⚡%dms", r.ResponseMs)))
} else if r.ResponseMs < 500 {
timeBadge = fmt.Sprintf(" %s", output.Yellow(fmt.Sprintf("⏱️%dms", r.ResponseMs)))
} else {
timeBadge = fmt.Sprintf(" %s", output.Red(fmt.Sprintf("🐢%dms", r.ResponseMs)))
}
}
fmt.Printf("\n%s %s%s%s\n", statusColor(statusIcon), output.BoldCyan(sub), statusBadge, timeBadge)
// Line 2: IPs
if len(r.IPs) > 0 {
ips := r.IPs
if len(ips) > 3 {
ips = ips[:3]
}
fmt.Printf(" %s %s\n", output.Dim("IP:"), output.White(strings.Join(ips, ", ")))
}
// Line 3: CNAME
if r.CNAME != "" {
fmt.Printf(" %s %s\n", output.Dim("CNAME:"), output.Blue(r.CNAME))
}
// Line 4: Location + ASN
if r.Country != "" || r.City != "" || r.ASN != "" {
loc := ""
if r.City != "" && r.Country != "" {
loc = r.City + ", " + r.Country
} else if r.Country != "" {
loc = r.Country
} else if r.City != "" {
loc = r.City
}
asnStr := ""
if r.ASN != "" {
asnStr = r.ASN
if len(asnStr) > 40 {
asnStr = asnStr[:37] + "..."
}
}
if loc != "" && asnStr != "" {
fmt.Printf(" Location: %s | %s\n", output.Cyan(loc), output.Blue(asnStr))
} else if loc != "" {
fmt.Printf(" Location: %s\n", output.Cyan(loc))
} else if asnStr != "" {
fmt.Printf(" ASN: %s\n", output.Blue(asnStr))
}
}
// Line 5: PTR
if r.PTR != "" {
fmt.Printf(" PTR: %s\n", output.Magenta(r.PTR))
}
// Line 6: HTTP Info (Title, Size)
if r.Title != "" || r.ContentLength > 0 {
httpInfo := " HTTP: "
if r.Title != "" {
title := r.Title
if len(title) > 50 {
title = title[:47] + "..."
}
httpInfo += fmt.Sprintf("\"%s\"", title)
}
if r.ContentLength > 0 {
sizeStr := ""
if r.ContentLength > 1024*1024 {
sizeStr = fmt.Sprintf("%.1fMB", float64(r.ContentLength)/(1024*1024))
} else if r.ContentLength > 1024 {
sizeStr = fmt.Sprintf("%.1fKB", float64(r.ContentLength)/1024)
} else {
sizeStr = fmt.Sprintf("%dB", r.ContentLength)
}
if r.Title != "" {
httpInfo += fmt.Sprintf(" (%s)", sizeStr)
} else {
httpInfo += sizeStr
}
}
fmt.Println(httpInfo)
}
// Line 7: Redirect
if r.RedirectURL != "" {
redirectURL := r.RedirectURL
if len(redirectURL) > 60 {
redirectURL = redirectURL[:57] + "..."
}
fmt.Printf(" Redirect: %s\n", output.Yellow(redirectURL))
}
// Line 8: Tech + Server
if len(r.Tech) > 0 || r.Server != "" {
techMap := make(map[string]bool)
var uniqueTech []string
for _, t := range r.Tech {
if !techMap[t] {
techMap[t] = true
uniqueTech = append(uniqueTech, t)
}
}
if len(uniqueTech) > 5 {
uniqueTech = uniqueTech[:5]
}
if len(uniqueTech) > 0 {
fmt.Printf(" Tech: %s\n", output.Yellow(strings.Join(uniqueTech, ", ")))
}
}
// Line 9: Security (WAF, TLS)
var securityInfo []string
if r.WAF != "" {
securityInfo = append(securityInfo, fmt.Sprintf("WAF: %s", output.Red(r.WAF)))
}
if r.TLSVersion != "" {
securityInfo = append(securityInfo, fmt.Sprintf("TLS: %s", output.Cyan(r.TLSVersion)))
}
if len(securityInfo) > 0 {
fmt.Printf(" Security: %s\n", strings.Join(securityInfo, " | "))
}
// Line 10: Ports
if len(r.Ports) > 0 {
var portStrs []string
for _, p := range r.Ports {
portStrs = append(portStrs, fmt.Sprintf("%d", p))
}
fmt.Printf(" Ports: %s\n", output.Magenta(strings.Join(portStrs, ", ")))
}
// Line 11: Extra files
var extras []string
if r.RobotsTxt {
extras = append(extras, "robots.txt")
}
if r.SitemapXml {
extras = append(extras, "sitemap.xml")
}
if r.FaviconHash != "" {
extras = append(extras, fmt.Sprintf("favicon:%s", r.FaviconHash[:8]))
}
if len(extras) > 0 {
fmt.Printf(" Files: %s\n", output.Green(strings.Join(extras, ", ")))
}
// Line 12: DNS Records
if len(r.MXRecords) > 0 {
mx := r.MXRecords
if len(mx) > 2 {
mx = mx[:2]
}
fmt.Printf(" MX: %s\n", strings.Join(mx, ", "))
}
// Line 13: Security Headers
if len(r.MissingHeaders) > 0 && len(r.MissingHeaders) < 7 {
// Only show if some headers are present (not all missing)
if len(r.SecurityHeaders) > 0 {
fmt.Printf(" Headers: %s | Missing: %s\n",
output.Green(strings.Join(r.SecurityHeaders, ", ")),
output.Yellow(strings.Join(r.MissingHeaders, ", ")))
}
} else if len(r.SecurityHeaders) > 0 {
fmt.Printf(" Headers: %s\n", output.Green(strings.Join(r.SecurityHeaders, ", ")))
}
// Line 14: Cloud Provider
if r.CloudProvider != "" {
fmt.Printf(" Cloud: %s\n", output.Cyan(r.CloudProvider))
}
// Line 15: Email Security (only for root domain)
if r.EmailSecurity != "" {
emailColor := output.Green
if r.EmailSecurity == "Weak" {
emailColor = output.Yellow
} else if r.EmailSecurity == "None" {
emailColor = output.Red
}
fmt.Printf(" Email: %s\n", emailColor(r.EmailSecurity))
}
// Line 16: TLS Alt Names
if len(r.TLSAltNames) > 0 {
altNames := r.TLSAltNames
if len(altNames) > 5 {
altNames = altNames[:5]
}
fmt.Printf(" TLS Alt: %s\n", output.Blue(strings.Join(altNames, ", ")))
}
// Line 17: S3 Buckets
if len(r.S3Buckets) > 0 {
for _, bucket := range r.S3Buckets {
if strings.Contains(bucket, "PUBLIC") {
fmt.Printf(" %s %s\n", output.Red("S3:"), output.Red(bucket))
} else {
fmt.Printf(" S3: %s\n", output.Yellow(bucket))
}
}
}
// Line 18: Security Issues (vulnerabilities found)
var vulns []string
if r.OpenRedirect {
vulns = append(vulns, "Open Redirect")
}
if r.CORSMisconfig != "" {
vulns = append(vulns, fmt.Sprintf("CORS: %s", r.CORSMisconfig))
}
if len(r.DangerousMethods) > 0 {
vulns = append(vulns, fmt.Sprintf("Methods: %s", strings.Join(r.DangerousMethods, ", ")))
}
if r.GitExposed {
vulns = append(vulns, ".git Exposed")
}
if r.SvnExposed {
vulns = append(vulns, ".svn Exposed")
}
if len(r.BackupFiles) > 0 {
files := r.BackupFiles
if len(files) > 3 {
files = files[:3]
}
vulns = append(vulns, fmt.Sprintf("Backup: %s", strings.Join(files, ", ")))
}
if len(vulns) > 0 {
fmt.Printf(" %s %s\n", output.Red("VULNS:"), output.Red(strings.Join(vulns, " | ")))
}
// Line 19: Discovery (admin panels, API endpoints)
var discoveries []string
if len(r.AdminPanels) > 0 {
panels := r.AdminPanels
if len(panels) > 5 {
panels = panels[:5]
}
discoveries = append(discoveries, fmt.Sprintf("Admin: %s", strings.Join(panels, ", ")))
}
if len(r.APIEndpoints) > 0 {
endpoints := r.APIEndpoints
if len(endpoints) > 5 {
endpoints = endpoints[:5]
}
discoveries = append(discoveries, fmt.Sprintf("API: %s", strings.Join(endpoints, ", ")))
}
if len(discoveries) > 0 {
fmt.Printf(" %s %s\n", output.Magenta("FOUND:"), output.Magenta(strings.Join(discoveries, " | ")))
}
// Line 20: JavaScript Analysis
if len(r.JSFiles) > 0 {
files := r.JSFiles
if len(files) > 3 {
files = files[:3]
}
fmt.Printf(" JS Files: %s\n", output.Blue(strings.Join(files, ", ")))
}
if len(r.JSSecrets) > 0 {
for _, secret := range r.JSSecrets {
fmt.Printf(" %s %s\n", output.Red("JS SECRET:"), output.Red(secret))
}
}
// Line 21: Takeover
if r.Takeover != "" {
fmt.Printf(" %s %s\n", output.BgRed(" TAKEOVER "), output.BoldRed(r.Takeover))
}
// Line 22: AI Findings
if len(r.AIFindings) > 0 {
severityColor := output.Cyan
severityLabel := "AI"
if r.AISeverity == "critical" {
severityColor = output.BoldRed
severityLabel = "AI:CRITICAL"
} else if r.AISeverity == "high" {
severityColor = output.Red
severityLabel = "AI:HIGH"
} else if r.AISeverity == "medium" {
severityColor = output.Yellow
severityLabel = "AI:MEDIUM"
}
for i, finding := range r.AIFindings {
if i == 0 {
fmt.Printf(" %s %s\n", severityColor(severityLabel+":"), finding)
} else {
fmt.Printf(" %s %s\n", output.Dim(" "), finding)
}
if i >= 4 { // Limit displayed findings
remaining := len(r.AIFindings) - 5
if remaining > 0 {
fmt.Printf(" %s (%d more findings...)\n", output.Dim(" "), remaining)
}
break
}
}
// Show model used
if r.AIModel != "" {
fmt.Printf(" %s model: %s\n", output.Dim(" "), output.Dim(r.AIModel))
}
}
// Line 23: CVE Findings
if len(r.CVEFindings) > 0 {
for _, cve := range r.CVEFindings {
fmt.Printf(" %s %s\n", output.BoldRed("CVE:"), output.Red(cve))
}
}
}
fmt.Println()
fmt.Println(output.BoldCyan("═══════════════════════════════════════════════════════════════════════════════"))
// Save output // Save output
if cfg.Output != "" { if cfg.Output != "" {
output.SaveOutput(cfg.Output, cfg.Format, results) output.SaveOutput(cfg.Output, cfg.Format, results)
} }
} }
// LoadWordlist loads words from a file
func LoadWordlist(path string) ([]string, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var words []string
scanner := bufio.NewScanner(file)
for scanner.Scan() {
word := strings.TrimSpace(scanner.Text())
if word != "" && !strings.HasPrefix(word, "#") {
words = append(words, word)
}
}
return words, scanner.Err()
}
// ScanPorts scans ports on an IP address
func ScanPorts(ip string, ports []int, timeout int) []int {
var openPorts []int
var mu sync.Mutex
var wg sync.WaitGroup
for _, port := range ports {
wg.Add(1)
go func(p int) {
defer wg.Done()
address := fmt.Sprintf("%s:%d", ip, p)
conn, err := net.DialTimeout("tcp", address, time.Duration(timeout)*time.Second)
if err == nil {
conn.Close()
mu.Lock()
openPorts = append(openPorts, p)
mu.Unlock()
}
}(port)
}
wg.Wait()
sort.Ints(openPorts)
return openPorts
}
// Helper functions for AI analysis
func countSubdomainsWithAI(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if len(r.AIFindings) > 0 {
count++
}
}
return count
}
func countActive(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if r.StatusCode >= 200 && r.StatusCode < 400 {
count++
}
}
return count
}
func countVulns(results map[string]*config.SubdomainResult) int {
count := 0
for _, r := range results {
if r.OpenRedirect || r.CORSMisconfig != "" || len(r.DangerousMethods) > 0 ||
r.GitExposed || r.SvnExposed || len(r.BackupFiles) > 0 {
count++
}
}
return count
}
func buildAISummary(results map[string]*config.SubdomainResult) string {
var summary strings.Builder
criticalCount := 0
highCount := 0
mediumCount := 0
for sub, r := range results {
if len(r.AIFindings) == 0 {
continue
}
switch r.AISeverity {
case "critical":
criticalCount++
summary.WriteString(fmt.Sprintf("\n[CRITICAL] %s:\n", sub))
case "high":
highCount++
summary.WriteString(fmt.Sprintf("\n[HIGH] %s:\n", sub))
case "medium":
mediumCount++
summary.WriteString(fmt.Sprintf("\n[MEDIUM] %s:\n", sub))
default:
continue // Skip low/info for summary
}
// Add first 3 findings
for i, finding := range r.AIFindings {
if i >= 3 {
break
}
summary.WriteString(fmt.Sprintf(" - %s\n", finding))
}
// Add CVE findings
if len(r.CVEFindings) > 0 {
summary.WriteString(" CVEs:\n")
for _, cve := range r.CVEFindings {
summary.WriteString(fmt.Sprintf(" - %s\n", cve))
}
}
}
header := fmt.Sprintf("Summary: %d critical, %d high, %d medium findings\n", criticalCount, highCount, mediumCount)
return header + summary.String()
}

View File

@@ -20,13 +20,12 @@ func CheckAdminPanels(subdomain string, timeout int) []string {
}, },
} }
// Common admin panel paths // Generic admin paths (common across all platforms)
// Note: Removed platform-specific paths like /wp-admin, /admin.php, /phpmyadmin
// These generate false positives on non-PHP/WordPress sites
paths := []string{ paths := []string{
"/admin", "/administrator", "/admin.php", "/admin.html", "/admin", "/administrator",
"/login", "/login.php", "/signin", "/auth", "/login", "/signin", "/auth",
"/wp-admin", "/wp-login.php",
"/phpmyadmin", "/pma", "/mysql",
"/cpanel", "/webmail",
"/manager", "/console", "/dashboard", "/manager", "/console", "/dashboard",
"/admin/login", "/user/login", "/admin/login", "/user/login",
} }
@@ -200,15 +199,16 @@ func CheckAPIEndpoints(subdomain string, timeout int) []string {
// WithClient versions for parallel execution // WithClient versions for parallel execution
func CheckAdminPanelsWithClient(subdomain string, client *http.Client) []string { func CheckAdminPanelsWithClient(subdomain string, client *http.Client) []string {
// Generic admin paths (common across all platforms)
paths := []string{ paths := []string{
"/admin", "/administrator", "/admin.php", "/admin.html", "/admin", "/administrator",
"/login", "/login.php", "/signin", "/auth", "/login", "/signin", "/auth",
"/wp-admin", "/wp-login.php",
"/phpmyadmin", "/pma", "/mysql",
"/cpanel", "/webmail",
"/manager", "/console", "/dashboard", "/manager", "/console", "/dashboard",
"/admin/login", "/user/login", "/admin/login", "/user/login",
} }
// Note: We removed platform-specific paths like /wp-admin, /admin.php, /login.php
// These generate false positives on non-PHP/WordPress sites
// The tech detection should be used to check platform-specific paths
var found []string var found []string
baseURLs := []string{ baseURLs := []string{

85
internal/sources/base.go Normal file
View File

@@ -0,0 +1,85 @@
package sources
import (
"context"
"fmt"
"io"
"net/http"
"regexp"
"strings"
"time"
)
// sharedClient is reused across all source fetches
var sharedClient = &http.Client{
Timeout: 30 * time.Second,
Transport: &http.Transport{
MaxIdleConns: 100,
MaxIdleConnsPerHost: 10,
IdleConnTimeout: 30 * time.Second,
},
}
// regexFetch performs a fetch and extracts subdomains using regex
// This reduces code duplication across many sources
func regexFetch(url string, domain string, timeout time.Duration) ([]string, error) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return []string{}, nil
}
req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/120.0.0.0 Safari/537.36")
resp, err := sharedClient.Do(req)
if err != nil {
return []string{}, nil
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return []string{}, nil
}
return extractSubdomains(string(body), domain), nil
}
// extractSubdomains extracts subdomains from text using regex
func extractSubdomains(text, domain string) []string {
// Compile regex once per call (could cache but domain changes)
pattern := fmt.Sprintf(`(?i)([a-z0-9][a-z0-9._-]*\.%s)`, regexp.QuoteMeta(domain))
re := regexp.MustCompile(pattern)
matches := re.FindAllStringSubmatch(text, -1)
seen := make(map[string]bool)
var subs []string
for _, match := range matches {
if len(match) > 1 {
name := strings.ToLower(match[1])
// Filter out invalid patterns
if !seen[name] && !strings.HasPrefix(name, ".") && strings.HasSuffix(name, domain) {
seen[name] = true
subs = append(subs, name)
}
}
}
return subs
}
// dedupeAndFilter filters subdomains and ensures they belong to the target domain
func dedupeAndFilter(subs []string, domain string) []string {
seen := make(map[string]bool)
var result []string
for _, sub := range subs {
sub = strings.ToLower(strings.TrimSpace(sub))
sub = strings.TrimPrefix(sub, "*.")
if sub != "" && !seen[sub] && strings.HasSuffix(sub, domain) {
seen[sub] = true
result = append(result, sub)
}
}
return result
}

460
internal/stealth/stealth.go Normal file
View File

@@ -0,0 +1,460 @@
package stealth
import (
"crypto/rand"
"math/big"
"net/http"
"sync"
"time"
)
// Mode defines the stealth level
type Mode int
const (
ModeOff Mode = iota // No stealth - maximum speed
ModeLight // Light stealth - reduced concurrency, basic delays
ModeModerate // Moderate - random delays, UA rotation, throttling
ModeAggressive // Aggressive - slow, distributed, evasive
ModeParanoid // Paranoid - ultra slow, maximum evasion
)
// Config holds stealth configuration
type Config struct {
Mode Mode
MinDelay time.Duration // Minimum delay between requests
MaxDelay time.Duration // Maximum delay (for randomization)
MaxReqPerSecond float64 // Rate limit per second
MaxReqPerHost int // Max concurrent requests per host
RotateUA bool // Rotate User-Agent
RandomizeOrder bool // Randomize request order
JitterPercent int // Jitter percentage (0-100)
DNSSpread bool // Spread DNS queries across resolvers
}
// Manager handles stealth operations
type Manager struct {
cfg Config
userAgents []string
uaIndex int
uaMutex sync.Mutex
hostLimiters map[string]*rateLimiter
hostMutex sync.RWMutex
globalLimiter *rateLimiter
}
// rateLimiter implements token bucket rate limiting
type rateLimiter struct {
tokens float64
maxTokens float64
refillRate float64 // tokens per second
lastRefill time.Time
mu sync.Mutex
}
// NewManager creates a new stealth manager
func NewManager(mode Mode) *Manager {
cfg := GetPreset(mode)
return NewManagerWithConfig(cfg)
}
// NewManagerWithConfig creates a manager with custom config
func NewManagerWithConfig(cfg Config) *Manager {
m := &Manager{
cfg: cfg,
userAgents: getUserAgents(),
hostLimiters: make(map[string]*rateLimiter),
}
if cfg.MaxReqPerSecond > 0 {
m.globalLimiter = newRateLimiter(cfg.MaxReqPerSecond, cfg.MaxReqPerSecond)
}
return m
}
// GetPreset returns configuration for a stealth mode
func GetPreset(mode Mode) Config {
switch mode {
case ModeLight:
return Config{
Mode: ModeLight,
MinDelay: 10 * time.Millisecond,
MaxDelay: 50 * time.Millisecond,
MaxReqPerSecond: 100,
MaxReqPerHost: 20,
RotateUA: true,
RandomizeOrder: false,
JitterPercent: 10,
DNSSpread: false,
}
case ModeModerate:
return Config{
Mode: ModeModerate,
MinDelay: 50 * time.Millisecond,
MaxDelay: 200 * time.Millisecond,
MaxReqPerSecond: 30,
MaxReqPerHost: 5,
RotateUA: true,
RandomizeOrder: true,
JitterPercent: 30,
DNSSpread: true,
}
case ModeAggressive:
return Config{
Mode: ModeAggressive,
MinDelay: 200 * time.Millisecond,
MaxDelay: 1 * time.Second,
MaxReqPerSecond: 10,
MaxReqPerHost: 2,
RotateUA: true,
RandomizeOrder: true,
JitterPercent: 50,
DNSSpread: true,
}
case ModeParanoid:
return Config{
Mode: ModeParanoid,
MinDelay: 1 * time.Second,
MaxDelay: 5 * time.Second,
MaxReqPerSecond: 2,
MaxReqPerHost: 1,
RotateUA: true,
RandomizeOrder: true,
JitterPercent: 70,
DNSSpread: true,
}
default: // ModeOff
return Config{
Mode: ModeOff,
MinDelay: 0,
MaxDelay: 0,
MaxReqPerSecond: 0, // unlimited
MaxReqPerHost: 0, // unlimited
RotateUA: false,
RandomizeOrder: false,
JitterPercent: 0,
DNSSpread: false,
}
}
}
// Wait applies stealth delay before a request
func (m *Manager) Wait() {
if m.cfg.Mode == ModeOff {
return
}
// Apply rate limiting
if m.globalLimiter != nil {
m.globalLimiter.wait()
}
// Apply random delay
if m.cfg.MaxDelay > 0 {
delay := m.randomDelay()
time.Sleep(delay)
}
}
// WaitForHost applies per-host rate limiting
func (m *Manager) WaitForHost(host string) {
if m.cfg.Mode == ModeOff || m.cfg.MaxReqPerHost <= 0 {
return
}
m.hostMutex.Lock()
limiter, exists := m.hostLimiters[host]
if !exists {
limiter = newRateLimiter(float64(m.cfg.MaxReqPerHost), float64(m.cfg.MaxReqPerHost))
m.hostLimiters[host] = limiter
}
m.hostMutex.Unlock()
limiter.wait()
}
// GetUserAgent returns a User-Agent string (rotated if enabled)
func (m *Manager) GetUserAgent() string {
if !m.cfg.RotateUA || len(m.userAgents) == 0 {
return "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
m.uaMutex.Lock()
defer m.uaMutex.Unlock()
ua := m.userAgents[m.uaIndex]
m.uaIndex = (m.uaIndex + 1) % len(m.userAgents)
return ua
}
// GetRandomUserAgent returns a random User-Agent
func (m *Manager) GetRandomUserAgent() string {
if len(m.userAgents) == 0 {
return m.GetUserAgent()
}
idx := secureRandomInt(len(m.userAgents))
return m.userAgents[idx]
}
// ApplyToRequest applies stealth settings to an HTTP request
func (m *Manager) ApplyToRequest(req *http.Request) {
// Set User-Agent
req.Header.Set("User-Agent", m.GetUserAgent())
// Add realistic browser headers
if m.cfg.Mode >= ModeModerate {
req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8")
req.Header.Set("Accept-Language", m.getRandomAcceptLanguage())
req.Header.Set("Accept-Encoding", "gzip, deflate, br")
req.Header.Set("Connection", "keep-alive")
req.Header.Set("Upgrade-Insecure-Requests", "1")
req.Header.Set("Sec-Fetch-Dest", "document")
req.Header.Set("Sec-Fetch-Mode", "navigate")
req.Header.Set("Sec-Fetch-Site", "none")
req.Header.Set("Sec-Fetch-User", "?1")
req.Header.Set("Cache-Control", "max-age=0")
}
}
// SelectResolver picks a DNS resolver (distributed if enabled)
func (m *Manager) SelectResolver(resolvers []string, index int) string {
if len(resolvers) == 0 {
return "8.8.8.8:53"
}
if m.cfg.DNSSpread {
// Random selection for distribution
return resolvers[secureRandomInt(len(resolvers))]
}
// Sequential selection
return resolvers[index%len(resolvers)]
}
// ShuffleSlice randomizes slice order if enabled
func (m *Manager) ShuffleSlice(items []string) []string {
if !m.cfg.RandomizeOrder || len(items) <= 1 {
return items
}
// Fisher-Yates shuffle
shuffled := make([]string, len(items))
copy(shuffled, items)
for i := len(shuffled) - 1; i > 0; i-- {
j := secureRandomInt(i + 1)
shuffled[i], shuffled[j] = shuffled[j], shuffled[i]
}
return shuffled
}
// GetEffectiveConcurrency returns adjusted concurrency for stealth mode
func (m *Manager) GetEffectiveConcurrency(requested int) int {
switch m.cfg.Mode {
case ModeLight:
return min(requested, 100)
case ModeModerate:
return min(requested, 30)
case ModeAggressive:
return min(requested, 10)
case ModeParanoid:
return min(requested, 3)
default:
return requested
}
}
// GetConfig returns current stealth configuration
func (m *Manager) GetConfig() Config {
return m.cfg
}
// GetModeName returns the name of the stealth mode
func (m *Manager) GetModeName() string {
return ModeName(m.cfg.Mode)
}
// ModeName returns human-readable mode name
func ModeName(mode Mode) string {
switch mode {
case ModeLight:
return "light"
case ModeModerate:
return "moderate"
case ModeAggressive:
return "aggressive"
case ModeParanoid:
return "paranoid"
default:
return "off"
}
}
// ParseMode converts string to Mode
func ParseMode(s string) Mode {
switch s {
case "light", "1":
return ModeLight
case "moderate", "medium", "2":
return ModeModerate
case "aggressive", "3":
return ModeAggressive
case "paranoid", "4":
return ModeParanoid
default:
return ModeOff
}
}
// randomDelay returns a random delay with jitter
func (m *Manager) randomDelay() time.Duration {
if m.cfg.MaxDelay <= m.cfg.MinDelay {
return m.cfg.MinDelay
}
// Calculate range
rangeNs := int64(m.cfg.MaxDelay - m.cfg.MinDelay)
randomNs := secureRandomInt64(rangeNs)
delay := m.cfg.MinDelay + time.Duration(randomNs)
// Apply jitter
if m.cfg.JitterPercent > 0 {
jitterRange := int64(delay) * int64(m.cfg.JitterPercent) / 100
jitter := secureRandomInt64(jitterRange*2) - jitterRange
delay = time.Duration(int64(delay) + jitter)
if delay < 0 {
delay = m.cfg.MinDelay
}
}
return delay
}
func (m *Manager) getRandomAcceptLanguage() string {
languages := []string{
"en-US,en;q=0.9",
"en-GB,en;q=0.9",
"en-US,en;q=0.9,es;q=0.8",
"de-DE,de;q=0.9,en;q=0.8",
"fr-FR,fr;q=0.9,en;q=0.8",
"es-ES,es;q=0.9,en;q=0.8",
"it-IT,it;q=0.9,en;q=0.8",
"pt-BR,pt;q=0.9,en;q=0.8",
"nl-NL,nl;q=0.9,en;q=0.8",
"ja-JP,ja;q=0.9,en;q=0.8",
}
return languages[secureRandomInt(len(languages))]
}
// Rate limiter implementation
func newRateLimiter(maxTokens, refillRate float64) *rateLimiter {
return &rateLimiter{
tokens: maxTokens,
maxTokens: maxTokens,
refillRate: refillRate,
lastRefill: time.Now(),
}
}
func (rl *rateLimiter) wait() {
rl.mu.Lock()
defer rl.mu.Unlock()
// Refill tokens based on elapsed time
now := time.Now()
elapsed := now.Sub(rl.lastRefill).Seconds()
rl.tokens += elapsed * rl.refillRate
if rl.tokens > rl.maxTokens {
rl.tokens = rl.maxTokens
}
rl.lastRefill = now
// Wait if no tokens available
if rl.tokens < 1 {
waitTime := time.Duration((1 - rl.tokens) / rl.refillRate * float64(time.Second))
rl.mu.Unlock()
time.Sleep(waitTime)
rl.mu.Lock()
rl.tokens = 0
} else {
rl.tokens--
}
}
// Secure random helpers
func secureRandomInt(max int) int {
if max <= 0 {
return 0
}
n, err := rand.Int(rand.Reader, big.NewInt(int64(max)))
if err != nil {
return 0
}
return int(n.Int64())
}
func secureRandomInt64(max int64) int64 {
if max <= 0 {
return 0
}
n, err := rand.Int(rand.Reader, big.NewInt(max))
if err != nil {
return 0
}
return n.Int64()
}
func min(a, b int) int {
if a < b {
return a
}
return b
}
// User-Agent pool
func getUserAgents() []string {
return []string{
// Chrome Windows
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
// Chrome macOS
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
// Firefox Windows
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:119.0) Gecko/20100101 Firefox/119.0",
// Firefox macOS
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:120.0) Gecko/20100101 Firefox/120.0",
// Safari macOS
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Safari/605.1.15",
// Edge Windows
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0",
// Chrome Linux
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
// Firefox Linux
"Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0",
"Mozilla/5.0 (X11; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/120.0",
// Mobile Chrome Android
"Mozilla/5.0 (Linux; Android 14; SM-S918B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.144 Mobile Safari/537.36",
"Mozilla/5.0 (Linux; Android 13; Pixel 7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.144 Mobile Safari/537.36",
// Mobile Safari iOS
"Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1",
"Mozilla/5.0 (iPhone; CPU iPhone OS 17_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Mobile/15E148 Safari/604.1",
// Brave
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Brave/120",
// Opera
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 OPR/106.0.0.0",
}
}