mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-02-26 03:14:17 +00:00
Compare commits
2 Commits
master
...
fix/cross-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2921c92732 | ||
|
|
ac6e8a8a34 |
521
CONTRIBUTING.md
521
CONTRIBUTING.md
@@ -1,17 +1,21 @@
|
|||||||
# Contributing to FuzzForge 🤝
|
# Contributing to FuzzForge OSS
|
||||||
|
|
||||||
Thank you for your interest in contributing to FuzzForge! We welcome contributions from the community and are excited to collaborate with you.
|
Thank you for your interest in contributing to FuzzForge OSS! We welcome contributions from the community and are excited to collaborate with you.
|
||||||
|
|
||||||
## 🌟 Ways to Contribute
|
**Our Vision**: FuzzForge aims to be a **universal platform for security research** across all cybersecurity domains. Through our modular architecture, any security tool—from fuzzing engines to cloud scanners, from mobile app analyzers to IoT security tools—can be integrated as a containerized module and controlled via AI agents.
|
||||||
|
|
||||||
- 🐛 **Bug Reports** - Help us identify and fix issues
|
## Ways to Contribute
|
||||||
- 💡 **Feature Requests** - Suggest new capabilities and improvements
|
|
||||||
- 🔧 **Code Contributions** - Submit bug fixes, features, and enhancements
|
|
||||||
- 📚 **Documentation** - Improve guides, tutorials, and API documentation
|
|
||||||
- 🧪 **Testing** - Help test new features and report issues
|
|
||||||
- 🛡️ **Security Workflows** - Contribute new security analysis workflows
|
|
||||||
|
|
||||||
## 📋 Contribution Guidelines
|
- **Security Modules** - Create modules for any cybersecurity domain (AppSec, NetSec, Cloud, IoT, etc.)
|
||||||
|
- **Bug Reports** - Help us identify and fix issues
|
||||||
|
- **Feature Requests** - Suggest new capabilities and improvements
|
||||||
|
- **Core Features** - Contribute to the MCP server, runner, or CLI
|
||||||
|
- **Documentation** - Improve guides, tutorials, and module documentation
|
||||||
|
- **Testing** - Help test new features and report issues
|
||||||
|
- **AI Integration** - Improve MCP tools and AI agent interactions
|
||||||
|
- **Tool Integrations** - Wrap existing security tools as FuzzForge modules
|
||||||
|
|
||||||
|
## Contribution Guidelines
|
||||||
|
|
||||||
### Code Style
|
### Code Style
|
||||||
|
|
||||||
@@ -44,9 +48,10 @@ We use conventional commits for clear history:
|
|||||||
|
|
||||||
**Examples:**
|
**Examples:**
|
||||||
```
|
```
|
||||||
feat(workflows): add new static analysis workflow for Go
|
feat(modules): add cloud security scanner module
|
||||||
fix(api): resolve authentication timeout issue
|
fix(mcp): resolve module listing timeout
|
||||||
docs(readme): update installation instructions
|
docs(sdk): update module development guide
|
||||||
|
test(runner): add container execution tests
|
||||||
```
|
```
|
||||||
|
|
||||||
### Pull Request Process
|
### Pull Request Process
|
||||||
@@ -65,9 +70,14 @@ docs(readme): update installation instructions
|
|||||||
|
|
||||||
3. **Test Your Changes**
|
3. **Test Your Changes**
|
||||||
```bash
|
```bash
|
||||||
# Test workflows
|
# Test modules
|
||||||
cd test_projects/vulnerable_app/
|
FUZZFORGE_MODULES_PATH=./fuzzforge-modules uv run fuzzforge modules list
|
||||||
ff workflow security_assessment .
|
|
||||||
|
# Run a module
|
||||||
|
uv run fuzzforge modules run your-module --assets ./test-assets
|
||||||
|
|
||||||
|
# Test MCP integration (if applicable)
|
||||||
|
uv run fuzzforge mcp status
|
||||||
```
|
```
|
||||||
|
|
||||||
4. **Submit Pull Request**
|
4. **Submit Pull Request**
|
||||||
@@ -76,65 +86,353 @@ docs(readme): update installation instructions
|
|||||||
- Link related issues using `Fixes #123` or `Closes #123`
|
- Link related issues using `Fixes #123` or `Closes #123`
|
||||||
- Ensure all CI checks pass
|
- Ensure all CI checks pass
|
||||||
|
|
||||||
## 🛡️ Security Workflow Development
|
## Module Development
|
||||||
|
|
||||||
### Creating New Workflows
|
FuzzForge uses a modular architecture where security tools run as isolated containers. The `fuzzforge-modules-sdk` provides everything you need to create new modules.
|
||||||
|
|
||||||
1. **Workflow Structure**
|
**Documentation:**
|
||||||
```
|
- [Module SDK Documentation](fuzzforge-modules/fuzzforge-modules-sdk/README.md) - Complete SDK reference
|
||||||
backend/toolbox/workflows/your_workflow/
|
- [Module Template](fuzzforge-modules/fuzzforge-module-template/) - Starting point for new modules
|
||||||
├── __init__.py
|
- [USAGE Guide](USAGE.md) - Setup and installation instructions
|
||||||
├── workflow.py # Main Temporal workflow
|
|
||||||
├── activities.py # Workflow activities (optional)
|
### Creating a New Module
|
||||||
├── metadata.yaml # Workflow metadata (includes vertical field)
|
|
||||||
└── requirements.txt # Additional dependencies (optional)
|
1. **Use the Module Template**
|
||||||
|
```bash
|
||||||
|
# Generate a new module from template
|
||||||
|
cd fuzzforge-modules/
|
||||||
|
cp -r fuzzforge-module-template my-new-module
|
||||||
|
cd my-new-module
|
||||||
```
|
```
|
||||||
|
|
||||||
2. **Register Your Workflow**
|
2. **Module Structure**
|
||||||
Add your workflow to `backend/toolbox/workflows/registry.py`:
|
```
|
||||||
|
my-new-module/
|
||||||
|
├── Dockerfile # Container definition
|
||||||
|
├── Makefile # Build commands
|
||||||
|
├── README.md # Module documentation
|
||||||
|
├── pyproject.toml # Python dependencies
|
||||||
|
├── mypy.ini # Type checking config
|
||||||
|
├── ruff.toml # Linting config
|
||||||
|
└── src/
|
||||||
|
└── module/
|
||||||
|
├── __init__.py
|
||||||
|
├── __main__.py # Entry point
|
||||||
|
├── mod.py # Main module logic
|
||||||
|
├── models.py # Pydantic models
|
||||||
|
└── settings.py # Configuration
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Implement Your Module**
|
||||||
|
|
||||||
|
Edit `src/module/mod.py`:
|
||||||
```python
|
```python
|
||||||
# Import your workflow
|
from fuzzforge_modules_sdk.api.modules import BaseModule
|
||||||
from .your_workflow.workflow import main_flow as your_workflow_flow
|
from fuzzforge_modules_sdk.api.models import ModuleResult
|
||||||
|
from .models import MyModuleConfig, MyModuleOutput
|
||||||
# Add to registry
|
|
||||||
WORKFLOW_REGISTRY["your_workflow"] = {
|
class MyModule(BaseModule[MyModuleConfig, MyModuleOutput]):
|
||||||
"flow": your_workflow_flow,
|
"""Your module description."""
|
||||||
"module_path": "toolbox.workflows.your_workflow.workflow",
|
|
||||||
"function_name": "main_flow",
|
def execute(self) -> ModuleResult[MyModuleOutput]:
|
||||||
"description": "Description of your workflow",
|
"""Main execution logic."""
|
||||||
"version": "1.0.0",
|
# Access input assets
|
||||||
"author": "Your Name",
|
assets = self.input_path
|
||||||
"tags": ["tag1", "tag2"]
|
|
||||||
}
|
# Your security tool logic here
|
||||||
|
results = self.run_analysis(assets)
|
||||||
|
|
||||||
|
# Return structured results
|
||||||
|
return ModuleResult(
|
||||||
|
success=True,
|
||||||
|
output=MyModuleOutput(
|
||||||
|
findings=results,
|
||||||
|
summary="Analysis complete"
|
||||||
|
)
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
3. **Testing Workflows**
|
4. **Define Configuration Models**
|
||||||
- Create test cases in `test_projects/vulnerable_app/`
|
|
||||||
- Ensure SARIF output format compliance
|
Edit `src/module/models.py`:
|
||||||
- Test with various input scenarios
|
```python
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from fuzzforge_modules_sdk.api.models import BaseModuleConfig, BaseModuleOutput
|
||||||
|
|
||||||
|
class MyModuleConfig(BaseModuleConfig):
|
||||||
|
"""Configuration for your module."""
|
||||||
|
timeout: int = Field(default=300, description="Timeout in seconds")
|
||||||
|
max_iterations: int = Field(default=1000, description="Max iterations")
|
||||||
|
|
||||||
|
class MyModuleOutput(BaseModuleOutput):
|
||||||
|
"""Output from your module."""
|
||||||
|
findings: list[dict] = Field(default_factory=list)
|
||||||
|
coverage: float = Field(default=0.0)
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Build Your Module**
|
||||||
|
```bash
|
||||||
|
# Build the SDK first (if not already done)
|
||||||
|
cd ../fuzzforge-modules-sdk
|
||||||
|
uv build
|
||||||
|
mkdir -p .wheels
|
||||||
|
cp ../../dist/fuzzforge_modules_sdk-*.whl .wheels/
|
||||||
|
cd ../..
|
||||||
|
docker build -t localhost/fuzzforge-modules-sdk:0.1.0 fuzzforge-modules/fuzzforge-modules-sdk/
|
||||||
|
|
||||||
|
# Build your module
|
||||||
|
cd fuzzforge-modules/my-new-module
|
||||||
|
docker build -t fuzzforge-my-new-module:0.1.0 .
|
||||||
|
```
|
||||||
|
|
||||||
|
6. **Test Your Module**
|
||||||
|
```bash
|
||||||
|
# Run with test assets
|
||||||
|
uv run fuzzforge modules run my-new-module --assets ./test-assets
|
||||||
|
|
||||||
|
# Check module info
|
||||||
|
uv run fuzzforge modules info my-new-module
|
||||||
|
```
|
||||||
|
|
||||||
|
### Module Development Guidelines
|
||||||
|
|
||||||
|
**Important Conventions:**
|
||||||
|
- **Input/Output**: Use `/fuzzforge/input` for assets and `/fuzzforge/output` for results
|
||||||
|
- **Configuration**: Support JSON configuration via stdin or file
|
||||||
|
- **Logging**: Use structured logging (structlog is pre-configured)
|
||||||
|
- **Error Handling**: Return proper exit codes and error messages
|
||||||
|
- **Security**: Run as non-root user when possible
|
||||||
|
- **Documentation**: Include clear README with usage examples
|
||||||
|
- **Dependencies**: Minimize container size, use multi-stage builds
|
||||||
|
|
||||||
|
**See also:**
|
||||||
|
- [Module SDK API Reference](fuzzforge-modules/fuzzforge-modules-sdk/src/fuzzforge_modules_sdk/api/)
|
||||||
|
- [Dockerfile Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
|
||||||
|
|
||||||
|
### Module Types
|
||||||
|
|
||||||
|
FuzzForge is designed to support modules across **all cybersecurity domains**. The modular architecture allows any security tool to be containerized and integrated. Here are the main categories:
|
||||||
|
|
||||||
|
**Application Security**
|
||||||
|
- Fuzzing engines (coverage-guided, grammar-based, mutation-based)
|
||||||
|
- Static analysis (SAST, code quality, dependency scanning)
|
||||||
|
- Dynamic analysis (DAST, runtime analysis, instrumentation)
|
||||||
|
- Test validation and coverage analysis
|
||||||
|
- Crash analysis and exploit detection
|
||||||
|
|
||||||
|
**Network & Infrastructure Security**
|
||||||
|
- Network scanning and service enumeration
|
||||||
|
- Protocol analysis and fuzzing
|
||||||
|
- Firewall and configuration testing
|
||||||
|
- Cloud security (AWS/Azure/GCP misconfiguration detection, IAM analysis)
|
||||||
|
- Container security (image scanning, Kubernetes security)
|
||||||
|
|
||||||
|
**Web & API Security**
|
||||||
|
- Web vulnerability scanners (XSS, SQL injection, CSRF)
|
||||||
|
- Authentication and session testing
|
||||||
|
- API security (REST/GraphQL/gRPC testing, fuzzing)
|
||||||
|
- SSL/TLS analysis
|
||||||
|
|
||||||
|
**Binary & Reverse Engineering**
|
||||||
|
- Binary analysis and disassembly
|
||||||
|
- Malware sandboxing and behavior analysis
|
||||||
|
- Exploit development tools
|
||||||
|
- Firmware extraction and analysis
|
||||||
|
|
||||||
|
**Mobile & IoT Security**
|
||||||
|
- Mobile app analysis (Android/iOS static/dynamic analysis)
|
||||||
|
- IoT device security and firmware analysis
|
||||||
|
- SCADA/ICS and industrial protocol testing
|
||||||
|
- Automotive security (CAN bus, ECU testing)
|
||||||
|
|
||||||
|
**Data & Compliance**
|
||||||
|
- Database security testing
|
||||||
|
- Encryption and cryptography analysis
|
||||||
|
- Secrets and credential detection
|
||||||
|
- Privacy tools (PII detection, GDPR compliance)
|
||||||
|
- Compliance checkers (PCI-DSS, HIPAA, SOC2, ISO27001)
|
||||||
|
|
||||||
|
**Threat Intelligence & Risk**
|
||||||
|
- OSINT and reconnaissance tools
|
||||||
|
- Threat hunting and IOC correlation
|
||||||
|
- Risk assessment and attack surface mapping
|
||||||
|
- Security audit and policy validation
|
||||||
|
|
||||||
|
**Emerging Technologies**
|
||||||
|
- AI/ML security (model poisoning, adversarial testing)
|
||||||
|
- Blockchain and smart contract analysis
|
||||||
|
- Quantum-safe cryptography testing
|
||||||
|
|
||||||
|
**Custom & Integration**
|
||||||
|
- Domain-specific security tools
|
||||||
|
- Bridges to existing security tools
|
||||||
|
- Multi-tool orchestration and result aggregation
|
||||||
|
|
||||||
|
### Example: Simple Security Scanner Module
|
||||||
|
|
||||||
|
```python
|
||||||
|
# src/module/mod.py
|
||||||
|
from pathlib import Path
|
||||||
|
from fuzzforge_modules_sdk.api.modules import BaseModule
|
||||||
|
from fuzzforge_modules_sdk.api.models import ModuleResult
|
||||||
|
from .models import ScannerConfig, ScannerOutput
|
||||||
|
|
||||||
|
class SecurityScanner(BaseModule[ScannerConfig, ScannerOutput]):
|
||||||
|
"""Scans for common security issues in code."""
|
||||||
|
|
||||||
|
def execute(self) -> ModuleResult[ScannerOutput]:
|
||||||
|
findings = []
|
||||||
|
|
||||||
|
# Scan all source files
|
||||||
|
for file_path in self.input_path.rglob("*"):
|
||||||
|
if file_path.is_file():
|
||||||
|
findings.extend(self.scan_file(file_path))
|
||||||
|
|
||||||
|
return ModuleResult(
|
||||||
|
success=True,
|
||||||
|
output=ScannerOutput(
|
||||||
|
findings=findings,
|
||||||
|
files_scanned=len(list(self.input_path.rglob("*")))
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def scan_file(self, path: Path) -> list[dict]:
|
||||||
|
"""Scan a single file for security issues."""
|
||||||
|
# Your scanning logic here
|
||||||
|
return []
|
||||||
|
```
|
||||||
|
|
||||||
|
### Testing Modules
|
||||||
|
|
||||||
|
Create tests in `tests/`:
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from module.mod import MyModule
|
||||||
|
from module.models import MyModuleConfig
|
||||||
|
|
||||||
|
def test_module_execution():
|
||||||
|
config = MyModuleConfig(timeout=60)
|
||||||
|
module = MyModule(config=config, input_path=Path("test_assets"))
|
||||||
|
result = module.execute()
|
||||||
|
|
||||||
|
assert result.success
|
||||||
|
assert len(result.output.findings) >= 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Run tests:
|
||||||
|
```bash
|
||||||
|
uv run pytest
|
||||||
|
```
|
||||||
|
|
||||||
### Security Guidelines
|
### Security Guidelines
|
||||||
|
|
||||||
- 🔐 Never commit secrets, API keys, or credentials
|
**Critical Requirements:**
|
||||||
- 🛡️ Focus on **defensive security** tools and analysis
|
- Never commit secrets, API keys, or credentials
|
||||||
- ⚠️ Do not create tools for malicious purposes
|
- Focus on **defensive security** tools and analysis
|
||||||
- 🧪 Test workflows thoroughly before submission
|
- Do not create tools for malicious purposes
|
||||||
- 📋 Follow responsible disclosure for security issues
|
- Test modules thoroughly before submission
|
||||||
|
- Follow responsible disclosure for security issues
|
||||||
|
- Use minimal, secure base images for containers
|
||||||
|
- Avoid running containers as root when possible
|
||||||
|
|
||||||
## 🐛 Bug Reports
|
**Security Resources:**
|
||||||
|
- [OWASP Container Security](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html)
|
||||||
|
- [CIS Docker Benchmarks](https://www.cisecurity.org/benchmark/docker)
|
||||||
|
|
||||||
|
## Contributing to Core Features
|
||||||
|
|
||||||
|
Beyond modules, you can contribute to FuzzForge's core components.
|
||||||
|
|
||||||
|
**Useful Resources:**
|
||||||
|
- [Project Structure](README.md) - Overview of the codebase
|
||||||
|
- [USAGE Guide](USAGE.md) - Installation and setup
|
||||||
|
- Python best practices: [PEP 8](https://pep8.org/)
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
- **fuzzforge-mcp** - MCP server for AI agent integration
|
||||||
|
- **fuzzforge-runner** - Module execution engine
|
||||||
|
- **fuzzforge-cli** - Command-line interface
|
||||||
|
- **fuzzforge-common** - Shared utilities and sandbox engines
|
||||||
|
- **fuzzforge-types** - Type definitions and schemas
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
1. **Clone and Install**
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/FuzzingLabs/fuzzforge-oss.git
|
||||||
|
cd fuzzforge-oss
|
||||||
|
uv sync --all-extras
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Run Tests**
|
||||||
|
```bash
|
||||||
|
# Run all tests
|
||||||
|
make test
|
||||||
|
|
||||||
|
# Run specific package tests
|
||||||
|
cd fuzzforge-mcp
|
||||||
|
uv run pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Type Checking**
|
||||||
|
```bash
|
||||||
|
# Type check all packages
|
||||||
|
make typecheck
|
||||||
|
|
||||||
|
# Type check specific package
|
||||||
|
cd fuzzforge-runner
|
||||||
|
uv run mypy .
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Linting and Formatting**
|
||||||
|
```bash
|
||||||
|
# Format code
|
||||||
|
make format
|
||||||
|
|
||||||
|
# Lint code
|
||||||
|
make lint
|
||||||
|
```
|
||||||
|
|
||||||
|
## Bug Reports
|
||||||
|
|
||||||
When reporting bugs, please include:
|
When reporting bugs, please include:
|
||||||
|
|
||||||
- **Environment**: OS, Python version, Docker version
|
- **Environment**: OS, Python version, Docker version, uv version
|
||||||
|
- **FuzzForge Version**: Output of `uv run fuzzforge --version`
|
||||||
|
- **Module**: Which module or component is affected
|
||||||
- **Steps to Reproduce**: Clear steps to recreate the issue
|
- **Steps to Reproduce**: Clear steps to recreate the issue
|
||||||
- **Expected Behavior**: What should happen
|
- **Expected Behavior**: What should happen
|
||||||
- **Actual Behavior**: What actually happens
|
- **Actual Behavior**: What actually happens
|
||||||
- **Logs**: Relevant error messages and stack traces
|
- **Logs**: Relevant error messages and stack traces
|
||||||
|
- **Container Logs**: For module issues, include Docker/Podman logs
|
||||||
- **Screenshots**: If applicable
|
- **Screenshots**: If applicable
|
||||||
|
|
||||||
Use our [Bug Report Template](.github/ISSUE_TEMPLATE/bug_report.md).
|
**Example:**
|
||||||
|
```markdown
|
||||||
|
**Environment:**
|
||||||
|
- OS: Ubuntu 22.04
|
||||||
|
- Python: 3.14.2
|
||||||
|
- Docker: 24.0.7
|
||||||
|
- uv: 0.5.13
|
||||||
|
|
||||||
## 💡 Feature Requests
|
**Module:** my-custom-scanner
|
||||||
|
|
||||||
|
**Steps to Reproduce:**
|
||||||
|
1. Run `uv run fuzzforge modules run my-scanner --assets ./test-target`
|
||||||
|
2. Module fails with timeout error
|
||||||
|
|
||||||
|
**Expected:** Module completes analysis
|
||||||
|
**Actual:** Times out after 30 seconds
|
||||||
|
|
||||||
|
**Logs:**
|
||||||
|
```
|
||||||
|
ERROR: Module execution timeout
|
||||||
|
...
|
||||||
|
```
|
||||||
|
```
|
||||||
|
|
||||||
|
## Feature Requests
|
||||||
|
|
||||||
For new features, please provide:
|
For new features, please provide:
|
||||||
|
|
||||||
@@ -142,33 +440,124 @@ For new features, please provide:
|
|||||||
- **Proposed Solution**: How should it work?
|
- **Proposed Solution**: How should it work?
|
||||||
- **Alternatives**: Other approaches considered
|
- **Alternatives**: Other approaches considered
|
||||||
- **Implementation**: Technical considerations (optional)
|
- **Implementation**: Technical considerations (optional)
|
||||||
|
- **Module vs Core**: Should this be a module or core feature?
|
||||||
|
|
||||||
Use our [Feature Request Template](.github/ISSUE_TEMPLATE/feature_request.md).
|
**Example Feature Requests:**
|
||||||
|
- New module for cloud security posture management (CSPM)
|
||||||
|
- Module for analyzing smart contract vulnerabilities
|
||||||
|
- MCP tool for orchestrating multi-module workflows
|
||||||
|
- CLI command for batch module execution across multiple targets
|
||||||
|
- Support for distributed fuzzing campaigns
|
||||||
|
- Integration with CI/CD pipelines
|
||||||
|
- Module marketplace/registry features
|
||||||
|
|
||||||
## 📚 Documentation
|
## Documentation
|
||||||
|
|
||||||
Help improve our documentation:
|
Help improve our documentation:
|
||||||
|
|
||||||
|
- **Module Documentation**: Document your modules in their README.md
|
||||||
- **API Documentation**: Update docstrings and type hints
|
- **API Documentation**: Update docstrings and type hints
|
||||||
- **User Guides**: Create tutorials and how-to guides
|
- **User Guides**: Improve USAGE.md and tutorial content
|
||||||
- **Workflow Documentation**: Document new security workflows
|
- **Module SDK Guides**: Help document the SDK for module developers
|
||||||
- **Examples**: Add practical usage examples
|
- **MCP Integration**: Document AI agent integration patterns
|
||||||
|
- **Examples**: Add practical usage examples and workflows
|
||||||
|
|
||||||
## 🙏 Recognition
|
### Documentation Standards
|
||||||
|
|
||||||
|
- Use clear, concise language
|
||||||
|
- Include code examples
|
||||||
|
- Add command-line examples with expected output
|
||||||
|
- Document all configuration options
|
||||||
|
- Explain error messages and troubleshooting
|
||||||
|
|
||||||
|
### Module README Template
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Module Name
|
||||||
|
|
||||||
|
Brief description of what this module does.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- Feature 1
|
||||||
|
- Feature 2
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
| Parameter | Type | Default | Description |
|
||||||
|
|-----------|------|---------|-------------|
|
||||||
|
| timeout | int | 300 | Timeout in seconds |
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
\`\`\`bash
|
||||||
|
uv run fuzzforge modules run module-name --assets ./path/to/assets
|
||||||
|
\`\`\`
|
||||||
|
|
||||||
|
## Output
|
||||||
|
|
||||||
|
Describes the output structure and format.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
Practical usage examples.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Recognition
|
||||||
|
|
||||||
Contributors will be:
|
Contributors will be:
|
||||||
|
|
||||||
- Listed in our [Contributors](CONTRIBUTORS.md) file
|
- Listed in our [Contributors](CONTRIBUTORS.md) file
|
||||||
- Mentioned in release notes for significant contributions
|
- Mentioned in release notes for significant contributions
|
||||||
- Invited to join our Discord community
|
- Credited in module documentation (for module authors)
|
||||||
- Eligible for FuzzingLabs Academy courses and swag
|
- Invited to join our [Discord community](https://discord.gg/8XEX33UUwZ)
|
||||||
|
|
||||||
## 📜 License
|
## Module Submission Checklist
|
||||||
|
|
||||||
By contributing to FuzzForge, you agree that your contributions will be licensed under the same [Business Source License 1.1](LICENSE) as the project.
|
Before submitting a new module:
|
||||||
|
|
||||||
|
- [ ] Module follows SDK structure and conventions
|
||||||
|
- [ ] Dockerfile builds successfully
|
||||||
|
- [ ] Module executes without errors
|
||||||
|
- [ ] Configuration options are documented
|
||||||
|
- [ ] README.md is complete with examples
|
||||||
|
- [ ] Tests are included (pytest)
|
||||||
|
- [ ] Type hints are used throughout
|
||||||
|
- [ ] Linting passes (ruff)
|
||||||
|
- [ ] Security best practices followed
|
||||||
|
- [ ] No secrets or credentials in code
|
||||||
|
- [ ] License headers included
|
||||||
|
|
||||||
|
## Review Process
|
||||||
|
|
||||||
|
1. **Initial Review** - Maintainers review for completeness
|
||||||
|
2. **Technical Review** - Code quality and security assessment
|
||||||
|
3. **Testing** - Module tested in isolated environment
|
||||||
|
4. **Documentation Review** - Ensure docs are clear and complete
|
||||||
|
5. **Approval** - Module merged and included in next release
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
By contributing to FuzzForge OSS, you agree that your contributions will be licensed under the same license as the project (see [LICENSE](LICENSE)).
|
||||||
|
|
||||||
|
For module contributions:
|
||||||
|
- Modules you create remain under the project license
|
||||||
|
- You retain credit as the module author
|
||||||
|
- Your module may be used by others under the project license terms
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Thank you for making FuzzForge better! 🚀**
|
## Getting Help
|
||||||
|
|
||||||
Every contribution, no matter how small, helps build a stronger security community.
|
Need help contributing?
|
||||||
|
|
||||||
|
- Join our [Discord](https://discord.gg/8XEX33UUwZ)
|
||||||
|
- Read the [Module SDK Documentation](fuzzforge-modules/fuzzforge-modules-sdk/README.md)
|
||||||
|
- Check the module template for examples
|
||||||
|
- Contact: contact@fuzzinglabs.com
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Thank you for making FuzzForge better!**
|
||||||
|
|
||||||
|
Every contribution, no matter how small, helps build a stronger security research platform. Whether you're creating a module for web security, cloud scanning, mobile analysis, or any other cybersecurity domain, your work makes FuzzForge more powerful and versatile for the entire security community!
|
||||||
|
|||||||
10
README.md
10
README.md
@@ -72,8 +72,8 @@ Instead of manually running security tools, describe what you want and let your
|
|||||||
|
|
||||||
If you find FuzzForge useful, please **star the repo** to support development! 🚀
|
If you find FuzzForge useful, please **star the repo** to support development! 🚀
|
||||||
|
|
||||||
<a href="https://github.com/FuzzingLabs/fuzzforge-oss/stargazers">
|
<a href="https://github.com/FuzzingLabs/fuzzforge_ai/stargazers">
|
||||||
<img src="https://img.shields.io/github/stars/FuzzingLabs/fuzzforge-oss?style=social" alt="GitHub Stars">
|
<img src="https://img.shields.io/github/stars/FuzzingLabs/fuzzforge_ai?style=social" alt="GitHub Stars">
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -135,8 +135,8 @@ If you find FuzzForge useful, please **star the repo** to support development!
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Clone the repository
|
# Clone the repository
|
||||||
git clone https://github.com/FuzzingLabs/fuzzforge-oss.git
|
git clone https://github.com/FuzzingLabs/fuzzforge_ai.git
|
||||||
cd fuzzforge-oss
|
cd fuzzforge_ai
|
||||||
|
|
||||||
# Install dependencies
|
# Install dependencies
|
||||||
uv sync
|
uv sync
|
||||||
@@ -246,7 +246,7 @@ class MySecurityModule(FuzzForgeModule):
|
|||||||
## 📁 Project Structure
|
## 📁 Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
fuzzforge-oss/
|
fuzzforge_ai/
|
||||||
├── fuzzforge-cli/ # Command-line interface
|
├── fuzzforge-cli/ # Command-line interface
|
||||||
├── fuzzforge-common/ # Shared abstractions (containers, storage)
|
├── fuzzforge-common/ # Shared abstractions (containers, storage)
|
||||||
├── fuzzforge-mcp/ # MCP server for AI agents
|
├── fuzzforge-mcp/ # MCP server for AI agents
|
||||||
|
|||||||
@@ -95,8 +95,8 @@ class DockerCLI(AbstractFuzzForgeSandboxEngine):
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
reference = f"{repo}:{tag}"
|
reference = f"{repo}:{tag}"
|
||||||
|
|
||||||
if filter_prefix and not reference.startswith(filter_prefix):
|
if filter_prefix and filter_prefix not in reference:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
images.append(
|
images.append(
|
||||||
|
|||||||
@@ -31,14 +31,15 @@ def get_logger() -> BoundLogger:
|
|||||||
|
|
||||||
def _is_running_under_snap() -> bool:
|
def _is_running_under_snap() -> bool:
|
||||||
"""Check if running under Snap environment.
|
"""Check if running under Snap environment.
|
||||||
|
|
||||||
VS Code installed via Snap sets XDG_DATA_HOME to a version-specific path,
|
VS Code installed via Snap sets XDG_DATA_HOME to a version-specific path,
|
||||||
causing Podman to look for storage in non-standard locations. When SNAP
|
causing Podman to look for storage in non-standard locations. When SNAP
|
||||||
is set, we use custom storage paths to ensure consistency.
|
is set, we use custom storage paths to ensure consistency.
|
||||||
|
|
||||||
Note: Snap only exists on Linux, so this also handles macOS implicitly.
|
Note: Snap only exists on Linux, so this also handles macOS implicitly.
|
||||||
"""
|
"""
|
||||||
import os # noqa: PLC0415
|
import os # noqa: PLC0415
|
||||||
|
|
||||||
return os.getenv("SNAP") is not None
|
return os.getenv("SNAP") is not None
|
||||||
|
|
||||||
|
|
||||||
@@ -48,11 +49,11 @@ class PodmanCLI(AbstractFuzzForgeSandboxEngine):
|
|||||||
This implementation uses subprocess calls to the Podman CLI with --root
|
This implementation uses subprocess calls to the Podman CLI with --root
|
||||||
and --runroot flags when running under Snap, providing isolation from
|
and --runroot flags when running under Snap, providing isolation from
|
||||||
system Podman storage.
|
system Podman storage.
|
||||||
|
|
||||||
The custom storage is only used when:
|
The custom storage is only used when:
|
||||||
1. Running under Snap (SNAP env var is set) - to fix XDG_DATA_HOME issues
|
1. Running under Snap (SNAP env var is set) - to fix XDG_DATA_HOME issues
|
||||||
2. Custom paths are explicitly provided
|
2. Custom paths are explicitly provided
|
||||||
|
|
||||||
Otherwise, uses default Podman storage which works for:
|
Otherwise, uses default Podman storage which works for:
|
||||||
- Native Linux installations
|
- Native Linux installations
|
||||||
- macOS (where Podman runs in a VM via podman machine)
|
- macOS (where Podman runs in a VM via podman machine)
|
||||||
@@ -69,16 +70,24 @@ class PodmanCLI(AbstractFuzzForgeSandboxEngine):
|
|||||||
:param runroot: Path to container runtime state.
|
:param runroot: Path to container runtime state.
|
||||||
|
|
||||||
Custom storage is used when running under Snap AND paths are provided.
|
Custom storage is used when running under Snap AND paths are provided.
|
||||||
|
|
||||||
|
:raises FuzzForgeError: If running on macOS (Podman not supported).
|
||||||
"""
|
"""
|
||||||
|
import sys # noqa: PLC0415
|
||||||
|
|
||||||
|
if sys.platform == "darwin":
|
||||||
|
msg = (
|
||||||
|
"Podman is not supported on macOS. Please use Docker instead:\n"
|
||||||
|
" brew install --cask docker\n"
|
||||||
|
" # Or download from https://docker.com/products/docker-desktop"
|
||||||
|
)
|
||||||
|
raise FuzzForgeError(msg)
|
||||||
|
|
||||||
AbstractFuzzForgeSandboxEngine.__init__(self)
|
AbstractFuzzForgeSandboxEngine.__init__(self)
|
||||||
|
|
||||||
# Use custom storage only under Snap (to fix XDG_DATA_HOME issues)
|
# Use custom storage only under Snap (to fix XDG_DATA_HOME issues)
|
||||||
self.__use_custom_storage = (
|
self.__use_custom_storage = _is_running_under_snap() and graphroot is not None and runroot is not None
|
||||||
_is_running_under_snap()
|
|
||||||
and graphroot is not None
|
|
||||||
and runroot is not None
|
|
||||||
)
|
|
||||||
|
|
||||||
if self.__use_custom_storage:
|
if self.__use_custom_storage:
|
||||||
self.__graphroot = graphroot
|
self.__graphroot = graphroot
|
||||||
self.__runroot = runroot
|
self.__runroot = runroot
|
||||||
@@ -98,8 +107,10 @@ class PodmanCLI(AbstractFuzzForgeSandboxEngine):
|
|||||||
if self.__use_custom_storage and self.__graphroot and self.__runroot:
|
if self.__use_custom_storage and self.__graphroot and self.__runroot:
|
||||||
return [
|
return [
|
||||||
"podman",
|
"podman",
|
||||||
"--root", str(self.__graphroot),
|
"--root",
|
||||||
"--runroot", str(self.__runroot),
|
str(self.__graphroot),
|
||||||
|
"--runroot",
|
||||||
|
str(self.__runroot),
|
||||||
]
|
]
|
||||||
return ["podman"]
|
return ["podman"]
|
||||||
|
|
||||||
|
|||||||
@@ -2,32 +2,41 @@
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
|
import sys
|
||||||
import uuid
|
import uuid
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest import mock
|
from unittest import mock
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from fuzzforge_common.exceptions import FuzzForgeError
|
||||||
from fuzzforge_common.sandboxes.engines.podman.cli import PodmanCLI, _is_running_under_snap
|
from fuzzforge_common.sandboxes.engines.podman.cli import PodmanCLI, _is_running_under_snap
|
||||||
|
|
||||||
|
|
||||||
|
# Helper to mock Linux platform for testing (since Podman is Linux-only)
|
||||||
|
def _mock_linux_platform() -> mock._patch[str]:
|
||||||
|
"""Context manager to mock sys.platform as 'linux'."""
|
||||||
|
return mock.patch.object(sys, "platform", "linux")
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def podman_cli_engine() -> PodmanCLI:
|
def podman_cli_engine() -> PodmanCLI:
|
||||||
"""Create a PodmanCLI engine with temporary storage.
|
"""Create a PodmanCLI engine with temporary storage.
|
||||||
|
|
||||||
Uses short paths in /tmp to avoid podman's 50-char runroot limit.
|
Uses short paths in /tmp to avoid podman's 50-char runroot limit.
|
||||||
Simulates Snap environment to test custom storage paths.
|
Simulates Snap environment to test custom storage paths.
|
||||||
|
Mocks Linux platform since Podman is Linux-only.
|
||||||
"""
|
"""
|
||||||
short_id = str(uuid.uuid4())[:8]
|
short_id = str(uuid.uuid4())[:8]
|
||||||
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
||||||
runroot = Path(f"/tmp/ff-{short_id}/run")
|
runroot = Path(f"/tmp/ff-{short_id}/run")
|
||||||
|
|
||||||
# Simulate Snap environment for testing
|
# Simulate Snap environment for testing on Linux
|
||||||
with mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
with _mock_linux_platform(), mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
||||||
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
||||||
|
|
||||||
yield engine
|
yield engine
|
||||||
|
|
||||||
# Cleanup
|
# Cleanup
|
||||||
parent = graphroot.parent
|
parent = graphroot.parent
|
||||||
if parent.exists():
|
if parent.exists():
|
||||||
@@ -48,21 +57,30 @@ def test_snap_detection_when_snap_not_set() -> None:
|
|||||||
assert _is_running_under_snap() is False
|
assert _is_running_under_snap() is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_podman_cli_blocks_macos() -> None:
|
||||||
|
"""Test that PodmanCLI raises error on macOS."""
|
||||||
|
with mock.patch.object(sys, "platform", "darwin"):
|
||||||
|
with pytest.raises(FuzzForgeError) as exc_info:
|
||||||
|
PodmanCLI()
|
||||||
|
assert "Podman is not supported on macOS" in str(exc_info.value)
|
||||||
|
assert "Docker" in str(exc_info.value)
|
||||||
|
|
||||||
|
|
||||||
def test_podman_cli_creates_storage_directories_under_snap() -> None:
|
def test_podman_cli_creates_storage_directories_under_snap() -> None:
|
||||||
"""Test that PodmanCLI creates storage directories when under Snap."""
|
"""Test that PodmanCLI creates storage directories when under Snap."""
|
||||||
short_id = str(uuid.uuid4())[:8]
|
short_id = str(uuid.uuid4())[:8]
|
||||||
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
||||||
runroot = Path(f"/tmp/ff-{short_id}/run")
|
runroot = Path(f"/tmp/ff-{short_id}/run")
|
||||||
|
|
||||||
assert not graphroot.exists()
|
assert not graphroot.exists()
|
||||||
assert not runroot.exists()
|
assert not runroot.exists()
|
||||||
|
|
||||||
with mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
with _mock_linux_platform(), mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
||||||
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
PodmanCLI(graphroot=graphroot, runroot=runroot)
|
||||||
|
|
||||||
assert graphroot.exists()
|
assert graphroot.exists()
|
||||||
assert runroot.exists()
|
assert runroot.exists()
|
||||||
|
|
||||||
# Cleanup
|
# Cleanup
|
||||||
shutil.rmtree(graphroot.parent, ignore_errors=True)
|
shutil.rmtree(graphroot.parent, ignore_errors=True)
|
||||||
|
|
||||||
@@ -72,15 +90,15 @@ def test_podman_cli_base_cmd_under_snap() -> None:
|
|||||||
short_id = str(uuid.uuid4())[:8]
|
short_id = str(uuid.uuid4())[:8]
|
||||||
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
||||||
runroot = Path(f"/tmp/ff-{short_id}/run")
|
runroot = Path(f"/tmp/ff-{short_id}/run")
|
||||||
|
|
||||||
with mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
with _mock_linux_platform(), mock.patch.dict(os.environ, {"SNAP": "/snap/code/123"}):
|
||||||
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
||||||
base_cmd = engine._base_cmd()
|
base_cmd = engine._base_cmd()
|
||||||
|
|
||||||
assert "podman" in base_cmd
|
assert "podman" in base_cmd
|
||||||
assert "--root" in base_cmd
|
assert "--root" in base_cmd
|
||||||
assert "--runroot" in base_cmd
|
assert "--runroot" in base_cmd
|
||||||
|
|
||||||
# Cleanup
|
# Cleanup
|
||||||
shutil.rmtree(graphroot.parent, ignore_errors=True)
|
shutil.rmtree(graphroot.parent, ignore_errors=True)
|
||||||
|
|
||||||
@@ -90,25 +108,26 @@ def test_podman_cli_base_cmd_without_snap() -> None:
|
|||||||
short_id = str(uuid.uuid4())[:8]
|
short_id = str(uuid.uuid4())[:8]
|
||||||
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
graphroot = Path(f"/tmp/ff-{short_id}/storage")
|
||||||
runroot = Path(f"/tmp/ff-{short_id}/run")
|
runroot = Path(f"/tmp/ff-{short_id}/run")
|
||||||
|
|
||||||
env = os.environ.copy()
|
env = os.environ.copy()
|
||||||
env.pop("SNAP", None)
|
env.pop("SNAP", None)
|
||||||
with mock.patch.dict(os.environ, env, clear=True):
|
with _mock_linux_platform(), mock.patch.dict(os.environ, env, clear=True):
|
||||||
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
engine = PodmanCLI(graphroot=graphroot, runroot=runroot)
|
||||||
base_cmd = engine._base_cmd()
|
base_cmd = engine._base_cmd()
|
||||||
|
|
||||||
assert base_cmd == ["podman"]
|
assert base_cmd == ["podman"]
|
||||||
assert "--root" not in base_cmd
|
assert "--root" not in base_cmd
|
||||||
|
|
||||||
# Directories should NOT be created when not under Snap
|
# Directories should NOT be created when not under Snap
|
||||||
assert not graphroot.exists()
|
assert not graphroot.exists()
|
||||||
|
|
||||||
|
|
||||||
def test_podman_cli_default_mode() -> None:
|
def test_podman_cli_default_mode() -> None:
|
||||||
"""Test PodmanCLI without custom storage paths."""
|
"""Test PodmanCLI without custom storage paths."""
|
||||||
engine = PodmanCLI() # No paths provided
|
with _mock_linux_platform():
|
||||||
base_cmd = engine._base_cmd()
|
engine = PodmanCLI() # No paths provided
|
||||||
|
base_cmd = engine._base_cmd()
|
||||||
|
|
||||||
assert base_cmd == ["podman"]
|
assert base_cmd == ["podman"]
|
||||||
assert "--root" not in base_cmd
|
assert "--root" not in base_cmd
|
||||||
|
|
||||||
@@ -116,7 +135,7 @@ def test_podman_cli_default_mode() -> None:
|
|||||||
def test_podman_cli_list_images_returns_list(podman_cli_engine: PodmanCLI) -> None:
|
def test_podman_cli_list_images_returns_list(podman_cli_engine: PodmanCLI) -> None:
|
||||||
"""Test that list_images returns a list (even if empty)."""
|
"""Test that list_images returns a list (even if empty)."""
|
||||||
images = podman_cli_engine.list_images()
|
images = podman_cli_engine.list_images()
|
||||||
|
|
||||||
assert isinstance(images, list)
|
assert isinstance(images, list)
|
||||||
|
|
||||||
|
|
||||||
@@ -125,6 +144,6 @@ def test_podman_cli_can_pull_and_list_image(podman_cli_engine: PodmanCLI) -> Non
|
|||||||
"""Test pulling an image and listing it."""
|
"""Test pulling an image and listing it."""
|
||||||
# Pull a small image
|
# Pull a small image
|
||||||
podman_cli_engine._run(["pull", "docker.io/library/alpine:latest"])
|
podman_cli_engine._run(["pull", "docker.io/library/alpine:latest"])
|
||||||
|
|
||||||
images = podman_cli_engine.list_images()
|
images = podman_cli_engine.list_images()
|
||||||
assert any("alpine" in img.identifier for img in images)
|
assert any("alpine" in img.identifier for img in images)
|
||||||
|
|||||||
@@ -136,7 +136,7 @@ class ModuleExecutor:
|
|||||||
# - fuzzforge-module-{name}:{tag} (OSS local builds with module prefix)
|
# - fuzzforge-module-{name}:{tag} (OSS local builds with module prefix)
|
||||||
# - localhost/fuzzforge-module-{name}:{tag} (standard convention)
|
# - localhost/fuzzforge-module-{name}:{tag} (standard convention)
|
||||||
# - localhost/{name}:{tag} (legacy/short form)
|
# - localhost/{name}:{tag} (legacy/short form)
|
||||||
|
|
||||||
# For OSS local builds (no localhost/ prefix)
|
# For OSS local builds (no localhost/ prefix)
|
||||||
for tag in tags_to_check:
|
for tag in tags_to_check:
|
||||||
# Check direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
# Check direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
||||||
@@ -146,7 +146,7 @@ class ModuleExecutor:
|
|||||||
if not module_identifier.startswith("fuzzforge-"):
|
if not module_identifier.startswith("fuzzforge-"):
|
||||||
if engine.image_exists(f"fuzzforge-{module_identifier}:{tag}"):
|
if engine.image_exists(f"fuzzforge-{module_identifier}:{tag}"):
|
||||||
return True
|
return True
|
||||||
|
|
||||||
# For registry-style naming (localhost/ prefix)
|
# For registry-style naming (localhost/ prefix)
|
||||||
name_prefixes = [f"fuzzforge-module-{module_identifier}", module_identifier]
|
name_prefixes = [f"fuzzforge-module-{module_identifier}", module_identifier]
|
||||||
|
|
||||||
@@ -166,17 +166,17 @@ class ModuleExecutor:
|
|||||||
|
|
||||||
"""
|
"""
|
||||||
engine = self._get_engine()
|
engine = self._get_engine()
|
||||||
|
|
||||||
# Try common tags
|
# Try common tags
|
||||||
tags_to_check = ["latest", "0.1.0", "0.0.1"]
|
tags_to_check = ["latest", "0.1.0", "0.0.1"]
|
||||||
|
|
||||||
# Check OSS local builds first (no localhost/ prefix)
|
# Check OSS local builds first (no localhost/ prefix)
|
||||||
for tag in tags_to_check:
|
for tag in tags_to_check:
|
||||||
# Direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
# Direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
||||||
direct_name = f"{module_identifier}:{tag}"
|
direct_name = f"{module_identifier}:{tag}"
|
||||||
if engine.image_exists(direct_name):
|
if engine.image_exists(direct_name):
|
||||||
return direct_name
|
return direct_name
|
||||||
|
|
||||||
# With fuzzforge- prefix if not already present
|
# With fuzzforge- prefix if not already present
|
||||||
if not module_identifier.startswith("fuzzforge-"):
|
if not module_identifier.startswith("fuzzforge-"):
|
||||||
prefixed_name = f"fuzzforge-{module_identifier}:{tag}"
|
prefixed_name = f"fuzzforge-{module_identifier}:{tag}"
|
||||||
@@ -189,7 +189,7 @@ class ModuleExecutor:
|
|||||||
prefixed_name = f"localhost/fuzzforge-module-{module_identifier}:{tag}"
|
prefixed_name = f"localhost/fuzzforge-module-{module_identifier}:{tag}"
|
||||||
if engine.image_exists(prefixed_name):
|
if engine.image_exists(prefixed_name):
|
||||||
return prefixed_name
|
return prefixed_name
|
||||||
|
|
||||||
# Legacy short form: localhost/{name}:{tag}
|
# Legacy short form: localhost/{name}:{tag}
|
||||||
short_name = f"localhost/{module_identifier}:{tag}"
|
short_name = f"localhost/{module_identifier}:{tag}"
|
||||||
if engine.image_exists(short_name):
|
if engine.image_exists(short_name):
|
||||||
@@ -198,7 +198,7 @@ class ModuleExecutor:
|
|||||||
# Default fallback
|
# Default fallback
|
||||||
return f"localhost/{module_identifier}:latest"
|
return f"localhost/{module_identifier}:latest"
|
||||||
|
|
||||||
def _pull_module_image(self, module_identifier: str, registry_url: str = "ghcr.io/fuzzinglabs", tag: str = "latest") -> None:
|
def _pull_module_image(self, module_identifier: str, registry_url: str, tag: str = "latest") -> None:
|
||||||
"""Pull a module image from the container registry.
|
"""Pull a module image from the container registry.
|
||||||
|
|
||||||
:param module_identifier: Name/identifier of the module to pull.
|
:param module_identifier: Name/identifier of the module to pull.
|
||||||
@@ -238,21 +238,30 @@ class ModuleExecutor:
|
|||||||
)
|
)
|
||||||
raise SandboxError(message) from exc
|
raise SandboxError(message) from exc
|
||||||
|
|
||||||
def _ensure_module_image(self, module_identifier: str, registry_url: str = "ghcr.io/fuzzinglabs", tag: str = "latest") -> None:
|
def _ensure_module_image(self, module_identifier: str, registry_url: str = "", tag: str = "latest") -> None:
|
||||||
"""Ensure module image exists, pulling it if necessary.
|
"""Ensure module image exists, pulling it if necessary.
|
||||||
|
|
||||||
:param module_identifier: Name/identifier of the module image.
|
:param module_identifier: Name/identifier of the module image.
|
||||||
:param registry_url: Container registry URL to pull from.
|
:param registry_url: Container registry URL to pull from (empty = local-only mode).
|
||||||
:param tag: Image tag to pull.
|
:param tag: Image tag to pull.
|
||||||
:raises SandboxError: If image check or pull fails.
|
:raises SandboxError: If image not found locally and no registry configured.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
logger = get_logger()
|
logger = get_logger()
|
||||||
|
|
||||||
if self._check_image_exists(module_identifier):
|
if self._check_image_exists(module_identifier):
|
||||||
logger.debug("module image exists locally", module=module_identifier)
|
logger.debug("module image exists locally", module=module_identifier)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
# If no registry configured, we're in local-only mode
|
||||||
|
if not registry_url:
|
||||||
|
raise SandboxError(
|
||||||
|
f"Module image '{module_identifier}' not found locally.\n"
|
||||||
|
"Build it with: make build-modules\n"
|
||||||
|
"\n"
|
||||||
|
"Or configure a registry URL via FUZZFORGE_REGISTRY__URL environment variable."
|
||||||
|
)
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"module image not found locally, pulling from registry",
|
"module image not found locally, pulling from registry",
|
||||||
module=module_identifier,
|
module=module_identifier,
|
||||||
@@ -260,7 +269,7 @@ class ModuleExecutor:
|
|||||||
info="This may take a moment on first run",
|
info="This may take a moment on first run",
|
||||||
)
|
)
|
||||||
self._pull_module_image(module_identifier, registry_url, tag)
|
self._pull_module_image(module_identifier, registry_url, tag)
|
||||||
|
|
||||||
# Verify image now exists
|
# Verify image now exists
|
||||||
if not self._check_image_exists(module_identifier):
|
if not self._check_image_exists(module_identifier):
|
||||||
message = (
|
message = (
|
||||||
@@ -332,6 +341,7 @@ class ModuleExecutor:
|
|||||||
try:
|
try:
|
||||||
# Create temporary directory - caller must clean it up after container finishes
|
# Create temporary directory - caller must clean it up after container finishes
|
||||||
from tempfile import mkdtemp
|
from tempfile import mkdtemp
|
||||||
|
|
||||||
temp_path = Path(mkdtemp(prefix="fuzzforge-input-"))
|
temp_path = Path(mkdtemp(prefix="fuzzforge-input-"))
|
||||||
|
|
||||||
# Copy assets to temp directory
|
# Copy assets to temp directory
|
||||||
@@ -341,16 +351,19 @@ class ModuleExecutor:
|
|||||||
if assets_path.suffix == ".gz" or assets_path.name.endswith(".tar.gz"):
|
if assets_path.suffix == ".gz" or assets_path.name.endswith(".tar.gz"):
|
||||||
# Extract archive contents
|
# Extract archive contents
|
||||||
import tarfile
|
import tarfile
|
||||||
|
|
||||||
with tarfile.open(assets_path, "r:gz") as tar:
|
with tarfile.open(assets_path, "r:gz") as tar:
|
||||||
tar.extractall(path=temp_path)
|
tar.extractall(path=temp_path)
|
||||||
logger.debug("extracted tar.gz archive", archive=str(assets_path))
|
logger.debug("extracted tar.gz archive", archive=str(assets_path))
|
||||||
else:
|
else:
|
||||||
# Single file - copy it
|
# Single file - copy it
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
shutil.copy2(assets_path, temp_path / assets_path.name)
|
shutil.copy2(assets_path, temp_path / assets_path.name)
|
||||||
else:
|
else:
|
||||||
# Directory - copy all files (including subdirectories)
|
# Directory - copy all files (including subdirectories)
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
for item in assets_path.iterdir():
|
for item in assets_path.iterdir():
|
||||||
if item.is_file():
|
if item.is_file():
|
||||||
shutil.copy2(item, temp_path / item.name)
|
shutil.copy2(item, temp_path / item.name)
|
||||||
@@ -363,19 +376,23 @@ class ModuleExecutor:
|
|||||||
if item.name == "input.json":
|
if item.name == "input.json":
|
||||||
continue
|
continue
|
||||||
if item.is_file():
|
if item.is_file():
|
||||||
resources.append({
|
resources.append(
|
||||||
"name": item.stem,
|
{
|
||||||
"description": f"Input file: {item.name}",
|
"name": item.stem,
|
||||||
"kind": "unknown",
|
"description": f"Input file: {item.name}",
|
||||||
"path": f"/data/input/{item.name}",
|
"kind": "unknown",
|
||||||
})
|
"path": f"/data/input/{item.name}",
|
||||||
|
}
|
||||||
|
)
|
||||||
elif item.is_dir():
|
elif item.is_dir():
|
||||||
resources.append({
|
resources.append(
|
||||||
"name": item.name,
|
{
|
||||||
"description": f"Input directory: {item.name}",
|
"name": item.name,
|
||||||
"kind": "unknown",
|
"description": f"Input directory: {item.name}",
|
||||||
"path": f"/data/input/{item.name}",
|
"kind": "unknown",
|
||||||
})
|
"path": f"/data/input/{item.name}",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
# Create input.json with settings and resources
|
# Create input.json with settings and resources
|
||||||
input_data = {
|
input_data = {
|
||||||
@@ -461,6 +478,7 @@ class ModuleExecutor:
|
|||||||
try:
|
try:
|
||||||
# Create temporary directory for results
|
# Create temporary directory for results
|
||||||
from tempfile import mkdtemp
|
from tempfile import mkdtemp
|
||||||
|
|
||||||
temp_dir = Path(mkdtemp(prefix="fuzzforge-results-"))
|
temp_dir = Path(mkdtemp(prefix="fuzzforge-results-"))
|
||||||
|
|
||||||
# Copy entire output directory from container
|
# Copy entire output directory from container
|
||||||
@@ -489,6 +507,7 @@ class ModuleExecutor:
|
|||||||
|
|
||||||
# Clean up temp directory
|
# Clean up temp directory
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||||
|
|
||||||
logger.info("results pulled successfully", sandbox=sandbox, archive=str(archive_path))
|
logger.info("results pulled successfully", sandbox=sandbox, archive=str(archive_path))
|
||||||
@@ -571,6 +590,7 @@ class ModuleExecutor:
|
|||||||
self.terminate_sandbox(sandbox)
|
self.terminate_sandbox(sandbox)
|
||||||
if input_dir and input_dir.exists():
|
if input_dir and input_dir.exists():
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
shutil.rmtree(input_dir, ignore_errors=True)
|
shutil.rmtree(input_dir, ignore_errors=True)
|
||||||
|
|
||||||
# -------------------------------------------------------------------------
|
# -------------------------------------------------------------------------
|
||||||
@@ -669,4 +689,5 @@ class ModuleExecutor:
|
|||||||
self.terminate_sandbox(container_id)
|
self.terminate_sandbox(container_id)
|
||||||
if input_dir:
|
if input_dir:
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
shutil.rmtree(input_dir, ignore_errors=True)
|
shutil.rmtree(input_dir, ignore_errors=True)
|
||||||
|
|||||||
@@ -60,10 +60,15 @@ class ProjectSettings(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class RegistrySettings(BaseModel):
|
class RegistrySettings(BaseModel):
|
||||||
"""Container registry configuration for module images."""
|
"""Container registry configuration for module images.
|
||||||
|
|
||||||
#: Registry URL for pulling module images.
|
By default, registry URL is empty (local-only mode). When empty,
|
||||||
url: str = Field(default="ghcr.io/fuzzinglabs")
|
modules must be built locally with `make build-modules`.
|
||||||
|
Set via FUZZFORGE_REGISTRY__URL environment variable if needed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#: Registry URL for pulling module images (empty = local-only mode).
|
||||||
|
url: str = Field(default="")
|
||||||
|
|
||||||
#: Default tag to use when pulling images.
|
#: Default tag to use when pulling images.
|
||||||
default_tag: str = Field(default="latest")
|
default_tag: str = Field(default="latest")
|
||||||
|
|||||||
20
ruff.toml
Normal file
20
ruff.toml
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
line-length = 120
|
||||||
|
|
||||||
|
[lint]
|
||||||
|
select = [ "ALL" ]
|
||||||
|
ignore = [
|
||||||
|
"COM812", # conflicts with the formatter
|
||||||
|
"D100", # ignoring missing docstrings in public modules
|
||||||
|
"D104", # ignoring missing docstrings in public packages
|
||||||
|
"D203", # conflicts with 'D211'
|
||||||
|
"D213", # conflicts with 'D212'
|
||||||
|
"TD002", # ignoring missing author in 'TODO' statements
|
||||||
|
"TD003", # ignoring missing issue link in 'TODO' statements
|
||||||
|
]
|
||||||
|
|
||||||
|
[lint.per-file-ignores]
|
||||||
|
"tests/*" = [
|
||||||
|
"ANN401", # allowing 'typing.Any' to be used to type function parameters in tests
|
||||||
|
"PLR2004", # allowing comparisons using unamed numerical constants in tests
|
||||||
|
"S101", # allowing 'assert' statements in tests
|
||||||
|
]
|
||||||
Reference in New Issue
Block a user