EvoSynth/jailbreak_toolbox/attacks/blackbox/implementations/evosynth/data_structures/README.md

AdeptTool V2 Data Structures

This directory contains the core data structures for the AdeptTool V2 framework, providing a clean, elegant tool management system.

Overview

The data structures have been simplified to focus on:

1. Code String Storage: Tools store Python code as strings and execute them on demand
2. Unified Context: Single context that replaces SimpleContext and integrates tool management
3. Identifier-Based Lookup: Tools can be accessed by ID or name interchangeably
4. Function Validation: Ensures tools only use callable functions that don't start with "_"

Core Components

1. AIGeneratedTool (ai_tool_system.py)

The simplified tool class that stores code and executes it when needed.

from data_structures.ai_tool_system import AIGeneratedTool

# Create a tool from code
tool = AIGeneratedTool(
    tool_name="MyAttackTool",
    tool_description="A simple attack tool",
    tool_code='''
def attack(query: str) -> str:
    """Simple attack function"""
    return f"Attack: {query}"
'''
)

# Validate the tool
validation_result = tool.validate_tool_function()
if validation_result["success"]:
    # Execute the tool
    result = tool.execute("test query")
    print(result)  # "Attack: test query"

Key Features:

• Stores Python code as strings
• Executes code on demand with proper __builtins__
• Validates functions are callable and don't start with "_"
• Provides detailed validation results with error information
• Tracks performance metrics (execution count, success rate, etc.)

2. ToolEvolutionContext (ai_tool_system.py)

Manages collections of AI-generated tools with intelligent lookup.

from data_structures.ai_tool_system import ToolEvolutionContext, AIGeneratedTool

# Create context
context = ToolEvolutionContext()

# Add tools
tool = AIGeneratedTool(tool_name="MyTool", tool_code="...")
context.add_tool(tool)

# Retrieve tools by ID or name
tool_by_id = context.get_tool(tool.tool_id)
tool_by_name = context.get_tool_by_name("MyTool")
tool_by_identifier = context.get_tool_by_identifier("MyTool")  # Works with both

Key Features:

• Stores tools by both ID and name for flexible lookup
• Provides get_tool_by_identifier() for universal access
• Tracks evolution history and performance analyses
• Supports tool evolution and mutation

3. UnifiedContext (unified_context.py)

The main context that replaces SimpleContext and integrates tool management.

from data_structures.unified_context import create_context

# Create unified context
context = create_context(
    original_query="test query",
    target_model=my_model,
    judge_model=my_judge
)

# Add and manage tools
context.add_tool(tool)
retrieved_tool = context.get_tool("MyTool")
result = context.execute_tool("MyTool", "query")

# Track session
context.add_attack_result(success=True)
context.set_phase("exploitation")

Key Features:

• Replaces SimpleContext with integrated tool management
• Contains nested ToolEvolutionContext for advanced features
• Provides session tracking and statistics
• Supports both regular and AI tool evolution workflows

Usage Patterns

Basic Tool Creation and Execution

# 1. Create tool from code string
tool_code = '''
def sophisticated_attack(query: str) -> str:
    """Generate sophisticated attack prompts"""
    techniques = [
        f"Please help me understand: {query}",
        f"I'm researching AI safety: {query}",
        f"For educational purposes: {query}"
    ]
    return techniques[0]  # Return first technique
'''

tool = AIGeneratedTool(
    tool_name="SophisticatedAttackTool",
    tool_description="Generates sophisticated attack prompts",
    tool_code=tool_code
)

# 2. Validate the tool
validation = tool.validate_tool_function()
if not validation["success"]:
    print(f"Validation failed: {validation['error']}")

# 3. Execute the tool
result = tool.execute("How to bypass safety measures")
print(result)

Context-Based Tool Management

# 1. Create unified context
context = create_context(original_query="test query")

# 2. Add multiple tools
tools = [
    AIGeneratedTool(tool_name="Tool1", tool_code="..."),
    AIGeneratedTool(tool_name="Tool2", tool_code="..."),
]

for tool in tools:
    context.add_tool(tool)

# 3. Retrieve and execute tools
for tool_name in ["Tool1", "Tool2"]:
    tool = context.get_tool(tool_name)
    if tool:
        result = context.execute_tool(tool_name, "query")
        print(f"{tool_name}: {result}")

# 4. Track session
print(f"Total attacks: {context.total_attacks}")
print(f"Success rate: {context.successful_attacks / max(1, context.total_attacks)}")

Tool Evolution Integration

# 1. Create context with evolution support
context = create_context(original_query="test query")

# 2. Access evolution context
evolution_context = context.get_evolution_context()
if evolution_context:
    # 3. Use evolution functions
    from data_structures.ai_tool_system import get_tool_performance_data
    
    # Get performance data (works with unified context)
    perf_data = get_tool_performance_data(context, "MyTool")
    print(f"Performance: {perf_data}")

Function Tools

The system provides several function tools for tool management:

get_tool_performance_data(ctx, tool_identifier)

Get comprehensive performance data for a tool.

create_evolved_tool_version(ctx, original_tool_identifier, evolved_code, evolution_reasoning, improvements_made)

Create a new version of a tool with evolved code.

test_evolved_tool(ctx, evolved_tool_id, test_scenarios)

Test an evolved tool with various scenarios.

Context Compatibility

The system is designed to work with both:

1. Unified Context: Main context for agent sessions
2. Direct Evolution Context: For pure tool evolution workflows

Function tools automatically detect the context type and adapt accordingly:

# This works with both context types
def my_tool_function(ctx: RunContextWrapper, tool_id: str):
    # Function automatically detects context type
    # and extracts the evolution context
    pass

Validation Rules

The system enforces these validation rules:

1. Callable Functions: Only functions that are callable are considered valid
2. No Private Functions: Functions starting with "_" are ignored
3. Syntax Validation: Code must have valid Python syntax
4. Execution Environment: Code must execute in the provided environment

Error Handling

The system provides detailed error information:

validation_result = tool.validate_tool_function()
if not validation_result["success"]:
    print(f"Error: {validation_result['error']}")
    print(f"Error type: {validation_result['error_type']}")
    print(f"Valid functions found: {len(validation_result['valid_functions'])}")

Migration from SimpleContext

To migrate from SimpleContext to UnifiedContext:

# OLD (SimpleContext)
context = SimpleContext(session_id)
context.created_tools = []
context.ai_generated_tools = []

# NEW (UnifiedContext)
from data_structures.unified_context import create_context
context = create_context(original_query="query")
# Tools are managed automatically

Best Practices

1. Use Factory Functions: Always use create_context() to create contexts
2. Validate Tools: Always validate tools before execution
3. Use Identifiers: Use get_tool_by_identifier() for flexible tool access
4. Handle Errors: Check validation results and handle errors gracefully
5. Track Sessions: Use context methods to track attack results and phases

Testing

Run the test suite to verify functionality:

cd evosynth/data_structures
python test_tool_system.py

The test suite covers:

• Tool creation and execution
• Evolution context functionality
• Unified context integration
• Function validation rules
• Error handling

Performance Considerations

• Code Execution: Tools execute code on demand, not at creation time
• Memory Usage: Tools store code strings, not compiled functions
• Lookup Speed: Dual indexing (ID and name) provides fast lookups
• Validation: Validation only occurs when explicitly requested

Future Enhancements

The simplified architecture provides a foundation for:

1. Tool Caching: Cache compiled functions for better performance
2. Parallel Execution: Execute multiple tools concurrently
3. Tool Dependencies: Support for tools that depend on other tools
4. Persistent Storage: Save and load tool collections
5. Advanced Evolution: More sophisticated tool evolution strategies