Overview

Add lightweight model capability detection with runtime learning to optimize tool execution strategies based on model type.

Motivation

• Currently TunaCode treats all models the same, leading to wasted API calls
• JSON fallback only triggers AFTER native tool calling fails (reactive, not proactive)
• No optimization of parallel execution based on model capabilities
• Need to improve performance for diverse model types (OpenAI, Claude, local models, etc.)

Implementation Details

Core Design

@dataclass
class ModelCapabilities:
    supports_native_tools: bool = True
    supports_parallel: bool = True
    max_parallel_tools: int = 4
    prefers_json_fallback: bool = False
    is_local_model: bool = False
    
    @classmethod
    def detect_from_string(cls, model_string: str) -> 'ModelCapabilities':
        # Pattern matching for known model types
        # Default to capable for unknown models (optimistic)

Deliverables

• Implement ModelCapabilities dataclass with detection logic
• Add runtime learning system that tracks success/failure
• Integrate with agent initialization in agent_config.py
• Adjust tool execution flow based on capabilities:
  • Skip native tools for models that need JSON fallback
  • Adjust parallel batch sizes based on model
  • Optimize timeout values

Pattern Matching Rules

• Full capabilities: gpt-4, claude, opus, sonnet, gemini-2
• Local/limited: ollama, llama, mistral, phi (JSON fallback, no parallel)
• Small/fast: mini, nano, flash, lite (reduced parallelization)
• Unknown: Default to capable (learn from runtime)

Runtime Learning

• Track tool execution success/failure per model
• After 3 failures, mark capability as unsupported
• Persist learned capabilities across sessions
• Merge detected + learned for accurate behavior

Files to Modify

• src/tunacode/core/agents/agent_components/agent_config.py - Detection integration
• src/tunacode/core/agents/agent_components/node_processor.py - Capability-based execution
• src/tunacode/types.py - ModelCapabilities dataclass
• New file: src/tunacode/core/agents/capability_detector.py - Detection logic

Success Criteria

• Model detection correctly identifies tool support for known models
• Runtime learning improves accuracy over time
• Reduced API calls for models that don't support native tools
• Parallel execution adapts to model capabilities
• No performance regression for capable models

Priority

Medium-High - Improves performance and compatibility across diverse models

Benefits

• Immediate performance gains by skipping failed attempts
• Better support for local and alternative models
• Self-improving through runtime learning
• No maintenance burden (optimistic defaults)