AI Context Engineering for Large Repositories

Transform massive repositories into crystallized, AI-consumable knowledge through systematic analysis and optimization.

Just as pressure transforms carbon into diamonds, Context Crystallizer applies systematic AI analysis to transform raw repositories into crystallized knowledge - structured, searchable, and optimized for AI consumption. Each file is analyzed to extract its purpose, key APIs, patterns, and relationships, creating a knowledge base that AI can efficiently search and understand.

Context Crystallizer was inspired by AI Distiller (aid), which pioneered the concept of intelligently extracting essential information from large codebases for AI consumption. While AI Distiller focuses on extracting public APIs and type information using tree-sitter parsers, Context Crystallizer takes a complementary approach by having AI agents generate comprehensive crystallized contexts about functionality, patterns, and relationships.

AI agents hit context length limits when working with large repositories. A typical enterprise repository has 10,000+ files, but LLMs can only process a fraction at once. This forces AI to work blindly or make assumptions about unfamiliar code.

Context Crystallizer creates a searchable crystallized context base of AI-optimized knowledge:

• 🔍 Search by functionality: "find authentication middleware"
• ⚡ Token-efficient: 5:1 compression ratio (source:crystallized context)
• 🤖 AI-optimized format: Structured specifically for LLM consumption
• 📊 Smart assembly: Combines multiple contexts within token limits
• 💎 Crystallized knowledge: Preserves essential information in optimized form

Simple 3-step crystallization process:

1. Initialize: Scan repository and prepare for crystallization
   • Automatically respects .gitignore patterns
   • Skips common build directories (node_modules, dist, build, .git)
   • Filters out binary files and very large files (>1MB)
2. Crystallize: AI analyzes each file to extract meaningful knowledge
3. Search: Find relevant crystallized contexts for any task

Developer using Claude Code with a large documentation repository:

Developer: "I need to understand how authentication works in this massive project"

Claude: "I'll crystallize this repository first to build a searchable knowledge base, then find all authentication-related information."

Claude crystallizes the repository - scanning and analyzing each file

Claude: "Crystallization complete! I found 5 files with authentication logic. The main JWT middleware handles token validation with Redis session caching. Here's how it works..."

Claude provides comprehensive explanation using crystallized contexts

Developer: "What files depend on the authentication system?"

Claude: "Let me search for related crystallized contexts..."

Claude uses find_related_crystallized_contexts() to discover dependencies

# Install globally via NPM
npm install -g context-crystallizer

# Navigate to your project
cd /path/to/your/project

# Start the MCP server
context-crystallizer

Add to your Claude Desktop configuration (~/claude_desktop_config.json):

{
  "mcpServers": {
    "context-crystallizer": {
      "command": "npx",
      "args": ["context-crystallizer"],
      "cwd": "/path/to/your/project"
    }
  }
}

Context Crystallizer provides dual access to crystallization functionality: AI agents can use MCP tools for conversation-driven analysis, while developers can use CLI commands for direct control.

AI agents interact with Context Crystallizer through MCP (Model Context Protocol) for conversation-driven crystallization and knowledge search.

Developers can use direct CLI commands for precise control over crystallization operations.

🔄 Initial Setup & Crystallization

Developer → Start: context-crystallizer
Developer → Configure: Claude Desktop with MCP
Developer → Request: "Crystallize this repository"
Claude → Calls: init_crystallization, get_next_file_to_crystallize, store_crystallized_context
Claude → Reports: Progress and completion

🔍 Daily Development Workflow

Developer → Ask: "How does feature X work?"
Claude → Calls: search_crystallized_contexts
Claude → Explains: Using found crystallized knowledge

Developer → Ask: "What will this change affect?"
Claude → Calls: find_related_crystallized_contexts  
Claude → Warns: About potential impacts

🔧 Maintenance & Updates

Developer → Notification: "I changed some files"
Claude → Calls: update_crystallized_contexts
Claude → Reports: "Updated 3 contexts, all current"

Developer → Question: "Is crystallization still good quality?"
Claude → Calls: validate_crystallization_quality
Claude → Reports: Quality metrics and suggestions

# Install globally
npm install -g context-crystallizer

# Navigate to your project directory
cd /path/to/your/project

# Start the MCP server (required for AI agent integration)
context-crystallizer

The server will start and display:

Context Crystallizer MCP server running... Ready to transform repositories into crystallized knowledge!

1. Claude Desktop Integration

Add to ~/claude_desktop_config.json:

{
  "mcpServers": {
    "context-crystallizer": {
      "command": "npx", 
      "args": ["context-crystallizer"],
      "cwd": "/path/to/your/project"
    }
  }
}

2. MCP-Compatible Clients

Any MCP-compatible client can connect to the server and use the 11 crystallization tools. The server implements the standard MCP protocol for tool discovery and execution.

3. Direct Development

For development and testing:

# Development with hot reload
npm run dev

# Production build
npm run build
npm start

# TypeScript development
npm run dev:mcp

• Server Lifecycle: The MCP server must be running for AI agent integration
• Project Context: Always start the server from your project root directory
• Persistent Storage: Crystallized contexts are saved in .context-crystallizer/ directory
• File Watching: Use update_crystallized_contexts tool after making code changes
• Quality Monitoring: Regular quality validation ensures accurate crystallized knowledge

We welcome contributions! Focus on AI workflow improvements:

• 🐛 Bug Reports: Use our issue templates
• 💡 Feature Requests: Enhance AI context engineering capabilities
• 🔧 Pull Requests: Include crystallization quality validation tests
• 💬 Discussions: Share AI integration patterns

See CONTRIBUTING.md for detailed guidelines.

MIT License - see LICENSE.md for details.

Transform your large repository into crystallized, AI-consumable knowledge. Enable AI agents to work with enterprise-scale projects efficiently.

⭐ Star this repo if Context Crystallizer helps your AI workflows!