feat: Add eight core packages for AI agent observability and project management

🚀 Overview

This PR introduces 8 new packages that add comprehensive AI agent observability, project management, dev server preview, task tracking, and system monitoring capabilities to VibeKit. These packages provide the foundation for monitoring AI agent behavior, managing development projects, tracking system performance, and ensuring secure sandboxed execution.

📦 New Packages Added

1. @vibe-kit/ai-chat - AI Conversation Interface (850KB)

Production-ready chat system with Vercel AI SDK v5 integration for streaming AI conversations.

• Core Features: Claude model support, OAuth + API key authentication, customizable UI components
• Developer Tools: useChat() and useAuthStatus() hooks, server-side handlers, Radix UI + Tailwind theming
• Observability: Real-time streaming monitoring, token usage tracking, comprehensive error boundaries
• Bundle Optimization: Heavy dependencies (react-syntax-highlighter, @anthropic-ai/claude-code) moved to peer dependencies
• 434 lines of React components + 579 lines of MCP-integrated handlers + 37 test files

2. @vibe-kit/mcp-client - Model Context Protocol Client (29KB)

Full-featured MCP client supporting stdio, SSE, and HTTP transports with zero framework dependencies.

• Connection Management: Automatic reconnection, request queuing, environment isolation
• Server Operations: Tool execution, resource management, prompt templates, server lifecycle monitoring
• Configuration: Flexible JSON storage, import/export, metadata separation, multi-transport support
• 283 lines of MCP client + 428 lines of configuration store + 398 lines of server management + 16 test files

3. @vibe-kit/mcp-server - VibeKit MCP Server (7KB)

FastMCP-based server exposing VibeKit's project management as MCP tools for AI agents.

• Available Tools: projects (list/get/search), project_manage (CRUD operations)
• Project Data: Full metadata support, tags, priorities, setup/dev/cleanup scripts, status tracking
• Transports: stdio (default) and HTTP streaming with configurable ports
• 156 lines of tool definitions + 117 lines of server implementation + 5 test files

4. @vibe-kit/taskmaster - Self-Contained Task Management (242KB)

JSON-based Kanban system with .taskmaster folder storage and real-time updates via Server-Sent Events.

• Task Features: 6 status types (pending/in-progress/review/done/deferred/cancelled), priorities, dependencies
• Real-time Sync: SSE-based watching, framework-agnostic API handlers (Next.js/Express/Fastify examples)
• Data Structure: Tag-based organization, subtasks, test strategies, metadata tracking
• Bundle Optimization: UI dependencies (@dnd-kit, chokidar) moved to optional dependencies for lighter core bundle
• 189 lines of task provider + 194 lines of API routes + 8 test files

5. @vibe-kit/projects - Zero-Dependency Project Management (8KB)

Lightweight project lifecycle management with persistent JSON storage in ~/.vibekit directory.

• Project Operations: CRUD operations, multiple query methods (ID/name/path)
• Data Management: Built-in validation, formatting utilities (table/detail views), 8-character ID generation
• File Structure: ~/.vibekit/projects.json database, automatic directory creation
• 128 lines of project manager + comprehensive validation/formatting utilities + 5 test files

6. @vibe-kit/logger - Environment-Aware Structured Logging (15KB)

Centralized logging system with automatic test suppression and component isolation.

• Smart Behavior: Auto-silent in test environments unless VIBEKIT_TEST_LOGS=true, component-scoped instances
• Log Levels: debug/info/warn/error with structured context objects and TypeScript safety
• Integration: Used across all 8 packages for consistent logging, PII sanitization capabilities
• 303 lines of structured logger + 201 lines of PII detection + 7 test files

7. @vibe-kit/preview - Dev Server Preview System (34KB)

Enterprise-grade development server management extracted from dashboard for modular reuse.

• Framework Detection: Automatic detection of Next.js, React (Vite), Vue, Express, Python, and static projects
• Process Management: Robust lifecycle management with proper SIGTERM/SIGKILL handling, port conflict resolution (8080-8129 range)
• Cross-Session Persistence: Lock files in ~/.vibekit/preview-locks/ ensure servers persist across browser sessions
• Resource Optimization: Automatic cleanup of idle servers (1-hour timeout), smart port allocation, memory-bounded log storage
• 805 lines of DevServerManager + 173 lines of SimpleProjectDetector + 144 lines of StaticServer + 161 lines of tests

8. @vibe-kit/monitor - System Performance & Health Monitoring (29KB)

Comprehensive system monitoring with in-memory metrics storage and real-time health checks.

• Performance Metrics: Request tracking, response times, throughput analysis, P95/P99 percentiles, error rate monitoring
• System Health: Component-level health checks, memory usage tracking, resource utilization, automated alerts
• In-Memory Storage: Efficient circular buffers for metrics retention, no database dependencies, automatic cleanup
• Framework Integration: Express/Next.js middleware for automatic request monitoring, health check endpoints, metrics export
• Enterprise Features: Event-driven architecture, graceful degradation, configurable retention periods, error context tracking
• 738 lines of MonitorService + 389 lines of middleware integrations + 182 lines of comprehensive tests

🖥️ Dashboard Integration

Unified Dashboard

• 903 lines of projects overview with analytics and resource usage monitoring
• 1,017 lines of individual project pages with real-time AI interaction tracking
• 878 lines of Kanban board implementation with AI-powered suggestions
• 726 lines of system monitoring dashboard powered by @vibe-kit/monitor package
• 603 lines of MCP server management interface

Package Integration

• Dashboard imports @vibe-kit/preview package for dev server management with clean separation of concerns
• Dashboard imports @vibe-kit/monitor package replacing internal performance monitoring with modular solution
• 569 lines of React preview panel UI components remain in dashboard for iframe rendering
• API routes provide thin layers over PreviewService and MonitorService for HTTP handling
• Enables future CLI integration for vibekit preview and vibekit monitor commands

🧪 Testing & Quality Assurance

Comprehensive Test Coverage

• 95+ test files across all 8 packages with extensive scenarios including system monitoring edge cases
• Integration tests for cross-package validation in /test/ directory
• Performance tests for MCP server operations, resource monitoring, and metrics collection
• Security validation with PII sanitization, vulnerability testing, and monitoring error handling

Quality Standards

• TypeScript strict mode with 100% compilation success
• Vitest for fast, modern testing with coverage reports
• ESLint configuration for consistent code quality

🔧 Technical Features

Cross-Package Integration

• MCP protocol enables AI agents to access project management tools directly
• Server-Sent Events provide real-time updates across chat, tasks, preview, and monitoring systems
• Centralized logging with PII sanitization ensures secure observability across all packages
• System monitoring provides health checks and performance metrics for all running services
• Native npm workspaces build system with tsup bundling for consistent compilation

Security & Validation

• Automatic input validation and structured error handling across all packages
• Environment-aware logging that auto-silences in test environments
• JSON-based configuration storage with metadata separation for clean version control

📈 Impact

Code Statistics

• 8 new packages with full TypeScript support and 95+ comprehensive test files
• Unified chat interface, visual project management, real-time preview system, and comprehensive monitoring
• Extensible architecture with plugin system ready for additional MCP servers and monitoring components
• Backward compatibility maintained with progressive enhancement approach

Bundle Size Analysis

• Total bundle size: ~1.2MB across 8 packages
• Ultra-lightweight core packages: Projects (8KB), MCP Server (7KB), Logger (15KB), Preview (34KB), Monitor (29KB), MCP Client (29KB) - optimized for minimal overhead
• Optimized UI packages: AI Chat (850KB), Taskmaster (242KB) - heavy dependencies externalized
• Modular architecture: Each package can be imported independently, allowing applications to use only needed functionality
• Production-ready: Extremely efficient bundle sizes suitable for edge deployment and CLI tools

📋 Summary

This PR creates a comprehensive ecosystem where AI agents can safely manage development projects through a unified observability layer. The 8 packages work together to form an integrated system:

AI Agent Integration: The ai-chat package provides the conversational interface while the mcp-client and mcp-server packages enable AI agents to execute project management tools directly through the Model Context Protocol, creating a seamless workflow where agents can create projects, manage tasks, and monitor execution status.

Project & Task Management: The projects package handles core project lifecycle operations, while taskmaster provides Kanban-based task tracking with real-time updates. Together, they create a complete project organization system that AI agents can interact with programmatically.

Development Environment: The preview package provides enterprise-grade dev server management with automatic framework detection and cross-session persistence, enabling AI agents to safely start, monitor, and manage development servers for projects they're working on.

System Monitoring & Safety: The monitor package provides comprehensive system health monitoring with in-memory metrics storage, tracking request performance, memory usage, and error rates across all services. Combined with the logger package that ensures all AI agent actions are tracked and sanitized for sensitive data, this creates a complete observability solution while the dashboard integrates all systems to provide real-time visibility into what agents are doing across development environments.

Unified Experience: From initial project creation through task execution to live preview deployment, these packages create an end-to-end development workflow where AI agents operate transparently within secure, observable boundaries while maintaining full project management capabilities.