🏠 Self-Hosted Telemetry Solution - Keep your data private, secure, and under your control

Malti is a comprehensive self-hosted telemetry collection and visualization system designed for monitoring application performance and load metrics. It provides real-time insights into HTTP request patterns, response times, error rates, and system health across multiple services and nodes - all running on your own infrastructure.

• 🏠 Self-Hosted: Complete control over your data and infrastructure
• 🔒 Data Privacy: No third-party services, all data stays on your servers
• 💰 Cost-Effective: No per-request fees or subscription costs
• ⚡ Real-time Telemetry Collection: Automatic HTTP request monitoring with minimal overhead
• 🌐 Multi-Service Support: Monitor multiple services and nodes
• 📊 Time-Series Analytics: Built on TimescaleDB for efficient time-series data storage and querying
• 🎨 Interactive Dashboard: Modern React-based web interface with Material-UI components
• 🔌 API-First Design: RESTful API for both data ingestion and querying
• 🛡️ Rate Limiting: Built-in protection against abuse and overload

Malti Dashboard - Real-time telemetry monitoring interface

• Real-time Metrics: Live telemetry data visualization with 1-minute resolution
• Interactive Filtering: Filter by service, node, endpoint, method, request context, and time range
• Multiple Visualizations:
  • Response time trends over time
  • Top endpoints by request count
  • Consumer distribution analysis
  • System health indicators
  • Error rate monitoring
• Responsive Design: Optimized for desktop, tablet, and mobile devices
• Time Range Selection: Quick access to 1h, 6h, 24h, 7d, 30d, 3m, 6m, 1y data
• Configurable Thresholds: Color-coded metrics based on customizable performance thresholds

• Automatic Aggregation: Continuous aggregates for 5-minute and 1-hour intervals
• Data Retention: Configurable retention policies for raw and aggregated data
• Batch Processing: Efficient batch ingestion with overflow protection
• Connection Pooling: Optimized database connections for high throughput

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Services      │    │   Malti API     │    │  TimescaleDB    │
│   (Your Apps)   │───▶│   (FastAPI)     │───▶│   (Database)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                              │
                              ▼
                       ┌─────────────────┐
                       │   Dashboard     │
                       │   (React)       │
                       └─────────────────┘

• API Layer: FastAPI with async/await support
• Database: TimescaleDB with hypertables and continuous aggregates
• Authentication: Service and user API key management
• Rate Limiting: SlowAPI-based request throttling
• Frontend: React with Material-UI and Recharts

🔒 Complete Data Control

• Your telemetry data never leaves your infrastructure
• No third-party access to sensitive application metrics
• Full compliance with data residency requirements

💰 Cost Predictability

• No per-request fees or usage-based pricing
• Predictable infrastructure costs
• No vendor lock-in or subscription dependencies

⚡ Performance & Reliability

• No network latency to external services
• No dependency on external service availability

🔧 Customization

• Modify and extend functionality as needed
• Integrate with your existing monitoring stack
• Custom retention policies and data processing

Minimum System Requirements:

• CPU: 2 cores
• RAM: 4GB (8GB recommended for production)
• Storage: 20GB SSD (more for longer retention)
• Network: Stable internet connection

Recommended for Production:

• CPU: 4+ cores
• RAM: 16GB+
• Storage: 100GB+ SSD with backup strategy
• Network: High-bandwidth, low-latency connection

• Docker and Docker Compose
• Node.js 20+ (for development)
• Python 3.12+ (for development)

1. Clone the repository:

   git clone <repository-url>
   cd malti

2. Configure for your environment:

   # Edit configuration file
   cp config/malti.toml config/malti.production.toml
   # Update API keys, service names, and settings

3. Start your self-hosted instance:

   docker-compose -f docker-compose.dev.yml up -d

4. Access your private dashboard: Open http://your-server:8000 in your browser

5. Login with your configured credentials:

1. Backend Development:

   # Install Python dependencies
   pip install -r requirements.txt
   
   # Start TimescaleDB
   docker-compose -f docker-compose.dev.yml up -d timescaledb
   
   # Run the backend
   uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

2. Frontend Development:

   cd dashboard
   npm install
   npm run dev

3. Run Tests:

   python test/run_tests.py

[services]
[services.user-service]
api_key = "user-service-key"
description = "User management service"

[services.test-service]
api_key = "test-service-key"
description = "Test service"

[users]
# Define users who can query metrics
[users.user1]
username = "user1"
api_key = "user-api-key"

[api]
base_url = "http://localhost:8000"
ingest_endpoint = "/api/v1/ingest"
metrics_endpoint = "/api/v1/metrics"

[aggregation]
batch_size = 1000
batch_interval_seconds = 60

[dashboard.thresholds]
# Error rate thresholds (in percentage) - used for coloring error rate displays
error_rate_success_threshold = 0.5     # Error rate <= 0.5% displays as green (success)
error_rate_warning_threshold = 1.0     # Error rate > 0.5% and <= 1.0% displays as yellow/orange (warning)
                                       # Error rate > 1.0% displays as red (error)

# Latency thresholds (in milliseconds) - used for coloring latency displays
latency_success_threshold = 300        # Latency <= 300ms displays as green (success)
latency_warning_threshold = 800       # Latency > 300ms and <= 800ms displays as yellow/orange (warning)
                                       # Latency > 800ms displays as red (error)

• DATABASE_URL: PostgreSQL connection string
• MALTI_CONFIG_PATH: Path to configuration file
• LOG_LEVEL: Logging level (DEBUG, INFO, WARNING, ERROR)

• MALTI_SERVICE_NAME: Service name for telemetry
• MALTI_API_KEY: Service API key
• MALTI_URL: Malti server URL
• MALTI_NODE: Node identifier
• MALTI_BATCH_SIZE: Batch size for telemetry (default: 500)
• MALTI_BATCH_INTERVAL: Batch interval in seconds (default: 60)

All API endpoints require authentication via the X-API-Key header.

Service API Keys: Used for data ingestion (/api/v1/ingest) User API Keys: Used for data querying (/api/v1/metrics/*)

POST /api/v1/ingest
Content-Type: application/json
X-API-Key: your-service-api-key

{
  "requests": [
    {
      "service": "auth-service",
      "method": "POST",
      "endpoint": "/api/v1/login",
      "status": 200,
      "response_time": 150,
      "consumer": "web-app",
      "node": "node-1",
      "created_at": "2025-01-01T12:00:00Z"
    }
  ]
}

GET /api/v1/metrics/aggregate?service=auth-service&start_time=2025-01-01T00:00:00Z&end_time=2025-01-01T23:59:59Z
X-API-Key: your-user-api-key

GET /api/v1/metrics/aggregate/realtime?service=auth-service
X-API-Key: your-user-api-key

GET /api/v1/auth/test
X-API-Key: your-user-api-key

# Configure via environment variables
import os
os.environ['MALTI_SERVICE_NAME'] = 'my-service'
os.environ['MALTI_API_KEY'] = 'my-service-api-key'
os.environ['MALTI_URL'] = 'http://localhost:8000'

from fastapi import FastAPI
from malti_telemetry.middleware import MaltiMiddleware

app = FastAPI()

# Add telemetry middleware (route patterns automatically extracted!)
app.add_middleware(MaltiMiddleware)

For Starlette-based frameworks (like FastAPI), check out our Python/Starlette integration package which provides middleware and utilities for seamless telemetry collection.

• 1 Hour: Real-time data with 1-minute resolution
• 6 Hours: 5-minute aggregated data
• 24 Hours: 5-minute aggregated data
• 7 Days: 1-hour aggregated data
• 30 Days: 1-hour aggregated data
• 3 Months: 1-hour aggregated data
• 6 Months: 1-hour aggregated data
• 1 Year: 1-hour aggregated data

• Service: Select specific services to monitor
• Node: Filter by specific nodes (requires service selection)
• Endpoint: Filter by specific API endpoints
• Time Range: Quick selection buttons for common time periods

• Metrics Cards: Overview of total requests, average latency, error rates
• Status Indicator: System health overview with color-coded status
• Latency Chart: Response time trends with min/max/average lines
• Endpoints Chart: Top endpoints by request count with error indicators
• Consumer Chart: Request distribution by consumer type

The dashboard uses configurable thresholds to provide visual indicators for system health and performance metrics. These thresholds are defined in your malti.toml configuration file and determine the color-coding of metrics throughout the dashboard.

Error rates are displayed with color-coded indicators:

• 🟢 Green (Success): Error rate ≤ 0.5%
• 🟡 Yellow/Orange (Warning): Error rate > 0.5% and ≤ 1.0%
• 🔴 Red (Error): Error rate > 1.0%

Response times are color-coded based on performance:

• 🟢 Green (Success): Average latency ≤ 300ms
• 🟡 Yellow/Orange (Warning): Average latency > 300ms and ≤ 800ms
• 🔴 Red (Error): Average latency > 800ms

These thresholds apply to all dashboard visualizations including metrics cards, charts, and status indicators.

• 5-minute aggregates: requests_5min (90-day retention)
• 1-hour aggregates: requests_1hour (720-day retention)

• Raw data: 6 hours
• 5-minute aggregates: 90 days
• 1-hour aggregates: 720 days

# Run all tests
python test/run_tests.py

# Run specific test modules
python -c "from test.test_health import TestHealthEndpoints; TestHealthEndpoints().run_all_tests()"
python -c "from test.test_ingest import TestIngestEndpoint; TestIngestEndpoint().run_all_tests()"
python -c "from test.test_metrics import TestMetricsEndpoints; TestMetricsEndpoints().run_all_tests()"

• ✅ Health endpoints
• ✅ Authentication and authorization
• ✅ Data ingestion validation
• ✅ Metrics querying with filters
• ✅ Rate limiting
• ✅ Error handling

1. Prepare Your Server:

   # Update system packages
   sudo apt update && sudo apt upgrade -y
   
   # Install Docker and Docker Compose
   curl -fsSL https://get.docker.com -o get-docker.sh
   sudo sh get-docker.sh
   sudo usermod -aG docker $USER

2. Configure Your Self-Hosted Instance:

   # Clone and configure
   git clone <repository-url>
   cd malti
   
   # Create production configuration
   cp config/malti.toml config/malti.production.toml
   
   # Edit with your settings:
   # - Generate secure API keys
   # - Configure your services
   # - Set up user accounts
   # - Adjust retention policies

3. Deploy with Caddy (Recommended - Automatic HTTPS):

   # Use the production Docker Compose with Caddy
   cp .env.prod.example .env.prod
   # Edit .env.prod with your domain and database password
   
   # Deploy with automatic HTTPS
   docker-compose -f docker-compose.prod.yml --env-file .env.prod up -d

4. Health Check Your Self-Hosted Instance:

   # Check if services are running
   docker-compose -f docker-compose.prod.yml ps
   
   # Test HTTPS endpoint
   curl https://yourdomain.com/health
   
   # Check Caddy logs
   docker-compose -f docker-compose.prod.yml logs caddy

System Monitoring:

# Monitor resource usage
docker stats

# Check logs (production with Caddy)
docker-compose -f docker-compose.prod.yml logs -f malti-backend
docker-compose -f docker-compose.prod.yml logs -f caddy
docker-compose -f docker-compose.prod.yml logs -f timescaledb

# Database monitoring
docker exec -it malti-timescaledb psql -U malti_user -d malti -c "SELECT * FROM timescaledb_information.hypertables;"

# Check Caddy automatic HTTPS status
docker exec -it malti-caddy caddy list-certificates

Backup Strategy:

# Database backup
docker exec malti-timescaledb pg_dump -U malti_user malti > backup_$(date +%Y%m%d_%H%M%S).sql

# Configuration backup
cp -r config/ backups/config_$(date +%Y%m%d_%H%M%S)/

Zero-Downtime Updates:

# Pull latest changes
git pull origin main

# Rebuild and restart (production with Caddy)
docker-compose -f docker-compose.prod.yml down
docker-compose -f docker-compose.prod.yml pull
docker-compose -f docker-compose.prod.yml --env-file .env.prod up -d

# Verify update
curl https://yourdomain.com/health

Network Security:

• Use firewalls to restrict access
• Implement VPN access for remote monitoring
• Use strong API keys and rotate regularly
• Caddy provides automatic SSL/TLS encryption

Data Security:

• Regular database backups
• Encrypted storage volumes
• Access logging and monitoring
• Regular security updates
• Caddy automatic certificate management

• Complete Data Ownership: All telemetry data remains on your infrastructure
• No External Dependencies: No third-party services or data sharing
• Compliance Ready: Meet GDPR, HIPAA, and other regulatory requirements
• Audit Trail: Full control over access logs and data retention

• Service API Keys: For data ingestion (configured in malti.toml)
• User API Keys: For dashboard access and metrics querying
• Rate Limiting: Prevents abuse of authentication endpoints
• Role-Based Access: Granular permissions for different user types

• Network Isolation: Run on private networks with VPN access
• API Key Management: Rotate keys regularly and use strong passwords
• SSL/TLS Encryption: Always use HTTPS in production
• Database Security: Restrict network access to TimescaleDB
• Regular Updates: Keep system and dependencies updated
• Backup Strategy: Regular encrypted backups of data and configuration
• Monitoring: Monitor authentication logs and access patterns
• Firewall Rules: Restrict access to necessary ports only

malti/
├── app/                   # FastAPI backend
│   ├── api/               # API routes
│   ├── core/              # Core functionality
│   ├── models/            # Data models
│   ├── services/          # Business logic
│   └── static/            # Built dashboard files
├── dashboard/             # React frontend
│   ├── src/               # Source code
│   ├── public/            # Static assets
│   └── package.json       # Dependencies
├── config/                # Configuration files
├── database/              # Database initialization
├── lib/                   # Client library
├── test/                  # Test suite
└── docker-compose.*.yml   # Docker configuration

1. Backend: Add routes in app/api/, services in app/services/
2. Frontend: Add components in dashboard/src/components/
3. Database: Update schema in database/init.sql
4. Tests: Add tests in test/ directory

• Python: Follow PEP 8, use type hints
• JavaScript: ESLint configuration included
• Database: Use proper indexing and query optimization

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run the test suite
6. Submit a pull request

This project is licensed under the MIT License - see the LICENSE file for details.

Common Self-Hosting Issues:

1. "Connection refused" errors:

   • Ensure TimescaleDB is running: docker-compose -f docker-compose.dev.yml ps
   • Check firewall settings and port accessibility
   • Verify Docker network connectivity

2. Authentication failures:

   • Check API keys in your malti.toml configuration
   • Verify service and user key configurations
   • Ensure proper file permissions on config files

3. Dashboard not loading:

   • Verify backend is running on port 8000
   • Check reverse proxy configuration if using one
   • Ensure SSL certificates are valid

4. No data in dashboard:

   • Check service API keys and data ingestion
   • Verify network connectivity from your services to Malti
   • Check database connection and data retention policies

5. Performance issues:

   • Monitor system resources: docker stats
   • Check database query performance
   • Review batch sizes and intervals

Self-Hosting Debugging:

# Check all services status
docker-compose -f docker-compose.prod.yml ps

# View logs
docker-compose -f docker-compose.prod.yml logs -f malti-backend
docker-compose -f docker-compose.dev.prod logs -f timescaledb

# Test database connectivity
docker exec -it malti-timescaledb psql -U malti_user -d malti -c "SELECT version();"

# Check API health
curl http://localhost:8000/health

# Verify configuration
docker exec -it malti-backend cat /app/config/malti.production.toml

For High-Volume Self-Hosting:

• Database Optimization: Increase TimescaleDB memory settings
• Batch Processing: Tune batch sizes and intervals for your workload
• Connection Pooling: Optimize database connection settings
• Storage: Use SSD storage for better I/O performance
• Network: Ensure high-bandwidth connections for data ingestion

Resource Monitoring:

# Monitor system resources
htop
docker stats

# Database performance
docker exec -it malti-timescaledb psql -U malti_user -d malti -c "SELECT * FROM pg_stat_activity;"

# Check TimescaleDB hypertables
docker exec -it malti-timescaledb psql -U malti_user -d malti -c "SELECT * FROM timescaledb_information.hypertables;"

Disclaimer: Malti is not affiliated with the services listed below and is not compatible with their libraries or APIs. These are independent alternatives with different features and integrations, offering a SaaS solution to API monitoring.

A modern API monitoring and analytics platform with:

• Real-time API metrics and error tracking
• Request logging with privacy controls
• Custom alerts and uptime monitoring
• Support for FastAPI, Flask, Django, Express, NestJS, and more
• Developer-friendly with minimal code changes

Open-source alternative for API monitoring with:

• Framework integrations for FastAPI, Flask, Django, Express, NestJS, and more
• Real-time analytics dashboard
• Error tracking and performance monitoring
• Automated active health checks for resources
• Self-hosted option available

Special thanks to Apitally and API Analytics for inspiring the development of Malti as a lightweight self-hosted alternative.