A browser-based peer-to-peer file sharing application utilizing WebRTC mesh networking and BitTorrent v2 protocol with custom extensions.
- Node.js/npm/bun (for client)
- mkcert (for local SSL certificates)
First, install and setup mkcert for local SSL certificates:
# Install mkcert (choose your platform)
# macOS
brew install mkcert
# Linux
curl -JLO "https://dl.filippo.io/mkcert/latest?for=linux/amd64"
chmod +x mkcert-v*-linux-amd64
sudo cp mkcert-v*-linux-amd64 /usr/local/bin/mkcert
# Windows (using Chocolatey)
choco install mkcert
# Windows (using Scoop)
scoop bucket add extras
scoop install mkcertInstall the local CA and generate certificates:
# Install the local CA in the system trust store
mkcert -install
# Generate certificates for kollator.local and wildcard subdomain
cd certs/
mkcert kollator.local "*.kollator.local"
# This creates:
# - kollator.local+1.pem (certificate)
# - kollator.local+1-key.pem (private key)# Start the client demo (backend services disabled for demo)
./start-testbed.sh- Client Demo: https://kollator.local:5173 (SSL certificate trusted via mkcert)
Note: This demo runs only the client application. Backend services (signaling server, worker, MinIO) are disabled for demonstration purposes.
For full development with backend services, you'll need additional prerequisites:
- Bun runtime (for servers)
- Git (for submodule initialization)
- Docker & Docker Compose (for MinIO S3 storage)
The project uses git submodules for node-datachannel dependencies:
# Clone the repository with submodules
git clone --recursive <repository-url>
# Or if you already cloned without --recursive, initialize submodules:
git submodule update --init --recursiveTo enable backend services, uncomment the relevant sections in start-testbed.sh.
Kollator implements a "Ticket Booth Architecture":
- Front Servers (
server/signaling/): Handle WebRTC signaling and client bootstrapping - Workers (
server/worker/): Provide persistent storage and data retrieval - Clients (
client/): Browser-based file management and P2P transfers
- ✅ Basic WebRTC signaling between components
- ✅ Binary startup scripts for performance
- ✅ SSL certificate generation
⚠️ File chunking needs optimization (see below)- ❌ BitTorrent DHT integration (planned)
- ❌ S3 storage integration (planned)
- ❌ OPFS client storage (planned)
The file chunking needs work due to WebRTC DataChannel limitations:
- Issue: feross/simple-peer#561
- Solution: Use 16KB chunks as recommended
- Module: https://www.npmjs.com/package/chunk-stream
var chunks = require('chunk-stream')
inputStream.pipe(chunks(16000)).pipe(peer)
// or with write:
var chunkStream = chunks(16000)
chunkStream.pipe(peer)
chunkStream.write(myData)References:
-
Certificate Trust Issues
If you're seeing SSL certificate warnings, ensure mkcert is properly installed:
# Check if mkcert CA is installed mkcert -CAROOT # If not installed, install the local CA mkcert -install # Regenerate certificates if needed cd certs/ mkcert kollator.local "*.kollator.local"
Manual Certificate Trust (if mkcert doesn't work):
# Chrome: Settings > Privacy > Manage Certificates > Authorities # Import: certs/_wildcard.kollator.local+3.pem # Firefox: Settings > Privacy & Security > Certificates > View Certificates # Import the certificate under "Authorities" tab # Safari: Double-click the certificate file and add to Keychain # Set trust to "Always Trust" in Keychain Access
-
DNS Resolution
# Ensure kollator.local resolves (done automatically by start-testbed.sh) echo "127.0.0.1 kollator.local" | sudo tee -a /etc/hosts
-
mkcert Installation Issues
macOS: If Homebrew installation fails:
# Alternative installation via direct download curl -JLO "https://dl.filippo.io/mkcert/latest?for=darwin/amd64" chmod +x mkcert-v*-darwin-amd64 sudo mv mkcert-v*-darwin-amd64 /usr/local/bin/mkcert
Linux: If you don't have sudo access:
# Install to user directory mkdir -p ~/.local/bin curl -JLO "https://dl.filippo.io/mkcert/latest?for=linux/amd64" chmod +x mkcert-v*-linux-amd64 mv mkcert-v*-linux-amd64 ~/.local/bin/mkcert export PATH="$HOME/.local/bin:$PATH"
Windows: If package managers aren't available:
# Download from GitHub releases # Visit: https://github.com/FiloSottile/mkcert/releases # Download mkcert-v*-windows-amd64.exe # Rename to mkcert.exe and add to PATH
-
Submodule Issues
If you're getting errors related to node-datachannel or missing dependencies:
# Check submodule status git submodule status # If submodules are not initialized or out of sync: git submodule update --init --recursive # If you need to update submodules to latest commits: git submodule update --remote --recursive # If submodule URLs have changed or you have SSH key issues: git submodule sync --recursive git submodule update --init --recursive
-
Port Conflicts
# Check what's using ports lsof -i :8000 # Signaling server lsof -i :5173 # Client dev server
Logs are stored in the logs/ directory:
client.log- Client dev server logs
If the startup script doesn't work, start the client manually:
# Start client dev server
cd client && npm run devFor full development with backend services, start components manually:
# Terminal 1: Signaling Server
cd server/signaling && bun run index.ts
# Terminal 2: Worker
cd server/worker && bun run index.ts
# Terminal 3: Client
cd client && npm run devTo fully implement the PDR requirements:
-
File Processing Pipeline
- Implement 5MB file chunking
- Add BitTorrent v2 hash generation
- Create .ko torrent file format
-
DHT Integration
- Add
webtorrent/bittorrent-dhtto all components - Implement chunk discovery and announcement
- Add
-
Storage Systems
- Integrate S3/Wasabi for worker persistent storage
- Implement OPFS for client-side storage
- Add LRU caching for workers
-
Advanced Features
- Consistent hashing for worker assignment
- cr-sqlite for front server synchronization
- Fault tolerance and automatic recovery
See individual component READMEs for detailed development information: