Origin Server for Transcoding + HLS Packaging
go build -o engo .
./engo videos/input.mp4+------------------+ +---------------------+
| Clients | -----> | Job Producer API |
| (Web / Mobile) | | (Receives request) |
+------------------+ +---------------------+
|
v
+------------------------+
| RabbitMQ Queue |
| (transcode_jobs) |
+------------------------+
|
+--------------------------+--------------------------+
| | |
v v v
+------------------+ +------------------+ +------------------+
| Worker 1 | | Worker 2 | ... | Worker N |
| (Go service) | | (Go service) | | (Go service) |
+------------------+ +------------------+ +------------------+
| | |
| +-------------------+------------------+ |
| | FFmpeg Transcoding & HLS Generation | |
| +-------------------+------------------+ |
| | |
| v |
| +------------------------------+ |
| | Azure Blob Storage |<----------+
+-------> | (Upload segments & playlists)|
+------------------------------+
|
v
+------------------------+
| Metadata Store / |
| Notification Service |
| (e.g. Postgres, Redis) |
+------------------------+
|
v
+------------------------+
| Client notified / |
| Job status updated |
+------------------------+
- Blob Fetcher: Fetch video file from blob storage, also has validation for video file.
- Transcoding Manager: Transcode video into multi bitrate hls chunks. Uses worker pool pattern for efficiency.
- HLS Packager:
- create folder for output videos and stores locally
- Generates master.m3u8 pointing to: 1080p/index.m3u8 720p/index.m3u8
- Output Uploader: Upload to blob
- Job Queue (Redis): Handle concurrent jobs, retries
- Webhook: on Complete Notify client when HLS is ready
- Presigned URLs: Secure temporary access to HLS files
- Database: Store job metadata, status tracking
- Health Checks: Production-grade deployment readiness