Amr Samir | Video Upload & Streaming Pipeline System Design

1. System Scale & Video Architecture Challenges

Video files are massive (hundreds of megabytes to gigabytes). Serving raw MP4 files directly from web servers causes buffering delays, high bandwidth costs, and poor user experience on mobile networks. A scalable video platform requires:

Direct chunked uploads to object storage.

Asynchronous transcoding pipelines generating multiple resolutions (1080p, 720p, 480p, 360p).

Chunked adaptive bitrate streaming via HTTP Live Streaming (HLS).

CDN edge caching.

code

+----------------------------------------------------------------------+
|                     End-to-End Video Pipeline                        |
|                                                                      |
|  [ Client ] ---> [ Direct S3 Upload ] ---> [ Transcoding Cluster ]   |
|                                                   |                  |
|                                                   v                  |
|  [ Player ] <--- [ CDN Edge (HLS) ] <--- [ HLS Manifests (.m3u8) ]   |
+----------------------------------------------------------------------+

2. Adaptive Bitrate Streaming (HLS) Architecture

HLS splits a video into short 2–6 second segments (.ts / .m4s) and generates a master playlist (master.m3u8):

code

#EXTM3U (Master Manifest)
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1920x1080
1080p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=1280x720
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=854x480
480p/index.m3u8

The client video player dynamically shifts resolutions based on the viewer's current real-time bandwidth.

3. Worker Transcoding Pipeline with FFmpeg

typescript

export async function transcodeToHLS(inputPath: string, outputDir: string) {
  // Transcode to 1080p, 720p, 480p with HLS segmentation
  const ffmpegCommand = `
    ffmpeg -i ${inputPath} \
    -filter_complex "[0:v]split=3[v1][v2][v3]; \
      [v1]scale=w=1920:h=1080[v1out]; \
      [v2]scale=w=1280:h=720[v2out]; \
      [v3]scale=w=854:h=480[v3out]" \
    -map "[v1out]" -c:v:0 libx264 -b:v:0 3000k \
    -map "[v2out]" -c:v:1 libx264 -b:v:1 1500k \
    -map "[v3out]" -c:v:2 libx264 -b:v:2 800k \
    -f hls -hls_time 4 -hls_playlist_type vod \
    -master_pl_name master.m3u8 \
    ${outputDir}/%v/index.m3u8
  `;
  await execAsync(ffmpegCommand);
}

Designing Video Upload & Adaptive Bitrate Streaming Pipelines (HLS/DASH)

Recommended Posts

Docker & Kubernetes — From Containerization to Cluster Orchestration

Designing Multi-Tenant SaaS Architecture: 100 to 10,000 Tenants Without Data Leaks

JWT Authentication in Production: Access Tokens, Refresh Token Rotation & Revocation

Related Projects

Related Certificates

Designing Video Upload & Adaptive Bitrate Streaming Pipelines (HLS/DASH)

1. System Scale & Video Architecture Challenges

2. Adaptive Bitrate Streaming (HLS) Architecture

3. Worker Transcoding Pipeline with FFmpeg

4. Key Takeaways