Skip to content
sysgeeks

Design a Video Processing Pipeline, stage 11 of 14: break it

Deleting a video mid-transcode

Deletion touches every store in the system: the video row, the job row, raw and rendered objects, and whatever the CDN has cached. A worker may be mid-flight. Evaluate each statement about this situation.

System so far· 8 parts
123456789CLIENTInstructorbrowserSERVICEVideo APIDATABASEPostgresOBJECT STOREObject storageWORKERTranscodeworkersWORKERReconcilerEDGECDNCLIENTStudent player

Select a component to see what it is responsible for and which state it owns.

  1. 1Instructor browser → Video API: Create upload, report parts, poll status
  2. 2Instructor browser → Object storage: Upload parts via presigned URLs
  3. 3Video API → Object storage: Complete multipart upload, verify object
  4. 4Video API → Postgres: Video row and job row in one transaction
  5. 5Transcode workers → Postgres: Claim lease, heartbeat, fenced completion
  6. 6Transcode workers → Object storage: Read raw upload, write attempt output
  7. 7Reconciler → Postgres: Find abandoned uploads and orphaned output
  8. 8CDN → Object storage: Origin fetch on cache miss
  9. 9Student player → CDN: Manifest and segments
  • Request / response
  • Bulk data

What you need to know

0 of 2 checks done

  1. Deleting a video touches several stores with no shared transaction: the database rows, objects in storage, and copies cached at CDN edges. A worker may also be in the middle of a job for it.

    A tombstone makes deletion a state rather than an absence: status = 'deleted'. Anything later that's conditioned on another status (like the worker's WHERE status = 'processing') quietly fails to apply.

  2. Check

    Which order is safe if the process can crash at any step?