Skip to main content

Workflows & workers

Long video pipelines are too slow for a single HTTP request. Streamdat pushes work to Celery workers and tracks status through the API and realtime channel.

Queues

Redis backs multiple named queues (for example video_queue, ai_queue, workflow_queue, and a default queue). Heavy FFmpeg and ML tasks drain from the video and AI queues so interactive API latency stays predictable.

JSON DAG engine

The workflows subsystem stores directed acyclic graphs as JSON. Nodes represent actions (ingest, score, render, post) and edges encode dependencies. Runs are recorded for replay, failure handling, and admin visibility.

User-visible flow

The marketing walkthrough on the homepage workflow section matches the product story end users see; this document describes the machinery underneath.

Where to read more

  • Architecture for how the API and workers share the database and object store.
  • Realtime for job progress delivery to the browser.