Skip to main content

Workflows & workers

Long video pipelines are too slow for a single HTTP request. Streamdat pushes work to Celery workers and tracks status through the API and realtime channel.

Availability

The workflow DAG engine is API-only today: there is no workflow builder in the web app yet. Creating and running workflows requires a plan that includes the workflows feature (Pro or Studio; lower tiers get a 402), and is done through the REST endpoints documented in Swagger. Existing workflows stay viewable after a plan downgrade, but not executable.

Queues

Redis backs multiple named queues (video_queue, ai_queue, io_queue, workflow_queue, and a default queue). Heavy FFmpeg and ML tasks drain from the video and AI queues so interactive API latency stays predictable.

JSON DAG engine

The workflows subsystem stores directed acyclic graphs as JSON. Nodes represent actions (ingest, score, render, post) and edges encode dependencies. Runs are recorded for replay, failure handling, and admin visibility.

User-visible flow

The marketing walkthrough on the homepage workflow section matches the product story end users see; this document describes the machinery underneath.

Where to read more

  • Architecture for how the API and workers share the database and object store.
  • Realtime for job progress delivery to the browser.