Run Rust-backed Python data pipelines with Rivers

Rivers is worth a bookmark if you build data or ML pipelines and want a Python authoring layer without putting the whole control plane on Python runtime behavior. The project defines assets and repositories in Python, then uses a Rust core for graph resolution, execution planning, scheduling, partition handling, and the web UI.

What to try

• Define simple assets as Python functions and materialize them locally with pip install rivers.
• Use rivers dev MODULE_PATH for a single-process local setup with embedded storage, scheduler, and UI.
• Evaluate it against Dagster-style asset orchestration needs where startup speed, typed execution planning, and Kubernetes deployment matter.
• Check the Kubernetes path if you need CodeLocation CRDs, one-pod-per-step execution, Helm charts, and a separate operator.
• Review the authentication guide before exposing the UI: Rivers documents none, native OIDC with PKCE, and forward-auth mode behind a trusted proxy.

Good fit

Rivers looks most useful for teams experimenting with data asset orchestration, ML pipeline scheduling, or internal pipeline platforms that want Python ergonomics plus a compiled Rust scheduler/control plane. It also has practical production hooks: SurrealDB-backed state, Server-Sent Events for UI updates, Kubernetes-native execution, backfills, sensors, schedules, concurrency limits, and pluggable IO.

Caveats

This is still a young project. The GitHub repository was created in May 2026 and the PyPI package is at 0.4.0, so treat it as an evaluation candidate rather than a drop-in replacement for mature orchestrators. The repository license is AGPL-3.0, which matters for internal platform and SaaS use. LinkLoot has not performed a hands-on review; verify installation, security posture, and operational behavior in a disposable environment before using it with production workloads.