Prefect

Prefect is a workflow orchestration platform for data, machine learning, and agent-based applications. It solves the problem of running Python functions reliably in production by handling scheduling, retries, caching, state management, and crash recovery automatically, so teams do not need to build and maintain their own infrastructure for these tasks.

The platform is built around an open-source Python framework where workflows are written as plain functions decorated with `@flow` and `@task`. Key capabilities include durable execution, observability across all runs and teams, scheduling via cron or events, global concurrency limits, and automation and alerting. Prefect Cloud provides a managed control plane with enterprise features such as role-based access control, SSO, directory sync, audit logs, and IP allowlisting. The platform supports hybrid execution where code and data remain in the customer’s own infrastructure, as well as serverless and fully managed execution models. Prefect also builds FastMCP, an open-source framework for creating MCP servers that connect agents to tools and data.

Prefect is designed for data engineers, ML engineers, and platform teams who need to orchestrate data pipelines, ML workflows, and agent-based processes. The open-source framework is available under the Apache 2.0 license. Prefect Cloud offers a free tier and paid editions with additional enterprise security and support features.