Seldon

Seldon is a Kubernetes-native MLOps platform for deploying, monitoring, and governing machine learning and LLM models in production. It addresses the operational challenges teams face once models leave the notebook, covering real-time inference, observability, and compliance. Seldon is now part of TrueFoundry.

The platform includes Seldon Core 2, an open-source framework for orchestrating ML components as Kubernetes microservices with REST and gRPC support; MLServer, a lightweight inference server compatible with the Open Inference Protocol; an LLM Module for GenAI workflows with prompt orchestration, observability, and guardrails; Alibi Detect and Alibi Explain for outlier, drift, and adversarial detection plus model explanation; MPM for model performance metrics; and an Enterprise Module for governance, authentication, audit trails, and team controls. It supports A/B tests, canary deployments, multi-armed bandits, multi-model serving with overcommit, and real-time observability via Prometheus, Grafana, Kafka, and Jaeger. Integrations include Istio, Envoy, ArgoCD, Flux, MLflow, Weights & Biases, Triton, Hugging Face, LangSmith, and storage backends across AWS EKS, Azure AKS, Google GKE, OpenShift, and Digital Ocean.

Seldon is aimed at AI platform engineering teams, enterprise architects, and organizations deploying models at scale across clouds or on-premise environments. Products are packaged as open-source components (Seldon Core 2, MLServer, Alibi) alongside modules (LLM, MPM) and an Enterprise edition for regulated industries, with a modular structure allowing teams to adopt only what they need.