Cortex

Cortex is a cloud infrastructure platform for deploying, managing, and scaling machine learning models in production. It addresses the operational work of running inference and data workloads by providing serverless compute that handles cluster provisioning, scaling, and resource management on the user's own AWS account.

The platform supports realtime, async, and batch workloads, with model serving across CPU or GPU instances, MLOps services for continuous retraining and evaluation, and compute-intensive microservices. Cluster autoscaling, spot instances with on-demand backups, and multiple configurable environments are included. Provisioning works through declarative configuration or a Terraform provider, while metrics and logs can be sent to external tools or used with pre-built Grafana dashboards and CloudWatch integration. Cortex runs on EKS within a VPC and integrates with IAM for authentication and authorization.

Cortex is aimed at teams running machine learning applications and data processing pipelines on AWS. It is offered in Core and Enterprise editions, each available as self-managed or fully-managed. The Core edition is described as paying only for AWS resources, while the Enterprise edition adds automated cluster upgrades, uptime SLAs, enterprise support, and an admin dashboard; a serverless option is listed as coming soon. A free starting point lets users create a cluster on their AWS account.