mrvitej.in

DevOps · CNCF daily

KEDA: Event-Driven Autoscaling for Kubernetes Workloads

CNCF Graduated. Scales Kubernetes workloads from zero to many replicas based on events such as queue depth, stream lag or database metrics, rather than CPU and memory alone.

7 min read

CNCF Graduated

  • KEDA
  • Kubernetes
  • HPA
  • Apache Kafka
  • Prometheus
CNCF project page

KEDA (Kubernetes Event-driven Autoscaling) is a lightweight component that adds event-driven scaling to any Kubernetes cluster. It reads metrics from external systems such as message queues, streams, databases and monitoring tools, and scales Deployments, StatefulSets, Jobs and other scalable resources up from zero, down to zero, or anywhere in between. It addresses a gap in the built-in Horizontal Pod Autoscaler, which reacts to CPU and memory and cannot see that a queue holds ten thousand unprocessed messages while the pods sit idle. Microsoft and Red Hat created KEDA in May 2019. It joined the CNCF Sandbox in March 2020, moved to Incubating in August 2021 and was announced as Graduated on August 22, 2023.

CNCF daily · Sprint 2, day 3 · Scheduling & Orchestration

At a glance
Category Scheduling & Orchestration
CNCF status Graduated. Sandbox March 2020; Incubating August 2021; Graduated August 2023
Written in Go
License Apache 2.0
Current release line 2.20 (June 2026; 2.20.2 in July 2026), with later releases published on the project’s GitHub Releases page
Custom resources ScaledObject, ScaledJob, TriggerAuthentication, ClusterTriggerAuthentication
Event sources Dozens of built-in scalers for cloud queues, streams, databases, Prometheus and more, plus an external scaler interface for custom sources

Architecture

flowchart TB
  SRC[Event sources<br/>queues, streams, databases, Prometheus] --> OP[keda-operator<br/>watches ScaledObject and ScaledJob]
  OP -->|activate and deactivate<br/>zero and one replicas| WL[Target workload<br/>Deployment, StatefulSet or Job]
  OP -->|create and manage| HPA[Horizontal Pod Autoscaler<br/>one to N replicas]
  HPA -->|query external metrics| MS[keda-metrics-apiserver<br/>external metrics adapter]
  MS -->|poll scaler| SRC
  HPA -->|set replicas| WL
  TA[TriggerAuthentication<br/>Secrets or workload identity] --> OP
  TA --> MS
  WH[Admission webhooks<br/>validate resources] --> OP
Component Responsibility
keda-operator Watches ScaledObject and ScaledJob resources, checks each trigger and handles activation: moving a workload between zero and one replica.
keda-metrics-apiserver Serves the Kubernetes external metrics API, so the standard HPA can scale from one replica to N using the same scaler values.
Admission webhooks Validate KEDA resources and block unsafe combinations, such as two ScaledObjects targeting one workload.
ScaledObject Binds a workload to one or more triggers with minimum and maximum replica counts, polling interval and cooldown.
ScaledJob Creates a Kubernetes Job per batch of events, suited to long-running tasks that should not be interrupted by scale-down.
TriggerAuthentication Holds credentials or workload identity references for a scaler, kept separate from the workload so secrets are not placed in pod specs.

Design principle. KEDA does not replace the HPA; it extends it. The operator handles the part Kubernetes cannot do natively, activation from and to zero, and feeds external metrics to a standard HPA for everything in between. Applications need no changes and no KEDA-specific code, and removing KEDA leaves the workload as an ordinary Deployment.

Production reference design

A typical deployment on EKS, GKE, AKS or on-premises Kubernetes scales a queue or stream consumer from event backlog, while keeping a minimum replica count for latency-sensitive services.

  1. Install KEDA with the official Helm chart, pinning the chart version: 
    helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace --version <chart-version>
    For high availability, run more than one operator replica; the operator uses leader election so only one instance acts at a time. Install the chart’s default webhooks.
  2. Store credentials separately with a TriggerAuthentication, preferring workload identity (IRSA on EKS, Workload Identity on GKE, Azure Workload Identity) over static secrets.
  3. Declare the scaling rule for a Kafka consumer, scaling on consumer group lag: 
    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: orders-consumer
  namespace: shop
spec:
  scaleTargetRef:
    name: orders-consumer
  minReplicaCount: 0
  maxReplicaCount: 30
  pollingInterval: 15
  cooldownPeriod: 300
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: events-kafka-bootstrap.kafka:9092
        consumerGroup: orders-consumer
        topic: orders
        lagThreshold: "100"
        activationLagThreshold: "1"
  4. Add a safety net with fallback replicas so the workload keeps a sensible size if the event source becomes unreachable, and an advanced.horizontalPodAutoscalerConfig.behavior block to slow scale-down.
  5. Scale batch work with ScaledJob where each message batch should run to completion as its own Job rather than being cut off by scale-in.
  6. Observe with KEDA’s Prometheus metrics (scaler errors, latency, active status) and alert on scaler failures, not only on replica counts.

Typical production uses are queue and stream consumers (SQS, Service Bus, Kafka, RabbitMQ, Pub/Sub), scheduled and cron-style scaling, scaling web workloads on Prometheus queries, and cost reduction by scaling idle workloads to zero.

Operational considerations

  • Scaling: pick lagThreshold or equivalent as messages per replica that a pod can clear in an acceptable time, then set maxReplicaCount below what the source and downstream services can take. For Kafka, replicas beyond the partition count add no throughput.
  • High availability: run multiple operator and metrics server replicas, and set fallback so an unreachable source does not scale a critical service to zero. Keep a nonzero minReplicaCount for services with strict latency needs, since cold starts add delay.
  • Security: use TriggerAuthentication with workload identity or Vault-backed secrets, restrict who can create ScaledObjects, and review RBAC changes when upgrading; the 2.20 release moved event recording to the events.k8s.io API group, which matters for restricted RBAC.
  • Cost and sizing: KEDA is light, but each ScaledObject polls its source every pollingInterval; at thousands of objects, lengthen intervals and watch load on the sources and the API server. Cloud queue APIs bill per request.
  • Common pitfalls: pairing a ScaledObject with a separate HPA on the same workload; combining CPU or memory triggers with scale-to-zero without understanding activation; scaler configuration fields removed across releases, which makes reading the release notes before each minor upgrade necessary.

Adoption

The CNCF incubation announcement named Alibaba, CastAI, KPMG, Meltwater and Microsoft among KEDA’s users, and the graduation announcement states that more than 45 end-user organisations are listed by the project. Microsoft offers KEDA as a managed add-on for Azure Kubernetes Service, and Red Hat supports it in OpenShift through the Custom Metrics Autoscaler operator. The project’s end-user list on keda.sh names further organisations.

Alternatives

Solution Model Best suited to
Horizontal Pod Autoscaler with custom metrics Built-in, using a Prometheus adapter Teams already exposing metrics and content to keep a minimum of one replica
Knative Serving (CNCF Graduated) Request-driven serverless platform with scale to zero HTTP workloads wanting a full serverless model
Karpenter (NON CNCF, listed in the CNCF Landscape) Node-level provisioning Scaling nodes to match pods, used alongside KEDA rather than instead of it
Cloud-native functions (AWS Lambda, Azure Functions, Cloud Run) Fully managed event handling Teams that prefer not to run containers for event workloads
Vertical Pod Autoscaler Adjusts pod requests and limits Rightsizing resources rather than replica counts

Strengths and limitations

Strengths Limitations
Scale to and from zero on any Kubernetes cluster Scale from zero adds cold-start delay
Large library of built-in scalers and an external scaler API Each trigger depends on the availability of its source
Extends the standard HPA instead of replacing it Only one ScaledObject per workload; mixing with a separate HPA conflicts
Credentials kept apart from workloads via TriggerAuthentication Scaler fields change across releases, so upgrades need review
No application code changes and vendor-neutral governance Does not scale nodes, which needs a tool such as Karpenter or Cluster Autoscaler

Recommendation

Adopt KEDA for any Kubernetes platform that runs queue, stream or schedule-driven workloads, or that wants idle workloads to cost nothing. Start with one consumer, a conservative maxReplicaCount, fallback replicas and workload identity for authentication, and pair it with a node autoscaler so new pods find capacity. Teams with only CPU-bound web services can stay with the standard HPA.

References: CNCF project page · KEDA documentation · KEDA graduation announcement · CNCF: KEDA moves to Incubating · KEDA releases on GitHub · KEDA on GitHub

Spotted something wrong or out of date?Suggest a correction. I review every suggestion before changing the page.