mrvitej.in

DevOps · CNCF daily

Cilium: eBPF-Based Networking, Security and Observability for Kubernetes

CNCF Graduated. A Kubernetes network plugin built on eBPF that provides pod connectivity, network policy, load balancing, a sidecarless service mesh and flow-level visibility through Hubble.

7 min read

CNCF Graduated

  • Cilium
  • Hubble
  • eBPF
  • Kubernetes
  • Gateway API
CNCF project page

Cilium is a networking, security and observability layer for cloud native platforms, implemented with eBPF, which lets small verified programs run inside the Linux kernel. It started as an eBPF-based implementation of the Container Networking Interface (CNI) and has grown into a full data plane: pod networking, identity-based network policy, kube-proxy replacement, load balancing, transparent encryption, multi-cluster connectivity, BGP, a sidecarless service mesh and Layer 3 to 7 flow visibility through Hubble. It addresses the limits of iptables-based networking, where rule lists grow with the number of services and policies are tied to IP addresses that change constantly; Cilium instead attaches policy to workload identity and resolves it in the kernel. The project was created by Isovalent, became a CNCF Incubating project in October 2021 and graduated in October 2023, announced on October 11, 2023.

CNCF daily · Sprint 1, day 4 · Cloud Native Network

At a glance
Category Cloud Native Network
CNCF status Graduated. Incubating since October 2021; graduated October 2023
Written in Go for the control plane, with eBPF programs written in C
License Apache 2.0
Data plane eBPF programs attached to network devices and sockets in the Linux kernel
Main features CNI, network policy, kube-proxy replacement, Cluster Mesh, BGP, WireGuard and IPsec encryption, Gateway API, service mesh
Related projects Hubble for observability and Tetragon for runtime security observability
Release line 1.19 (patch 1.19.5 published June 2026), with 1.20 following

Architecture

flowchart TB
  API[Kubernetes API server<br/>pods, services, policies] -->|watch| OP
  subgraph CP[Control plane]
    OP[Cilium operator<br/>IPAM and cluster tasks]
    KV[Identity store<br/>CRDs or kvstore]
  end
  subgraph NODE[Each node]
    AG[Cilium agent<br/>DaemonSet] -->|compiles and loads| EBPF[eBPF programs<br/>in the kernel]
    EBPF --> NET[Pod traffic<br/>routing, policy, load balancing]
    EBPF --> FLOW[Flow events]
  end
  OP --> AG
  KV --> AG
  FLOW --> HUB[Hubble<br/>per node observer]
  HUB --> REL[Hubble Relay<br/>cluster view]
  REL --> UI[Hubble UI and CLI<br/>Prometheus metrics]
  AG --> CM[Cluster Mesh<br/>other clusters]
Component Responsibility
Cilium agent Runs on every node, learns about pods and policies from the API server, and programs the eBPF data plane.
eBPF programs Handle packet forwarding, policy enforcement, service load balancing and encryption in the kernel, without iptables chains.
Cilium operator Handles cluster-wide tasks such as IP address management and garbage collection.
Identities Each set of pods with the same labels gets a numeric identity, and policy is expressed and enforced against identities rather than IP addresses.
Hubble Observes flows, DNS and HTTP events from the data plane and exports them through a relay, a UI, a CLI and Prometheus metrics.
Cluster Mesh Connects several clusters so services and policies can span them.
Envoy Cilium embeds Envoy for Layer 7 features such as HTTP policy, ingress and Gateway API routing.

Design principle. Cilium moves networking logic out of user space and iptables and into eBPF programs that run at the earliest point in the kernel’s packet path. Policy follows workload identity rather than IP address, so rules stay valid as pods move, and the same data plane that forwards traffic also produces the flow records that Hubble exposes, giving enforcement and visibility from one source.

Production reference design

A common pattern on EKS, GKE, AKS and on-premises clusters, where Cilium is the CNI and replaces kube-proxy. Several managed services offer Cilium directly, and Azure’s CNI integrates it, so check what your provider supports before installing it yourself.

  1. Check prerequisites. Use a recent Linux kernel (Cilium documents minimum versions per feature), and decide whether the cluster is created without kube-proxy and without another CNI. Replacing the CNI on a running cluster is a planned migration, not a casual change.
  2. Install with Helm, enabling kube-proxy replacement, Hubble and encryption in a values file: 
    # values.yaml
kubeProxyReplacement: true
k8sServiceHost: <api-server-host>
k8sServicePort: 6443
operator:
  replicas: 2
hubble:
  relay: { enabled: true }
  ui: { enabled: true }
encryption:
  enabled: true
  type: wireguard
    helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium --version <version> -n kube-system -f values.yaml
cilium status --wait
  3. Verify the data plane with cilium status and the connectivity test (cilium connectivity test) from the Cilium CLI before moving workloads.
  4. Start with default-deny and add policy. Apply identity-based rules, including Layer 7 rules where useful: 
    apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: orders-allow-frontend, namespace: orders }
spec:
  endpointSelector: { matchLabels: { app: orders-api } }
  ingress:
    - fromEndpoints: [{ matchLabels: { app: frontend } }]
      toPorts:
        - ports: [{ port: "8080", protocol: TCP }]
          rules: { http: [{ method: GET, path: "/orders.*" }] }
    Check field names against the Cilium version you install.
  5. Expose services with Cilium’s Gateway API implementation for ingress, and use BGP or the cloud load balancer integration to advertise addresses on bare metal.
  6. Observe and connect. Use Hubble to see denied flows while writing policy, export its metrics to Prometheus, and add Cluster Mesh if services must reach each other across clusters.

Typical production uses include the cluster CNI for large clusters, kube-proxy replacement for faster service load balancing, zero-trust network policy between namespaces, flow visibility for troubleshooting, and multi-cluster service connectivity.

Operational considerations

  • Kernel and platform fit. Features depend on kernel versions and cloud networking modes. Check the compatibility table for your distribution and managed service before choosing features such as native routing or encryption.
  • Roll out policy gradually. Use Hubble to observe real traffic first, apply policies in audit mode where available, and then enforce. Default-deny without that preparation causes outages.
  • Treat CNI changes as migrations. Replacing the CNI or enabling kube-proxy replacement on a live cluster needs a tested plan, node-by-node rollout and a rollback path.
  • Size and monitor the agent. The agent runs on every node, so set resource requests, watch its memory and map usage in large clusters, and alert on agent health, endpoint regeneration and dropped packets.
  • Upgrade carefully. Cilium releases a minor version regularly and documents upgrade notes per version. Read them, upgrade the agent and operator together, and test connectivity afterwards.

Adoption

  • The CNCF graduation announcement listed case studies from Bell Canada, Bloomberg, DB Schenker, S&P Global, Sky and The New York Times, and quoted adopters including Datadog, Tietoevry and Schuberg Philis.
  • The announcement reported more than 800 contributors and maintainers from seven companies, 46 public case studies and well over 100 organisations in the project’s USERS file, and placed Cilium second only to Kubernetes among CNCF projects by commits.
  • Microsoft Azure integrated open-source Cilium into its CNI, and the project’s own ten-year retrospective cites OVHcloud and Zynga, the latter replacing kube-proxy with eBPF load balancing to avoid conntrack scaling problems.

Alternatives

Solution Model Best suited to
Calico CNI with network policy, using iptables or an eBPF data plane Clusters that want a long-established policy engine and flexible data planes
Flannel Simple overlay CNI Small clusters that need basic connectivity only
Cloud provider CNIs (AWS VPC CNI, Azure CNI, GKE) Native cloud networking Teams that want provider support and VPC-native addressing
Antrea Open vSwitch based CNI VMware environments and Windows node support
Istio ambient or Linkerd Service mesh layered on a CNI Teams whose main need is mTLS and Layer 7 traffic management

Strengths and limitations

Strengths Limitations
Identity-based policy and kernel-level enforcement that scale with cluster size Depends on kernel versions and eBPF features, so platform support needs checking
kube-proxy replacement with efficient service load balancing Debugging eBPF behaviour needs specific tools and skills
Hubble provides flow and DNS visibility without extra agents Migrating an existing cluster from another CNI is a careful project
One platform for CNI, policy, encryption, ingress and multi-cluster Many features, so configuration choices and upgrades need discipline
Large community and wide managed-service support Advanced features and support often come through commercial offerings

Recommendation

Adopt Cilium as the CNI for new Kubernetes clusters, particularly large or security-sensitive ones, using the managed offering your cloud provides where it exists. Enable Hubble from the start, roll out network policy from observed traffic, and plan upgrades with the release notes. For small clusters with simple needs, a provider CNI or Flannel may be enough, and an existing Calico estate does not need to move without a clear gain.

References: CNCF project page · CNCF: Cilium graduation announcement · Cilium documentation · Cilium releases on Go packages · Cilium on GitHub

Spotted something wrong or out of date?Suggest a correction. I review every suggestion before changing the page.