etcd is a distributed, strongly consistent key-value store. It keeps small amounts of critical data, such as cluster state, configuration and locks, replicated across several members using the Raft consensus algorithm, and exposes it through a gRPC API with watches, leases and transactions. It addresses the problem of coordination: distributed systems need one reliable place to agree on who the leader is, what the configuration is and which members are alive. etcd began at CoreOS in 2013, joined the CNCF as an incubating project in December 2018 and graduated on November 24, 2020. It is best known as the datastore behind every standard Kubernetes control plane. The 3.7 line was announced on July 8, 2026.
CNCF daily · Sprint 2, day 4 · Coordination & Service Discovery
At a glance
| Category | Orchestration & Management › Coordination & Service Discovery |
| CNCF status | Graduated. Incubating December 2018; graduated November 24, 2020 |
| Written in | Go |
| License | Apache 2.0 |
| Current release lines | 3.7 (July 8, 2026), 3.6 (May 15, 2025) and 3.5, which is supported until July 2027 |
| Consensus and storage | Raft for replication; bbolt for the on-disk, multi-version key-value store |
| API | gRPC with range, put, delete, transactions, watches, leases, locks and elections |
Architecture
flowchart TB
CLIENT[Clients<br/>kube-apiserver, etcdctl, apps] ==>|gRPC over TLS, writes| L
CLIENT -->|serializable reads| F1
CLIENT -->|serializable reads| F2
subgraph CL[etcd cluster of three members]
L[Leader member]
F1[Follower member 1]
F2[Follower member 2]
L ==>|Raft log replication| F1
L ==>|Raft log replication| F2
end
L ==> WAL[(Write ahead log<br/>and snapshots)]
L ==> DB[(bbolt database<br/>MVCC key value store)]
DB --> WATCH[Watch and lease services]
DB -.->|etcdutl snapshot save| OBJ[(Object storage<br/>backups)]
classDef actor fill:#eceff1,stroke:#455a64,stroke-width:1.5px,color:#1d1c1a
classDef core fill:#0f766e,stroke:#064e4a,stroke-width:2px,color:#ffffff
classDef ctrl fill:#ede7f6,stroke:#5e35b1,stroke-width:1.5px,color:#1d1c1a
classDef data fill:#e3f2fd,stroke:#1565c0,stroke-width:1.5px,color:#1d1c1a
classDef store fill:#fff3e0,stroke:#e65100,stroke-width:1.5px,color:#1d1c1a
classDef sec fill:#fce4ec,stroke:#ad1457,stroke-width:1.5px,color:#1d1c1a
classDef obs fill:#e8f5e9,stroke:#2e7d32,stroke-width:1.5px,color:#1d1c1a
class CLIENT actor
class L,F1,F2,WATCH core
class WAL,DB,OBJ store
style CL fill:#0f766e14,stroke:#0f766e,stroke-width:2px
Key: teal blocks are the project itself, blue are workloads and proxies, purple are control and configuration, amber are storage, red are security and identity, green are observability, grey are external actors. Thick arrows show the primary path, thin solid arrows other data flows, dashed arrows control, discovery and telemetry.
| Component | Responsibility |
|---|---|
| Raft consensus | Elects a leader and replicates every write to a majority of members before acknowledging it. |
| Write-ahead log | Persists each Raft entry to disk before it is applied, so a member recovers its state after a restart. |
| MVCC store (bbolt) | Keeps revisions of every key, which allows consistent reads, watches from a revision and compaction of old versions. |
| Watch service | Streams changes to clients from a given revision; Kubernetes controllers depend on it. |
| Leases | Time-bound keys that expire unless renewed, used for leader election, locks and liveness. |
| etcdctl and etcdutl | etcdctl talks to a running cluster; etcdutl works on data directories for snapshots, restores and defragmentation. |
Design principle. etcd prefers correctness over throughput. A write succeeds only after a majority of members have persisted it, so the cluster tolerates the loss of a minority of members without losing acknowledged data, at the price of write latency set by disk and network round trips between members.
Production reference design
A typical Kubernetes platform on-premises or on cloud virtual machines runs three etcd members, or five for stricter availability, on dedicated SSD-backed nodes, either stacked on the control plane nodes (the kubeadm default) or as an external cluster.
- Size the cluster for failure tolerance. Three members tolerate one failure and five tolerate two. Use an odd count and spread members across zones that have low latency between them; adding members does not increase write throughput.
- Give etcd fast, dedicated disks. Use local or provisioned-IOPS SSD storage, because fsync latency on the write-ahead log is the main performance factor. Do not share the volume with noisy workloads.
- Configure TLS for clients and peers and bound the database size, for example in the etcd configuration file:
name: etcd-1 data-dir: /var/lib/etcd listen-client-urls: https://10.0.1.11:2379 advertise-client-urls: https://10.0.1.11:2379 listen-peer-urls: https://10.0.1.11:2380 initial-advertise-peer-urls: https://10.0.1.11:2380 initial-cluster: etcd-1=https://10.0.1.11:2380,etcd-2=https://10.0.2.12:2380,etcd-3=https://10.0.3.13:2380 initial-cluster-state: new initial-cluster-token: prod-k8s quota-backend-bytes: 8589934592 auto-compaction-mode: periodic auto-compaction-retention: "1h" client-transport-security: cert-file: /etc/etcd/pki/server.crt key-file: /etc/etcd/pki/server.key client-cert-auth: true trusted-ca-file: /etc/etcd/pki/ca.crt peer-transport-security: cert-file: /etc/etcd/pki/peer.crt key-file: /etc/etcd/pki/peer.key client-cert-auth: true trusted-ca-file: /etc/etcd/pki/ca.crt - Back up on a schedule with a snapshot and copy it off the cluster:
ETCDCTL_API=3 etcdctl --endpoints=https://10.0.1.11:2379 \ --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/client.crt --key=/etc/etcd/pki/client.key \ snapshot save /backup/etcd-$(date +%F-%H%M).db etcdutl snapshot status /backup/etcd-2026-10-16-0600.db -w table - Monitor with the built-in Prometheus metrics: leader changes, WAL fsync and backend commit duration, database size against quota, and proposal failures, and alert on them.
- Rehearse restore. Restore into a fresh data directory with
etcdutl snapshot restore, and practise member replacement, because these are the procedures needed in an incident.
Typical uses are the Kubernetes control plane datastore, service discovery and configuration for other distributed systems (CoreDNS and Rook use it, among others), leader election and distributed locks.
Operational considerations
- Scaling: etcd scales vertically and by limiting what is stored. The recommended data size is a few gigabytes; keep large objects, high-churn data and events out of it, and consider a separate etcd cluster for Kubernetes events on very large clusters.
- High availability: keep a quorum healthy at all times; replace failed members promptly with
etcdctl membercommands, upgrade one member at a time, and never run two members on the same failure domain. - Security: require mutual TLS on client and peer ports, restrict network access to the client port to the API servers, encrypt Kubernetes Secrets at rest in the API server, and protect snapshots because they contain every Secret.
- Cost and sizing: disk latency and network latency between members drive performance more than CPU. Provision for low fsync latency, and expect higher cost for multi-zone replication traffic in the cloud.
- Common pitfalls: the database reaching its quota (etcd then rejects writes until compaction and defragmentation), forgetting compaction and defragmentation, slow disks causing leader elections, and upgrading across minor versions without following the guide. Version 3.6 requires first upgrading to 3.5.20 or later, and 3.7 removes the deprecated
--experimental-*flags, so read the release notes before each minor upgrade.
Adoption
The CNCF graduation announcement named Alibaba, Amazon, Baidu, Cisco, EMC, Google, Huawei, IBM, Red Hat, Uber and Verizon among organisations using etcd, and projects including Kubernetes, CoreDNS, M3, Rook and TiKV as building on it. It reported 200 distinct contributors over the prior twelve months, a Trail of Bits security audit in July 2020 that found no significant issues in the core components, and Jepsen consistency testing in January 2020. Managed Kubernetes services from the major clouds run etcd on behalf of their customers.
Alternatives
| Solution | Model | Best suited to |
|---|---|---|
| Apache ZooKeeper | Mature coordination service using ZAB | Existing ZooKeeper-based systems such as older Kafka and Hadoop stacks |
| HashiCorp Consul (NON CNCF) | Service discovery with a key-value store, Raft-based | Service discovery and mesh with a built-in UI |
| Kine with an SQL database | etcd API on top of PostgreSQL, MySQL or SQLite | Small Kubernetes distributions, as used by K3s |
| Managed control planes (EKS, GKE, AKS) | Provider-run etcd | Teams that prefer not to operate etcd |
| Kafka KRaft and cloud databases | Built-in or managed metadata stores | Systems that embed their own consensus layer |
Strengths and limitations
| Strengths | Limitations |
|---|---|
| Strong consistency with a well-tested Raft implementation | Small data limits; not a general database |
| Watches and leases suit controllers and leader election | Write throughput does not grow by adding members |
| Graduated, vendor-neutral and used by Kubernetes everywhere | Sensitive to disk and network latency |
| Simple gRPC API and clear operational tooling | Needs scheduled compaction, defragmentation and tested backups |
| Downgrade support and feature gates since 3.6 | Minor upgrades carry migration steps and removed flags |
Recommendation
Adopt etcd as the datastore for self-managed Kubernetes and for coordination needs that require strong consistency. Run three or five members on dedicated SSD nodes with mutual TLS, scheduled compaction and tested snapshot restores, and plan minor upgrades one version at a time, noting that 3.5 remains supported until July 2027. Teams on managed Kubernetes do not operate etcd directly and need not run their own.
References: CNCF project page · etcd documentation · CNCF: etcd graduation announcement · Announcing etcd v3.7.0 · Announcing etcd v3.6.0 · endoflife.date: etcd