K3s: Production Kubernetes in a Single Lightweight Binary

K3s is a lightweight, fully conformant Kubernetes distribution that packages the entire control plane, container runtime and essential add-ons into a single binary of under 100 MB. It removes the main operational barriers to running Kubernetes outside large cloud data centres: heavyweight dependencies, high memory use and complex installation. Created by Rancher Labs in 2019 and now maintained under SUSE, K3s was accepted into the CNCF Sandbox in August 2020. It is the engine behind tools such as k3d and Rancher Desktop, and a common choice for edge sites, on-premises clusters and CI environments.

At a glance

Category	Kubernetes distribution: lightweight and edge
CNCF status	Sandbox, accepted August 2020
Written in	Go, distributed as a single binary for x86_64, ARM64 and ARMv7
License	Apache 2.0
Conformance	Certified Kubernetes, tracking upstream releases
Default datastore	SQLite via Kine; embedded etcd, MySQL, PostgreSQL or external etcd for HA
Bundled add-ons	containerd, Flannel, CoreDNS, Traefik, ServiceLB, local-path provisioner, metrics-server

Architecture

flowchart TB
  subgraph SRV[K3s server node]
    API[API server]
    SCH[Scheduler]
    CM[Controller manager]
    KINE[Kine or embedded etcd]
    DB[(SQLite or etcd)]
    SKL[Kubelet and containerd]
    API --> KINE --> DB
    SCH --> API
    CM --> API
  end
  subgraph AG1[K3s agent node]
    KL1[Kubelet]
    CR1[containerd]
    FL1[Flannel CNI]
  end
  subgraph AG2[K3s agent node]
    KL2[Kubelet]
    CR2[containerd]
    FL2[Flannel CNI]
  end
  KL1 -->|websocket tunnel| API
  KL2 -->|websocket tunnel| API
  USERS[kubectl and CI pipelines] --> API
  LB[Traefik ingress<br/>ServiceLB] --> AG1
  LB --> AG2

Component	Responsibility
k3s server	Runs the API server, scheduler and controller manager in one process, and can also run workloads.
Kine	Translates the etcd API to SQLite, MySQL or PostgreSQL, so a single node needs no etcd cluster.
Embedded etcd	Optional built-in etcd for highly available multi-server clusters (--cluster-init).
k3s agent	Worker node running kubelet, kube-proxy and containerd. It registers with the server over a secure tunnel.
Flannel / Kube-router	Default pod networking (VXLAN) and network-policy enforcement. Can be replaced with Cilium or Calico.
Traefik / ServiceLB	Bundled ingress controller and a simple LoadBalancer implementation for bare-metal clusters.
Local-path provisioner	Dynamic PersistentVolumes backed by node-local storage.

Design principle. K3s is upstream Kubernetes, not a fork. It removes legacy and in-tree cloud-provider code, replaces etcd with a pluggable datastore by default, and bundles the add-ons a bare cluster needs to be useful. The result is one process and one binary to install, upgrade and secure. Applications see standard Kubernetes APIs, and every bundled component can be disabled and replaced.

Production reference design

A three-server, highly available on-premises cluster with dedicated workers, for example on Ubuntu VMs in a private data centre or at an edge site:

1. Bootstrap the first server with embedded etcd:

   curl -sfL https://get.k3s.io | K3S_TOKEN=<shared-secret> sh -s - server --cluster-init

2. Join two more servers for a three-member etcd quorum, which tolerates the loss of one server:

   curl -sfL https://get.k3s.io | K3S_TOKEN=<shared-secret> sh -s - server --server https://<server-1>:6443

3. Add worker nodes as agents:

   curl -sfL https://get.k3s.io | K3S_URL=https://<server-1>:6443 K3S_TOKEN=<shared-secret> sh -

4. Standardise configuration in /etc/rancher/k3s/config.yaml on each server rather than using long command lines. For example, add the load-balancer name to the API certificate and disable bundled components you replace:

   write-kubeconfig-mode: "0644"
   tls-san:
     - k3s.internal.example.com
   disable:
     - traefik
     - servicelb
   node-label:
     - "site=dc-1"

5. Put a fixed registration address (a virtual IP via kube-vip, or an external load balancer) in front of the three servers. Install your chosen ingress (for example NGINX Gateway Fabric) and MetalLB, and manage workloads with Argo CD or Flux.

Typical production uses include on-premises replicas of cloud environments, retail, factory and telecom edge sites, ephemeral CI clusters, ARM devices, and developer environments through k3d or Rancher Desktop.

Operational considerations

• Choose the datastore deliberately. SQLite suits single-server clusters only. Use embedded etcd with three or five servers, or an external database, for HA, and schedule snapshots (etcd-snapshot-schedule-cron) to object storage.
• Upgrades can be automated with the system-upgrade-controller and declarative Plan resources, upgrading servers before agents and one minor version at a time.
• Harden for production. Follow the K3s CIS hardening guide: enable secrets encryption, apply Pod Security Admission and restrict access to the node token.
• Review bundled defaults. Traefik, ServiceLB and local-path storage are convenient but basic. Production clusters often replace them with a preferred ingress, MetalLB and a replicated storage system such as Longhorn.
• Resource sizing. Servers start at about 2 CPU and 2 GB RAM. Etcd-backed servers need low-latency SSD storage, since slow disks are the most common cause of instability.

Adoption

• K3s is the Kubernetes engine behind k3d (K3s in Docker) and Rancher Desktop, and a core part of SUSE’s edge and Rancher product line.
• It is widely used for edge, IoT and homelab clusters on ARM hardware, and for disposable clusters in CI pipelines.
• The CNCF project page does not list named enterprise case studies, so public, attributable adopter data is limited compared with Graduated projects.

Alternatives

Solution	Model	Best suited to
RKE2	Hardened Kubernetes distribution from the same team	Regulated or government environments that need FIPS and CIS defaults
k0s	Single-binary distribution	Similar use cases, with fewer bundled opinions
MicroK8s	Snap-packaged distribution from Canonical	Ubuntu-centric teams, with add-ons enabled by command
Talos Linux	Immutable OS built only to run Kubernetes	API-managed bare-metal clusters without SSH or a general-purpose OS
KubeEdge (CNCF Graduated)	Cloud control plane with edge nodes	Large fleets of edge devices managed from a central cluster
kind / minikube	Local test clusters	Development and CI only, not production

Strengths and limitations

Strengths	Limitations
Installs in under a minute with one command; single binary to patch	Bundled defaults (Traefik, ServiceLB, local-path) often need replacing in production
Low memory footprint; runs well on ARM and small VMs	Sandbox status: less formal maturity than Graduated distributions
Fully conformant upstream Kubernetes, so standard tooling works	Not designed for very large clusters with thousands of nodes
Flexible datastore: SQLite, embedded etcd or external SQL	SQLite mode has no high availability
Large community, plus k3d for identical local and CI clusters	HA still requires planning: a fixed registration address, etcd quorum and backups

Recommendation

Adopt K3s for on-premises, edge and CI clusters, and wherever a full upstream distribution is too heavy. It is also a strong choice for replicating cloud environments locally. Plan for embedded etcd with three servers, scheduled snapshots and replaced ingress and storage components before production use. For regulated environments, evaluate RKE2 from the same maintainers.

References: CNCF project page · K3s documentation · K3s on GitHub