Archive · Kubernetes

On-Prem Platform, Part 1: Ubuntu on WSL2, Docker, k3s and the Data Layer

Build a local or on-premises platform on Windows with WSL2: systemd, networking, Docker, a k3s cluster, namespaces, and Kubernetes manifests for MySQL, MongoDB, Redis and Kafka.

8 min read

  • Ubuntu 24.04 LTS on WSL2
  • Docker Engine
  • k3s
  • MySQL 8.4 LTS
  • MongoDB 8.0
  • Redis
  • Apache Kafka 4

A full copy of a cloud application stack on one machine is useful for development, demos, training and testing changes without paying for a cloud environment. This two-part series builds one on a Windows machine using WSL2 Ubuntu, Docker and k3s. Part 1 covers the platform and the data layer. Part 2 covers the application: Angular, Node.js, S3 uploads with presigned URLs, a CDN, SMTP and notifications.

Architecture

flowchart TB
  subgraph WIN[Windows host]
    BR[Browser and tools]
    subgraph WSL[WSL2 Ubuntu 24.04]
      DK[Docker Engine<br/>image builds]
      subgraph K3S[k3s cluster]
        ING[Traefik ingress<br/>TLS]
        subgraph APPS[namespace apps]
          UI[Angular UI]
          API[Node.js API]
          WK[Notification worker]
        end
        subgraph DATA[namespace data]
          MY[MySQL 8.4]
          MG[MongoDB 8.0]
          RD[Redis]
          KF[Kafka 4 KRaft]
        end
        subgraph SUP[namespace platform]
          LS[LocalStack S3]
          MP[Mailpit SMTP]
        end
      end
    end
  end
  BR --> ING
  ING --> UI
  ING --> API
  API --> MY
  API --> MG
  API --> RD
  API --> KF
  KF --> WK
  WK --> MP
  API --> LS
Layer Choice Why
OS Ubuntu 24.04 LTS on WSL2 Same Linux as the servers, on a normal Windows laptop
Container builds Docker Engine inside WSL Builds images for k3s; no Docker Desktop licence questions
Orchestration k3s Real Kubernetes API in one binary, light on memory
Data MySQL, MongoDB, Redis, Kafka Mirrors the production data stores
Cloud stand-ins LocalStack (S3), Mailpit (SMTP) Test AWS and email flows offline

1. Prepare WSL2

From an elevated PowerShell prompt:

wsl --install -d Ubuntu-24.04
wsl --update

Give the WSL VM enough resources and use mirrored networking (Windows 11 22H2 or later), so services listening in WSL are reachable on localhost from Windows and the LAN behaves predictably. Create %UserProfile%\.wslconfig:

[wsl2]
memory=12GB
processors=6
swap=4GB
networkingMode=mirrored
dnsTunneling=true
autoProxy=true

[experimental]
autoMemoryReclaim=gradual
sparseVhd=true

Inside Ubuntu, enable systemd so services (Docker, k3s) start normally. Edit /etc/wsl.conf:

[boot]
systemd=true

[network]
hostname=onprem-dev

Restart WSL from PowerShell with wsl --shutdown, open Ubuntu again and confirm:

systemctl is-system-running    # running or degraded is fine
free -h && nproc

Kubernetes keeps a lot of data. Keep it on the Linux filesystem (/var/lib, ~), not under /mnt/c, where file access is much slower.

2. Install Docker Engine

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER && newgrp docker
docker run --rm hello-world

Docker here is used to build images. k3s runs them with its own containerd, so images built by Docker are imported into k3s (shown below) or pushed to a registry.

3. Install k3s

curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=stable sh -s - \
  --write-kubeconfig-mode 644 \
  --node-name onprem-dev

mkdir -p ~/.kube && cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
kubectl get nodes -o wide
kubectl get pods -A

k3s includes:

  • Traefik as the ingress controller, listening on ports 80 and 443.
  • local-path as the default storage class, which stores volumes under /var/lib/rancher/k3s/storage.
  • CoreDNS and metrics-server.

Pin the version you test with (INSTALL_K3S_VERSION=v1.xx.y+k3s1) once things work, so a reinstall gives the same cluster.

Import a locally built image into k3s:

docker build -t shop-api:dev ./api
docker save shop-api:dev | sudo k3s ctr images import -

4. Namespaces, quotas and secrets

# 00-namespaces.yaml
apiVersion: v1
kind: Namespace
metadata: { name: data }
---
apiVersion: v1
kind: Namespace
metadata: { name: apps }
---
apiVersion: v1
kind: Namespace
metadata: { name: platform }
---
apiVersion: v1
kind: ResourceQuota
metadata: { name: data-quota, namespace: data }
spec:
  hard:
    requests.memory: 6Gi
    limits.memory: 8Gi
    persistentvolumeclaims: "10"

Create secrets from the command line, never from files committed to Git:

kubectl apply -f 00-namespaces.yaml
kubectl -n data create secret generic mysql-auth \
  --from-literal=root-password="$(openssl rand -base64 18)" \
  --from-literal=app-password="$(openssl rand -base64 18)"
kubectl -n data create secret generic mongo-auth \
  --from-literal=username=admin --from-literal=password="$(openssl rand -base64 18)"
kubectl -n data create secret generic redis-auth \
  --from-literal=password="$(openssl rand -base64 18)"

5. MySQL 8.4 LTS

MySQL 8.0 reached end of life in April 2026, so new setups use the 8.4 LTS line.

# 10-mysql.yaml
apiVersion: v1
kind: Service
metadata: { name: mysql, namespace: data }
spec:
  clusterIP: None
  selector: { app: mysql }
  ports: [{ port: 3306 }]
---
apiVersion: apps/v1
kind: StatefulSet
metadata: { name: mysql, namespace: data }
spec:
  serviceName: mysql
  replicas: 1
  selector: { matchLabels: { app: mysql } }
  template:
    metadata: { labels: { app: mysql } }
    spec:
      containers:
        - name: mysql
          image: mysql:8.4
          args: ["--character-set-server=utf8mb4", "--collation-server=utf8mb4_0900_ai_ci"]
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom: { secretKeyRef: { name: mysql-auth, key: root-password } }
            - name: MYSQL_DATABASE
              value: shop
            - name: MYSQL_USER
              value: shop
            - name: MYSQL_PASSWORD
              valueFrom: { secretKeyRef: { name: mysql-auth, key: app-password } }
          ports: [{ containerPort: 3306 }]
          readinessProbe:
            exec: { command: ["sh", "-c", "mysqladmin ping -uroot -p$MYSQL_ROOT_PASSWORD"] }
            periodSeconds: 10
          resources:
            requests: { cpu: 250m, memory: 768Mi }
            limits: { memory: 1536Mi }
          volumeMounts: [{ name: data, mountPath: /var/lib/mysql }]
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 10Gi } }

6. MongoDB 8.0

# 11-mongodb.yaml
apiVersion: v1
kind: Service
metadata: { name: mongodb, namespace: data }
spec:
  clusterIP: None
  selector: { app: mongodb }
  ports: [{ port: 27017 }]
---
apiVersion: apps/v1
kind: StatefulSet
metadata: { name: mongodb, namespace: data }
spec:
  serviceName: mongodb
  replicas: 1
  selector: { matchLabels: { app: mongodb } }
  template:
    metadata: { labels: { app: mongodb } }
    spec:
      containers:
        - name: mongodb
          image: mongo:8.0
          env:
            - name: MONGO_INITDB_ROOT_USERNAME
              valueFrom: { secretKeyRef: { name: mongo-auth, key: username } }
            - name: MONGO_INITDB_ROOT_PASSWORD
              valueFrom: { secretKeyRef: { name: mongo-auth, key: password } }
          ports: [{ containerPort: 27017 }]
          readinessProbe:
            exec: { command: ["mongosh", "--quiet", "--eval", "db.adminCommand('ping').ok"] }
            periodSeconds: 10
          resources:
            requests: { cpu: 250m, memory: 768Mi }
            limits: { memory: 1536Mi }
          volumeMounts: [{ name: data, mountPath: /data/db }]
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 10Gi } }

For a single-node development copy one member is enough. To test replica-set behaviour such as transactions and change streams, start it with --replSet rs0 and run rs.initiate() once.

7. Redis

# 12-redis.yaml
apiVersion: v1
kind: Service
metadata: { name: redis, namespace: data }
spec:
  selector: { app: redis }
  ports: [{ port: 6379 }]
---
apiVersion: apps/v1
kind: Deployment
metadata: { name: redis, namespace: data }
spec:
  replicas: 1
  selector: { matchLabels: { app: redis } }
  template:
    metadata: { labels: { app: redis } }
    spec:
      containers:
        - name: redis
          image: redis:8
          command: ["sh", "-c"]
          args:
            - exec redis-server --requirepass "$REDIS_PASSWORD"
              --maxmemory 256mb --maxmemory-policy allkeys-lru --appendonly no
          env:
            - name: REDIS_PASSWORD
              valueFrom: { secretKeyRef: { name: redis-auth, key: password } }
          ports: [{ containerPort: 6379 }]
          resources:
            requests: { cpu: 50m, memory: 128Mi }
            limits: { memory: 384Mi }

Used as a cache, Redis needs no persistence here. If your production uses Valkey instead, swap the image; the protocol and commands are the same.

8. Kafka 4 in KRaft mode

A single combined broker and controller is enough locally. The official apache/kafka image is configured through environment variables:

# 13-kafka.yaml
apiVersion: v1
kind: Service
metadata: { name: kafka, namespace: data }
spec:
  clusterIP: None
  selector: { app: kafka }
  ports:
    - { name: client, port: 9092 }
    - { name: controller, port: 9093 }
---
apiVersion: apps/v1
kind: StatefulSet
metadata: { name: kafka, namespace: data }
spec:
  serviceName: kafka
  replicas: 1
  selector: { matchLabels: { app: kafka } }
  template:
    metadata: { labels: { app: kafka } }
    spec:
      containers:
        - name: kafka
          image: apache/kafka:4.0.0
          env:
            - { name: KAFKA_NODE_ID, value: "1" }
            - { name: KAFKA_PROCESS_ROLES, value: "broker,controller" }
            - { name: KAFKA_LISTENERS, value: "PLAINTEXT://:9092,CONTROLLER://:9093" }
            - { name: KAFKA_ADVERTISED_LISTENERS, value: "PLAINTEXT://kafka-0.kafka.data.svc.cluster.local:9092" }
            - { name: KAFKA_CONTROLLER_LISTENER_NAMES, value: "CONTROLLER" }
            - { name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP, value: "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT" }
            - { name: KAFKA_CONTROLLER_QUORUM_VOTERS, value: "1@localhost:9093" }
            - { name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR, value: "1" }
            - { name: KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR, value: "1" }
            - { name: KAFKA_TRANSACTION_STATE_LOG_MIN_ISR, value: "1" }
            - { name: KAFKA_LOG_DIRS, value: "/var/lib/kafka/data" }
            - { name: KAFKA_NUM_PARTITIONS, value: "3" }
          ports: [{ containerPort: 9092 }, { containerPort: 9093 }]
          resources:
            requests: { cpu: 250m, memory: 1Gi }
            limits: { memory: 1536Mi }
          volumeMounts: [{ name: data, mountPath: /var/lib/kafka/data }]
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 10Gi } }

Create the topics the application needs:

kubectl -n data exec -it kafka-0 -- /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 --create --topic notifications --partitions 3
kubectl -n data exec -it kafka-0 -- /opt/kafka/bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 --create --topic notifications-dlq --partitions 1

9. Apply and verify

kubectl apply -f 10-mysql.yaml -f 11-mongodb.yaml -f 12-redis.yaml -f 13-kafka.yaml
kubectl -n data get pods,pvc -w

Quick checks from a throwaway pod:

kubectl -n data run t --rm -it --image=busybox:1.36 -- sh -c \
  "nc -zv mysql 3306; nc -zv mongodb 27017; nc -zv redis 6379; nc -zv kafka 9092"

Applications reach the services by DNS: mysql.data.svc.cluster.local, mongodb.data.svc.cluster.local, redis.data.svc.cluster.local and kafka-0.kafka.data.svc.cluster.local:9092.

10. Backups and resets

Even a local platform deserves a one-command backup, so a broken experiment does not cost a day:

kubectl -n data exec mysql-0 -- sh -c 'mysqldump -uroot -p"$MYSQL_ROOT_PASSWORD" --single-transaction shop' > shop-mysql.sql
kubectl -n data exec mongodb-0 -- sh -c 'mongodump -u "$MONGO_INITDB_ROOT_USERNAME" -p "$MONGO_INITDB_ROOT_PASSWORD" --authenticationDatabase admin --archive --gzip' > shop-mongo.archive.gz

To reset everything, delete the data namespace, which removes its volumes with the local-path provisioner, and apply the manifests again.

Limits to keep in mind

  • WSL2 sleeps when idle and its IP and services stop with Windows. It is a development and test platform, not a server.
  • local-path volumes live on one disk with no replication.
  • Resource limits matter: the whole stack fits comfortably in about 8 to 10 GB of RAM with the values above.

Part 2 deploys the application on top: the Angular UI, the Node.js API, presigned S3 uploads, the CDN, SMTP and notifications.