Elasticsearch 8 and Kibana on Ubuntu: Secure Single-Node Setup

Elasticsearch stores and searches documents, and Kibana is the browser tool for exploring them. Since version 8, security (TLS and passwords) is switched on automatically, which is a big change from the old “open on port 9200” days. This post sets up a single node properly.

Versions in use on this date. Elasticsearch and Kibana 8.14.3, released in July 2024; 8.15 is expected in August. Both ship with their own bundled Java, so no separate JDK is needed. Ubuntu 22.04 LTS with 4 GB of RAM or more.

What you are building

flowchart TB
  B[Browser] ==>|HTTP 5601| K[Kibana]
  K ==>|HTTPS 9200 with service token| E[Elasticsearch<br/>single node]
  E --> D[(Data folder<br/>/var/lib/elasticsearch)]
  A[Apps and curl] -->|HTTPS 9200 with API key| E
  E -.-> S[(Snapshot repository<br/>/mnt/es-backups)]

System preparation

sudo sysctl -w vm.max_map_count=262144
echo 'vm.max_map_count=262144' | sudo tee /etc/sysctl.d/99-elasticsearch.conf
sudo swapoff -a                                # and remove swap from /etc/fstab

Elasticsearch memory-maps its index files, so the default vm.max_map_count of 65530 is too low and the node refuses to start in production mode. Swap makes a search node slow and unpredictable, so turn it off.

Install from the Elastic repository

sudo apt install -y apt-transport-https gnupg curl
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic.gpg
echo "deb [signed-by=/usr/share/keyrings/elastic.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" \
  | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update && sudo apt install -y elasticsearch

At the end of the install the package prints the generated password for the built-in elastic user. Save it now. If you miss it:

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

Configure the node

Edit /etc/elasticsearch/elasticsearch.yml:

cluster.name: lab
node.name: es-1
discovery.type: single-node
network.host: 127.0.0.1        # use the server IP only if other hosts need access
path.data: /var/lib/elasticsearch

Set the heap to about half the RAM, never more than 31 GB, with the same minimum and maximum:

printf -- '-Xms2g\n-Xmx2g\n' | sudo tee /etc/elasticsearch/jvm.options.d/heap.options
sudo systemctl enable --now elasticsearch
sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200

The reply with the cluster name and version shows it is up. The certificate authority file in /etc/elasticsearch/certs/http_ca.crt is how clients trust the node’s self-signed TLS certificate; do not use -k outside a quick test.

Install and enrol Kibana

sudo apt install -y kibana
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
sudo /usr/share/kibana/bin/kibana-setup --enrollment-token <token>

The token is valid for 30 minutes. kibana-setup writes the connection details and a service account token into kibana.yml, so Kibana never uses the elastic superuser. Then set server.host in /etc/kibana/kibana.yml (leave it on localhost and use an SSH tunnel or an Nginx proxy with TLS rather than exposing port 5601) and start it:

sudo systemctl enable --now kibana

Open http://localhost:5601 and log in as elastic. If the page asks for a verification code, get it with sudo /usr/share/kibana/bin/kibana-verification-code.

Index and search something

In Kibana, open Dev Tools:

PUT /orders/_doc/1
{ "customer": "asha", "total": 1250, "status": "paid", "@timestamp": "2024-07-23T09:30:00Z" }

GET /orders/_search
{ "query": { "match": { "status": "paid" } } }

GET _cat/indices?v
GET _cluster/health

A single node reports yellow health once an index has replicas, because a replica can never be placed on the same node as its primary. For a lab set index.number_of_replicas: 0 on the index; in production, use three nodes.

Create an API key for applications

POST /_security/api_key
{ "name": "orders-writer",
  "role_descriptors": { "w": { "indices": [ { "names": ["orders*"], "privileges": ["create_index","write"] } ] } } }

Applications use the returned key and have no more rights than that role, which is much safer than sharing the elastic password.

Snapshots and housekeeping

PUT _snapshot/local
{ "type": "fs", "settings": { "location": "/mnt/es-backups" } }
PUT _snapshot/local/snap-2024-07-23?wait_for_completion=true

For logs, use index lifecycle management (ILM) to roll indices over and delete old ones, or the disk fills. Watch free space: at 85% disk use Elasticsearch stops allocating new shards, and at 95% it makes indices read-only.