ELK Stack installation

This guide walks you through installing and configuring the ELK Stack (Elasticsearch, Logstash, and Kibana) with security features enabled.

Elasticsearch

Step 1: Add the Elasticsearch APT Repository

Elasticsearch requires an official package repository to be added before installation. Run the following commands to add the repository and update your package list:

curl  -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/elastic.gpg
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
apt update

Step 2: Install Elasticsearch

apt install elasticsearch

Step 3: Security Configuration

Elasticsearch 8.x comes with authentication, authorization, and TLS encryption enabled by default. The autogenerated CA and TLS certificates are stored under:

ls -1l /etc/elasticsearch/certs/

Expected output:

http_ca.crt
http.p12
transport.p12

Elasticsearch 8.x Autogenerated CA and TLS Certificates

See etc/elasticsearch/elasticsearch.yml for more details. The CA and self signed TLS certificates are generated by default and stored under:

--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : Zcjz-oCdnxtnK*vyj1ps

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
sudo systemctl start elasticsearch.service

Check the elastic password in the output: Zcjz-oCdnxtnK*vyj1ps

Each file serves a different purpose:

• http_ca.crt: Self-signed CA certificate for Elasticsearch.
• http.p12: Encrypts client communication (e.g., Kibana to Elasticsearch).
• transport.p12: Encrypts cluster communications.

Elastic search Keystore is used to store secrets information for example your p12 passwords, you can look at the contents of the Keystore by using the following command:

/usr/share/elasticsearch/bin/elasticsearch-keystore list

Step 4: Configure Elasticsearch

Modify the Elasticsearch configuration file /etc/elasticsearch/elasticsearch.yml and set the following parameters: Change cluster.name, node.name and network.host in /etc/elasticsearch/elasticsearch.yml file.

grep -Ev '^#|^$' /etc/elasticsearch/elasticsearch.yml

cluster.name: logs
node.name: logs
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: [ "elk-01" ]
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"

Step 5: Start and Enable Elasticsearch Service

systemctl daemon-reload # reload systemd
systemctl enable elasticsearch # enable elasticsearch service
systemctl start elasticsearch # start elasticsearch service
systemctl status elasticsearch # check status of elasticsearch service

In case of error, check the logs:

journalctl -fu elasticsearch --since yesterday

Step 6: Verify Elasticsearch

To ensure Elasticsearch is running and responding, use the following command:

curl https://localhost:9200 --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps

A successful response should return JSON output with cluster details:

{
  "name": "logs",
  "cluster_name": "logs",
  "cluster_uuid": "dID2u4CmRT6-9fR2YXlOzA",
  "version": {
    "number": "8.7.0",
    "build_flavor": "default",
    "build_type": "deb",
    "build_hash": "09520b59b6bc1057340b55750186466ea715e30e",
    "build_date": "2023-03-27T16:31:09.816451435Z",
    "build_snapshot": false,
    "lucene_version": "9.5.0",
    "minimum_wire_compatibility_version": "7.17.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "You Know, for Search"
}

autoconfiguration.password_hash
keystore.seed
xpack.security.http.ssl.keystore.secure_password
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password

You can list them by using the following command:

/usr/share/elasticsearch/bin/elasticsearch-keystore show keystore.seed
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password # this one is used to encrypt client communication 
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password 
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.truststore.secure_password 

To open the http.p12 file and look at its contents you can type, password is IFQzPruaSwWIBactv0fIlg:

openssl pkcs12 -nokeys -info -in /etc/elasticsearch/certs/http.p12 

Step 1: Install Kibana

Install Kibana using the package manager:

apt install kibana

Step 2: Configure Kibana

Modify /etc/kibana/kibana.yml to set up Kibana’s host, name, and Elasticsearch connection:

server.port: 5601
server.host: "0.0.0.0"
server.name: "kibana-node"
elasticsearch.hosts: [ "https://localhost:9200" ]

Step 3: Start and Enable Kibana

Enable Kibana to start on boot and start the service:

systemctl enable kibana
systemctl start kibana
systemctl status kibana

Logstash

Step 1: Install Logstash

Install Logstash using the following command:

apt install logstash

Step 2: Configure Logstash

Create a logstash_writer role in Elasticsearch to define necessary privileges:

curl -X POST "https://localhost:9200/_security/role/logstash_writer" \
  -H 'Content-Type: application/json' \
  -u elastic:your_password \
  -d'{"cluster": ["manage_index_templates", "monitor", "manage_ilm"], "indices": [ { "names": [ "*" ], "privileges": ["write","create","create_index","manage","manage_ilm"] }]}'

Next, create a Logstash user and assign it the logstash_writer role:

curl -X POST "https://localhost:9200/_security/user/logstash" \
  -H 'Content-Type: application/json' \
  -u elastic:your_password -d'{"password" : "logstash", "roles" : ["logstash_writer"], "full_name" : "Internal Logstash User"}'

Step 3: Start and Enable Logstash

Enable Logstash to start on boot and start the service:

systemctl enable logstash
systemctl start logstash
systemctl status logstash

Then in /etc/kibana/kibana.yml change server.host, server.name and elasticsearch.hosts:

grep -Ev '^#|^$' /etc/kibana/kibana.yml

server.port: 5601
server.host: "0.0.0.0"
server.name: "logs"
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid

Then use elasticsearch-create-enrollment-token to create token for Kibana:

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTAuMi4yMDAuMjQ1OjkyMDAiXSwiZmdyIjoiM2JlOWQ0NDA0N2U5ZmYzYjExODhmYTEzODA2MmZlMzJiNjMzNjg5ZGFlYTFjNGRhZWJhMGYyOTZhODU4MTFhMSIsImtleSI6InRveHk5WlVCejh4cy1aQXR4STFVOlBZVmt3WUJjUUxTbEJPSFd6eWlPRmcifQ==

And setup kiabana with this token:

/usr/share/kibana/bin/kibana-setup --enrollment-token eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTAuMi4yMDAuMjQ1OjkyMDAiXSwiZmdyIjoiM2JlOWQ0NDA0N2U5ZmYzYjExODhmYTEzODA2MmZlMzJiNjMzNjg5ZGFlYTFjNGRhZWJhMGYyOTZhODU4MTFhMSIsImtleSI6InRveHk5WlVCejh4cy1aQXR4STFVOlBZVmt3WUJjUUxTbEJPSFd6eWlPRmcifQ==

There should be created another *.crt files in /var/lib/kibana/ directory:

curl https://localhost:9200 --cacert /var/lib/kibana/ca_1743522857893.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps

and in /etc/kibana/kibana.yml should be automatically added following lines:

elasticsearch.hosts: [ 'https://10.2.200.245:9200' ]
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE2ODI2MTEyMTQ4NjM6RzU5aWotOVVULWFWVkk4eHUzaDFPUQ
elasticsearch.ssl.certificateAuthorities: [ /var/lib/kibana/ca_1743522857893.crt ]
xpack.fleet.outputs: [ { id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: [ 'https://10.2.200.245:9200' ], ca_trusted_fingerprint: 93e437ecdede3340ddae02b9156af68310458bd9807f1e34b25fc5c62c8ef568 } ]

systemctl enable kibana
systemctl start kibana
systemctl status kibana

First time you should login with elastic user using password Zcjz-oCdnxtnK*vyj1ps. Then you can create new users with Kibana - under Management > Security > Users.

Create new users

Then you can add multiple users (e.g. OzzyCzech) with superuser role:

curl -s --cacert /etc/logstash/ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps \
  -X POST "https://localhost:9200/_security/user/OzzyCzech" -H 'Content-Type: application/json' \
  -d'{"password" : "123456", "roles" : ["superuser"], "full_name" : "Roman Ožana"}'

Filebeat

first create filebeat_writer role:

curl -s --cacert /etc/logstash/ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps \
  -X POST "https://localhost:9200/_security/role/filebeat_writer" \
  -H 'Content-Type: application/json' \
  -d'{"cluster": ["all"], "indices": [ { "names": [ "*" ], "privileges": ["all"] }]}'

and filebeat user:

curl -s --cacert /etc/logstash/ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps \
  -X POST "https://localhost:9200/_security/user/filebeat" \
  -H 'Content-Type: application/json' \
  -d'{"password" : "filebeat", "roles" : ["filebeat_writer"], "full_name" : "Internal Filebeat User"}'

then check if you can login with this user:

curl https://localhost:9200 --cacert /etc/elasticsearch/certs/http_ca.crt -u filebeat:filebeat

Filebeat certificate

You have to get http_ca.crt password from elasticsearch keystore:

# this one is used to encrypt client communication
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

Then you have to dump the keys to a PEM file:

openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -nocerts -nodes \
  -out /certs/http_ca_key.pem

The next step is to package up the CA key into its own p12 file:

openssl pkcs12 -export -out /certs/http_ca_key.p12 \
 -inkey /certs/http_ca_key.pem \
 -in /etc/elasticsearch/certs/http_ca.crt

You will be prompted to enter an export password. Feel free to use the same password as for /etc/elasticsearch/certs/http.p12.

Now we can generate a key-certificate pair for the logstash instance:

/usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca /certs/http_ca_key.p12 --ip $(ip -o route get to 8.8.8.8 | sed -n 's/.*src \([0-9.]\+\).*/\1/p') --name client --out /certs/client.p12

Since client.crt will contain both the CA and client certificates, you will need to separate them out into different files. Copy the CA cert into a file named http_ca.crt.

mkdir -p /certs/filebeat
cp /etc/elasticsearch/certs/http_ca.crt /certs/filebeat

Extract the certificates from the p12 file as well:

openssl pkcs12 -in /certs/client.p12 -nocerts -nodes -out /certs/filebeat/filebeat.key

openssl pkcs12 -in /certs/client.p12 -nokeys -nodes -out /certs/filebeat/filebeat.crt

Download certificates to the filebeat host and update the configuration:

output.elasticsearch:
  hosts: [ "http://elk-01.int.wikidi.net:9200" ]
  username: "filebeat"
  password: "filebeat"
  ssl.enabled: true
  ssl.certificate_authorities: [ "/etc/filebeat/certs/http_ca.crt" ]
  ssl.certificate: "/etc/filebeat/certs/filebeat.crt"
  ssl.key: "/etc/filebeat/certs/filebeat.key"

Logstash

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic-keyring.gpg
sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elastic-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list

apt update
apt install apt-transport-https

Fist copy the certificates to logstash directory:

openssl s_client -showcerts -connect localhost:9200 </dev/null 2>/dev/null | openssl x509 > /etc/logstash/ca.crt

or you can use the http_ca.crt certificate from elasticsearch:

mkdir /etc/logstash/certs
cp /etc/elasticsearch/certs/http_ca.crt /etc/logstash/ca.crt

then you have to create logstash_writer role:

curl -s --cacert /etc/logstash/ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps \
-X POST "https://localhost:9200/_security/role/logstash_writer" -H 'Content-Type: application/json' \
-d'{"cluster": ["manage_index_templates", "monitor", "manage_ilm"], "indices": [ { "names": [ "*" ], "privileges": ["write","create","create_index","manage","manage_ilm"] }]}'

and logstash user:

curl -s --cacert /etc/logstash/ca.crt -u elastic:Zcjz-oCdnxtnK*vyj1ps \
-X POST "https://localhost:9200/_security/user/logstash" -H 'Content-Type: application/json' \
-d'{"password" : "logstash", "roles" : ["logstash_writer"], "full_name" : "Internal Logstash User"}'

then check if you can login with this user:

curl https://localhost:9200 --cacert /etc/elasticsearch/certs/http_ca.crt -u logstash:logstash

{
  "name" : "logs",
  "cluster_name" : "logs",
  "cluster_uuid" : "dID2u4CmRT6-9fR2YXlOzA",
  "version" : {
    "number" : "8.7.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "09520b59b6bc1057340b55750186466ea715e30e",
    "build_date" : "2023-03-27T16:31:09.816451435Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

then update the logstash.yml file:

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash
xpack.monitoring.elasticsearch.password: logstash

journalctl -fu logstash --since "5 minute ago"

Filebeat

Step 1: Create Filebeat User and Role

Define the filebeat_writer role in Elasticsearch:

curl -X POST "https://localhost:9200/_security/role/filebeat_writer" \
  -H 'Content-Type: application/json' \
  -u elastic:your_password -d'{"cluster": ["all"], "indices": [ { "names": [ "*" ], "privileges": ["all"] }]}'

Create a Filebeat user with appropriate permissions:

curl -X POST "https://localhost:9200/_security/user/filebeat" \
  -H 'Content-Type: application/json' \
  -u elastic:your_password -d'{"password" : "filebeat", "roles" : ["filebeat_writer"], "full_name" : "Internal Filebeat User"}'

Step 2: Configure Filebeat

Modify the Filebeat configuration file /etc/filebeat/filebeat.yml to enable secure communication with Elasticsearch:

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: "filebeat"
  password: "filebeat"
  ssl.enabled: true
  ssl.certificate_authorities: ["/etc/filebeat/certs/http_ca.crt"]
  ssl.certificate: "/etc/filebeat/certs/filebeat.crt"
  ssl.key: "/etc/filebeat/certs/filebeat.key"

Resources

• Setting up a secure Elasticsearch pipeline for logs analysis
• Elasticsearch 8.x Deployment
• Installing Logstash
• Install Logstash 8 on Ubuntu/Debian
• Start the Elastic Stack with security enabled automatically
• Easily Configure Elasticsearch HTTPS Connection
• Install Elastic Stack 8