How to install Graylog server on Ubuntu 20.04

Welcome to today’s tutorial where we are going to learn how to install the Graylog server on Ubuntu 20.04.

Graylog is a free and open-source log management solution for capturing, storing and enabling real-time analysis of terabytes of machine data. The main purpose of creating Graylog was that log management was a nightmare, so the creators of Graylog were on a mission to make log management and Security Information and Event Management (SIEM) easier, affordable, and more efficient.

What is Graylog used for

Graylog’s main purpose is for log storage. How it does this is it collects log data and does analysis and whenever you need certain data, you only have to ask Graylog to give you full analysis. This had help data analysts give timely and effective information.

Take an example like when you want to do a security audit in your systems, What you need to do is to collect all the data from all users like the login timestamps on every server and workstations. After you have obtained all the logs you then need to analyze using Graylog and at the end, you will get insightful information knowing when your users get accessed to the needed resources, you will know what needs to be done to improve every outcome.

Graylog helps remove the complexity of crunching data using excel sheets or going a manual way of collecting data. It is a good thing to have so that you can focus on other things that bring value to the company.

How does Graylog store data?

Each country in the world has got regulations on how to store data and how long you should stay with that data before disposing of it. This brings a challenge to many institutions because you will need to decide on the best solution to store your data be it on the cloud or the premises. This cuts along all lines including Graylog.

Graylog stores its data in two ways, online and archives data.

Online. It employs the use of Elasticseach which is searchable using the GUI

Archived. It is stored in a compressed format, either on a Graylog server or on a network file share. It is searchable via GREP but must be reconstituted on Graylog to be searchable via GUI.

Graylog default ports

In case you are wondering what are the default ports for Graylog, here are a few open ports used by Graylog.

How to install Graylog server on Ubuntu 20.04

Before we can jump into installation, there are prerequisites to consider first.

Prerequisites

• Make sure you have Elasticsearch installed on your Ubuntu system. Check out this on how to install Elasticsearch on Ubuntu 20.04
• Make sure you have MongoDB installed. Use this article to install MongoDB on Ubuntu 20.04
• Ubuntu 20.04 server up and running
• Make sure you have Java Openjdk and jre installed. Use this article on how to install Java 17 on Ubuntu 20.04

Let’s now dive into the installation steps.

Install System updates

Installing updates will ensure that we have up-to-date repositories, so it is always necessary to do so.

$ sudo apt update && apt upgrade -y
# if necessary reboot your system

We are going to do a single server setup as an example

Install Java openjdk and jre-headless

You need java OpenJDK before we can install the Graylog server. To install OpenJDK 11 use the following code. Before you can install any java you can check if it’s installed with the following command.

$ java --version
Command 'java' not found, but can be installed with:

apt install openjdk-11-jre-headless  # version 11.0.13+8-0ubuntu1~20.04, or
apt install default-jre              # version 2:1.11-72
apt install openjdk-13-jre-headless  # version 13.0.7+5-0ubuntu1~20.04
apt install openjdk-16-jre-headless  # version 16.0.1+9-1~20.04
apt install openjdk-17-jre-headless  # version 17.0.1+12-1~20.04
apt install openjdk-8-jre-headless   # version 8u312-b07-0ubuntu1~20.04

The above will tell you if you have java installed, in my case I will have to do a fresh install.

$ sudo apt install apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen

When you check the version of java installed it will show the following

Output
openjdk 11.0.13 2021-10-19
OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)

Install MongoDB on Ubuntu 20.04

We also need to install MongoDB in our system as a database handler for Graylog logs.

The easiest way to install MongoDB is to use the following command

$ sudo apt install mongodb-server

After installation, you need to start and enable MongoDB

# Enable MongoDB
$ sudo systemctl enable mongodb
#start MongoDB 
$ Sudo systemctl start mongodb

You can check the status if it is running with the following command

$ sudo systemctl status mongodb

Output
● mongodb.service - An object/document-oriented database
     Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-01-14 09:08:20 UTC; 3min 4s ago
       Docs: man:mongod(1)
   Main PID: 18980 (mongod)
      Tasks: 23 (limit: 9513)
     Memory: 42.4M
     CGroup: /system.slice/mongodb.service
             └─18980 /usr/bin/mongod --unixSocketPrefix=/run/mongodb --config /etc/mongodb.conf

Jan 14 09:08:20 ubuntu systemd[1]: Started An object/document-oriented database.

Install Elasticsearch on Ubuntu 20.04

Here we need to install the open source version of Elasticsearch and also ensure that you run Elasticsearch version 7.x

PGP key is for signing the Elasticsearch. We need to download it which helps with security concerns.

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

You may need to install apt-transport-https as prerequisites even though Ubuntu 20.04 comes preinstalled but its always necessary as a precaution. To install run the following command

$ sudo apt install apt-transport-https

Then we need to add Elastic to source.list.d with the following command

$ echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

After, you need to update your system repositories again

$ sudo apt update

Then install Elasticsearch with the following command:

$ sudo apt install elasticsearch-oss

Now you need to configure Elasticsearch by modifying the elasticsearch.yml file.

Open your preferred editor tool.

$ sudo nano /etc/elasticsearch/elasticsearch.yml

Add cluster name to graylog and uncomment action.auto_create_index to false

cluster name: graylog
action.auto_create_index: false

Save your file and exit

Reload for the changes to take effect.

$ sudo systemctl daemon-reload
$ sudo systemctl enable elasticsearch.service
$ sudo systemctl start elasticsearch.service

You can now check the status if it’s running with the following

$ sudo systemctl status elasticsearch

output
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-01-14 09:54:13 UTC; 2min 17s ago
       Docs: https://www.elastic.co
   Main PID: 2292 (java)
      Tasks: 36 (limit: 9508)
     Memory: 1.2G
     CGroup: /system.slice/elasticsearch.service
             └─2292 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.net>

Jan 14 09:53:51 ubuntu systemd[1]: Starting Elasticsearch...
Jan 14 09:54:13 ubuntu systemd[1]: Started Elasticsearch.
lines 1-12/12 (END)

We can now test if Elasticsearch is running on port 9200 with the following curl command

$ curl -X GET 'http://localhost:9200'

If you get the below output, know you are ok to continue

output
{
  "name" : "ubuntu",
  "cluster_name" : "graylog",
  "cluster_uuid" : "UzLRwMkYTA6qa_WBqsIi6A",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

The last step is to install Graylog server

Install Graylog server on Ubuntu 20.04

First, install Graylog repository

# Get Graylog repository
$ wget https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.deb

# Configure the installer 
$ sudo dpkg -i graylog-4.2-repository_latest.deb

Run system updates again for the changes to take effect.

$ sudo apt update

Now you can install Graylog server

$ sudo apt install graylog-server -y

This is the output you will get

Output
The following NEW packages will be installed:
  graylog-server
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 197 MB of archives.
After this operation, 218 MB of additional disk space will be used.
Get:1 https://packages.graylog2.org/repo/debian stable/4.2 amd64 graylog-server all 4.2.5-1 [197 MB]
Fetched 197 MB in 4s (49.4 MB/s)         
Selecting previously unselected package graylog-server.
(Reading database ... 96009 files and directories currently installed.)
Preparing to unpack .../graylog-server_4.2.5-1_all.deb ...
Unpacking graylog-server (4.2.5-1) ...
Setting up graylog-server (4.2.5-1) ...
################################################################################
Graylog does NOT start automatically!

Please run the following commands if you want to start Graylog automatically on system boot:

    sudo systemctl enable graylog-server.service

    sudo systemctl start graylog-server.service

################################################################################
Processing triggers for systemd (245.4-4ubuntu3.15) ...

$ echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1

Output
Enter Password: ?>3v-6$6!M&pyE>E
73d6d6bb0a27d3fdf83bfc2182039b860f0cc1c70f68036b2d32ef371432ef41

$ pwgen -N 1 -s 96

output
ZofFscZW22SZTj8n8AcOQ0UVms0fokzUoAKWb2X8OXDjkQg9sdgqHp5xLIy7Rnlen8223Xb0K5pbNKe1ZJv4kVUicJG3xGuF

$ sudo nano /etc/graylog/server/server.conf 

# root_password_sha2
73d6d6bb0a27d3fdf83bfc2182039b860f0cc1c70f68036b2d32ef371432ef41
# password_secret
ZofFscZW22SZTj8n8AcOQ0UVms0fokzUoAKWb2X8OXDjkQg9sdgqHp5xLIy7Rnlen8223Xb0K5pbNKe1ZJv4kVUicJG3xGuF
# set bind address to your server Ip address eg. for my case 
http_bind_address= 142.93.46.143

Enable and start Graylog server

# reload graylog server
# sudo systemctl daemon-reload
# enbale Graylog
$ sudo systemctl enable graylog-server.service
# start Graylog
$ sudo systemctl start graylog-server.service

Now you can check the status of Graylog server

$ sudo systemctl status graylog-server

The out put will be as follows

Output
● graylog-server.service - Graylog server
     Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-01-14 11:34:44 UTC; 28s ago
       Docs: http://docs.graylog.org/
   Main PID: 10528 (graylog-server)
      Tasks: 115 (limit: 9508)
     Memory: 622.2M
     CGroup: /system.slice/graylog-server.service
             ├─10528 /bin/sh /usr/share/graylog-server/bin/graylog-server
             └─10578 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:-OmitStackTraceInFas>

Jan 14 11:34:44 ubuntu systemd[1]: Started Graylog server.
Jan 14 11:34:44 ubuntu graylog-server[10578]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was de>
Jan 14 11:34:45 ubuntu graylog-server[10578]: WARNING: sun.reflect.Reflection.getCallerClass is not supported. T>
Jan 14 11:34:52 ubuntu graylog-server[10578]: WARNING: An illegal reflective access operation has occurred
Jan 14 11:34:52 ubuntu graylog-server[10578]: WARNING: Illegal reflective access by retrofit2.Platform (file:/us>
Jan 14 11:34:52 ubuntu graylog-server[10578]: WARNING: Please consider reporting this to the maintainers of retr>
Jan 14 11:34:52 ubuntu graylog-server[10578]: WARNING: Use --illegal-access=warn to enable warnings of further i>
Jan 14 11:34:52 ubuntu graylog-server[10578]: WARNING: All illegal access operations will be denied in a future 

Go to your favourite browser and type the following

http://<your ip address>:9000

Use Admin as username and the password you set while configuring root_password_sha2

Conclusion

Congratulations, you have successfully installed and configured Graylog server on Ubuntu 20.04. If you experience any difficulty always consult Graylog documentation.