Install Graylog 3.0 on CentOS 7

Dear 丶 2023-10-18 19:24 81阅读 0赞

Graylog is an open-source log management tool which helps you to collect, index and analyze any machine logs centrally. This guide will help you to install Graylog on CentOS 7 / RHEL 7.  
Components  
1. MongoDB – Being a database to store the configurations and meta information.  
2. Elasticsearch – It stores the log messages received from the Graylog server and provides a facility to search them whenever required. Elasticsearch is a resource monger as it does indexing of data, so allocate more memory and use SAS or SAN disks.  
3. Graylog server – This does the parsing of logs that are coming from various inputs and provides built-in Web Interface to handle those logs.  
Prerequisites  
Make sure your system is configured with EPEL repository.  
yum install epel-release -y  
Install the required packages for Graylog installation.  
yum -y install wget pwgen  
Elasticsearch requires Java to be installed on the machine. So, install either OpenJDK or Oracle JDK.  
yum install -y java-1.8.0-openjdk-headless  
Verify the Java version.  
java -version  
Output:  
openjdk version "1.8.0\_191"  
OpenJDK Runtime Environment (build 1.8.0\_191-b12)  
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)  
Install Elasticsearch  
Elasticsearch is one of the important components in the Graylog setup. It stores the data coming from Graylog input and displays the messages whenever a user request over the Graylog built-in web interface.  
Elasticseach is mainly used here to index the data/logs and provide the searching functionality when the Graylog web interface request for any information.  
This guide covers the basic configuration required for Graylog.  
Import the GPG signing key before the installation.  
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch  
Create a repository file to get the latest version of Elasticsearch package from the official repository.  
vi /etc/yum.repos.d/elasticsearch.repo  
Add the below content to above file.  
\[elasticsearch-6.x\]  
name=Elasticsearch repository for 6.x packages  
baseurl=https://artifacts.elastic.co/packages/6.x/yum  
gpgcheck=1  
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch  
enabled=1  
autorefresh=1  
type=rpm-md  
Now, install the Elasticsearch package using yum command.  
yum install -y elasticsearch  
Reload the systemctl daemon and enable Elasticsearch to start automatically on the system startup.  
systemctl daemon-reload  
systemctl enable elasticsearch  
To make Elasticsearch work with Graylog setup, we need to set the cluster name to graylog. Edit the elasticsearch.yml file.  
vi /etc/elasticsearch/elasticsearch.yml  
Update it like shown below.  
cluster.name: graylog  
Restart the Elasticsearch.  
systemctl restart elasticsearch  
Give a minute to let the Elasticsearch get fully restarted. Elastisearch should be now listening to 9200 for processing HTTP requests. Use the CURL command to check the response.  
curl -X GET http://localhost:9200  
Cluster name should be graylog.  
\{  
"name" : "DF8QK3-",  
"cluster\_name" : "graylog",  
"cluster\_uuid" : "\_wAgUfN9RJeQ0npCKBswVA",  
"version" : \{  
"number" : "6.6.0",  
"build\_flavor" : "default",  
"build\_type" : "rpm",  
"build\_hash" : "a9861f4",  
"build\_date" : "2019-01-24T11:27:09.439740Z",  
"build\_snapshot" : false,  
"lucene\_version" : "7.6.0",  
"minimum\_wire\_compatibility\_version" : "5.6.0",  
"minimum\_index\_compatibility\_version" : "5.0.0"  
\},  
"tagline" : "You Know, for Search"  
\}  
Perform a health check of Elasticsearch cluster.  
curl -XGET 'http://localhost:9200/\_cluster/health?pretty=true'  
Make sure the cluster status comes as green.  
\{  
"cluster\_name" : "graylog",  
"status" : "green",  
"timed\_out" : false,  
"number\_of\_nodes" : 1,  
"number\_of\_data\_nodes" : 1,  
"active\_primary\_shards" : 0,  
"active\_shards" : 0,  
"relocating\_shards" : 0,  
"initializing\_shards" : 0,  
"unassigned\_shards" : 0,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 100.0  
\}  
Install MongoDB

MongoDB acts as a database to store the configurations and meta information. We will use MongoDB’s official repository to get the latest version of MongoDB.  
vi /etc/yum.repos.d/mongodb-org-4.0.repo  
Add repository information to the above file.  
\[mongodb-org-4.0\]  
name=MongoDB Repository  
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86\_64/  
gpgcheck=1  
enabled=1  
gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc  
Install the community edition of MongoDB using the following command.  
yum install -y mongodb-org  
Start the MongoDB service and enable it to the system start-up.  
systemctl start mongod  
systemctl enable mongod  
Install Graylog

Graylog server accepts and processes the log messages coming from the various inputs, displays data to requests that come from the graylog web interface.  
Download and Install graylog 3.x repository.  
rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository\_latest.rpm  
Install the Graylog server using the following command.  
yum -y install graylog-server  
Edit the server.conf file to begin the graylog configuration.  
vi /etc/graylog/server/server.conf  
Use the following command to create a secret  
pwgen -N 1 -s 96  
Output:  
OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP  
Place the secret like below in server.conf  
password\_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP  
Set the hash password for the root user, i.e., admin of graylog. You would need this password to login into the graylog web.  
If you ever want to change/reset the forgotten password of admin, you can edit/update the server.conf with a hashed password.  
Generate a hashed password using the below command. Replace yourpassword with the choice of yours.  
echo -n yourpassword | sha256sum  
Output:  
e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951  
Place the hash password.  
root\_password\_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951  
You can set up an email address for the admin user.  
root\_email = "itzgeek.web@gmail.com"  
Set timezone of root (admin) user.  
root\_timezone = UTC  
Set only one master node by defining the below variable, the default setting is true.  
If this server is your second Graylog node in your environment, set it to false to make this node as a slave. Master node does some periodic tasks that slave nodes won’t perform.  
is\_master = true  
Set the number of log messages to keep per index. It is recommended to have several smaller indices instead of larger ones.  
elasticsearch\_max\_docs\_per\_index = 20000000  
The following parameter defines to have a total number of indices. If this number is reached, the old index will be deleted.  
elasticsearch\_max\_number\_of\_indices = 20  
Shards setting rely on the number of nodes in the particular Elasticsearch cluster. If you have only one node, set it as 1.  
elasticsearch\_shards = 1  
This defines the number of replicas for your indices. If you have only one node in the Elasticsearch cluster, set it as 0.  
elasticsearch\_replicas = 0  
Install Graylog web interface  
From the Graylog version 2.x, the web interface is served directly by the Graylog server. Enable the Graylog web interface by editing the server.conf file.  
vi /etc/graylog/server/server.conf  
Modify the entries to let Graylog Web Interface to connect to the Graylog server. Replace 192.168.1.10 with your system IP address.  
http\_bind\_address = 192.168.1.10:9000  
Restart Graylog service.  
systemctl daemon-reload  
systemctl restart graylog-server  
Make Graylog server to start automatically on system startup.  
systemctl enable graylog-server  
You can check out the server startup logs; it will be useful for you to troubleshoot Graylog for any issues.  
tailf /var/log/graylog-server/server.log  
If everything goes well, you should see below message in the server.log file.  
2019-02-25T21:00:23.157+05:30 INFO \[ServerBootstrap\] Graylog server up and running.  
Firewall  
In CentOS 7 / RHEL 7, firewall rules are set to block most of the traffic coming from the external machines by default. You would need to add allow rules for Graylog to work correctly.  
firewall-cmd --permanent --add-port=9000/tcp  
Reload firewalld to take an effect of the new rules.  
firewall-cmd --reload  
Access Graylog web interface  
The web interface will now be listening on port 9000. Point your browser to.  
http://ip.add.re.ss:9000  
Login with username admin and the password you configured at root\_password\_sha2 on server.conf.  
Install Graylog 3.0 on CentOS 7 – Gralog Login Screen  
Once you logged in, you would see the getting started page.  
Install Graylog 3.0 on CentOS 7 – Graylog Dashboard  
Click on System >> Overview to know the status of Graylog server.  
Install Graylog 3.0 on CentOS 7 – System Overview  
Create Graylog Inputs  
Graylog input needs to be configured to receive the logs from external sources, i.e., syslog or any logging system. For this example, we will set up an input for receiving logs from syslog on port number UDP 514.  
But, there is a problem. If you start to try an input on one of the privileged ports, any TCP/UDP port number below 1024, you would see permission denied error in Graylog logs.  
To resolve the issue, we will create an input to start on port 1514, and we will redirect the traffic comes to 514 to 1514 with firewall rules.  
Click System >> Inputs >> select Syslog UDP and then click Launch new input.  
Install Graylog 3.0 on CentOS 7 – Syslog Input  
Fill the values shown below.  
Node: Select your Graylog Node  
Title: Name your input  
Bind address: 0.0.0.0 (Leave the default one)  
Port: 1514  
Install Graylog 3.0 on CentOS 7 – Create Syslog UDP Input  
Ensure the created Syslog input is in running state.  
Install Graylog 3.0 on CentOS 7 – Graylog Input Status  
Configure Rsyslog  
Once you have created the inputs, configure Rsyslog or forward any system logs to your Graylog server.  
Edit the Rsyslog configuration file.  
vi /etc/rsyslog.conf  
Typical Rsyslog configuration will look like below to send logs on UDP 514.  
If you do not wish to use UDP 514 and want to send logs directly to UDP 1514 of Graylog, then you can change the port number in the Rsyslog configuration file  
\*.info;mail.none;authpriv.none;cron.none @192.168.1.10:514  
Restart the Rsyslog service to send logs.  
systemctl restart rsyslog  
Firewall  
Configure the firewall based on Rsyslog configuration you did on the previous step.  
Port 514  
Execute the below commands on the Graylog server to redirect the traffic that comes on port UDP 514 to UDP 1514 of Graylog input.  
firewall-cmd --add-masquerade --permanent  
firewall-cmd --add-forward-port=port=514:proto=udp:toport=1514 --permanent  
firewall-cmd --reload  
Port 1514  
Execute the below commands on the Graylog server to allow port number UDP 1514 on the firewall to receive logs from Rsyslog.  
firewall-cmd --permanent --add-port=1514/udp  
firewall-cmd --reload  
View Logs using Graylog  
Wait for a few minutes. You should start receiving log messages from the client machine when the event is generated.  
Following screenshot shows the logs received by Graylog (Graylog console >> Search).  
Install Graylog 3.0 on CentOS 7 – View Syslog Messages using Graylog  
Conclusion  
You have successfully installed Graylog 3.0 on CentOS 7 / RHEL 7 and configured Graylog to receive Rsyslog logs from external sources. As a further read, you can try configuring Nginx or Apache as a reverse proxy and set up HTTPS for the Graylog web interface.