Configuring syslog Logging with HAProxy

Configure rsyslog to Accept HAProxy Log Data

We're going to configure HAProxy to feed log data to the rsyslog instance running on our HAProxy server.

Before we get started, let's take a look at the /var/log directory just to show we have no HAProxy logs.

The first thing we need to do is add some configuration code to our /etc/rsyslog.d directory. We'll create a file called 99-haproxy.conf in this directory.

Create a configuration file (/etc/rsyslog.d/99-haproxy.conf) with the following characteristics:

• Listen for log data on 127.0.0.1, port 514, using UDP.
• Log all levels from the local0 facility to the file /var/log/haproxy-traffic.log.
• Log only notice from the local0 facility to the file /var/log/haproxy-admin.log.

Save, exit, and restart the rsyslogd service.

Let's take another look at the /var/log directory, just to show we have no HAProxy logs.

No logs yet! Let's change that.

Configure HAProxy to Send Log Data to rsyslog

Now that rsyslog is configured to accept log data, we're going to go ahead and configure logging in HAProxy.

Modify our HAProxy configuration at /etc/haproxy/haproxy.cfg:

• Replace the existing log entry in the global section to send log data to the rsyslog server at 127.0.0.1, port 514, using the local0 facility.
• Scroll down to the defaults section and confirm there is a line, log global, which is the same as putting log global in all the frontend and backend sections.
• Scroll down to the frontend ssh-in section and confirm we have option tcplog set. This provides a log format that's more useful for Layer 4 traffic.

Save and exit, and then restart the haproxy service.

Let's check out our new logs!

Check if we have HAProxy logs in /var/log. We should see our new HAProxy log files!

Take a look at the contents of the HAProxy log files. Compare them and confirm it appears that we are logging the correct error levels to the correct logs.

We need to generate some traffic to fill our logs with data.

Let's generate some HTTP traffic:

wget --no-check-certificate -O - http://www.site1.com/test.txt

wget --no-check-certificate -O - http://www.site2.com/test.txt

Let's generate some HTTPS traffic:

wget --no-check-certificate -O - https://www.site1.com/test.txt

wget --no-check-certificate -O - https://www.site2.com/test.txt

Let's try pulling a file from our SSH container, via HAProxy:

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -P 2222 [email protected]:/sshfiles/ssh-test.txt .

Let's keep doing the scp operation until we hit our session limit (10) and log some errors, then take a look at the contents of our HAProxy log files again.

We see our traffic logged in the /var/log/haproxy-traffic.log file, both HTTP and TCP.

We'd like to have individual logs for the HTTP and TCP traffic, plus a combined traffic log. Let's make some changes.

Separating HTTP and TCP Logs

Modify the /etc/rsyslog.d/99-haproxy.conf as follows:

• Add a line in the frontend http-https-in section to send log data to the rsyslog server at 127.0.0.1, port 514, using the local1 facility.
• Add a line in the frontend sshd-in section to send log data to the rsyslog server at 127.0.0.1, port 514, using the local2 facility.

So, we're using the local0 facility to pass log traffic defined on the global level, local1 for the HTTP log data, and local2 for TCP log data.

Let's stop rsyslog and remove the current HAProxy log files, then start syslog again.

There should be no HAProxy log files in /var/log. Let's change that. Restart the haproxy service.

Test HAProxy Logging Again

Let's see if we have all our HAProxy logs now. We should see our new HAProxy log files!

Let's take a look at the contents of our log files. We should see everything starting up. Let's generate some traffic to fill our logs with data!

Let's generate some HTTP traffic:

wget --no-check-certificate -O - http://www.site1.com/test.txt

wget --no-check-certificate -O - http://www.site2.com/test.txt

Let's generate some HTTPS traffic:

wget --no-check-certificate -O - https://www.site1.com/test.txt

wget --no-check-certificate -O - https://www.site2.com/test.txt

Let's try pulling a file from our SSH container, via HAProxy:

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -P 2222 [email protected]:/sshfiles/ssh-test.txt .

Let's keep doing the scp operation until we hit our session limit (10) and log some errors.

Take a look at the contents of the HAProxy log files again. We should see that we have 4 logs now, the admin log, one with HTTP traffic only, another with TCP traffic only, and a fourth with combined traffic.

Let's see what happens when our backend goes down. Use the podman stop -a command to stop all the containers. You can use the podman ps -a command to confirm all the containers have been stopped.

Take a look at the HAProxy log contents again. We should see the error messages indicating that our web servers are down.

Let's restart all our containers.

Starting all the web containers:

podman start site{1..2}_server{1..3}

Starting the SSH container:

podman start sshd-server

Take a look at the HAProxy log contents again. We should see our servers are back up.

Congratulations, Cloud Guru! You just configured HAProxy logging!