Setup Audit Events with Elasticsearch SIEM

Excited about your brand new Elasticsearch SIEM setup, but trying to figure out which data to start with? Look no further than the landing page of the SIEM section in Kibana.

Notice the two categories:

• Host Events
• Network Events

These are reasonably self explanatory, but the idea is that with this SIEM capability, you bridge the connections between the hosts and the network to reveal emergent properties of malicious activity, and build attack chains more efficiently than looking at the data separately.

More information on the Elasticsearch SIEM and the use of different beats to ingest the additionally compatible events the other Elasticsearch SIEM guides is here. In this guide, you will install, configure, and ingest logs from the most prominent category in the host data section, Auditbeats. You can find more information on the Elasticsearch SIEM and the use of different beats to ingest the additionally compatible events the other Elasticsearch SIEM guides here.

The Auditbeat module from Elasticsearch is an agent that is loaded on to an endpoint, Linux, MacOS, or Windows that uses different modules to provide events to the Elasticsearch SIEM.

The events that are compatible with the Elasticsearch SIEM are shown on the overview page, but not all modules and datasets work on all endpoint OS systems.

Here is an overview:

To gather information from a previous Elasticsearch SIEM installation, you will need the exposed IP of the Elasticsearch and the Kibana service found in the services respective YAML configuration files.

Assuming the elastic instance was set up to expose 192.168.218.139 and the firewall set rules to allow Elasticsearch (9200) and Kibana (5601) to listen externally, as shown in the elasticsearch setup guide, these will be the endpoint settings used for the Auditbeat configuration in this guide as well.

Though not all features are supported on Windows and MacOS, the file integrity, process, and host monitoring capabilities are significant. To get started with audit beats on Windows, first navigate to the Auditbeat downloads page within your browser of choice and select the correct distro from the list.

For this guide, I am installing Auditbeats on a 64bit Windows 10 system and chose that respective package for download.

Make sure to also download the sha file for integrity checking and then open up Powershell, navigating to the download location of the two files.

An alternative command line is invoke-webrequest. https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-7.3.2-windows-x86_64.zip

Open Powershell and navigate to the download location, hash the downloaded zip file, and print the contents from the sha integrity file and ensure the two values match before continuing.

The results should be two hashes printed that match and look something like this:

With verified installation media downloaded from Elasticsearch, it is time to install the Auditbeat client.

Open Windows Explorer to the location containing the Auditbeat zip file, right-click the file, and select extract all.

Choose the location to extract the contents of the archive and click extract.

Then verify successful extraction of folder and files to the desired location.

You can install Auditbeat as a service by running the supplied ps1 file from an elevated command prompt. It will set up an automatic start service and use the default auditbeat.yml config file. Note: Install service Powershell module for later .

Before running, you need to let Auditbeat know where to send the data. In this case, it is the Elasticsearch SIEM set up previously.

Open the auditbeat.yml file located in the extracted folder with Powershell ISE using the run as admin option. Powershell ISE is the choice here because it will display the yaml text in a easily readable and editable format while providing a Powershell console to run the commands that will verify the configurations were properly implemented.

Once you have Powershell ISE open, select the file menu button, then open. Navigate to the extracted Auditbeat folder location and change the file type selector to Any File . You should now see the auditbeat.yml file available for selection.

Now that the file is open, you need to scroll down until you see the Kibana section,which will have the default values commented out and look like this:

Next, continue scrolling to the Output section of the configuration file to the first subheading named Elasticsearch.

Similar to the Kibana section configuration, you will want to remove the # comment character and replace the localhost:9200 value with the value of your Elasticsearch node's listening address. In this case, it will be 192.168.218.139:9200.

Note: This config differs from the kibana configuration in that it does not require the https:// prefix.

Now save the configuration, but do not close the Powershell ISE Window.

In the bottom of the Powershell ISE window, run the following command to change to the extracted Auditbeat directory and run the setup.

Once run, the the Auditbeat program will run checks agains the config and the environment and reach out to the Elasticsearch and Kibana node configured to create an index and index template, pattern, policy, and upload pre-built dashboards.

With the setup successfully complete, it is time to run Auditbeats for the first time. It is always good troubleshooting practice to run the Auditbeats executable not only from the command line, but with the -e option, which tells the program to output to standard out, allowing you to see the activity and troubleshoot problems as they arise.

Once you are happy with the configuration and stability of the Auditbeats program, you can then add it as a service that runs automatically with the start of system.

Run Auditbeats with following command:

As it runs, you can see the successful connections to Elasticsearch and Kibana, but once initialized, you can also see the JSON output of the metrics of events sent to the Elasticsearch SIEM with the [monitoring] event category shown at the bottom of the console print out.

The moment you have been waiting for is here. You will finally get your hands on that juicy Auditbeat host data. Though you trust the console output from the client, browse to the Kibana node and click the SIEM window to verify successful transmission and collection of data.

In the hosts section of the Elasticsearch SIEM, you now have one host populated and listed by name in the All Hosts card resulting from the default configuration for the system module to include the host dataset.

Event Data is also included as the output of the file integrity module.

Uncommon Processes is populated from the process data set of the system module.

You probably noticed that the authentication and IP data that is shown in the hosts card is missing, and if you took a look at the overview page you would see that under the Auditbeat audit, login, package, user, and socket modules there is still a 0, indicating that Elasticsearch SIEM has not received any of this type of data within the given time period. This makes sense because those are the exact modules that are not compatible with the Windows operating system. To get data from the rest of the modules, you need to install and configure the Auditbeat agent on a Linux OS device.

No reason to leave any data analysis profit on the table. Following the same installation process and centOS 7 system as the Filebeats program in the Setting Up Elasticsearch SIEM guide, you will be able to leverage all the available modules.

First open a terminal and use the following commands to download the rpm and install the Auditbeat 7.3.2 agent.

Then edit the auditbeat.yml file located in /etc/auditbeat/*.

The configuration file has some different modules available, but otherwise is identical to the configuration file for Windows. Navigate to the Kibana section and change it to the external listening address of the Kibana node, and then do the same in the Elasticsearch section under output.

Press ctrl+x, then y, then return/enter to save the file from nano.

Now, just as in Windows, you need to run the setup. The Linux version of the Auditbeat agent will have different index patterns and dashboard to upload because it supports many more modules and datasets than the agent for MacOS or Windows.

Successful setup will again end with the Loaded Dashboards text and return you to the command prompt.

Next, use screen to create a new terminal session, and run Auditbeat with the -e flag to monitor the progress and look for the `monitoring] event category to indicate successful transmission of events to the Elasticsearch SIEM.

Once you see the indication that events from the other modules are being sent, open up the browser to the Kibana node and the Elasticsearch SIEM host window and behold!

The second host is listed in the All Hosts card, and you now have data populating the User Authentications and Unique IPs section. If you pivot to the SIEM overview tab, you can see there are events in every single category of Auditbeat data sources.

You might notice in this environment the Auditbeat Audit category only has one event, which stands out due to the earlier errors you may have also seen in the auditd module.

If you did not have those errors, and if you have more than one event in the Audit beat, then disregard the following instructions.

Take a look at this one event from the auditd module. You can see that it is simply an event saying that it failed to start this module and that an audit process was already running. This is confirmed on the centOS 7 endpoint being used in this guide but may vary depending on distribution.

Also, remember that in the logs generated to standard out from the -e flag, when this executable was run on the Linux device mentioned that no auditd rules were set.

Back in the centOS7 device running Auditbeat, press ctrl+c to exit the process to terminal, and open up the configuration file.

Notice the modules sections, beginning with file integrity. This is the module that scans for changes in files and folder locations that you are concerned with. The values can be changed and customized, and the options for this module can be found in detail on Elastic's website here.

Below, you can see the system module running now with every possible data set, which was not the case on the Windows system due to the reduced compatibility and supported features. More options for the system module can be found here.

All of the modules are configured by default on Linux systems with the exception of the auditd module, which requires you to either write custom rules or uncomment (remove the #) rules that are already in the configuration.

Once you have uncommented the rules, you have eliminated the first error that was recognized for the auditd module during the running of the Auditbeat process. Save the file from nano and exit by pressing ctrl+x, then y then return.

Back at the terminal, you now need to stop the process that is currently running monitoring audit events. This process, known as auditd, is run on the local OS in centOS 7 by default to record audit logs to disk. You have to stop this service, disable it, and optimize the disk space usage by sending the audit events to devel null, so that the auditd module can run in its place.

Run:

Confirm that the auditd status is indeed running and is the reason that you cannot run the auditd module from Auditbeat.

Then stop the service and disable it to prevent it from starting automatically at startup.

Note: If you are using a debian-based linux, you may need to run chkconfig auditd off to disable the auditd service permanently.

Now optimize the OS and disk preventing the audit socket from writing logs to disk.

With the OS taken care of, you need to also make sure you have your bases covered with the auditd module configuration. Start by showing the audit rules to ensure the rules were properly uncommented and recognized.

Check the status of the auditd module.

Test your configuration file changes.

And with the okay from Auditbeat itself, it is time to re-run.

This time, you should not see any errors for auditd, and once you see good events in the monitor category, shift over to kiban and the Elasticserch SIEM overview tab.

Now you can see that there are 73 messages from auditd, which makes more sense from something that should be actively sending events and alerts from the kernel.

Just like that, you have working data from every Auditbeat module for the Elasticsearch SIEM. You can view them in the host tab, network (which includes Auditbeat socket), or in the form of a custom timeline of select events. The SIEM has optimized queries and parsing for these log formats, which can be seen in the search bar.

• Elastic SIEM Guide - https://www.elastic.co/guide/en/siem/guide/current/index.html
• Auditbeat Guide - https://www.elastic.co/guide/en/beats/auditbeat/current/index.html
• Elastic SIEM Dat Sources - https://www.elastic.co/products/siem

• SET: Detecting Malicious Traffic with Signature and Session Analysis
• SET: Detecting Network Anomalies with Behavioral Analysis
• SET: Monitoring Assets and Topology
• SET: Detecting System Anomalies