Setup NetFlow Monitoring with Elasticsearch SIEM

In modern data centers, many of the services you operate your businesses with are virtualized, and taping or spanning ports on physical infrastructure is not always an option. It is, in fact, not an option at all in cloud environments. However, NetFlow 1 through IPFIX(v10) is a standard format of session data from virtual and non-virtual switches located in the datacenter, vSphere, or cloud environments. Leveraging Netflow as a data source for security provides you the opportunity to have the least impact on the operations of the company while gaining visibility into the virtual network layer that is otherwise obscured.

This guide uses the 7.4 version of Filebeats; you should, of course, use whatever version you Elastic SIEM is currently operating. Not all of the dashboards and fields are backwards compatible, and mix-matching Filebeat versions cause issues with the indexes used for visualizations.

If you followed along with the Setting Up Elasticsearch for the Elastic SIEM Guide and the subsequent Kibana installation and configuration, you have specific IP addresses that are exposed in your environment, waiting to receive information. Before you get started, make sure you collect and record that information for further use. For this guide, the IP:Port endpoint for the Elasticsearch node is 192.168.218.139:9200, and for Kibana is 192.168.218.139:5601.

Filebeat is one of the most versatile of the beat family, with a long list of modules supporting the shipping of data to an Elastic stack.

Filebeat acts as a collector rather than a shipper for NetFlow logs, so you are setting it up to receive the NetFlow logs from your various sources. That being so, you can install Filebeat on whatever platform you wish as long as it is configured to send the data it collects and parses to the appropriate Kibana and Elastic nodes.

Download the correct architecture installation file for Filebeat from Elastic.

For this guide, follow along installing the rpm package for CentOS 7.

From the command line, use curl to download the rpm package and install with the rpm command.

Filebeat installs in the /etc/filbeat folder and, just like the other Elasticsearch products, requires some configuration and file modification to get going.

Edit the config file to point to the previously configured Elasticsearch and Kibana instances with nano.

In the Kibana section, find the host variable.

Change it to the exposed external IP address configured for the Kibana service. If you install Filebeats on the same device as the Kibana node, it is not listening on localhost, but instead on the external IP configured with 0.0.0.0.

Then find Elasticsearch in the outputs section and change the hosts variable.

Set it to the external IP address configured for the Elasticsearch service. In a similar fashion, if the Elastic node is on the same device interface that Filebeat is installed on, configure it with 0.0.0.0

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "Filebeat.yml.

Now it is time to enable and configure the Netflow module and run the Filebeat setup to connect to the Elasticsearch stack and upload index patterns and dashboards.

Note: This input supports NetFlow versions 1, 5, 6, 7, 8 and 9, as well as IPFIX. For NetFlow versions older than 9, fields are mapped automatically to NetFlow v9.

First, enable the NetFlow module.

Find the netflow.yml configuration located in the modules.d directory inside the /etc/Filebeat install location. Notice that it is the only file without the appending .disabled designator.

Edit this configuration file with nano.

To ensure the Filebeat service is listening on an externally available port, change the localhost to 0.0.0.0 and leave the default 2055 port.

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "netflow.yml.

The rest of the options can be found [here]https://www.Elastic.co/guide/en/beats/Filebeat/master/Filebeat-input-netflow.html.

Test the configuration files.

If your configuration checks out, run Filebeat initial setup to load dashboards, templates, indexes, and further validate configuration settings.

Note: If you are running Filebeat 7.4 with a lower version Elasticsearch, the dashboards are not all compatible, notably the Filebeat-aws-s3access-overview.json dashboard, and it throws an error. Either upgrade Elastic SIEM or navigate to the /usr/share/Filebeat/Kibana/7/dashboard directory and move that file to a different location. Then re-run Filebeat setup.

To allow the UDP traffic from the NetFlow sources into the device running Filebeats, you have to create a firewall rule for that port and protocol by running the following commands.

Then run Filebeats using the -e flag to output the activity to the console. Again, just to make sure everything is working properly.

Filebeat now sits and listen on the 2055 UDP port for a NetFlow source to send it data.

If you are comfortable that everything is working properly, you can run the Filebeats service, and the configurations still apply.

[user]$ sudo systemctl start Filebeat -e
``

Now you need to configure your Netflow source. Many products have NetFlow output that you can configure to point towards the Filebeat NetFlow collector. For this guide, you are going to walk through setting up a virtual distributed switch on vSphere vCenter and providing a window into the virtual traffic management without impact on business resources.

Other potential sources include Cisco devices (it is a proprietary Cisco standard after all), Juniper devices, VMware Virtual Networking, AWS VPC, and more!

First, login to vCenter and navigate to the networking section. Select the virtual distributed switch you want to configure and choose the Netflow section, and then click edit configuration.

Type the Collector IP address and Collector port of the NetFlow collector. You can contact the NetFlow collector by IPv4 or IPv6 address. If you are going to set up more than one change, the identifier accordingly, and leave the switch IP blank. Leave all other settings default.

Next, select the uplinks for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

Next, select the port group for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

Now it is time to check back in the Elastic SIEM by browsing to the https://x.x.x.x:5601 address and clicking the lock icon that represents the SIEM overview section. Now you should see events posting in NetFlow, as long as there are VM's using the distributed switch.

Click on view network, and you get much of the same data that is populated form packet flow except directly from a NetFlow source.

Pivot to the dashboards. In the search bar type "Netflow". You see a list of ten available dashboards for the Filebeat NetFlow modules.

Happy hunting!

• Elastic SIEM Guide - https://www.Elastic.co/guide/en/siem/guide/current/index.html

• Filebeat Guide - https://www.Elastic.co/guide/en/beats/Filebeat/current/configuring-howto-Filebeat.html

• Filebeat Netflow Module - https://www.Elastic.co/guide/en/beats/Filebeat/master/Filebeat-input-netflow.html

• SET: Detecting Malicious Traffic with Signature and Session Analysis

• SET: Detecting Network Anomalies with Behavioral Analysis

• SET: Monitoring Assets and Topology

• SET: Detecting System Anomalies