Skip to content

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.

Setup NetFlow Monitoring with Elasticsearch SIEM

Jan 21, 2020 • 11 Minute Read

Introduction

In modern data centers, many of the services you operate your businesses with are virtualized, and taping or spanning ports on physical infrastructure is not always an option. It is, in fact, not an option at all in cloud environments. However, NetFlow 1 through IPFIX(v10) is a standard format of session data from virtual and non-virtual switches located in the datacenter, vSphere, or cloud environments. Leveraging Netflow as a data source for security provides you the opportunity to have the least impact on the operations of the company while gaining visibility into the virtual network layer that is otherwise obscured.

Technology

This guide uses the 7.4 version of Filebeats; you should, of course, use whatever version you Elastic SIEM is currently operating. Not all of the dashboards and fields are backwards compatible, and mix-matching Filebeat versions cause issues with the indexes used for visualizations.

ProductVersionLink
Filebeat7.4https://www.Elastic.co/guide/en/beats/Filebeat/7.4/index.html

Verify Elastic SIEM Configuration

If you followed along with the Setting Up Elasticsearch for the Elastic SIEM Guide and the subsequent Kibana installation and configuration, you have specific IP addresses that are exposed in your environment, waiting to receive information. Before you get started, make sure you collect and record that information for further use. For this guide, the IP:Port endpoint for the Elasticsearch node is 192.168.218.139:9200, and for Kibana is 192.168.218.139:5601.

Filebeat for Netflow Collection

Filebeat is one of the most versatile of the beat family, with a long list of modules supporting the shipping of data to an Elastic stack.

Filebeat acts as a collector rather than a shipper for NetFlow logs, so you are setting it up to receive the NetFlow logs from your various sources. That being so, you can install Filebeat on whatever platform you wish as long as it is configured to send the data it collects and parses to the appropriate Kibana and Elastic nodes.

Download and Install Filebeat

Download the correct architecture installation file for Filebeat from Elastic.

For this guide, follow along installing the rpm package for CentOS 7.

From the command line, use curl to download the rpm package and install with the rpm command.

      user]$ curl -L -O https://artifacts.Elastic.co/downloads/beats/Filebeat/Filebeat-7.4.0-x86_64.rpm
[user]$ sudo rpm -vi Filebeat-7.4.0-x86_64.rpm
    

Filebeat installs in the /etc/filbeat folder and, just like the other Elasticsearch products, requires some configuration and file modification to get going.

Configure Filebeat

Edit the config file to point to the previously configured Elasticsearch and Kibana instances with nano.

      user]$ sudo nano /etc/Filebeat/filbeat.yml
    

In the Kibana section, find the host variable.

Change it to the exposed external IP address configured for the Kibana service. If you install Filebeats on the same device as the Kibana node, it is not listening on localhost, but instead on the external IP configured with 0.0.0.0.

Then find Elasticsearch in the outputs section and change the hosts variable.

Set it to the external IP address configured for the Elasticsearch service. In a similar fashion, if the Elastic node is on the same device interface that Filebeat is installed on, configure it with 0.0.0.0

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "Filebeat.yml.

Configure Netflow Module

Now it is time to enable and configure the Netflow module and run the Filebeat setup to connect to the Elasticsearch stack and upload index patterns and dashboards.

Note: This input supports NetFlow versions 1, 5, 6, 7, 8 and 9, as well as IPFIX. For NetFlow versions older than 9, fields are mapped automatically to NetFlow v9.

First, enable the NetFlow module.

      user]$ sudo Filebeat modules enable netflow
    

Find the netflow.yml configuration located in the modules.d directory inside the /etc/Filebeat install location. Notice that it is the only file without the appending .disabled designator.

Edit this configuration file with nano.

      user]$ sudo nano /etc/Filebeat/modules.d/netflow.yml
    

To ensure the Filebeat service is listening on an externally available port, change the localhost to 0.0.0.0 and leave the default 2055 port.

Exit nano, saving the config with ctrl+x, y to save changes, and enter to write to the existing filename "netflow.yml.

The rest of the options can be found [here]https://www.Elastic.co/guide/en/beats/Filebeat/master/Filebeat-input-netflow.html.

Test the configuration files.

      user]$ sudo Filebeat test config
    

If your configuration checks out, run Filebeat initial setup to load dashboards, templates, indexes, and further validate configuration settings.

Note: If you are running Filebeat 7.4 with a lower version Elasticsearch, the dashboards are not all compatible, notably the Filebeat-aws-s3access-overview.json dashboard, and it throws an error. Either upgrade Elastic SIEM or navigate to the /usr/share/Filebeat/Kibana/7/dashboard directory and move that file to a different location. Then re-run Filebeat setup.

      user]$ sudo Filebeat setup -e
    

Configure Firewall

To allow the UDP traffic from the NetFlow sources into the device running Filebeats, you have to create a firewall rule for that port and protocol by running the following commands.

      user]$ firewall-cmd --permanent --add-port 2055/udp
[user]$ firewall-cmd --reload
[user]$ firewall-cmd --list-all
    

Run File Beats

Then run Filebeats using the -e flag to output the activity to the console. Again, just to make sure everything is working properly.

      user]$ Filebeat -e
    

Filebeat now sits and listen on the 2055 UDP port for a NetFlow source to send it data.

If you are comfortable that everything is working properly, you can run the Filebeats service, and the configurations still apply.

[user]$ sudo systemctl start Filebeat -e
``

Configure Netflow Source

Now you need to configure your Netflow source. Many products have NetFlow output that you can configure to point towards the Filebeat NetFlow collector. For this guide, you are going to walk through setting up a virtual distributed switch on vSphere vCenter and providing a window into the virtual traffic management without impact on business resources.

Other potential sources include Cisco devices (it is a proprietary Cisco standard after all), Juniper devices, VMware Virtual Networking, AWS VPC, and more!

First, login to vCenter and navigate to the networking section. Select the virtual distributed switch you want to configure and choose the Netflow section, and then click edit configuration.

Type the Collector IP address and Collector port of the NetFlow collector. You can contact the NetFlow collector by IPv4 or IPv6 address. If you are going to set up more than one change, the identifier accordingly, and leave the switch IP blank. Leave all other settings default.

Next, select the uplinks for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

Next, select the port group for the VDS, properties, and click edit. Select the monitoring section and use the drop-down to change the NetFlow setting to enabled.

Netflow in Elastic SIEM

Now it is time to check back in the Elastic SIEM by browsing to the https://x.x.x.x:5601 address and clicking the lock icon that represents the SIEM overview section. Now you should see events posting in NetFlow, as long as there are VM's using the distributed switch.

Click on view network, and you get much of the same data that is populated form packet flow except directly from a NetFlow source.

Pivot to the dashboards. In the search bar type "Netflow". You see a list of ten available dashboards for the Filebeat NetFlow modules.

Happy hunting!

References

  • Elastic SIEM Guide - https://www.Elastic.co/guide/en/siem/guide/current/index.html

  • Filebeat Guide - https://www.Elastic.co/guide/en/beats/Filebeat/current/configuring-howto-Filebeat.html

  • Filebeat Netflow Module - https://www.Elastic.co/guide/en/beats/Filebeat/master/Filebeat-input-netflow.html

Related Courses

Aaron Rosenmund

Aaron R.

Aaron M. Rosenmund is a cyber security operations subject matter expert, with a background in federal and business defensive and offensive cyber operations and system automation. Leveraging his administration and automation experience, Aaron actively contributes to multiple open and closed source security operation platform projects and continues to create tools and content to benefit the community. As an educator & cyber security researcher at Pluralsight, he is focused on advancing cyber security workforce and technologies for business and national enterprises alike. In support of the Air National Guard, he contributes those skills part time in various initiatives to defend the nation in cyberspace. Certifications: GIAC GCIA, GIAC GCED, CCNA Cyber Operations, Pentest+, CySa+, CASP www.AaronRosenmund.com @arosenmund "ironcat"

More about this author