Working with Logstash

Install Elasticsearch with default settings:

1. Install Java:

   	yum install java-1.8.0-openjdk -y
   

2. Import Elastic's GPG key:

   	rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
   

3. Download the Elasticsearch RPM:

   	curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm
   

4. Install Elasticsearch:

   	rpm --install elasticsearch-6.2.3.rpm
   

5. Enable and start Elasticsearch:

   	systemctl daemon-reload
   	systemctl enable elasticsearch
   	systemctl start elasticsearch
   

Install Logstash with default settings:

1. Import the Logstash key:

   	rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
   

2. Add the Logstash repo:

   	vi /etc/yum.repos.d/logstash.repo
   
   [logstash-6.x]
   name=Elastic repository for 6.x packages
   baseurl=https://artifacts.elastic.co/packages/6.x/yum
   gpgcheck=1
   gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   autorefresh=1
   type=rpm-md
   

3. Install Logstash

   yum install logstash -y
   

4. Enable and start Logstash:

   	systemctl enable logstash
   	systemctl start logstash
   

Install Kibana with default settings:

1. Download Kibana:

   	curl -O https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-x86_64.rpm
   

2. Install Kibana:

   	rpm --install kibana-6.2.3-x86_64.rpm
   

3. Enable and start Kibana:

   	systemctl enable kibana
   	systemctl start kibana
   

Install Filebeat with default settings and use the system module:

1. Download Filebeat:

   	curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.3-x86_64.rpm
   

2. Install Filebeat:

   	rpm --install filebeat-6.2.3-x86_64.rpm
   

3. Edit the system module to convert timestamp timezones to UTC:

   In /etc/filebeat/modules.d/system.yml.disabled, change:

   	# Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
   	#var.convert_timezone: false
   

   To:

   	# Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
   	var.convert_timezone: true
   

   For both the syslog and auth sections.

4. Enable the system Filebeat module:

   	filebeat modules enable system
   

5. Install the ingest-geoip filter plugin for Elasticsearch ingest node:

   	/usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
   

6. Restart Elasticsearch so it can use the new ingest-geoip plugin:

   	systemctl restart elasticsearch
   

7. Once Elasticsearch starts up, push module assets to Elasticsearch and Kibana:

   	filebeat setup
   

8. Enable and start Filebeat:

   	systemctl enable filebeat
   	systemctl start filebeat
   

Connect to Kibana and explore your system log data:

1. From your local machine, SSH with port forwarding to your cloud server's public IP:

   	ssh user_name@public_ip -L 5601:localhost:5601
   

2. Navigate to localhost:5601 in your web browser.

3. Go to the Dashboard plugin via the side navigation bar.

4. Search for system to filter to your system dashboards.

5. Explore your system log data with the supplied dashboards.