Manage Linux with sed and awk
In this lab you will practice filtering and editing text at the Linux command line. When you have finished with this lab, you'll have knowledge and experience in using the commands: grep, sed, and awk. You will be able to filter text, programmatically edit text, and create reports from log files; all using free and open source tools.
Terms and conditions apply.
Challenge
Getting Started in the Lab Environment
Here are the initial instructions and explanation of the lab environment. Read this while your environment is busy creating itself from nothing. Yes, this violates physics; we know. How fun!
Challenge
Filter Text Using grep
In this challenge, you'll be using the command-line tool "grep" to extract data from text files. Specifically, you'll be searching for pure text within two standard files: /etc/passwd and /etc/os-release. The examples provided will be useful for any system, as they demonstrate how to effectively use grep to search for specific text patterns within these files. With this knowledge, you'll be able to efficiently search for and extract the information you need from text files on any system.
Challenge
Use Regular Expressions in Searches
In this task, you will delve into the realm of powerful search capabilities by harnessing the potential of regular expressions. Although clear text searches can be useful, regular expressions take things up a notch, offering an array of advanced search and pattern matching functionalities. By leveraging regular expressions with command line tools and scripting languages, you can undertake more intricate searches and extract valuable information. Throughout this challenge, we will delve into the usage of grep and regular expressions, enabling you to effectively search for and extract complex text patterns from files. By the end of this endeavor, you will have acquired the skills to navigate and conquer even the most intricate textual landscapes.
Challenge
Work with sed
You now enter the realm of the stream editor, sed, where files are programmatically transformed, making it an influential tool for manipulating text. Although sed may appear daunting initially, you shall embark on a journey to acquaint yourself with its multitude of options. Throughout this challenge, you will gain proficiency in utilizing sed's options proficiently, progressively honing your skills to effortlessly edit files programmatically. As the challenge unfolds, you will establish a firm groundwork in sed, empowering you to employ it effectively for an array of text editing endeavors.
Challenge
sed Command and Edits
So far, you have used a few sed commands: p to print the pattern match, d to delete lines, and s to substitute or replace a pattern. There are more commands, though, to investigate: i to insert a new line above, and a to append a new line below. In this challenge, you'll be working with the /etc/hosts file, and using the -i option with sed to edit it in-place. Your first task will be to remove all empty and commented lines from the file. Next, you'll add a header to indicate that the file is edited and managed using sed, and then add a new server entry. To achieve this, you'll create a sed file to manage the editing process. You'll test your results before proceeding by using the "sudo" command and sed -i to make in-place edits to the /etc/hosts file. By the end of this challenge, you'll have gained valuable experience with sed, and be able to apply these skills to other text editing tasks.
Challenge
Understand awk
Prepare to wield the mighty power of awk, an esteemed text processing tool often revered as the "big brother" of grep and sed. While its core function lies in generating reports, awk possesses the remarkable ability to create reports and manipulate text. Your journey begins with an exploration of the foundational concepts of awk, utilizing it to process command text files. Throughout this challenge, you will acquire the expertise to harness awk's potential in manipulating text data and extracting specific information from files. By the culmination of this endeavor, you will have gained invaluable experience with awk, equipping you to proficiently employ its capabilities across a vast array of text processing tasks.
Challenge
Work with awk and Flat Files
Delve into the depths of the /etc/passwd file, where entries are separated by colons. However, there are instances where alternative data interchange formats like CSV or YAML prove more convenient. Embark on this challenge as you unravel the realm of awk and its prowess in printing the contents of the passwd file in both CSV and YAML formats. This challenge will empower you to utilize awk in transforming data from one format to another, tailoring it to specific use cases. By the culmination of this endeavor, you will have gained invaluable expertise in awk, enabling you to wield its capabilities adeptly for data transformation into a diverse range of formats.
Challenge
Analyze web logs with awk
When systems have public access, it's important to be aware of the potential insecurities and vulnerabilities that can arise, especially when it comes to SSH access. In this challenge, you'll work with a log file that provides examples of possible malicious intent, and use it to create a list of IPs that have attempted to access the root account or invalid users. This list can then be used as a blocklist for firewalls to help protect against potential security breaches. To ensure your own IP is excluded from the list, you'll demonstrate how to use a simple grep filter. By the end of this challenge, you'll have gained valuable experience in analyzing log files and identifying potentially harmful activity, as well as how to create a blocklist to help secure your system.
Challenge
The Last Challenge
Welcome to the final challenge! This is your last chance to experiment in the environment. Clicking Finish Lab will end this little world that flittered into existence just for you.
Provided environment for hands-on practice
We will provide the credentials and environment necessary for you to practice right within your browser.
Guided walkthrough
Follow along with the author’s guided walkthrough and build something new in your provided environment!
Did you know?
On average, you retain 75% more of your learning if you get time for practice.
Recommended prerequisites
- Command line experience in Linux
- Edit text files using vim or nano