How to use BASH AWK command on Linux

When I was first starting out as a sysadmin and I wanted to know what it took to get promoted, one of the things I was told was that I needed to know the BASH utilities, such as AWK. The Linux job market continues to grow and expand, and our LFCS course will help prepare you for one of the standard industry Linux administration certifications.

I'm not sure AWK is actually a utility — it was originally intended to be a data-driven programming language when Alfred Aho, Peter Weinberger, and Brian Kernighan wrote it at Bell Labs. The name comes from the initials of their last names, but the program we use in BASH on Linux is (G)AWK or Gnu AWK, and there are many derivatives.

Now that we have the trivia out of the way, what can we do with AWK? AWK can be invoked from the command prompt by simply typing awk. On the command line, it is all lower case.

AWK is most often used for processing files. AWK processes a condition if one is provided and then takes an action. The default action is to print whatever meets the criteria of the condition. To search a file for a regular expression pattern match:

   awk ‘/regex/’  /etc/passwd 

This would result in the matching lines from the /etc/passwd file being printed to the command line. This is fairly basic, and we are using the default behavior of printing the matches.

AWK can also be used to search columns of data and cleanly output information. To know if a specific service is running and display the name if located:

   ps -ax | awk ‘/systemd/ {print $5}’

One of the nice things about awk is that it tokenizes the lines that are fed into it. This means for items that have a separator, you can select a column. You can specify a field separator on the command line.

One example would be to print a list of users from /etc/passwd, the entries are separated by a colon (:) and the username is in the first column. Remember that $0 is the entire line.

  awk -F: ‘{print $1}’ /etc/passwd

This is great, but we want to do some work with AWK, right? So if you have a directory (we'll use /etc for this one) and want to know the size of that directory:

  ls -l /etc | awk ‘{x += $5} END {print “total bytes:” x}'

Here, we take the output of the ls -l command and add each line to the variable x, since variables in AWK are initialized to 0 and the fifth column is the file size we are adding each file's size to x as we parse the output.

The END keyword indicates an action we want to take once the input parsing has completed. In this case, we print the value of x.In kilobytes, we divide the result x by 1024:

  ls -l /etc | awk ‘{x += $5} END {print “total Kilobytes:” (x/1024)}

AWK can also be used as a full scripting language, and you can pass script files to it on the command line using the -f flag.

Where to Get Started: As I said in the beginning, command line usage is something that really separates people from their contemporaries when it comes to job interviews. Companies are into automation and everything as code. The place this starts is with knowledge of the conditions in BASH shell scripting and all of its features.

If you want to set yourself apart and show you are comfortable in what you are doing, you might consider taking a look at the Admin's Guide to Bash Scripting. This will take you to a place where you can really show your comfort level with the command line.

A Cloud Guru makes it easy (and awesome) to get certified and master modern tech skills — whether you’re new to cloud or a seasoned pro. Check out ACG’s current free courses or get started now with a free trial.

• How to write a backup script with Bash
• How to use SCP and SSH Linux Commands
• 5 things you should know about Real-Time Analytics
• Protect Script Secrets with Azure Key Vault
• How to Monitor a VPN Connection in AWS using Lambda
• Conditions in bash if statements