How to use Awk command on Linux

Hello, in this guide, you will learn how to use Awk command on Linus systems. The awk command is a Linux tool and programming language that enables users to analyze and manipulate data as well as generate prepared reports.
The tool offers a variety of advanced text processing techniques and makes it easier to explain complex data selections.

The awk command was named after three programmers who created the first version in 1977: Alfred Aho, Peter Weinberger, and Brian Kernighan.

Awk Operations

Users can use awk to execute a variety of actions on an input file or text. Among the operations available are:

Go line by line through a file.
Divide the input file or line into fields.
Comparing the input line or fields to the pattern specified (s).
Take different actions on the matched lines.
Prepare the output lines.
Run string and arithmetic operations.
Use loops and control flow for output.
File and data transformation to a predetermined structure
Prepare reports in a certain format.

Using the Awk Command

In this guide, we will cover some of the use cases and examples of the AWK command. Below is the syntax for the awk command:

awk [options] 'selection_criteria {action}' input-file > output-file

The support for regular expression is one of the main characteristics that make awk a potent tool (regex, for short). A string that reflects a certain character pattern is known as a regular expression. The most popular regular expression syntaxes are listed below.

Basic characters. include anything that is alphanumeric, such as the underscore (_).
Character set: The regex has character groups to make things simpler. For instance, capital letters (A-Z), lowercase letters (a-z), and digits (0-9).
Meta-characters: Characters that explain alternative ways to build on the typical characters.
Period (. ): The position is valid for any character that matches it (except a newline).
Asterisk (*): It is valid if zero or more instances of the immediately preceding character occur.
Bracket ([]): If any character from the bracket matches at the position, the match is considered valid.
Character sets can be combined with it.
Caret (^). The match must be at the beginning of the line.
Dollar($).The match must be placed at the very end of the line.
Backslash (\): If a meta-character.

Here is a sample text file that can be used as a demo. There are 10 lines in it, with 2 words each.

$  cat tutornix.txt               
 dog end
 The new
 blue fox
 The team
 The clerk
 dog end
 The team
 The new
 dog end
 The clerk

Now depending on what you want to achieve, you can use awk in the following ways:

1. Printing text

Use the print command to print a text file’s whole contents. There is no defined pattern for the search pattern. Awk then prints each line. The AWK command “print” prints the contents of the input.

$ awk '{print}' tutornix.txt
dog end
 The new
 blue fox
 The team
 The clerk
 dog end
 The team
 The new
 dog end
 The clerk

2. String search

On the given text, AWK may do a rudimentary text search. It has to be the text to find in the pattern area. Awk will scan all lines of the file tutornix.txt for the text “team” in the following command.

$ awk '/team/' tutornix.txt
The team
 The team

3. Carret (^) Pattern

The following command will print all lines with the word “dog” at the beginning.

$ awk '/^ dog/ ' tutornix.txt
dog end 
dog end

4. Dollar ($) Pattern

What about looking for something at the end of a line? The following command will print all lines that contain the word “new” at the end.

$ awk '/new$/' tutornix.txt
 The new
 The new

5. Period pattern

The following example will demonstrate the use of the period (.).Any two characters can come before the character “e” in this case.

awk '/..e/' tutornix.txt

Sample output:

6. Asterisk (*) Pattern

What if the location can accommodate any amount of characters? Use the asterisk (*) to match any conceivable character at the place.
AWK will match all lines with any number of characters after “The” in this case.

$ awk '/The*/' tutornix.txt
 The new
 The team
 The clerk
 The team
 The new
 The clerk

7. Bracket expression

The bracket expression will be demonstrated in the following example. The bracket expression indicates that the match will be legitimate at the place if it matches the set of characters encompassed by the brackets.
For instance, the following command returns “The” and “Tee” as valid matches.

$ awk '/T[he]e/' tutornix.txt
 The new
 The team
 The clerk
 The team
 The new
 The clerk

Consider the following use of character sets with bracket expressions.

[0-9]: Indicates a single digit
[a-z]: Indicates a single lowercase letter
[A-Z]: Indicates a single uppercase letter
[a-zA-z]: Indicates a single letter
[a-zA-z 0-9]: A single character or digit is indicated.

For example, “A-Z” refers to the set of all uppercase letters. Awk will match any terms that begin with an uppercase letter in the following command.

$ awk '/[A-Z]/'  tutornix.txt
 The new
 The team
 The clerk
 The team
 The new
 The clerk

8. Pre-defined variables in Awk

AWK includes a slew of pre-defined and automated variables. These variables can help you write AWK programs and scripts more easily.

Note:$0– displays the complete line.

Awk is used to divide a line into fields and output the results. For example, if you want to print the first and third lines of the tutornix.txt file, the command will be:

awk ‘{print $1,$3}’ tutornix.txt

Using NR will print lines as well as line numbers. For example:

awk '{print NR,$0}' tutornix.txt

Sample Output:

Additionally, it can be used to define printing from one number to another.
Let’s display lines 3 through 6 in the display.

$ awk 'NR==3, NR==6 {print NR,$0}' tutornix.txt
3  blue fox
4  The team
5  The clerk
6  dog end

NF – returns the total number of fields in each record (line).

$ awk '{print NF}'   tutornix.txt             
2
2
2
2
2
2
2
2
2
2

$NF– prints the most recent columns.

$ awk '{print $NF}' tutornix.txt
end
new
fox
team
clerk
end
team
new
end
clerk

OFS – Output field separator – specifies the separation between output values.

$ awk 'OFS="/" {print $1,$2}' tutornix.txt
dog/end
The/new
blue/fox
The/team
The/clerk
dog/end
The/team
The/new
dog/end
The/clerk

9. Awk BEGIN and END rules

The BEGIN rule is anticipated to be executed once before any text processing and is executed before anything else.

awk 'BEGIN {print "my tutor file"} {print $0}' tutornix.txt

Sample Output:

10. Pattern addition

For a value larger than the output. For instance, in our file’s last field, which contains salary information, we should only publish data when the number is more than 15,000.

$ awk '$NF>=15000 {print $1,$2}' tutor.text
Ezy 20000
Gidy 25000
Frank 30000

Final reflection

This guide should have helped you gain a solid understanding of the fundamentals of AWK.
In terms of the power it bestows, mastering AWK is incredibly gratifying even though it could take some time.

Other manuals: