🎁 New User? Get 20% off your first purchase with code NEWUSER20 Register Now →
Menu

Categories

awk Command

Advanced Text Processing man(1)

Pattern scanning and text processing language

👁 8 views 📅 Updated: Mar 15, 2026
SYNTAX
awk 'PATTERN {ACTION}' FILE

What Does awk Do?

awk is a powerful text processing language and command for pattern scanning, field extraction, and data transformation. It processes input line by line, splitting each line into fields, making it ideal for working with structured text like CSV files, log files, and command output.

awk programs consist of pattern-action pairs: when a line matches the pattern, the corresponding action executes. The default field separator is whitespace, but it can be changed with -F. Built-in variables like NR (record number), NF (number of fields), and FS (field separator) provide context for processing.

While sed is designed for stream editing (find-and-replace), awk excels at columnar data extraction and computation. It supports variables, arrays, arithmetic, string functions, and control flow — making it a complete programming language for text processing tasks.

Options & Flags

OptionDescriptionExample
-F Set field separator (default: whitespace) awk -F',' '{print $1}' data.csv
-v Assign a variable before execution awk -v threshold=100 '$3 > threshold' data.txt
-f Read awk program from a file awk -f script.awk input.txt
BEGIN Execute block before processing any input awk 'BEGIN{print "Header"} {print}' file
END Execute block after all input is processed awk '{sum+=$1} END{print sum}' numbers.txt
NR Built-in variable: current record (line) number awk 'NR>=10 && NR<=20' file.txt
NF Built-in variable: number of fields in current record awk '{print NF}' file.txt
-OFS Output field separator awk -F',' -v OFS='\t' '{$1=$1; print}' file.csv

Practical Examples

#1 Print specific columns

Extracts the first and third fields from each line.
$ awk '{print $1, $3}' data.txt

#2 Process CSV files

Uses comma as field separator to extract columns from a CSV file.
$ awk -F',' '{print $2, $4}' users.csv

#3 Sum a column of numbers

Adds up all values in the first column and prints the total.
$ awk '{sum += $1} END {print "Total:", sum}' sales.txt
Output: Total: 15420

#4 Filter lines by condition

Shows only lines where the third field exceeds 1000.
$ awk '$3 > 1000' transactions.txt

#5 Count lines matching a pattern

Counts lines containing ERROR in the log file.
$ awk '/ERROR/ {count++} END {print count}' app.log
Output: 42

#6 Print lines between two patterns

Prints all lines from START to END inclusive — useful for extracting sections.
$ awk '/START/,/END/' config.txt

#7 Format output as a table

Formats /etc/passwd into aligned columns showing username, UID, and shell.
$ awk -F':' '{printf "%-15s %-6s %s\n", $1, $3, $7}' /etc/passwd
Output: root 0 /bin/bash

#8 Calculate average

Computes the average of values in the first column.
$ awk '{sum += $1; n++} END {print "Average:", sum/n}' scores.txt
Output: Average: 85.3

Tips & Best Practices

Print the last field: Use $NF to always get the last field regardless of how many fields exist: awk '{print $NF}' file.txt
awk vs cut: For simple column extraction, cut is simpler and faster. Use awk when you need conditions, calculations, or complex formatting.
Quote carefully: Wrap awk programs in single quotes to prevent the shell from interpreting $1, $2, etc. as shell variables instead of awk field variables.

Frequently Asked Questions

How do I print specific columns from a file?
Use awk '{print $1, $3}' file.txt where $1 is the first column and $3 is the third. Use -F to change the delimiter: awk -F',' for CSV files.
What is the difference between awk and sed?
sed is a stream editor best for find-and-replace and line-level editing. awk is a full programming language best for column-based data extraction, calculations, and formatted reporting.
How do I use awk with a pipe?
Pipe output from any command into awk: ps aux | awk '{print $1, $11}' extracts the user and command name from process listings.

Master Linux with Professional eBooks

Curated IT eBooks covering Linux, DevOps, Cloud, and more

Browse Books →