awk

Reference:

• The AWK programming language - Alfred Aho, Brian Kernighan, Peter J. Weinberger
• https://www.gnu.org/software/gawk/manual/gawk.html

Line number starts from 0.

Predefined variables

https://www.gnu.org/software/gawk/manual/gawk.html#Built_002din-Variables

• NR: line/record number (from all files)
• FNR: line number (from current file)
• RS: record separator. ie, separates lines
  • Default \n.
• END:
• $0: current line text

BEGIN and END

Special patterns.

• BEGIN: run before file is processed
• END: run after file is processed

Positional specifier

Follwing two statements have the same effect:

printf "%s %s\n", "hello", "world"
printf "%2$s %1$s\n", "world", "hello"

and prints 'hello world'.

Appending to a string variable

$ echo 'a' | awk 'BEGIN {a="1"} /.+/ {a=a$0} END{print a}'
1a

awk function

function <name> {

}

https://faculty.cs.niu.edu/~berezin/330/N/awk-fun.html

Arithmetic

• Integer division: int()
• Can treat strings as if they are numbers

String substitution

• sub
• gsub

https://www.gnu.org/software/gawk/manual/html_node/String-Functions.html

Matching non-empty lines

A trick:

length { <code> }

This works because length of empty line gives 0, which is considered as false value.

Thanks to tirnanog from #awk.

Regex

• [[:digit:]]: like \d in sed
• regex match operatioin: var ~ /rgx/

Example:

# -F is used to make field separator comma instead of space
# Prints lines whose first column starts with 'T' 
awk -F "," '$1 ~ /^T/ {print $0}'

Another:

$ cut -f4,5 -d, pincodes.csv \
  | sed "s/\"//g"  \
  | awk -F "," '$2 ~ /5[[:digit:]]5[[:digit:]]2[[:digit:]]/ && $1 ~ /^T/ {print $0}' \
  | sort -k1 \
  | less -N

Tips

• print "": outputs a blank line (with \n?)
• Indexing starts from 1
• Formatting functions: sprintf, printf

Jargon