f(x)=x

Here’s a short collection of one-liners that I’ve come up with over time. As I come across/up with more of these, I’ll try to remember to add them here.

Count/Sort the number of requests in an Apache Combind logfile by User Agent:

cat access.log | sed -e 's/^.*".*" "\(.*\)"$/\1/' | sort | uniq -c | sort -n

The break down:

• cat – It’s cat
• sed -e 's/^.*".*" "\(.*\)"$/\1/' – Use the stream editor (sed) to remove everything but the User Agent.
• sort – Sort rows
• uniq -c – Eliminate consecutive duplicate rows and count them.
• sort -n – Sort the rows numerically, putting the largest number at the bottom.

Count all things instances of words in a file:

cat ${FILES}  | tr [:upper:] [:lower:] | sed -r 's/\t/ /g' | sed -r "s/'s//g" | sed -r 's/ /\n/g' | tr -d [:punct:] | sort | uniq -c | sort -n

The break down:

• cat – It’s cat
• tr [:upper:] [:lower:] – Make everything lower case.
• sed -r 's/\t/ /g' – Convert tabs to spaces. All we care about is a word separator.
• sed -r "s/'s//g" – Remove “‘s”.
• sed -r 's/ /\n/g' – Convert spaces to newlines so we can use the sort | uniq -c | sort -n trick.
• sed -r "s/'s//g" – Remove “‘s”.
• tr -d [:punct:] – Strip all punctuation.
• sort | uniq -c | sort -n – As described above: sort so uniq works, count unique rows, sort result numerically.

Note: This will treat white space as a word like thing and count it as well.