Unix CLI One-Liners

Here’s a short collection of one-liners that I’ve come up with over time. As I come across/up with more of these, I’ll try to remember to add them here.

Count/Sort the number of requests in an Apache Combind logfile by User Agent:

cat access.log | sed -e 's/^.*".*" "\(.*\)"$/\1/' | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • sed -e 's/^.*".*" "\(.*\)"$/\1/' – Use the stream editor (sed) to remove everything but the User Agent.
  • sort – Sort rows
  • uniq -c – Eliminate consecutive duplicate rows and count them.
  • sort -n – Sort the rows numerically, putting the largest number at the bottom.

 

Count all things instances of words in a file:

cat ${FILES}  | tr [:upper:] [:lower:] | sed -r 's/\t/ /g' | sed -r "s/'s//g" | sed -r 's/ /\n/g' | tr -d [:punct:] | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • tr [:upper:] [:lower:] – Make everything lower case.
  • sed -r 's/\t/ /g' – Convert tabs to spaces. All we care about is a word separator.
  • sed -r "s/'s//g" – Remove “‘s”.
  • sed -r 's/ /\n/g' – Convert spaces to newlines so we can use the sort | uniq -c | sort -n trick.
  • sed -r "s/'s//g" – Remove “‘s”.
  • tr -d [:punct:] – Strip all punctuation.
  • sort | uniq -c | sort -n – As described above: sort so uniq works, count unique rows, sort result numerically.

Note: This will treat white space as a word like thing and count it as well.

Leave a Reply