Unix CLI One-Liners

Here’s a short collection of one-liners that I’ve come up with over time. As I come across/up with more of these, I’ll try to remember to add them here.

Count/Sort the number of requests in an Apache Combind logfile by User Agent:

cat access.log | sed -e 's/^.*".*" "\(.*\)"$/\1/' | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • sed -e 's/^.*".*" "\(.*\)"$/\1/' – Use the stream editor (sed) to remove everything but the User Agent.
  • sort – Sort rows
  • uniq -c – Eliminate consecutive duplicate rows and count them.
  • sort -n – Sort the rows numerically, putting the largest number at the bottom.

 

Count all things instances of words in a file:

cat ${FILES}  | tr [:upper:] [:lower:] | sed -r 's/\t/ /g' | sed -r "s/'s//g" | sed -r 's/ /\n/g' | tr -d [:punct:] | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • tr [:upper:] [:lower:] – Make everything lower case.
  • sed -r 's/\t/ /g' – Convert tabs to spaces. All we care about is a word separator.
  • sed -r "s/'s//g" – Remove “‘s”.
  • sed -r 's/ /\n/g' – Convert spaces to newlines so we can use the sort | uniq -c | sort -n trick.
  • sed -r "s/'s//g" – Remove “‘s”.
  • tr -d [:punct:] – Strip all punctuation.
  • sort | uniq -c | sort -n – As described above: sort so uniq works, count unique rows, sort result numerically.

Note: This will treat white space as a word like thing and count it as well.

2 Responses to “Unix CLI One-Liners”

  1. Hey there would you mind letting me know which web host you’re utilizing?
    I’ve loaded your blog in 3 different internet browsers and
    I must say this blog loads a lot quicker then most. Can you recommend a good hosting provider at a reasonable
    price? Kudos, I appreciate it!

    Feel free to visit my web page read what he said [Sabrina]

    • It’s a custom stack running on a Digital Ocean 512mb droplet. Their SSD’s are nice! If you have the time and skill to deal with server maintenance, I’d recommend going with them. Right now, I’m running a combination of Apache/Varnish on Ubuntu Linux, but I’m probably going to switch it over to php-fpm and nginx here at some point.

Leave a Reply