Unix CLI One-Liners

Here’s a short collection of one-liners that I’ve come up with over time. As I come across/up with more of these, I’ll try to remember to add them here.

Count/Sort the number of requests in an Apache Combind logfile by User Agent:

cat access.log | sed -e 's/^.*".*" "\(.*\)"$/\1/' | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • sed -e 's/^.*".*" "\(.*\)"$/\1/' – Use the stream editor (sed) to remove everything but the User Agent.
  • sort – Sort rows
  • uniq -c – Eliminate consecutive duplicate rows and count them.
  • sort -n – Sort the rows numerically, putting the largest number at the bottom.

 

Count all things instances of words in a file:

cat ${FILES}  | tr [:upper:] [:lower:] | sed -r 's/\t/ /g' | sed -r "s/'s//g" | sed -r 's/ /\n/g' | tr -d [:punct:] | sort | uniq -c | sort -n

The break down:

  • cat – It’s cat
  • tr [:upper:] [:lower:] – Make everything lower case.
  • sed -r 's/\t/ /g' – Convert tabs to spaces. All we care about is a word separator.
  • sed -r "s/'s//g" – Remove “‘s”.
  • sed -r 's/ /\n/g' – Convert spaces to newlines so we can use the sort | uniq -c | sort -n trick.
  • sed -r "s/'s//g" – Remove “‘s”.
  • tr -d [:punct:] – Strip all punctuation.
  • sort | uniq -c | sort -n – As described above: sort so uniq works, count unique rows, sort result numerically.

Note: This will treat white space as a word like thing and count it as well.

4 Responses to “Unix CLI One-Liners”

  1. Hey there would you mind letting me know which web host you’re utilizing?
    I’ve loaded your blog in 3 different internet browsers and
    I must say this blog loads a lot quicker then most. Can you recommend a good hosting provider at a reasonable
    price? Kudos, I appreciate it!

    Feel free to visit my web page read what he said [Sabrina]

    • It’s a custom stack running on a Digital Ocean 512mb droplet. Their SSD’s are nice! If you have the time and skill to deal with server maintenance, I’d recommend going with them. Right now, I’m running a combination of Apache/Varnish on Ubuntu Linux, but I’m probably going to switch it over to php-fpm and nginx here at some point.

  2. An outstanding share! I’ve just forwarded this onto a colleague who
    was conducting a little research on this. And he in fact bought me lunch due to
    the fact that I stumbled upon it for him… lol. So let me reword this….
    Thank YOU for the meal!! But yeah, thanx for spending time to discuss this matter here on your web
    page.

Leave a Reply