Monday, August 30, 2010

Five Text Processing Tools You Should Know

In the world of UNIX text is king. Almost anything one wants to know about a system can be gathered from reading a file. Often times that file can be a few thousand lines long. Sometimes it can be twelve million lines long.
This is where text processing comes in. Text processing aims to do one thing: given a source of information answer a question. Everybody has questions for their systems. “How many 404 errors has my website gotten?” “Has anyone tried to access my server without me knowing?” “What program is using the most CPU right now?”
This article will explore the most common ways to answer these questions with a set of tools that will by extension provide a method to answer hundreds of other questions. Questions will be answered with one-liners, or the stringing together of these tools to provide answers without writing a script itself.