by ·3 Comments
GoAccess - web log analyzer

GoAccess - web log analyzer

Description

GoAccess is a very handy tool, that runs in the terminal on most *nix Systems (e.g. Linux, Mac OS X, BSD etc.) and displays relatime data gathered from Apache’s access logfile.

It offers a very structured display with different “modules” that break down the parsed access.log data into sections:

  • General Statistics – Information analyzed from log file – Unique totals
  • Unique visitors per day – Including spiders
  • Requested files (Page-URL)
  • Requested static files (images, stylesheets etc.)
  • Referrer URLs
  • HTTP 404 Not Found response code
  • Operating Systems
  • Browsers
  • Hosts
  • HTTP Status Codes
  • Top Referring Sites
  • Top key phrases used on Google

Each module has a detail view that can be opened by pressing “o”. The detail view gives you a scrollable list with more information on the currently selected data.

Installation

Since I couldn’t find a package for GoAccess in Debian lenny’s apt sources, I took the manual compile route. But the installation is still really easy and fast.

  • Make sure you’ve got the GLIB2, ncurses & GeoIP packages installed. 
    aptitude install libglib2.0-dev, libncurses5-dev, libgeoip-dev
  • Download the latest GoAccess tarball from http://goaccess.prosoftcorp.com/download and uncompress it on your webserver.
    At the time of writing the current version was GoAccess 0.3.3.
  • Go into the newly created folder where the uncompressed files are and type “./configure”
    (include the dot and slash before the command!)
  • If all tests pass, you should see those three lines at the end of the output: 
    config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
  • Type “make” to start the compile process.
  • When the program is readily compiled, it is time to install the binaries on your system.
    For that gain root access (or use “sudo”) and type “make install“.

Running GoAccess

Once you are done installing you can start the program by typing in “goaccess” on the command prompt and pressing enter. If you don’t add any command line options, GoAccess will print a few lines of help text.

GoAccess - 0.3.3

Usage: goaccess [ -b ][ -s ][ -e IP_ADDRESS][ -f log_file ]

The following options can also be supplied to the command:

  -f  - Path to input log
  -b  - Enable total bandwidth consumption. To achieve
        faster parsing, do not enable this flag.
  -s  - Enable/report HTTP status codes. To avoid overhead
        while parsing, this has been disabled by default.
  -e  - Exclude an IP from being counted under the HOST
        module. This has been disabled by default. 

For more details visit: http://goaccess.prosoftcorp.com

If you want all the features you can start it with:

goaccess -b -s -f /var/log/apache/access.log

Give it some time to parse the logfile (could take a very long time, depending on your logfile size), then you should see the mainscreen with live updating data and can start to use GoAccess.

GoAccess - Operating Systems module report

GoAccess - Operating Systems module report

by ·0 Comments

This handy piece of code colors errors and warnings in make output.
(put it in your .bash_profile) 

make()
{
  pathpat="(/[^/]*)+:[0-9]+???
  ccred=$(echo -e ???\033[0;31m")
  ccyellow=$(echo -e "\033[0;33m")
  ccend=$(echo -e "\033[0m")
  /usr/bin/make "$@" 2>&1 | sed -E -e "/[Ee]rror[: ]/ s%$pathpat%$ccred&$ccend%g??? -e ???/[Ww]arning[: ]/ s%$pathpat%$ccyellow&$ccend%g???
  return ${PIPESTATUS[0]}
}

via shawnwilsher.com

An alternative would be to install colormake and use that instead of the normal make command.

by ·1 Comment

If you need to direct output into the void in your program on a *NIX system you simply pipe or redirect it to the null device /dev/null.
Needing a similar functionality today in a C# .NET project I found the following equivalent:

Stream.Null

Seemed fairly useful to me 😉

by ·1 Comment

Log4Twitter is very similar to an idea Herbert an me had a couple of years ago, to use a Jabber messaging framework to get distributed logging and alerts from several servers/applications.
It is a Java class that allows you to “log” to Twitter. This would easily allow you to set up an application that sends messages to a private Twitter Account that you can subscribe to. Retrieving those messages should then be possible by IM or even SMS, seems like a perfect set up for me.

Using Log4Twitter is as easy as the following:

Setting Classpath
Add log4twitter-1.0.jar to your application’s classpath.
Note that log4twitter-1.0.jar has to be loaded by the same classloader that will load the logging framework.

Setting Logger
Edit your logging framework’s configuration to enable Log4Twitter.
The fully qualified class name of log4twitter is “log4twitter.FRAMEWORK_NAME.TwitterAppender”.

See the Log4Twitter page for some examples.

Now who’s got the time to code a Linux syslogd replacement or supplement, so I can receive important log messages via Twitter? 🙂

by ·0 Comments

Jeff Atwood wrote a funny (and very true) article about what’s new or not so new in these Web 2.0 days. I especially had to laugh at the idea that “new” ideas on the net are merely replicas of old UNIX tools introduced by Marc Hedlund.

talk, finger

ICQ
LISTSERV

DejaNews
ls

Yahoo! directory
find, grep

Google
rn

Bloglines
pine

Google Mail
mount

Amazon S3
bash

Yahoo! Pipes
wall

Twitter

Go and check out the full posting: Coding Horror: When In Doubt, Make It Public.
And make sure you also read the comments for some more funny ideas.