GoAccess – Visual Apache Log Analyzer

GoAccess - web log analyzer

GoAccess - web log analyzer

Description

GoAccess is a very handy tool, that runs in the terminal on most *nix Systems (e.g. Linux, Mac OS X, BSD etc.) and displays relatime data gathered from Apache’s access logfile.

It offers a very structured display with different “modules” that break down the parsed access.log data into sections:

  • General Statistics – Information analyzed from log file – Unique totals
  • Unique visitors per day – Including spiders
  • Requested files (Page-URL)
  • Requested static files (images, stylesheets etc.)
  • Referrer URLs
  • HTTP 404 Not Found response code
  • Operating Systems
  • Browsers
  • Hosts
  • HTTP Status Codes
  • Top Referring Sites
  • Top key phrases used on Google

Each module has a detail view that can be opened by pressing “o”. The detail view gives you a scrollable list with more information on the currently selected data.

Installation

Since I couldn’t find a package for GoAccess in Debian lenny’s apt sources, I took the manual compile route. But the installation is still really¬†easy and fast.

  • Make sure you’ve got the GLIB2, ncurses & GeoIP packages installed.
    aptitude install libglib2.0-dev, libncurses5-dev, libgeoip-dev
  • Download the latest GoAccess tarball from http://goaccess.prosoftcorp.com/download and uncompress it on your webserver.
    At the time of writing the current version was GoAccess 0.3.3.
  • Go into the newly created folder where the uncompressed files are and type “./configure
    (include the dot and slash before the command!)
  • If all tests pass, you should see those three lines at the end of the output:
    config.status: creating Makefile
    config.status: creating config.h
    config.status: executing depfiles commands
  • Type “make” to start the compile process.
  • When the program is readily compiled, it is time to install the binaries on your system.
    For that gain root access (or use “sudo”) and type “make install“.

Running GoAccess

Once you are done installing you can start the program by typing in “goaccess” on the command prompt and pressing enter. If you don’t add any command line options, GoAccess will print a few lines of help text.

GoAccess - 0.3.3

Usage: goaccess [ -b ][ -s ][ -e IP_ADDRESS][ -f log_file ]

The following options can also be supplied to the command:

  -f  - Path to input log
  -b  - Enable total bandwidth consumption. To achieve
        faster parsing, do not enable this flag.
  -s  - Enable/report HTTP status codes. To avoid overhead
        while parsing, this has been disabled by default.
  -e  - Exclude an IP from being counted under the HOST
        module. This has been disabled by default. 

For more details visit: http://goaccess.prosoftcorp.com

If you want all the features you can start it with:

goaccess -b -s -f /var/log/apache/access.log

Give it some time to parse the logfile (could take a very long time, depending on your logfile size), then you should see the mainscreen with live updating data and can start to use GoAccess.

GoAccess - Operating Systems module report

GoAccess - Operating Systems module report

Morning News Update, November 4 2010

  • Google released an Apache module that auto tunes performance
    Google has announced that they are launching a module for the Apache HTTP server called mod_pagespeed, that will automatically tune several properties to optimize page load speed. I haven’t tried it out yet, but will definitely do so.
    Amongst other things, mod_pagespeed will:

    • recompress images
    • modify cache lifetime for static elements
    • make changes to pages built by CMS

    I don’t know if I’ll see a great performance on this server, as I already took care of a lot of speed improvements myself, but this module is a great way to implement common best-practices without having to dig into Apache’s configuration too much.
    Read more and download the module on the Page Speed Google Code website.

  • Facebook Mobile Announcements
    Just a short summary of what Facebook announced yesterday, there’s already tons of coverage out there:

    • Facebook for iPhone updated (as I hoped in my tweet just minutes before the event started).
    • Facebook for Android updated.
    • Single Sign-On for mobile platforms.
      So you don’t have to deal with entering your password in every app that uses Facebook connect.
    • Facebook Places API Update. Full read and write access for third parties.
    • Deals. This sounds kind of like Groupon mixed with Foursquare.
      Could be very interesting, once it reaches the critical mass of user adoption.
    • No dedicated iPad App in the near future.
      Although I’ve read about speculations of a HTML5 based website/app that could offer the same experience on multiple tablet based platforms.

Installing php5-cgi on Debian

For a long time I wanted to be able to run both PHP 4 and PHP 5 on the same server. Now I finally managed to come up with a solution that is easy to install and easy to use.

  1. Add the Debian Backports repository to your /etc/apt/sources.list
    deb http://www.backports.org/debian/ sarge-backports main contrib non-free
  2. Issue a package update: aptitude update
  3. Install PHP 5 as CGI module: aptitude install php5-cgi
  4. Configure Apache by adding the following lines to your main config file. In my case this was /etc/apache2/apache2.conf
    ScriptAlias /php5-cgi /usr/lib/cgi-bin/php5
    Action php5-cgi /php5-cgi
  5. Now find the Virtual Host you want to run on PHP 5 and simply put the following line into the <Directory> directive.
    AddHandler php5-cgi .php
    This overrides the default handler for files ending in .php which would be the normal PHP 4 module. Whit this directive it is now parsed and executed by the PHP 5 CGI binary.
  6. Reload your webserver config with
    /etc/init.d/apache2 reload

Fixing non working Permalinks on WordPress

After struggling almost one week to find the cause for my not entirely working Permalinks on this WordPress installation, I finally found the cure on the WordPress Codex. I already read this page about five times and I just can’t tell how this one paragraph could always slip away… anyway the solution is rather simple.

As I’m a decently paranoid Server Admin I once changed the way Apache reports it’s server software. This poses some troubles with WordPress which checks to see if the host is running Apache or IIS to determine wether to write a .htaccess file with some mod_rewrite rules or not. As WordPress can’t find out on it’s own on my host I just had to help it a little ;)

In wp-includes/vars.php on line 40 is the check for the hosting webserver, I changed the entire line to read

$is_apache = 1;

After this small modification Worpress happily updatet the .htaccess file with the apropriate mod_rewrite rules (given the file permissions are correct on the server).

One small sidenote: If you are using FAlbum and have “friendly urls” enabled you have to make sure that the rewrite rules made by FAlbum are above the ones from WordPress.