One of my most used means of monitoring what’s going on at a remote Linux server, is looking at the log lines that each daemon or client program writes to the disk. Using this technique you can see a real time view about what is happening and find out if there are any outstanding errors or optimizations in your configuration. I am going to show how I tackle this task and what tricks I use to make tons of lines of dull text more readable.
GoAccess is a very handy tool, that runs in the terminal on most *nix Systems (e.g. Linux, Mac OS X, BSD etc.) and displays relatime data gathered from Apache’s access logfile.
It offers a very structured display with different “modules” that break down the parsed access.log data into sections:
- General Statistics – Information analyzed from log file – Unique totals
- Unique visitors per day – Including spiders
- Requested files (Page-URL)
- Requested static files (images, stylesheets etc.)
- Referrer URLs
- HTTP 404 Not Found response code
- Operating Systems
- HTTP Status Codes
- Top Referring Sites
- Top key phrases used on Google
Each module has a detail view that can be opened by pressing “o”. The detail view gives you a scrollable list with more information on the currently selected data.
Since I couldn’t find a package for GoAccess in Debian lenny’s apt sources, I took the manual compile route. But the installation is still really easy and fast.
- Make sure you’ve got the GLIB2, ncurses & GeoIP packages installed.
aptitude install libglib2.0-dev, libncurses5-dev, libgeoip-dev
- Download the latest GoAccess tarball from http://goaccess.prosoftcorp.com/download and uncompress it on your webserver.
At the time of writing the current version was GoAccess 0.3.3.
- Go into the newly created folder where the uncompressed files are and type “./configure”
(include the dot and slash before the command!)
- If all tests pass, you should see those three lines at the end of the output:
config.status: creating Makefile config.status: creating config.h config.status: executing depfiles commands
- Type “make” to start the compile process.
- When the program is readily compiled, it is time to install the binaries on your system.
For that gain root access (or use “sudo”) and type “make install“.
Once you are done installing you can start the program by typing in “goaccess” on the command prompt and pressing enter. If you don’t add any command line options, GoAccess will print a few lines of help text.
GoAccess - 0.3.3 Usage: goaccess [ -b ][ -s ][ -e IP_ADDRESS][ -f log_file ] The following options can also be supplied to the command: -f - Path to input log -b - Enable total bandwidth consumption. To achieve faster parsing, do not enable this flag. -s - Enable/report HTTP status codes. To avoid overhead while parsing, this has been disabled by default. -e - Exclude an IP from being counted under the HOST module. This has been disabled by default. For more details visit: http://goaccess.prosoftcorp.com
If you want all the features you can start it with:
goaccess -b -s -f /var/log/apache/access.log
Give it some time to parse the logfile (could take a very long time, depending on your logfile size), then you should see the mainscreen with live updating data and can start to use GoAccess.