My configuration for bash, vim, git, tmux and a few other tools can be found on GitHub now.
I have spent a considerable amount of time to get them working on OS X and Linux.
(Tested on Mac OS X Yosemite 10.10.3 and Debian 7.8 – YMMV!)
One of my most used means of monitoring what’s going on at a remote Linux server, is looking at the log lines that each daemon or client program writes to the disk. Using this technique you can see a real time view about what is happening and find out if there are any outstanding errors or optimizations in your configuration. I am going to show how I tackle this task and what tricks I use to make tons of lines of dull text more readable.
Recently I’ve had to re-engineer my whole backup strategy because of a looming hardware issue on my old backup server. Since I am operating servers at Hetzner, I purchased some backup space there and based my new system around their available system. The new strategy consists of several key components:
- Mounting a remote directory via sshfs
- Ensuring the remote mountpoint is always available using autofs
- Creating a loopback filesystem on the remote storage
- Incremental rsync snapshots using my Hactar backup script
With the introduction of Apple’s newest operating system OS X 10.7 Lion, the company has taken one step further to disconnect their users from other competing user interaction schemes.
Let me elaborate on this rather strong statement by going back in history a bit.
I can still remember, when I got my first Mac, it was a 12″ iBook G4 with OS X Panther. Being a rather curious person, I started tweaking the user interface and tested out all available options, until I was satisfied with the speed and metaphors of my personal set up. On thing that I started using very frequently almost immediately was Exposé (now extinct due to Lion’s Mission Control). I had several hot corners set up, so windows would fly out of the way and reveal the Desktop whenever I moved my mouse in the lower left corner of the screen or reveal all opened windows when I moved to the lower right corner. This method of interacting with my programs quickly became very natural to me. Going so far, that I started moving my mouse into corners on Windows systems several times in vain, until I realized that I had to use the taskbar for those similar tasks. This wasn’t more than a little annoyance to me. I just found out that OS X permitted me to work faster than what I’d be able to achieve in the rare cases, where I’d need to use another system.
But the problem also ran in the other direction. People not used to Mac systems and their concept of hot corners seemed to have a much more clumsy way of moving their mouse around on the screen. Thus they inadvertently triggered the “Show Desktop” or “Show all windows” functions all of the time and were completely irritated by windows flying around all of the sudden. I merely laughed about that fact back then, but from my point of view today, that’s where the gap began to widen. I was used to a UI paradigm, that wasn’t logical at all for users not familiar with OS X.
Skip a few years forward and now we’ve got OS X 10.7 Lion with many more ways of interaction, that are nowhere near to be seen on any other platform. Take gestures for instance: they save a lot of time and clicks when used properly, but setting ones muscle memory further apart from the rest of computer users, who don’t have or use gestures. Now this isn’t a very big problem, because gestures need special hardware to be applicable and add new ways of interaction that don’t overrule the way of using a computer you got used to before. A much bigger problem could be the way Apple changed the scrolling direction in Lion. They call it “natural scrolling” and if you think about it, it makes a lot of sense and there isn’t much of a learning curve, once you’ve wrapped your head around it. But think of the ramifications that arise when others happen to sit in front of your computer or if you want to use another persons system.
If I think about a typical day at work, there are several times when a colleague might sit down at my Mac to look something up, load some spreadsheet or notes about what we are just discussing, or similar examples. In the past this was no big deal, they might only be annoyed by triggering hot-corners or when they searched for the “close window” button on the wrong side of a window’s title bar. But now there is also the reversal of scrolling, which not only is new and “weird” to them, but is completely contradictory to how the same metaphor worked for them over the last decades.
Don’t be mistaken, I’m not against the new “natural scrolling”. I’m getting used to it rather quickly and enjoy how I can now interact with content more logically. But on the other hand it will make it more difficult for me, whenever I have to use a Windows or Linux Desktop and now even when I use an older OS X system. At the same time other users will find it more irritating and difficult to use my machine. It will be interesting where this will take us in the future. Will the gap grow even more? Or will operating system manufacturers decide on a common ground for user interactions?
I don’t mind adapting to the new principles Apple comes up with, as long as they make sense for me and help me interact with my computer in a more efficient way. But this also means I’m separating myself further from the herd (aka millions of Windows users), which might not be entirely bad in itself 😉
GoAccess is a very handy tool, that runs in the terminal on most *nix Systems (e.g. Linux, Mac OS X, BSD etc.) and displays relatime data gathered from Apache’s access logfile.
It offers a very structured display with different “modules” that break down the parsed access.log data into sections:
- General Statistics – Information analyzed from log file – Unique totals
- Unique visitors per day – Including spiders
- Requested files (Page-URL)
- Requested static files (images, stylesheets etc.)
- Referrer URLs
- HTTP 404 Not Found response code
- Operating Systems
- HTTP Status Codes
- Top Referring Sites
- Top key phrases used on Google
Each module has a detail view that can be opened by pressing “o”. The detail view gives you a scrollable list with more information on the currently selected data.
Since I couldn’t find a package for GoAccess in Debian lenny’s apt sources, I took the manual compile route. But the installation is still really easy and fast.
- Make sure you’ve got the GLIB2, ncurses & GeoIP packages installed.
aptitude install libglib2.0-dev, libncurses5-dev, libgeoip-dev
- Download the latest GoAccess tarball from http://goaccess.prosoftcorp.com/download and uncompress it on your webserver.
At the time of writing the current version was GoAccess 0.3.3.
- Go into the newly created folder where the uncompressed files are and type “./configure”
(include the dot and slash before the command!)
- If all tests pass, you should see those three lines at the end of the output:
config.status: creating Makefile config.status: creating config.h config.status: executing depfiles commands
- Type “make” to start the compile process.
- When the program is readily compiled, it is time to install the binaries on your system.
For that gain root access (or use “sudo”) and type “make install“.
Once you are done installing you can start the program by typing in “goaccess” on the command prompt and pressing enter. If you don’t add any command line options, GoAccess will print a few lines of help text.
GoAccess - 0.3.3 Usage: goaccess [ -b ][ -s ][ -e IP_ADDRESS][ -f log_file ] The following options can also be supplied to the command: -f - Path to input log -b - Enable total bandwidth consumption. To achieve faster parsing, do not enable this flag. -s - Enable/report HTTP status codes. To avoid overhead while parsing, this has been disabled by default. -e - Exclude an IP from being counted under the HOST module. This has been disabled by default. For more details visit: http://goaccess.prosoftcorp.com
If you want all the features you can start it with:
goaccess -b -s -f /var/log/apache/access.log
Give it some time to parse the logfile (could take a very long time, depending on your logfile size), then you should see the mainscreen with live updating data and can start to use GoAccess.