The web is an oddly one-way medium. You can have a site that's read by thousands of people each day and -- unless you take some special effort -- you could have no idea it was even happening. It doesn't have to be this way. Web servers keep track of each hit they get. The problem is just that the data is never put together in a useful way.

front page

The stats system should have a front page that immediately communicates as much important information as possible:

On the top we have basic traffic information. How many hits / unique visitors this hour, day, month, year, with sparklines showing context (so you can see at a glance if these are above/below average).

Then we have a two column split.

Column one shows top pages. Where is traffic going to? The top 10 most popular pages (again, with sparklines showing context). At the bottom is a link "More..." that shows the top 100 or so.

Column two shows top referrers. Where is traffic coming from? The top 10 biggest referrers (with sparklines and a "More..." link).

When presenting referrers, we want to do some basic analysis to make them useful. A lot of referrers are search engine queries. Presenting a URL like http://www.google.com/search?q=foobar&sourceid=mozilla2&ie=utf-8&oe=utf-8 isn't useful at all. Instead, this should be presented as:

Google: foobar
  John's Foobars (/) is #5

meaning that it's a Google query for foobar and our homepage comes up #5 in the listings.

As for other pages, instead of just dumping the URL, we'll want to provide a little snippet like Google does:

FoobarWorld
  ...Another good source for foobar information
  is John's Foobars; he does a great job...
foobarworld.org/links

where the bolded portion is the link to our site.

Then underneath the columns we have the latest hits. Just a listing of the last ten hits or so, featuring the IP address (perhaps with reverse DNS if possible and whois info in parentheses), the page they hit, and their referrer. Again, a link to more.

individual pages

Everything in the front page should be a link that goes to more information. Clicking on the name of a page on our site goes to a special page showing the same sort of info as the front page, only filtered for just that particular page on our site.

Clicking on an IP address shows us more info about that IP (full whois information, geolocation, perhaps it tries an HTTP request on the reverse DNS) along with the full (paginated) history of that IP's hits on our site.

bonus features

Those are the basics, but once those are done there's lots of other fun stuff.

Deviation from the mean. Keep track of historical averages for things and have the front page note when there are significant deviations. An obscure page might suddenly be getting 10x as much traffic as usual, but still not get enough to make it to the front page's top 10. We'll still want to see that and see why. If traffic from Zimbabwe suddenly doubles, that's pretty interesting too.

Ajax push. Use Ajax push to live update the front page's latest hits, so you can watch them scroll by in your web browser. Even cooler if it live updates the other traffic data.

A name: We should come up with a name.

And there's plenty more where that came from.

last updated 2 years ago # history