-1

I operate a number of content websites that have several million user sessions and need a reliable way to monitor some real-time metrics on particular pieces of content (key metrics being: pageviews/unique pageviews over time, unique users, referrers).

The use case here is for the stats to be visible to authors/staff on the site, as well as to act as source data for real-time content popularity algorithms.

We already use Google Analytics, but this does not update quickly enough (4-24 hours depending on traffic volume). Google Analytics does offer a real-time reporting API, but this is currently in closed beta (I have requested access several times, but no joy yet).

New Relic appears to offer a few analytics products, but they are quite expensive ($149/500k pageviews - we have several times this).

Other answers I found on StackOverflow suggest building your own, but this was 3-5 years ago. Any ideas?

Adam McCann
  • 197
  • 2
  • 8

4 Answers4

0

If the metrics that you need to track are just limited to the ones that you have listed (Page Views, Unique Users, Referrers) you may think of collecting the logs of your web servers and using a log analyzer.

There are several free tools available on the Internet to get real-time statistics out of those logs. Take a look at www.elastic.co, for example.

Hope this helps!

AbZe31
  • 11
  • 1
  • This is effectively doing it myself - the last suggestion - which is of course possible, but not very quick (quite a bit of devops involved to collect all the logs together, I imagine, and then pipe this data to a database etc). If there are no services already built for this, I would go for this option, but I find that unlikely! – Adam McCann Dec 15 '15 at 15:47
0

Google Analytics offers real time data viewing now, if that's what you want?

https://support.google.com/analytics/answer/1638635?hl=en

I believe their API is now released as we are now looking at incorporating this!

Andrew
  • 26
  • 3
  • Yeah, I covered this in the text of my post. The Real Time API (which is what I would need to have it on site) is not yet public. – Adam McCann Dec 15 '15 at 15:49
0

If you have access to web server logs then you can actually set up Elastic Search as a search engine and along with log parser as Logstash and Kibana as Front end tool for analyzing the data.

For more information: please go through the elastic search link.

Elasticsearch weblink

Shantesh
  • 1,470
  • 1
  • 16
  • 26
0

Heard some good things about Woopra and they offer 1.2m page views for the same price as Relic.

https://www.woopra.com/pricing/

If that's too expensive then it's live loading your logs and using an elastic search service to read them to get he data you want but you will need access to your logs whilst they are being written to.

A service like Loggly might suit you which would enable you to "live tail" your logs (view whilst being written) but again there is a cost to that.

Failing that you could do something yourself or get someone on freelancer to knock something up for you enabling logs to be read and displayed in a format you recognise.

https://www.portent.com/blog/analytics/how-to-read-a-web-site-log-file.htm

Andrew
  • 26
  • 3