What is the best way to exclude bots from view counts?
I'm glad you know there isn't going to be a bulletproof way to accomplish this. That means your outlook is at least realistic.
Since JavaScript is not an option I would say you're left with:
Check the user-agent for the word "bot" in it. That will catch most of them.
Compile a list of known bots and filter them based on some kind of unique identifier, probably their user-agent.
Put a hidden link in the footer of your website that links to a page that collects user-agents and/or IP addresses. Users won't see this but bots will. So anyone who visits that page will be a bot. Record them and then block them from your stats.
There are three fairly simple ways:
- Use Google Analytics, which will process and handle all the data for you, and present you with detailed statistics for visitors and how they got to your site. This is by far the easiest solution.
- Use Javascript to do the counting. When the page has loaded, generate an AJAX request to your counting script. Robots and spiders don't run Javascript.
- Detecting "bot" in the user agent string is actually fairly reliable. Alternatively, you could stick to known bots only such as Googlebot, Yahoo, MSNbot etc. Checking those three should cover 99% of your bot traffic. This page has some others but it looks quite out of date.
UPDATE: Googlebot and some major bots do run JavaScript these days. So using option #2 alone is no longer viable. However, this does mean using it in conjunction with #3 should be quite reliable, as you can easily exclude most bots by using JS, then on the server-side exclude major bots like Googlebot that do run JS.
Also as mentioned in the comments you could try using the Google Analytics API to display views for each page.
If you use Javascript to count views then most bots won't run it and so won't be included in your view counts. This answer may be close to what you want https://stackoverflow.com/questions/1973448/how-can-i-count-a-page-views