26
Feb
Feb
I have a database logging all website visits. How can I separated between real visitors and crawlers?
You can use Google Analytics for the best results, and the most data regarding your visitors. To have an idea what information they can offer you: http://www.icantinternet.org/2009/01/stats-show-you-what/
If you want to track it yourself, then you should use the “user agent string”, which holds the identification of the browser (or the spider in that case). I've written an article on how to use this string to identify an iPhone visitor, but by simply changing the string to look for, it also works for spiders: http://www.icantinternet.org/2009/01/recognizing-the-iphone-for-webpages/
This entry was posted
on Thursday, February 26th, 2009 at 5:03 pm and is filed under Website Traffic.
You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or TrackBack URI from your own site.
3 Responses so far to "How do I distinguish real visitors from crawlers in my website traffic logs?"
February 26th, 2009 at 10:22 pm
You can Advertise Your website For Free @
http://post2world.com
This is a free website on which i also promote my website for free because they are having i.5 million visitors per minute, there you can post your classifieds for free and attract more ultimate visitors to your website.
References :
February 26th, 2009 at 10:55 pm
You should use the Google analytics to help you on it.
References :
http://www.neuracom.com
February 26th, 2009 at 11:43 pm
You can use Google Analytics for the best results, and the most data regarding your visitors. To have an idea what information they can offer you: http://www.icantinternet.org/2009/01/stats-show-you-what/
If you want to track it yourself, then you should use the “user agent string”, which holds the identification of the browser (or the spider in that case). I've written an article on how to use this string to identify an iPhone visitor, but by simply changing the string to look for, it also works for spiders: http://www.icantinternet.org/2009/01/recognizing-the-iphone-for-webpages/
References :
http://www.icantinternet.org/2009/01/stats-show-you-what/
http://www.icantinternet.org/2009/01/recognizing-the-iphone-for-webpages/