Hits or Misses: Who is Your Website's Audience?

securitas writes "The Christian Science Monitor's Gregory M. Lamb wrote a story interesting to anyone who runs a website: How do you accurately and reliably measure the audience for your website? From the article: 'Most websites have no idea how many people view their content. This inherent fuzziness is causing problems for commercial websites, especially online publications desperate to make money from Internet advertising... How can you charge for ads when it's nearly impossible to tell advertisers how many people will see them?' The article discusses the flaws and problems with Nielsen/NetRatings and comScore Media Metrix - they grossly undersample workplace users - and the rise in the number of sites requiring user registration."

use cookies?

[Anonymous Coward]

I always just set a cookie with a tracking ID, and then use that to keep track of the anon user. counting the number of tracking cookies given out each day, and the time they were used for seems to work sufficiently for me... or is there some problem with that I don't know about?

Re:use cookies?

[PhxBlue]

Only when you consider browsers that let you reject cookies, such as Firefox. But then, that's more the web developer's problem than mine, since I'd just as soon remain anonymous.

Re:Cookies?

[RetroGeek]

Call me oblivious, but wasn't this one of the reasons why cookies were created?

Using the Mozilla cookie control, I regularily go through my cookies. Anything that looks like it is coming from an ad site I delete and block.

Any site which I do not recognize gets the same treatment.

I have not had any problems from any site because of this.

Re:use cookies?

[Stephen]

Read the article. They are complaining that one user may read the content from work and from home, and so count as two users. One might also point out that sometimes two people may use the same computer, and only count as one person.

Re:You shouldn't care how many people visit ...

[lukewarmfusion]

Your first line is that advertisers shouldn't care how many people visit... but then you go on to talk about how you increase traffic to your own website.

If your site uses an ad-supported business model, you (and your advertisers) should care how many people are visiting your site. Advertisers want to spend their money somewhere that they know will be seen.

The Super Bowl charges more for a 30-second spot than your local cable channel; that's because of the sheer number of people that will be watching. If you (and your advertisers) know how many people are visiting the site, then you can put some numbers to your business model - and that's a smart way to run a business.

Re:use cookies?

[Anonymous Coward]

Don't forget, a lot of people who use spyware tools like adaware and spybot delete tracking cookies on a regular basis, further skewing results.

Re:use cookies?

[Anonymous Coward]

I'm glad I didn't waste my time reading such a stupid artical then.

That's like saying a billboard company wants to know the exact number of people that look at their billboard while driving by. Some things just can't be done, what a waste of time.

Re:Isn't it obvious

[cshark]

No.

That's why the standard is per impression CPM (cost per thousand). One user even from home could generate hundreds of impressions if the content is interesting enough, and the pages are chocked full of useful ads!

Per click is another methodology, but until Google came along, it really wasn't the standard on the ad sales end. Still isn't outside of Google and the search engines.

That said, most web sites do know exactly what demographics are visiting their web sites and when. If it's important enough to buy software to do it, and most do, there are several useful software packages that come to mind. Web Trends is the first one I think of. That program in particular actually catches many of the problems described in the article, and it's not unusual. Many such programs have similar functionality.

Honestly, it would have been nice to see them do their home work before writing yet another scare piece.

Re:use cookies?

[cmoney]

But how dynamic is your IP? Does it change every single time you access a web site? Or is it pretty much stable, changing when you power down your connection? My cable connection hasn't changed IP since 2 months ago when I installed a new router.

Re:Stupid crackhead mods

[FesterDaFelcher]

and they could call it metamoderation [slashdot.org]? Yeah, they should implement that.

Re:Cookies?

[frodo from middle ea]

Am I the only person here who uses privoxy [privoxy.org] ?
here is my setup,
Behind a NAT box, with no ports opened.
Use firefox as browser and privoxy as ad filtering proxy server. and zone alarm as FW
I have ad-aware, spybot and grisoft free antivirus, but in last 2 years I haven't had a single trojan/virus/spyware hit me.

Besides using privoxy will save you the trouble of going through your cookies, as it filters almost all of them.
Forget pop-up ads, it even filters in-line ads

Re:Uh, No...

[Pionar]

Ok, there's a difference between counting hits and counting visitors. You're wasting money if the same guy sees your ad over and over again, because he'll get desensitized to it and will ignore it after a while. But, you can't tell if those 47 hits from the same ISP is one guy or 47 guys, as he gets a different IP each time he dials in. Is that a unique visitor, or a refresh? Why is this person viewing this one page 12 times?

In short, sure, you can always count quantity using logs, but it's impossible to count quality with them. That's the point.

The other point is it's stupid to display an ad for MS Server 2003 to a person who doesn't deal with that kind of product. Why would Oracle want to display ads to the guy that maintains the Exchange server?

One problem is that right now, a lot of web advertising is hit-and-miss. You pay thousands for "targeted" advertising, just hoping someone who will actually need your product will see the ads.

Relevance and quality is the key in online advertising these days, not how many eyeballs you get. Counting is easy. Analyzing is hard. So, you're backwards, not the article.

Unique vistiors from web logs - Detailed anaylsis

[the frizz]

It was my task to develop log anaylsis software to count visitors for large web sites. I was not only surprised to find out how inaccurate the art was, but also had difficulty in convincing other web-experienced collegues on how impossible it was. "All the other web analysis programs display number of visitors" they said. Well they all make guesses is the answer. The current best practice is to count unique login names, but most sites don't use authenicated logins and even then you can have many hotmail accounts.Here's the disclaimer I eventually wrote for my sites unique visitor stats.

The number of visitors displayed does not accurately represent the number of actual people visiting your site. Many people can appear as a single IP address by sharing proxies, caches, NAT firewalls or even simply sharing the same computer at home. One person can also appear as many IP addresses by using dynamic IP addressing (most dialup and PPPoE users), being load balanced across proxies and caches or simply using multiple computers (e.g., at home and at work.)

Other reasons for overcounting include: robots; rogue client software that keeps changing its ID string; users that delete cookies, upgrade software or use multiple client software agents.

Other reasons for undercounting are clients that don't (or have been set not to) accept cookies or operate through anonymizers. If authenticated logins are used, determining the number of real people from server-side logs may be best derived from a cookie which is only set after authenticated login that only holds a value which uniquely corresponds to the user (e.g., an user name or account number).

When the statistics for more than one day is selected, the peak daily number of visitors is displayed rather than a sum of the daily visitors.

Since the above was written I discovered a common practice of sysadmins and help desks is the suggesting manually deleting all cookies (since you can't do it selectively with MS-IE) to get over site bugs. And now the increasing popular spyware removal tools (E.g., spybot) remove 3rd party cookies used just to count unique visitors in the name of removing sypware and viruses from your computer.

Originally I thought of defining a visitor for HTTP domains as the cookie if it exists, and the client IP address otherwise. But the flaw in this is that it will double count first time HTTP visitors. Once for the log line of their first hit with no cookie. And again for the subsequent hit. With streaming logs, using the GUID (effectively a cookie these days) and the client IP address is more useful as a unique visitor. The log lines in streaming are actually the summary of a sequence or request/reply transacations and so the first "hit" log line does have a GUID/cookie logged.

What follows is addition research I turned up:

• ABCi [abcinteractiveaudits.com] (a web traffic auditor)
  says: `` A visitor is defined as "a unique IP address with heuristic." To properly account for visits, the Web site needs to identify a "visitor" so that visitor activity is properly tracked. Registration and/or cookies are the best way to track a visitor's activity through the Web site. Unfortunately, a lot of Web sites do not require registration, nor do they use cookies [and browsers can disable cookies] If cookies are used, it is the clients' responsibility to provide the auditor with details on how the server sets the cookie, the cookie format and how the cookies are used. An alternative that has been suggested is to use the IP address AND user-agent in combination, to identify a unique visitor. The interaction with the site by this "visitor" is then analyzed to determine the number of visits which should be recorded. Using only the IP address to identify a visitor is not acceptable due to the number of visitors that may not be accurately reported because they are operating behind a proxy server or firewall. ''