Browser Stats For The BBC Homepage 260
Lord_Scrumptious writes "An interesting article titled 'The software used to access the BBC homepage' has recently been published on a blog by a BBC employee. It's all about the different browsers and operating systems accessing the BBC's homepage. The analysis is from a week of page requests in September 2005. Not surprisingly, Internet Explorer accounted for 85% of site visits, but Firefox had a very respectable 9.7% share. Even requests from Sony's handheld PSP device were recorded, but interestingly there's no mention of mobile phone devices."
Finally.... (Score:5, Insightful)
Re:Finally.... (Score:2, Insightful)
No, it's still unreliable. You simply can't correlate traffic to visitors. That's not the way HTTP works. httpd log analysis can tell you many interesting things, but mainly concerning the load on the server. Any attempt to read more into it is based on assumptions that are not only wrong, but wrong by an unknowable amount.
This is true every time somebody posts some bullshit story about how Firefox has a growing portion of the market, and every time somebody posts some bullshit story about how Firef
Re:Finally.... (Score:2)
I dare bet that FireFox users are MUCH more willing to fill in a survey about webbrowser usage than MSIE users.
I would honestly trust traffic logs a lot more than a survey on this matter. Say 10% of people use FireFox, then log stats might show anything from 5 to 20%, in a survey, it might easily show as 50% simply because FireFox users typically care more about their browser, afterall, they took the time and effort to install a different brow
Re:Finally.... (Score:2)
Where did you get these figures from? Correct me if I'm wrong, but it sounds like you just made them up.
And if surveys were typically presenting in such a way, then you might have a point. But they are usually one question amongst dozens, and people usually comple
Re:Finally.... (Score:2)
The first word in the sentence, "say" should have made it clear that it was an example only. Logical, since nobody has any true figures.
And if surveys were typically presenting in such a way, then you might have a point. But they are usually one question amongst dozens, and people usually complete surveys because there's a prize or payment of some kind, not because they want to evangelise whichever browse
Re:Finally.... (Score:5, Informative)
However, given the headers returned by the BBC site, caches should NOT cache the HTML, as the headers say the content expires immediately:
Expires: Mon, 24 Oct 2005 11:57:59 GMT
Date: Mon, 24 Oct 2005 11:57:59 GMT
Content-Type: text/html
Server: Zeus/4.2
Cache-Control: max-age=0
So, the BBC figures may be more accurate than you think.
Re:Finally.... (Score:3, Interesting)
Actually, it doesn't say that caches should not cache the resource, it says that caches should revalidate the resource before serving it again, IIRC.
Which BBC site are we talking about anyway? I'm getting completely different headers for www.bbc.co.uk:
Re:Finally.... (Score:2)
Re:Finally.... (Score:2, Informative)
Re:Finally.... (Score:2)
Also, why should I (as a webserver admin) need to modify my server config to cater to buggy browser behaviour? Surely this is a browser bug that needs fixing rather than a hassle for every webserver admin in the world.
Re:Finally.... (Score:2)
No it doesn't. Even if you've seen a more recent version of the same page, with the same URL, it will show the page you originally got if it is still in its cache (i.e. most of the time, unless the server requests otherwise).
Re:Finally.... (Score:2)
TFA is about the www.bbc.co.uk homepage.
There's no reliable way of counting visitors short of requiring registration and banning people from viewing the pages until they are logged in, and even then you have to deal with things like BugMeNot.
Re:Finally.... (Score:2)
No, but the simple expedient of setting a cookie, then ignoring as a repeat visitor any hit that either has the cookie or is not specifying a cookie but is using the same user agent and coming from the same IP address within some time limit (e.g. 5 minutes) works in over 99% of cases, which is good enough for
Trend matters, not snapshot (Score:3, Insightful)
While you are right that an accurate snapshot is impossible, snapshots only matter to magazine writers facing a deadli
Re:Nelson Ratings (was Finally....) (Score:2)
What, they know the frequency to which you are tuned? I thought Nielsen paid people to put a special box between the cable and the TV... in other words, "took a survey".
Re:Finally.... (Score:2)
Re:Finally.... (Score:2)
Re:Finally.... (Score:2)
Re:Finally.... (Score:2)
Mobile devices (Score:5, Informative)
Many people go to BBC, CNN and other major sites through their mobile service provider's front pages. These would naturally point to the dedicated mobile editions too.
Re:Mobile devices (Score:2, Insightful)
Re:Mobile devices (Score:2)
Ditto. Thats exactly what I do. He may mention something about this on the other pages in his blog, but I can't tell cos of the
Re:Mobile devices (Score:2)
I use Firefox's Live Bookmarks to visit pages on the BBC site at least once a day. But I never use their homepage, so I guess those won't be counted.
Re:Mobile devices (Score:2)
Yep. That WAP site is the home page on my mobile, and I probably use their traffic info page more than the others. I use it almost daily when walking to the car to check for reports of problems on the M4 before going to/from work.
errr (Score:5, Insightful)
Linux (various distributions) 0.41%
Windows Vista 0.15%
MSFT's unreleased os has nearly the same market share as linux?
We've got a long way to go.
BBC news, typically read at work (Score:5, Insightful)
Re:errr (Score:3, Funny)
By that logic, Windows 98 has nearly the same market share as Windows 2000.
Windows 2000 16.5%
Windows 98 6.6%
Re:errr (Score:4, Insightful)
I think not.
Re:errr (Score:2, Insightful)
It was once useful to make sites think that you were visiting using a different browser other than IE, but, for the vast majority of web sites, those days are long gone. I have never, on the other hand, had to pretend to be using another OS to visit a site, never.
I would be greatly intrigued if you could give some examples that require you to be identified as using Windows.
Re:errr (Score:5, Insightful)
Errr... no. Most Linux users will use the default setting for their browser, which for most people will not identify them at using Windows or IE. Yes, a very small number of people will do this, but to claim that it's "most" is just laughable.
Re:errr (Score:3, Insightful)
Two reasons: first sites started working, in that at least they removed the check and just fed their HTML, whether or not it worked on non-IE. Second is that the newer browsers support *temporarily* changing the string in a user-friendly way, old browsers would be permanently switched to IE as soon as the user fixed it to display one page.
Actually I suspect a large percentage of those very old IE versions they list are actually alternative browsers permanently swi
Re:errr (Score:3, Insightful)
Make no mistake, slashdot is big traffic-wise, but the BBC is much, much bigger (especially if you consider the whole bbc.co.uk domain, and not just news.bbc.co.uk)
If we all set up some bots... (Score:5, Funny)
Re:If we all set up some bots... (Score:2)
Re:If we all set up some bots... (Score:3, Insightful)
Have a look at alexa [alexa.com] and you'll see that the bbc site deals with 20 to 30 BILLION hits a day. Slashdots 1 billion is not going to make much difference to their servers.
Point them at various microsoft.com pages (Score:2)
Re: (Score:2)
Representative of Overall Market Share (Score:5, Informative)
The BBC's numbers are simply representative of this, as any large web site would be.
Re:Representative of Overall Market Share (Score:2)
Re:Representative of Overall Market Share (Score:2)
Re:Representative of Overall Market Share (Score:2)
Well, they'll at least be representative of the overall British marketshare. (And yes, I know not just Brits visit the BBC, but still...)
I imagine for a large German or Finnish media site, you'd see a much higher portion of Firefox users. Many countries in continental Europe seem to have much higher Firefox adoption rates.
Re:Representative of Overall Market Share (Score:2)
That's a load of crap, I don't know *anywhere* to buy firefox, Firefox has a 0% market share. It has a 9.5% consumer webbrowsing share though - if that is important.
Re:Representative of Overall Market Share (Score:2, Informative)
I assume after it went free, flood of newbie user got the browser and they didn't change ID String yet. I can't blame anyone, you can still live problems with identifying as Opera to some sites. While I was using Windows, I was identifying as Mozilla browser.
Its kind of stupid to identify browser features wi
mobile devices (Score:4, Informative)
Re:mobile devices (Score:2)
The front page is way too big/rich for a limited handset.
If it doesn't feature advertising, it must be communist.
Opera (Score:3, Interesting)
Re:Opera (Score:2, Informative)
Re:Opera (Score:5, Informative)
Yes, they are.
Old versions of Opera that identify themselves as IE by default use a user agent string like this:
So the "Opera" string is here and easily identifiable.
New versions should simply use the proper Opera UA string by default [slashdot.org].
If you use Opera I suggest to check that it sends the "correct" Opera UA string: the sky will (mostly) not fall down.
Re:Opera (Score:2)
Re:Opera (Score:5, Insightful)
We need to remember that people who do unusual things with unusual browsers are an incredibly small fraction of all internet users. The message of the article is that there's very rougly a 8/1/1 split between IE, firefox and 'other'. That message is not affected in the slightest by Opera, lynx or any other niche browser.
Yet another Opera is undercounted theory coming! (Score:2, Interesting)
It is plausible that some IE users have BCC homepage set as browser's start page and create large number of hits.
but other browsers have alternative mechanisms, that allow user to visit homepage even less often than usual. For example Opera on each start reopens previously open tabs, from cache, so rarely anyone uses start page feature. Opera and FF have RSS that leads users directly to articles, etc.
Re:Opera (Score:2)
We need to see Opera identify itself properly and give a big middle finger to broken web sites, at least after giving the webmaster a chance to fix their issue. Remember there are some stupid webmasters, but there are many that will fix the issue if the get told about it. If you have to go through a help-desk to sort th is
Variability by site (Score:5, Insightful)
On a related note, I hosted some pictures on my website last week that were posted into a fark.com forum, 47.6% of fark readers seem to use Firefox (from some 14,000 hits in two days) - I bet slashdot beats this though!
Re:Variability by site (Score:2)
Of
Re:Variability by site (Score:2)
You don't need to be running FF to know the value of the BBC.
Re:Variability by site (Score:4, Insightful)
I doubt it makes much difference. The BBC news site is read by a lot of Normal People who either couldn't care less about what browser they're using, or have no power to change it because it's a work computer.
I'm really surprised that firefox has such a high share. Of course there have been similar stats released by sites like i-am-a-1337-linux-doodz.com and windoxxors-is-teh-suxxors.com, but to get them from a mainstream site like the BBC must be very encouraging for the developers
Re:Variability by site (Score:2)
Take a look at my stats. [freeshell.org]
Then again, maybe I don't get enough visitors for any kind of accuracy. I keep getting somebody from "cups.cs.cmu.edu," and I've got no idea who that is. They're visiting enough to be statistically significant, though.
Fatally Flawed (Score:5, Insightful)
The author does point this out:
And I must stress again, these figures don't represent the breakdown of visitors to the BBC site as a whole, they are based on requests to the homepage alone, over the course of one week in September. Nevertheless I think they provide an interesting snapshot of web activity.
but it should have been avoided
Re:Fatally Flawed (Score:2)
Slashdot stats?` (Score:5, Interesting)
Re:Slashdot stats?` (Score:3, Insightful)
Bottom line - the beeb gives us a good painting; it's not a picture, true, but it is a good picture. Mozilla folk should be pleased with themselves; their strategy has worked rather well.
Mirror (Score:4, Informative)
How about Slashdot generating a mirror link via a neat little "mirror" icon next to the links?
Re:Mirror (Score:4, Informative)
Only nyud.net links [nyud.net] may help then, although my experiences with those aren't the best and why I tried to avoid it in the first case.
Slashdotted.... (Score:4, Funny)
I have a hunch this guy's web stats are going to show a MASSIVE influx of FireFox users, then a long period of downtime...
It depends upon your site content (Score:2, Interesting)
My site and blog mostly related to Linux and Open source stuff, and here is my exprince so far:
OS
Most of the corporate users, uses Windows XP/2000 desktop
Individual user uses Linux/BSD/Mac OS desktop
Browser
Firefox rules
IE (6.x/5.x)
So it depend upon your site content, if you wanna see this stats they are here [cyberciti.biz]
No MSI build for Firefox - no mass deployment (Score:5, Insightful)
Re:No MSI build for Firefox - no mass deployment (Score:2)
here at uni they deploy firefox on demand through zenworks like pretty much every other app they have avilible. I think they build thier own packages for that though.
Preferred at many (most?) companies (Score:2)
My guess is that this is the preferred method at most companies of > 50 people. I've worked at a number of companies over the past 3 years. This is by far the primary reason given for not deploying Mozilla/Firefox. MS gives tools to easily customize IE and push it out to everyone on the network very quickly. I'm working with a company now that realizes the
Re:No MSI build for Firefox - no mass deployment (Score:5, Informative)
http://www.frontmotion.com/Firefox/ [frontmotion.com]
MSI installers for Mozilla Firefox! Useful for installing Firefox on a single computer for the home user or deploy across thousands of computers automatically with Microsoft's Active Directory. Use Firefox on your corporate computers to decrease virus incidents and increase overall security. Save time and frustration with our installer that is targeted toward the corporate IT administrator with manageability and upgradeability in mind. This is not just a wrapper around the exe installer nor is it another half baked 'captured' install.
Re:No MSI build for Firefox - no mass deployment (Score:2)
Stop blaming software makers for that. Microsoft pushed for a _long_ time the _crazy_ idea that "programs must have their own
Re:No MSI build for Firefox - no mass deployment (Score:2)
What's the installer got to do with DLL hell? That's a problem of the application packager. To be honest, I haven't seen DLL hell for years.
It makes complete sense to use MSI for Firefox. Leverage both the deployment infrastructure that exists and the installer functionality that doesn't need to be reinvented.
As for your point about some installers requiring admin rights - wtf? Apt on my Debian system won't even run unless I'm root, so how is this any different?
Most o
Re:No MSI build for Firefox - no mass deployment (Score:2)
It would definitely be nice to have an official MSI package (as it would attract many more admins) coupled with something like ADM XPI
Super Respectable (Score:5, Insightful)
I use firefox and even I can't keep a strait face reading that line. I mean have some self-worth, man. There's nothing respectable about that. Can't we aim just a tad higher here? Especially if we're gonna tag on the word "very"?
All is relative. (Score:2)
As always, defaults play a role (Score:3, Informative)
Re:As always, defaults play a role (Score:5, Insightful)
Maybe so, but that's not the homepage, which is from where the stats were taken
Re:As always, defaults play a role (Score:2)
Firefox comes with a "Live Bookmark" to the BBC (Score:2, Insightful)
I run a website with 20% Firefox usage as of now! (Score:2)
Re:I run a website with 20% Firefox usage as of no (Score:2)
The table, reformatted:
* Before this date, Firefox & other Mozilla were
Re:I run a website with 20% Firefox usage as of no (Score:2)
Month IE Firefox Safari Mozilla Netscape
May 64.2 16.7 5.3 0* 1.3
June 64.7 17.9 5.4 0* 1
July 65.9 15.6 4.6 3.8 2.1
Aug 66.8 16.6 5 2.5
Re:I run a website with 20% Firefox usage as of no (Score:2)
Now we can discuss the actual numbers! IE UNDER 60%!!!
Obvious solution (Score:5, Funny)
Small AuPair website in the UK (Score:2, Interesting)
For this month, this is the breakdown of browser access
5250 Views this month:
* 77.5% Internet Explorer (inc. Maxthon & AOL) = 4070 Views
v5 (57 views) v5.5 (27 views) v6 (3703 views)
* 10.9% Mozilla Firefox (inc. Netscape & SeaMonkey) = 574 Views
* 02.3% Apple Safari (inc. Linux Konqueror) = 122 Views
* 00.4% Opera Browser = 22 Views
* 08.8% Other (Unknown, bots
Microsofts true competitor... (Score:2)
Re:Microsofts true competitor... (Score:2)
Most visited site in the UK (Score:2, Interesting)
Re:Most visited site in the UK (Score:3, Insightful)
Default? (Score:3, Insightful)
Anyone considered that, maybe, that might have influenced the results? Having a default bookmark as the page of the study? You wouldnt take browser results from MSN.com or whatever IE's default home page is.
Nevermind me though, I just suggested that a pro-Firefox poll might be biased. Karma be dammed!
I donno about this... (Score:2, Interesting)
Windows 16400 70 %
Linux 5497 23.4 %
MS Internet Explorer No 14012 59.8 %
Firefox No 7579 32.3 %
Though windows is the dominator in this respect but it goes to show the website content does really depend on who visits the site and therefore produces the stats.
Lets face it BBC is a news network business people and general interest users are reading these articals; ofcourse n
older stats (Score:2, Informative)
Then it got to about 95% IE, so 85% is quick a marked drop in IE support.
Do remember that the BBC is hardly a generic site for your average Internet user, it attracts a significant quantity of beginners and is dull for anyone technical (there are a higher proportion of technical users on the Internet than you'd meet on a street). So these stats are quite good.
I know the way they are worked out should be quite fair.
Re:Sampling? (Score:4, Interesting)
All I could get of the article (page 1 and 2) (Score:5, Informative)
Our old stats reporting tool at the BBC gives a breakdown of requests from different user agent strings, which is where the browsers and operating systems people use to navigate around the web leave their digital fingerprints. It is about to be phased out in favour of a new solution, but I'm not sure that the new system gives the same granularity of data, so once I'd started, I thought I'd look at the figures in some detail before the old system gives up the ghost.
Now if you've never looked at user agent strings, they are rather dull and geeky, and full of lots of technical gubbins like these examples:
* Mozilla/5.0 (Windows; U; Windows NT 5.2; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6
* Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.5
* Mozilla/4.0 (compatible; MSIE 6.0; America Online Browser 1.1; Windows NT 5.1; SV1;
* Mozilla/4.0 NETIKUS.NET GetHttp v1.0
* Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.5.1.0)
* Mozilla/4.0 (compatible; MSIE 6.0; Windows CE)
* Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 Firebird/0.7
* Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; BT [build 60A])
There are of course some caveats around the figures I'm about to talk about.
User agent strings aren't an exact science. Or rather, they ought to be, but in the real world the come out a right mess. I've done my best to untangle them, but I still ended up with a significant number of user agents that I could not identify properly. And that is before we get started on the corporate networks that use the UA string to broadcast their corporate branding to the world whilst masking their operating system. Or requests claiming to come from both Internet Explorer 6 and Internet Explorer 5.5. Or that claim to be from a particular Linux distribution and Windows 98 at the same time. Or the plain weird like the inadvisably named KummClient from Hungary that proudly proclaims 'Linux rulez' to anyone like me dull enough to be delving through their logfiles.
User agent statistics on something as big as the BBC homepage could almost be the very definition of the long tail. The most popular user agent string - IE6 on Windows XP - clocked up nearly 6 million requests. I only counted user agents that had made more than 50 requests, but between 6 million and 50 requests there were nearly 11,000 different user agents to look at. Examining that number of requests accounted for 95% of the reported traffic, but only around 1/3 of the stats report. I initially suspected that counting the whole of the tail was likely to increase the market share I derived for the quirkier set-ups, but a random sample showed that a large proportion of the tail consisted of the most popular browsers and operating systems, but with different installed toolbars or corporate network messages that distinguished them as a unique string.
And I must stress again, these figures don't represent the breakdown of visitors to the BBC site as a whole, they are based on requests to the homepage alone, over the course of one week in September. Nevertheless I think they provide an interesting snapshot of web activity.
In total I've examined around 32 million requests to the BBC servers - although some of these have been discounted as 'unknowns' and some originate from crawlers and spiders.
The complete dominance of Windows XP and Internet Explor
Re:All I could get of the article (page 1 and 2) (Score:2)
I would be interested to know what percentage were discounted as 'unknowns'
... and this is page 3 (Score:4, Informative)
Windows Operating System Share
Concentrating on just Windows alone we can see that Microsoft have done a very thorough job of converting their user base to the most recent iteration of the software. Windows XP accounts for just under 70.5% of the Windows requests, and Windows 2000 a further 17.4%. That means in total around 88% of users of Microsoft Operating System products are using the two most recent consumer releases.
Windows 98 features in 7% of requests made from a computer running a version of Windows, and after that the figures are very small in terms of market share. In fact the next largest figures is a clump of 'Windows other' including Windows CE, and various unspecific Windows NT user-agents that I couldn't pin down to a precise version.
Mac Operating System Share
I was frustrated in my attempts to similarly breakdown the different versions of the Mac OS that people were using to request the BBC homepage. I established that from the requests we saw I could identify Panther as supplying 31%, Tiger supplying 21%, with Jaguar lagging behind at 3%. However there were 41% of requests where I could identify that the computer was a Mac, but not the specific version. That is because Safari helpfully supplies in the user agent string the WebKit build, allowing the precise version of the OS to be identified, but most other browsers do not.
Linux Requests To The BBC Homepage
The number of Linux requests to the BBC homepage was very small, representing only 0.41% - less than 100,000 - of the 32 million requests included in this study. With such a comparatively low number I didn't take the time to delve into which different distributions were driving the requests.
The figures may, however, mask a slightly higher use of Linux. Since the user agents generated are more likely to be unique, they are more likely to have fallen into the statistical long tail. However I should add that my random samples of the tail did not show that it consisted entirely of Linux, in fact as I mentioned earlier, a lot of corporate-branded Windows networks show up in the tail.
Legacy OS Systems
We have some fairly strict standards for supporting legacy technology at the BBC on the client-side - but the long tail of older OS software visiting the BBC homepage is amazing. We still saw over 300 requests for the BBC homepage coming from machines claiming to be running Windows 3.1, and around 200 requests from machines claiming to be persevering with 0S/2 Warp.
Re:All I could get of the article (page 1 and 2) (Score:2)
Re:User Agent Switcher (Score:5, Funny)
Re:User Agent Switcher (Score:3, Informative)
The googlebot UA string currently is (remove the space in "ht tp"):
or:
Yahoo:
MSN:
Re:User Agent Switcher (Score:2)
Or, if you use a real browser like Konqueror you can just do it without having to hunt down an extension to let you.
Re:LATE BREAKING NEWS!!! (Score:5, Insightful)
Yes. It is probably the broadest cross section of mostly British web users you are likely to find on a single site.
The fact that nearly 10% of those users use firefox is particularly relevant, and is a good weapon for those of us who do commercial web design to persuade our clients that the extra work to support alternative browsers properly *is* worth it.