Are Ad Servers Bogging Down the Web? 387
blackbearnh writes "The work of making high-volume web sites perform well is an ongoing challenge, and one that continues to evolve as the nature of web content changes. According to Google Performance Guru Steve Souders, fat JavaScript libraries and rich content are creating new problems for web site tuning, but one of the biggest problems lies outside the control of web site administrators — ad servers. In an interview previewing the upcoming Velocity Online conference run by O'Reilly, Souders talks at length about the real causes of poor web performance today, and in particular, the effect that poorly performing ad servers are creating. 'We adopted a framework of inserting ads, of creating ads, that's pretty simple. And because it's pretty simple, it's not highly tuned. That's one reason why we shouldn't be too surprised that we see performance issues in third party ads. The other reason is that ad services are not focused on technology. Certainly companies like Yahoo and Google and Microsoft, we're technology companies. We focus on technology. So it's not surprising that our web developers are on the leading edge of adopting these performance best practices. And it's also not surprising that ad services might lag two, three or four years behind where these web technology companies are.'"
AdBlock Plus (Score:3, Informative)
Re:Make it a statistic and they'll care (Score:3, Informative)
Yup. Second biggest offender is usually Google Analytics. Hell, I've often had Firefox hang while trying to pull up something from GA.
Re:I don't even need to read the summary. (Score:3, Informative)
Even more embarassing for Google - As I mentioned in another post, Google Analytics is one of the biggest offenders in the "makes page load stop and browser freeze for a few seconds" category.
Re:Make it a statistic and they'll care (Score:5, Informative)
The solution is simple: /etc/hosts
vi
add:
127.0.0.1 ad.doubleclick.net
127.0.0.1 twx.doubleclick.net
127.0.0.1 ad.uk.doubleclick.net
127.0.0.1 www.google-analytics.com
127.0.0.1 googleads.g.doubleclick.net
127.0.0.1 pagead2.googlesyndication.com
127.0.0.1 partner.googleadservices.com
127.0.0.1 analytics.live.com
127.0.0.1 ads1.msn.com
etc.
Re:Why? Why not: HOSTS files, give this a read... (Score:1, Informative)
"One of the things that pisses me off to no end, are third party ads that are spewing crap/malware to driveby web browsing. I don't personally get infecgted by them, because I run all the latest anti-malware defenses (adblock, noscript, firefox etc). But I'm in IT, and I see way too many machines compromized by the lastest "Antivirus 2010" styple crap/malware all the time. Websites that house such malware should be blacklisted. Screw them if they can't make a living without using dubious adverts - by Archangel Michael (180766) on Monday November 30, @12:33PM (#30271632)
Archangel Michael, meet "the LORD OF HOSTS" (just in keeping with your nick/handle here, AND the fact that much of what you note is covered by another tool you omitted mentioning that is easily edited, everyone has one (if their OS IP stack is BSD based, most all are iirc), & eats no CPU cycles like a local DNS server can (& are not as security vulnerable either if you protect write access to a HOSTS file also):
I use a custom HOSTS file, in addition to the tools you noted (which only really function for FireFox/Mozilla products, but don't extend globally to all other webbound applications, & that is part of what HOSTS files give you above the methods you extoll + utilize: "GLOBAL COVERAGE", & of ALL webbound apps, not just FireFox/Mozilla ones via the addons you noted + use yourself...).
HOSTS files can be used to blockout KNOWN "bad" adserves, maliciously coded sites or adbanners, and "botnet C&C servers" too!
You can obtain reliable HOSTS files from reputable lists for more security online, but also for speed!
(More on that later & WHY/HOW (I use reliable lists for that, such as these HOSTS @ Wikipedia.com -> http://en.wikipedia.org/wiki/Hosts_file [wikipedia.org] or those from mvps.org (a good one this one))
I also further populate & keep current my custom HOSTS file with up to date information in regards to all of those threats, via:
----
A.) Spybot "Search & Destroy" updates (populates HOSTS and browser block lists)
B.) Sites like ZDNet's Mr. Dancho Danchev's blog -> http://ddanchev.blogspot.com/ [blogspot.com]
C.) Sites like FireEye -> http://blog.fireeye.com/ [fireeye.com]
D.) SRI -> http://mtc.sri.com/ [sri.com]
----
My HOSTS file incorporates ALL of the entries from the HOSTS files shown @ wikipedia as well... gaining me speed online (by blocking adbanners, which have been compromised many times the past few years now by malscripted exploits (examples below)).
(I combined ALL reputable HOSTS files with one of my own (30,000 entries), & I removed duplicates removed via a Borland Delphi app I wrote to do so called "APK HOSTS File Grinder 4.0++". That program also functions to change the default larger & SLOWER 127.0.0.1 blocking 'loopback adapter' IP address to either 0.0.0.0 (for VISTA/Windows Server 2008/Windows 7, smaller & thus faster than 127.0.0.1 default) or the smallest & fastest 0 "blocking 'IP ADDRESS'" (for Windows 2000/XP/Server 2003 which can STILL use it (& it was added in a service pack on Windows 2000, only on 12/09/2008 MS patch tuesday was it removed for VISTA onwards (& now all these "phunny little bugs" are showing up as FLAWS in this new NDIS6 approach via WFP as well in the firewall, which ROOTKIT.COM has stated (with code too no less on how it is done) -> http://www.rootkit.com/newsread.php?newsid=952 [rootkit.com] [rootkit.com] that it is EASIER TO UNHOOK (than was the design used in Windows 2000/XP/Server 2003))
Another EXCELLENT benefit of HOSTS file usage? More speed online, & also more security + reliability (especially in the case of DNS servers today, per folks like Dan Kaminsky &/or Moxie Marlinspike finding various security vuln
Re:Make it a statistic and they'll care (Score:4, Informative)
Second biggest offender is usually Google Analytics.
That's why it's usually interesting to host the JavaScript file, that seldom changes on your webserver, and only have the img that conveys the data retrieved from the foreign host.
JavaScript loading is usually blocking the rendering whereas img loading usually not.
Re:Why would an ad server slow down a site? (Score:3, Informative)
The last time I profiled some topsites in Firefox, well north of 50% of the CPU time was spent dealing with the Flash ads (as in, 50% of the time the CPU was busy during the pageload the program counter was inside the Flash plugin). Given the typical latency of the ad networks, I'd estimate over 40% of total load time on those sites was taken up by the ads.
Re:Make it a statistic and they'll care (Score:5, Informative)
The solution is simple: /etc/hosts
...
vi
add:
127.0.0.1 ad.doubleclick.net
etc.
Even easier, as I thankfully learned from Slashdot a long time ago, this downloadable MVPS hosts file [mvps.org] instantly prevents connecting to ads, spyware as well as other "parasites" and is constantly updated.
Since adopting it, I no longer wait for ad servers and a side benefit is not even being exposed to the ads!
Re:Kind of Fitting (Score:3, Informative)
For the Flash cookies, maybe? Dunno what the trade-off is vs. users who block Flash by default.
Also, SWF can uses vector graphics and the animated files are tiny.
Re:Make it a statistic and they'll care (Score:4, Informative)
I believe (from a little experience dabbling in web design) that browsers generally run inline javascript as they encounter it - so since the ads are usually inline JS at/near the top of the page it prevents further loading while it's being handled.
Theoretically if you put the ads in the footer this wouldn't be such an issue since most of the page would load first - most places won't do that though since they want the ads prominently on the top/side of the site so you're more likely to click them (but seriously - who are these people that click ads??)
Re:Kind of Fitting (Score:2, Informative)
Advertisers using flash because it's not as easily blockable
Uh... FlashBlock? Available now for the Chrome [chromeextensions.org] as well.
Re:Make it a statistic and they'll care (Score:3, Informative)
Or, even simpler, use NoScript and AdBlock (with the auto-update list) in Firefox. Editing hosts is simple, I'm not arguing that. It's also free and relatively effective. With one change, your tip even works in Windows.
But NoScript and AdBlock are far more effective and even simpler to use and maintain.
hosts doesn't let you know when a new site wants to "get through", you have to see the ad and block it. It also allows everything you don't explicitly block. And it requires a complete match on the URL. I played the game for quite some time, then decided the few little scraps of my sanity that were left weren't worth ditching over ads.
AdBlock has a predefined list that gets updated if you ask it to, so once you've installed it they are pretty good about catching "www432.adserver.com" and other constantly morphing URLs (plus you can use wildcards to block "*.adserver.com" and not have to worry when they add the www433 subdomain).
NoScript defaults to "no permissions" and forces you to ALLOW things you want, so it's a "positive confirmation" security. It's a bit of a pain the first time you visit a site that needs of scripts, but with a good number of them I also decide "you know what, the content isn't important to me to run 238 scripts from 19 sites just to read a news article" and move on.
Flashblock is also great if you want to allow companies to show ads, but you are concerned about "webilepsy" (the sudden and acute onset of epileptic symptoms caused by flash ads).
Webmasters hate slow ads too (Score:3, Informative)
There are quite a few webmasters who run their ads inside of iframes, as that usually avoids a slow ad holding up the rest of the page loading. The bad thing about that is that expandable ads (even polite, user-initiated) do not work. There are also some other tricks webmasters use, such as creating division tags and then using a bit of javascript trickery to move the ad loading to a point after the content loads.
Webmasters do hate slow ads (not to mention bad ads). I love direct sale campaigns on my site, because they almost always are run from my ad server. If that is slow, my whole site is slow anyway - and that happens very, very rarely (it has been months).
Re:Why? (Score:3, Informative)
I realize that most websites run some version or another of "adverts", but generally speaking, most of those sites are marginal value to start. The sites I frequent usually use text ads, and not the flash (pun intended) graphical ads on some of the more questionable sites.
Do you even realize that the hosting/bandwidth for your marginal and questionable comment was paid for by adverts?
Re:Make it a statistic and they'll care (Score:2, Informative)
Actually, this isn't usually Google Analytics. Firefox shows the "Waiting for..." thing for the last resource it requests, which is usually GA because GA is the last thing in the page. This is a good thing: your page can render while the google analytics stuff is being loaded (which is very very fast, like you'd expect from google), but if some ad image elsewhere in your page doesn't load, firefox will continue to show "Waiting for Google Analytics...".
Re:Kind of Fitting (Score:4, Informative)
Flash objects store cookies in a location that is not covered by browser privacy controls.
These cookies stick with you even after you uninstall/reinstall the plugin, and can only be managed through a web interface on the flash website. So you're correct - flash bypasses traditional browser controls and provides advertisers a more persistent method of following a user across multiple domains.
Re:Kind of Fitting (Score:5, Informative)
I have FlashBlock, but that doesn't stop sites from using Flash cookies [wikipedia.org], whether or not a flash movie is even played.
If you use Firefox, upgrade to version 3.5+ and install Better Privacy [mozilla.org] and you can blow away these nasties (each one can be up to 100kb binary data by default, with no expiration, ever), which btw are OS- and browser-independent. You will be shocked at the baggage they've saddled you with till now...
Top 3 addins for privacy: Better Privacy, AdBlock Plus, and NoScript, hands down imo.
Re:I don't even need to read the summary. (Score:2, Informative)
You are wrong. Google Analytics plays nice by loading their code last, which means they get the blame for other people's slow loading. Check firebug's resource tracker next time, or check out this blog post: http://www.analyticsmarket.com/blog/tracking-code-slows-my-site
Re:Kind of Fitting (Score:5, Informative)
From what I understand, it is. It's shown to those who have a high Karma, moderate, and meta-moderate. So, the good users. :)
It's been on mine for several months, so I've been happy. :) I'm guessing it was about the time they implemented it, since I've been doing all the stuff above for years.
Re:Kind of Fitting (Score:2, Informative)
Two ways Flash is used at tracking even individuals who have privacy modes turned on in the browser:
1: Flash Shared objects. Want them gone? You need to manually clear them out, or link the directory to /dev/null.
2: Supposedly, you can use Actionscript to pull specific information about a machine, such as BIOS revision, Windows install ID, and other specific data and send it up to the web server. This way, even if someone zaps the Flash objects, a website can still use cookies, ActiveX, Java cached objects, or flash shared objects to ensure a box is tracked across websites even with someone who clears stuff out religiously.
Just look at what gets stored in the "Flash Player" directory sometime.
Re:Kind of Fitting (Score:1, Informative)
Adblock and Noscript have served the same function well enough.
Re:Make it a statistic and they'll care (Score:3, Informative)
I used to do this, but got irritated with it blocking things that I wanted to do, e.g., filling out a survey about how dissatisfied I am with AT&T. They would redirect it through doubleclick and voila I'd get my 404. I'd have to fix the hosts file and then go back and refresh the browser. Very annoying.
I much prefer the flashblock approach, though I do allow ads to run in order to support the sites I visit (no AdBlock, etc.). There is no question that these ad servers need to be faster than the sites that link to them.
-l
Re:Make it a statistic and they'll care (Score:2, Informative)
Set up your own DNS, and do:
zone "doubleclick.net" IN { type master; file "master.d/null"; };
zone "gostats.com" IN { type master; file "master.d/null"; };
zone "hitbox.com" IN { type master; file "master.d/null"; };
zone "google-analytics.com" IN { type master; file "master.d/null"; };
and so on. 'master.d/null' contains SOA and NS records identifying your own DNS server, and nothing else.
Any domain I've got in there is either something that has caused repeated page-load delays, or "cookie bombs" where 30 "The site somesiteotherthantheoneyouaretryingtovisit.com wishes to set a cookie" to come up before I get to say "no" to the first one.
Merely providing ads isn't enough to get banned from my LAN.
Re:So I'll ask again (Score:2, Informative)
Why can't the serving of ads be done from the primary website's server?
Control.
The ad server outfit wants to control the ad content and the ad count from their end to avoid fraud by the content site owner.
Re:Make it a statistic and they'll care (Score:1, Informative)
Try Ghostery and Request Policy. A little annoying at first, but well worth it. Don't forget Better Privacy for when you do allow Flash to play to remove the trackers everyone uses.
Re:Kind of Fitting (Score:3, Informative)