How CDNs and Alternative DNS Services Combine For Higher Latency 187
The_PHP_Jedi writes "Alternative DNS services, such as OpenDNS and Google Public DNS, are used to bypass the sluggishness often associated with local ISP DNS servers. However, as more websites, particularly smaller ones, use content distribution networks via embedded ads, widgets, and other assets, the effectiveness of non-ISP DNS servers may be undermined. Why? Because CDNs rely on the location of a user's DNS server to determine the closest server with the hosted content. Sajal Kayan published a series of test results which demonstrates the difference, and also provided the Python script used so you can test which is the most effective DNS service for your own Internet connection."
Re:Do you even know what a CDN is? (Score:3, Interesting)
As a case in point, Slashdot is perfectly fine without images or Javascript (as long as you request Javascript-free pages, which are readily delivered).
Re:Is this a problem? (Score:5, Interesting)
Seven or so years ago, before I retired from one of the large cable companies, CDNs were hosting the relatively static parts for a surprisingly large number of broadly popular sites. I had an opportunity to see the list when we were approached by the then-largest CDN, who wanted to place servers in many of our head-end locations for the obvious performance benefit. I was the one who pointed out that all of our internal DNS requests were routed to one of two data centers, one on the East Coast and one on the West, creating exactly the situation described in the OP: the CDN would have no idea where the original request came from, so would be unable to direct the end user to the appropriate server.
I was one of the few engineers who argued for less centralization in our network. I wanted broader distribution for reliability purposes: at that time, the massive centralized mail servers had a tendency to fail at the drop of a hat. But it would also have given us the ability to work with companies like the CDNs in order to provide better service.
Re:Is this a problem? (Score:5, Interesting)
This is exactly the problem. Most people have probably not heard about a little company called Akamai, but chances are if you're downloading content from a large site, you're using Akamai's content delivery network. Go view a trailer on Apple's site for instance and you'll see the host is actually served off edgesuite.net (which is Akamai). They use a distributed system of caching mirror servers to serve up content to a server closest to you geographically.
The one reason I use an open DNS server instead of my cable provider's (Cox Cable) servers is because they have an Akamai server for Cox and it was horribly overloaded. I was getting 512 Kbps anytime I was trying to download something from Apple. I switched my DNS to a combination of Level3's and Cisco's open DNS servers and I started hitting another Akamai server outside Cox and started getting 15 Mbps. It was night and day going from barely being able to watch a standard definition movie trailer on Apple's site while it buffered buffered, played, buffered, play buffered, etc. to being able to watch a 1080p HDTV stream with the buffer way ahead of my realtime viewing.
Re:Is this a problem? (Score:4, Interesting)
Re:This is not accurate (Score:3, Interesting)
Hi David. Isn't it possible for you to just cooperate with Akamai and resolve according to the client location based on IP address?
Re:Poor application design (Score:3, Interesting)
Like you couldn't redirect on GET instead of serving up the app?
Re:Is this a problem? (Score:1, Interesting)
The servers within the CDN farm utilize reverse DNS lookup to balance and serve traffic from the correct place.
No, they don't. The way this works is that there is a separate domain name for the content which is to be served by the CDN. The response DNS resource record is dynamically chosen to point to the CDN server closest (in routing terms, not geographically) to the source of the request. The reverse lookup (i.e. the canonical name of the IP address) does not play a part in this, only the routing paths to the resolver's IP address.
The request usually comes from a recursive resolver which doesn't run on the user's computer. Most often it is located in one of the end-user ISP networks though. If the user chooses to use a resolver which is located across the world from him, then the CDN will also point his browser to download from a CDN server close to the resolver, not close to the user.
Since it's not a good idea to use a resolver which is on the other side of the planet, because DNS requests are frequent and invariably must finish before anything useful can be done, meaning high latency DNS sucks, this problem is not as big as it looks. For example, you can use Google DNS (8.8.8.8 and 8.8.4.4) and the resolver will not be far away from you, because those are IP anycast addresses. There are many resolvers across the world which all respond to these same addresses and BGP (the Border Gateway Protocol) routes your packets to the closest one.
Re:This is not accurate (Score:2, Interesting)
awesome! thank you for your reply. BUT wouldn't giving client ip away in the dns request reduce privacy?
Re:Poor application design (Score:3, Interesting)
There's various tricks you can do to decide later, if you have significant content other than the raw HTML page itself, though they do require some server processing. The initial HTML request will be based on DNS, but once the user's hit your servers, you have their IP, so you can rewrite the URLs of embedded content / AJAX requests / whatever, so that they hit a geographically nearby server.
Re:You're a man after my own heart! apk (Score:3, Interesting)
Re:This is not accurate (Score:3, Interesting)
Re:This is not accurate (Score:3, Interesting)
If anyone would like to see this for themselves on any servers they use, check out namebench
http://code.google.com/p/namebench/ [google.com]
Tests to figure out which DNS servers you should use from a speed perspective mostly, but does all sorts of neat checks for DNS hijacking like OpenDNS does.
Its bad enough to do NXDOMAIN hijacking, but flat out stealing google traffic and running it threw their own servers is just bullshit.
Re:This is not accurate (Score:1, Interesting)
While I question the utility of hiding one's IP from a DNS server (or anything on the web, really), it's entirely possible that your IP won't ever be displayed to the resolved server.
In cases of SMTP resolution, it's not meaningful for the DNS server to know the client IP. Also, I frequently DNS sites to see where they are, get IP block info, and possibly block access to them. I don't access their services as I do this, and thre's no reason for them to know I'm looking into them. An antivirus firm, when looking up the IP blocks of hostnames, could have an NXDOMAIN returned if the DNS server matches one of their IP's, thus throwing off the investigative trail.
The obvious resolution to this would be to set a privacy flag on the query. If I tell the resolver on my machine I want to be private (maybe for load testing issues such as was done in a comment below), the receiving DNS server should both 1. not forward my IP to the upstream DNS server and 2. forward the privacy flag to the upstream server so that the downstream server IP is not forwarded to the next upstream server. Problem resolved.... right? it would then be a matter of updating nslookup, host, etc to use the privacy flag.. but the protocol itself wouldn't be preventing the privacy.
Actually, reading the given link,
"Users who wish their full IP address to be hidden can include an edns-client-subnet option specifying the wildcard address 0.0.0.0/0 (i.e. FAMILY set to 1 (IPv4), SOURCE NETMASK to 0 and no ADDRESS). As described in previous sections, this option will be forwarded across all the Recursive Resolvers supporting edns-client-subnet, which MUST NOT modify it to include the network address of the client."
So... what's the problem here? ;-)
Re:This is not accurate (Score:3, Interesting)
You have summarized the privacy concern well. That's exactly the issue. The fear that is held is that implementations won't respect someone who includes 0.0.0.0/0 and instead will replace it with the actual client's source_addr when forwarding a request along. Think hotel, cafe, wifi hotspot vendors, etc... Those folks tend to implement for ease, not privacy. And sometimes they opt against privacy.
The critics of the proposal think that there is no assurance of privacy, and they feel that's a reason to not move forward. In my world, there are much better ways to violate real privacy than to see a client IP address in a DNS request, but maybe I'm less sensitive about it. I think it's certainly worthy of discussion and attempting to find a solution.
Re:This is not accurate (Score:2, Interesting)
Re:Leave Canada Alone (Score:2, Interesting)