Open Source Router on Par With Cisco, Users Say

Jane Walker writes "On a mission to avoid paying top dollar for Cisco routers, two users say Vyatta's Open Flexible Router is a viable alternative to the proprietary norm. Find out about the pluses and minor hassles involved in deploying this alternative." This probably won't surprise the users of (much lower end) networking gear like the famously hackable Linksys WRT54G, which — like a number of internally similar routers — can be reconfigured with one of several open-source firmwares to do things impossible with the hardware as delivered.

Difference between hardware and software....

[gweihir (88907)]

It is not surprising that low-end software routers can offer most things a proper Cisco router can. However when you need hgher speeds, a software router can not cut it. It is then when hardware routers show their strenght. A 100Mbps line usually does not require a hardware router. A 10Gbps line does.

Re:Difference between hardware and software....

[Shaman (1148)]

Bleah. This is tripe. Most Cisco routers have cheap, slow Intel processors in them.

Until you get up into the gigabit speeds, regular PC hardware is just as good or better. The only thing you have to watch for in the multi-hundred-megabit routing loads is that you don't have a lot of access control lists - which is also an issue you will run into with any router you might choose. Spending some time sizing the buffers and other kernel parameters is also important, because a stock Linux kernel is not set up to be a network core router.

I've got over 2,000 L2TP connections going into a single 2.4Ghz Intel box running Linux. Performance is significantly better than the Cisco 7204 that it replaced, and it's a lot cheaper and more flexible to support.

Now, in the multi-gigabit routing tasks, do yourself a favour and get a L7 switch with custom ASICs. Extreme, Foundry and others will be happy to sell you one. Cisco's stuff is crap, right up until you get their million dollar badasses which they bought from another party (go figure).

Re:Difference between hardware and software....

[Shaman (1148)]

FYI, I had a 7204 VXR and the Linux solution easily outperforms it.

Still have it, I never throw anything away...

cisco 7204VXR (NPE400) processor (revision A) with 114688K/16384K bytes of memory.
Processor board ID 21280102
R7000 CPU at 350Mhz, Implementation 39, Rev 3.3, 256KB L2, 4096KB L3 Cache
4 slot VXR midplane, Version 2.1
Last reset from power-on
Bridging software.
X.25 software, Version 3.0.0.

PCI bus mb0_mb1 (Slots 0, 1, 3 and 5) has a capacity of 600 bandwidth points.
Current configuration on bus mb0_mb1

Re:

[Cally (10873)]

You've heard of CEF, right?

Re:Difference between hardware and software....

[Shaman (1148)]

Certainly have heard of CEF. And have witnessed first-hand how bad it usually works in a big Internet environment.

Let's see...

--

IP CEF with switching (Table Version 271518), flags=0x0
    1030 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 3
    1033 leaves, 27 nodes, 152040 bytes, 269271 inserts, 268238 invalidations
    0 load sharing elements, 0 bytes, 0 references
    universal per-destination load sharing algorithm, id 26B36E8A
    2(0) CEF resets, 1425 revisions of existing leaves
    Resolution Timer: Exponential (currently 1s, peak 1s)
    2250 in-place/0 aborted modifications
    refcounts: 9206 leaf, 7168 node

Adjacency Table has 888 adjacencies
    2 incomplete adjacencies

--

It does speed things greatly. Load on the 2.4Ghz Linux box that replaced it is 0.07 right now, with 1800 L2TP connections.

Re:

[gweihir (88907)]

Sorry, I just meant that for lower speeds, buying Cisco is defeinitely a waste of money.

For high speeds get a hardware router and not necessarily from Cisco. In fact I don't like Cico for several reasons, which I will not go into here.

Re:

[saridder (103936)]

What part of Cisco's switch is crap? And which switch? Quantify that statement with some solid facts please.

Cisco routers don't have any Intel processors in them. Some of their network modules that run LINUX do, but their not the router. Open one up and look. In fact they never have and never were x86-based. They were Sun boxes way back when created in Stanford's labs, but that was before Bush Sr. was president. Regular PC's may or may not be able to forward packets as well as a Cisco router, I'm sure

Re:Difference between hardware and software....

[element-o.p. (939033)]

Ummm....no. In anything more complicated than what a switch can do, you are using software to process packets.

Yes, Cisco (and others) have routers that use ASICs to handle immediate in/out "routing" in hardware, but as soon as you start putting any kind of ACL, any kind of port/IP translation, or anything else that requires any intelligence on the router, you bring in software, and all of the processing overhead that goes with it.

So....if you are going to do anything *useful* with a router would you rather have a 50-200MHz Cisco box running a bloated IOS (do you *really* use X.25, for example???), or a server-class x86 motherboard running a 1GHz processor with a kernel optimized for routing and software optimized for the protocols you actually use?

We use http://www.imagestream.com/ [imagestream.com]ImageStream Linux-based routers where I work, and they absolutely run circles around the 2600, 3000, and as5000 -series routers that we have. Their support is absolutely phenomenal. When we have a problem with an ImageStream router, we frequently talk with their programmer, and he works with us until we have a patch installed on the box that fixes the problem. If there's a software bug in your Cisco router, it's "yeah, that will be fixed in the next IOS release"...which unless you paid out the <bodily orifice of your choice> for SmartNet you have to *buy*, even though their product was broken when you bought it.

You can use overpriced Cisco iron if you want; I'll stick with the Linux-based routers, thanks.

Re:

[netik (141046)]

Wrong.

Foundry ServerIrons handles ACLs in hardware. So do Cisco Catalysts. If you turn on logging, they switch back to software ACLs, but with logging turned off, ACLs ar ein hardware.

Link to Vyatta

[QuantumRiff (120817)]

Perhaps a link to the actual product would be in order?
Vyatta Open Flexible Router [vyatta.com]

Avoiding "License Transfer" Fees

[evansvillelinux (621123)]

Isn't this a way to avoid paying for the licensed software on Cisco equipment when it's sold second hand? (Not trolling or anything, I think it's ridiculous for Cisco to demand payment for software that's already been paid for once.)

if it is only "Standard PC Hardware"

[MerlynEmrys67 (583469)]

It isn't comparable with Cisco.

But then again for SMB - you don't need 100 MBit routing - many of your internal clients are slamming into your sub 10 Mbit internet connection anyway (that is probably further BW limited by the cable/phone company). Now for true enterprise - you really do need switching/routing at the ASIC level - real switching fabrics (not a glorified PCI bus) in the hardware etc. to handle the multiple GBit links, multiple OC12/OC48 connections to the world, etc.

This is where Cisco shines and I don't see "software only solutions" coming anywhere close

Re:

[Cheeze (12756)]

OC3 and greater are probably not in 98% of the companies right now. Most of these companies probably have a weak DSL line with a linksys device. As companies get larger, they move up to T1 and need new hardware, it's probably a shock to have to pay the Cisco tax. What's the price of a 7200VXR class of router these days? If you could provide the same service with an old PC and a few PCI cards at 1/10th the price, you just made a nice cut-rate market for yourself.

Re:

[MerlynEmrys67 (583469)]

Strange - why would you expect companies to step down from decent DSL speeds to T1 rates. I assume small companies stay on DSL/Cable for the download rates (host their web server somewhere else - what do you need upload speeds for anyway), then move up to OC12/48 rates (or multiple OC3) when there are enough employees to justify the need for the upload rates these speeds provide - or the reliability of multi homing the network. Again this trade off happens somewhere north of 100 tech workers, or multiple

Re:

[aesiamun (862627)]

Because even commercial Cable and DSL sales have very little to no QOS. Read your agreement with your local cable co at some point...

You aren't guaranteed uptime as a business cable company anymore than you are guaranteed uptime as a residential customer.

Absolutely true.

[Shaman (1148)]

If you're not paying at least $100s of dollars a month, you aren't getting any sort of guarantees.

Why change from DSL? Reliability...

[WoodstockJeff (568111)]

Strange - why would you expect companies to step down from decent DSL speeds to T1 rates.

When you need reliability, you have to give up on DSL/cable, because no DSL or cable provider is going to give you service guarantees. If a DSL/cable line doesn't provide it's advertised 2Mb/s download throughput, that's too bad; you might be able to negotiate your bill down. And if it goes down, it's going to be you reporting it to your ISP, not the other way around...

But a T1 circuit (generally) has both through throughput and uptime guarantees written into the contract. And automated monitoring of its performance, and fast notification that something's wrong, 24 hours a day. I've had DSL circuits be out for days; the longest a T1 circuit was down was 8 hours, and there were severe financial penalties proscribed for that event.

That's not to say a T1 circuit is perfect; we use a bonded pair of them to feed one site. One went down, due to an incident with a trencher. Verizon promptly fixed it... by moving the circuit to another pair that tested good in the cable. Guess which pair got used... If you guessed the pair that the second circuit lived on, you'd be right, and it went down. This went on for a day, alternating which circuit was up and down, until one of our people met the Verizon tech at the repair site. "You do know that there are TWO T1 circuits here, don't you?" "Oooops..."

Re:

[WoodstockJeff (568111)]

True, there is no way to guarantee uptime completely, because it all involves wires or radio or something else that can fail in ways that you're not going to be able to fix quickly. Our T1s aren't bonded for reliability, but for speed... a fractional fiber just wasn't available to that site, so multiple T1s is the only way to increase speed. We're hosting, not surfing, so uplink speed is our bottleneck.

But bonded DSLs have the same problem that a single DSL has - no guarantee of service. Period. And you ca

Re:

[Shaman (1148)]

Cisco has the worst-performing L7 switches on the market until you get into the really large-dollar stuff (which they bought from another company). Use Extreme, Foundry or Big Iron and be much happier.

Cisco's routers are cheap, mostly Intel-based systems with PC-quality hardware and low performance for the dollar. If you are routing mostly Ethernet (which most do these days), you can build a multi-hundred-megabit Linux router very inexpensively and get more performance out of it than a 7x00 series Cisco r

Re:if it is only "Standard PC Hardware"

[Gadzinka (256729)]

If I had one dollar for every time I give this answer, I'd be frelling rich:

99% of businesses use sub 10Mb connection to the Internet and yet they are told the Cisco is the only way to connect them professionally. Moreover, the sub-$10k Cisco gear is a crap when it comes to performace, on par with good PCIe PC running on multiple Gbit eth interfaces.

That about sums it up.

Robert

Re:if it is only "Standard PC Hardware"

[Cally (10873)]

Gosh, someone who knows what they're talking about ;)

If your internet link is DSL, you do not need a real router :)

I should point out that this topic comes up every couple of years on NANOG, ummmmmm... here's a reasonable selection from the last decade [google.com]. These people have forgotten more about routing than most of us here will ever know. And until generic PCs come with multi-gig backplanes, it ain't happening anywhere except the low end. And at the low end, you're better off either leaving it to your ISP or using a few whitebox "desktop" switches/routers. They're cheap, cheerful, work, and you don't need to know the difference between "sh ip bgp run" and "sh bgp ip run"...

Re:

[Tweekster (949766)]

Most businesses simply dont have those demands...Yes, large businesses do, but that isnt of concern for the hundreds of thousands of small to medium sized businesses in the US.

Re:

[pe1chl (90186)]

Cisco's extended access lists are extremely limited when compared to Linux iptables.
The first thing that is apparent is that they are lists. A linear list is not a very convenient way to express your access policy, especially when you have more than one external interface.

I LOVE DD-WRT

[celardore (844933)]

I rent a housemate cable internet, which we had terrible problems with before. The problem is a bad cable causing a load of bad packets to 'clog' the router. It is the only cable long enough I have though, but the DD-WRT firmware worked a treat. It does allow some cool features, such as increasing the number of IP connections from 512 (the default) to 4,096 which is ideal for p2p. You can also boost wireless power from the 28mW default to 250mW+. Anyway, my problem with it clogging up was solved by setting up a cron job within the router so that it reboots at 5am each day. Not ideal, but the solution works until he gets off his ass and finaly buys a wireless card.

ASICs

[Rekolitus (899752)]

This seems to be an entirely software router that just runs on a standard x86 machine.

Isn't half the point of buying a dedicated-hardware router that you get ASICs and whatnot that do the job faster than software?

Re:ASICs Issues

[mpapet (761907)]

get ASICs and whatnot that do the job faster than software

I agree with you in principal(sp?) but I have a question:

As we upgrade some machines, I've got dual cpu (1.5ghz =/-) and 2+GB RAM being replaced by dual cores. Would server hardware be able to handle as much, if not more than the cisco asics (2800's mostly) I've got?

I get a damn good router for free. And I've got a spare parts inventory + redundancy. What am I missing?

English, please!

[Goaway (82658)]

Can we have that article again, this time in English, please?

* As long as we're not switching half the U.S.

[mpapet (761907)]

Summary: Works great, supposed problem sounds like it was a driver issue more than an application issue.

Reads like a well-placed article-vertisement.

The "as long as we're not switching half the US" comment are the one's I grow tired of. It's a well-wrapped insult.

I'm not saying Linux is the best tool for routing half the nation, but the comment points out some things that do prevent more linux adoption.

1. "free" is not as good as something I paid for
2. Don't fsck with the status quo.

I admin a company 100% cisco routers/firewalls and I know for a fact Linux can do what gets done.

I'm not going to tell the boss to "just" switch or evangelize too much because of the social/economic implications of doing so may impact my future. I like my employer, they like me, so when we need another router, it's a cisco. I am personally disappointed by this, but I think it explains why innovation takes -so- long to come to the data center. (at least in the U.S.)

Let's not forget that cisco can fire most of their software devs and use a linux-based router project if it ever got close to competing with some Cisco products. Does that qualify as innovation? I'd say no. It's not cheaper or better.

Advertorial

[HKcastaway (985110)]

It is great that someone is out there tyring to put some preassure on Cisco however this company is not it. I think the average Linux installation with NAT is a bigger threat that this project.

PC hardware is a joke, slow backplanes, limitation on how many interfaces you can plug in. On the techspecs the number of interfaces types they use is well very very limited. Then reliability of PCs a joke compared to a Cisco box.

Where is this product used?
- Is this a bloated replacement for the US$20 taiwan PPPoE rou

No huge suprise

[peterdaly (123554)]

In or around 1999 I had a 1000 device network routing through a 133Mhz PC running Linux. The 133Mhz system practically thought is was sittle idle as it shuffled packets between three 100 megabit networks.

I'm not suprised at all that these Open Source solutions are on par with Cisco for many users. My only real concern would be support. At least back then (I have not dealt with them recently), Cisco had great support and would "own" network problem resolution in a way that made it worth paying their price.

Re:No huge suprise

[macdaddy (38372)]

There is nothing like calling Cisco TAC at 04:00. You get an Aussie TAC engineer that knows 1) you're network is seriously fscked up or you wouldn't be calling him at 04:00, 2) you've already removed what little hair you have from your head and your scalp is bleeding. The nightshift engineer is highly experienced in working under said conditions and is more than capable or resolving the problem. I've been in that position twice in the last month. All I can say is I want to move to Austrailia, mate.

I wish the SmartNet prices were a little more reasonable. They should cut the prices dramatically for the lower-end 8x5x4-day replacement support so that more people can afford it. This would be a solid recurring business for Cisco whereas only a small percentage of Cisco customers bother buying support nowadays.

Re:

[nuintari (47926)]

So, you have a couple of subnets, all through one central router that has one default route, so your routing table has what? All of four entries? And you think this proves that a PC can beat a 'real' router?

Sorry, that just doesn't need much power to work. Try adding in OSPF, and some redundant links into your internal network. Get a second ISP and become multihomed, run BGP and add all 194,000+ entires from the global routing table into the mix, watch that P133 slow to a crawl.

Software routers handle every

Re:

[Shaman (1148)]

You only get that kind of performance out of a Cisco when you spend big bucks. Really, really big bucks. You figure a quad-processor Compaq with same-day service is going to be more expensive than a big Cisco router with tepid performance these days?

Re:

[nuintari (47926)]

AS the AC already mentioned to you, and as I already mentioned, cisco routers route 98%+ of their traffic directly between the line cards, so the CPU can handle important stuff, like handling the routing and express forwarding tables. You can get much better performance out of a cisco because of this architecture because even your quad proc pc based router still has to shove everything through the CPU, and will buckle under the load once you add more than a few line cards. Try plugging your quad proc server

in other news

[atarione (601740)]

a small truck can replace a semi truck.... if you are moving small amounts of items.

Support, Support, Support

[bstory (89087)]

Ok, I haven't looked at the performance numbers, but as a network administrator of a medium sized corporate network I could care less. Whether it be Cisco, Juniper, Nortel or 3Com the difference is in the support. When my wan interface or network interface dies at 2am I don't think anyone from the OSS community is going to have a parts depot within 4 hours to fix the problem. I also don't see 24x7 tech support phone numbers manned by volunteers anytime soon. Vendors don't make the money on the hardware, they make it on services and support. I love OSS, but Linux and OSS are not the magic pill for everything.

Re:

[HKcastaway (985110)]

You also forgot to mention the fact that the likelyhood of a hardware failure on a PC to a Cisco unit is like 20:1 (for most products).

Cisco has a far fatter margins on the hardware than PC vendors and can provide a much higher quality product, can afford to underclock the machines for higher reliability etc.

Re:

[LWATCDR (28044)]

I tend to agree with you but...
With the cost of commodity PCs these days you could probably have an entire second router on hot standby for the cost of a single year's support contract.
If it is a T-1 then just move the cable over. If it is an Ethernet connection the fall over could be entirely automatic http://linux-ha.org/ [linux-ha.org]
You will also have a trade off of in house time to test and configure vs just buying Cisco.
Of course their are times where generic hardware will not cut it. However this does offer some i

Re:

[macdaddy (38372)]

Well said. This easily says 80% of what needs to be said. Without support you have an OSS space heater at 02:00, assuming it will power up at all.

Huh, Samba file sharing?

[IpSo_ (21711)]

"The feature set was comparable to your standard Cisco router," Knox said. "They were offering translating, gateway capability, Samba file sharing, VLAN trunking to 11q ... it really looked like a corporate-level router," he said.

Since when do "corporate-level routers" offer samba file sharing? This seems like the LAST thing I would ever want to put on a router. The only thing I could possibly see Samba being useful for is downloading log/config files. But on a router that is kinda scary, SCP seems much more secure and just as useful.

Open source routing is definitely an option now though. Over 3 years ago the web hosting company I worked for swithced out their Cisco routers that couldn't handle the slighest DDoS attack for a couple AMD based Linux boxes that could easily handle wirespeed DDoS attacks with ease. Not to mention they were a fraction of the cost.

So... a Cisco router as good as a Cisco router?

[Kenja (541830)]

Hate to break it to ya, but Linksys is owned by Cisco.

Since when ... ?

[MrNougat (927651)]

Since when do we listen to "users?"

You're buying the hardware

[thesandbender (911391)]

I use Debian at home for a general purpose router and firewall and it is very flexible. There have been times when I've been tempted to deploy it as a small/medium business router in lieu of cisco but it's not just about the software, it's about the hardware as well. For a reliable system you need reliable parts... which are more expensive... preferable a cpu with a low thermal dissipation but still fast enough to handle the load, which is going to cost you money and either a RAID system or (ideally) a flash based storage system, which is going to cost money. You can build a system that will beat Cisco's cost/feature set easily. Building a system that can compete on cost/mtbf ... not so easy... and generally just not worth the effort. The article referenced a "still servicable pc" ... which roughly translate into "a machine that we picked up from behind the receptionists desk and cleaned all the dust bunnies out of.... *shudders*

Hardware support lacking

[drwho (4190)]

Too bad they only support Sangoma serial cards.

Cisco switch performance review

[Alex (342)]

http://blogs.sun.com/sbullen/entry/who_buys_cisco_ 3750_s [sun.com]

Re:

[Rekolitus (899752)]

Huh? What?

It's my hardware. If I buy a Cisco router via eBay, you're telling me I'm not allowed to put Linux on it if I can figure out how?

Re:

[blackbear (587044)]

According to Cisco it is in fact your hardware, but it's still their software, and you can't sell it or transfer it.

This bit me a couple of years ago when I bought a 2611 on e-bay, and wanted to put the latest security fixes on it. Not being Cisco certified, I contacted Cisco to find out about getting or purchasing updates. I was told that my router was "gray market" and that I would need to buy another license for it.

"How much is that?", I asked.
"$1500.00."
"Holy shit!" (hangs up phone, lest they send the s

Stupid lawyers

[nuggz (69912)]

So you're claiming to be a lawyer, but I have a few questions.
Why post as an anonymous coward?
Are you violating your client confidentiality with the parent post?
If I wipe their firmware (which I have a license to use) how am I violating their copyright?

Re:

[Skapare (16644)]

Your understanding of technology is obviously zilch, zippo, nada, nothing. And that leaves me with the feeling that your understanding of law is also generally diminished. And I presume you've never changed any software on the PC you own?

Re:

[sirket (60694)]

Let me be clear- Linux _can_ offer almost all of the same services as Cisco- but not in a single unified way. You need to use Zebra for BGP and add in other software for the other protocols. And what happens if you disappear tomorrow. If your environment is Cisco then you can call any other Cisco admin and they can admin it immediately. Try that with a cobbled together Linux solution.

-sirket