OpenSolaris From a Linux Admin and User Perspective 370
MSa writes "How does OpenSolaris, Sun's effort to free its big-iron OS, fare from a Linux user's point of view? Is it merely a passable curiosity right now, or is it truly worth installing? Linux Format takes OpenSolaris for a test drive, examining the similarities and differences between the OS and a typical Linux distro. If you want to sample the mighty ZFS filesystem, OpenSolaris is definitely the way to go."
Re:From an experienced Admin's perspective (Score:5, Informative)
Horses for courses, but Solaris has much to offer even for shops that aren't traditionally tied to Sun. Hell, even my private ``1U box in someone else's datacentre'' server for my family is now a Solaris machine.
ian
Excerpts from the article: (Score:4, Informative)
"... I found OpenSolaris significantly slower than Ubuntu or OpenSUSE..."
"There are fewer packages available than for a mainstream Linux distro, although they do have over a thousand (and certainly enough for a fully-functioning system). The package naming is slightly odd; package names begin with a handful of capital letters (eg SUNW or FSW)."
"ZFS is transactional, meaning that the filesystem is always consistent (so fsck or equivalent isn't used or needed), and snapshots are intentionally both easy and cheap in terms of disk space."
"I'm very impressed with the concepts behind ZFS, but I'm also concerned that cross-functionality with Linux is limited."
"I did find it frustrating to have to relearn commands that I've been using without thinking for years now (eg ifconfig), and right now I'm not convinced that for me it's worth the mental effort, especially given the relative scarcity of external software available."
The syntax for ipconfig is different? (Score:3, Informative)
That's because the Linux folks were worried about the pending USG/CSRG lawsuit so they reimplemented TCP instead of using the BSD TCP stack and utilities like almost everyone else (including Microsoft) did.
Just about any non-Linux UNIX implementation is going to have the BSD TCP.
On the upside the lawsuit did set SCO up the bomb. Oh, it wasn't the only thing by any means (did they actually do ANYTHING right in that lawsuit?), but one of the side effects of the USG/CSRG lawsuit was that a lot of early UNIX code code was open-sourced. Including some of the SCO claimed were examples of "infringing code" in Linux. Come on, folks, wasn't it great to have Dennis Ritchie himself point that out?
Solaris has packaging design issues (Score:3, Informative)
Trying to harden Solaris is a nightmare. Mostly because so many packages in the Solaris install are interdependent. It is either install 90% of the packages or install nothing. Why do they even bother breaking the software packages if this is the end result? Getting rid of RPC can create so many problems it isn't even funny. Both BSD and Linux offer the option of only installing the base package and only choose the services you want with little to no other packages to depend on. This however absolutely cannot be the case for solaris because a single needed software package will require you to install nearly all services.
Re:From an experienced Admin's perspective (Score:5, Informative)
Solaris scales far better than any BSD or Linux distro out there.
Yeah, you know, the roadrunner team would like a word with you, as would pretty much everyone in the Top 500. For some business loads Solaris scales better. But the claim the "it scales far better" in general is as absurd as it is patently untrue.
Ask any *real* Unix admin who uses both and more than likely they will say Linux is great for small jobs but Solaris is king for anything else.
Ah, and no true scotsman^W UNIX admin would run a supercomputer, right?
Re:From an experienced Admin's perspective (Score:4, Informative)
Re:From an experienced Admin's perspective (Score:3, Informative)
Solaris handles threading a little differently than Linux
It used to be that Solaris used an N:M model while Linux used an N:1 model. Now both use a 1:1 model. There are lots of reasons for this (Matt Dillon gave a really detailed description when explaining why Dragonfly BSD went 1:1 instead of N:M). Basically, it boils down to the fact that debugging threaded C code is such a bitch that people tend not to use high levels of parallelism in C code (which is where N:M really shines). If a language has better support for parallelism then it is easy build an N:M model on top of a 1:1 model (this is what Erlang does, and I believe Java does as well in some versions).
Re:From an experienced Admin's perspective (Score:1, Informative)
"Since release 10, Solaris has been pretty well stomping the competition in price, performance and throughput. With Solaris supporting pretty much every type of virtualization (including some not offered anywhere else), it's hard to beat."
I think you have been drinking a bit too much of the marketing cool-aid. Well, you haven't ever been in my shop. I'm a Solaris guy - Admin, instructor, blah... These days our Sun boxes just don't cut it. They are being out performed by *AIX* machines!!! (yes, solaris 10)
"It's cheaper to buy Solaris support from Sun than to buy Linux support from RedHat."
You had better not rely on either. Sun support SUCKS!!!! (at least since 2003) RedHat doesn't do much better either. You need to have experienced admins. Platinum support means nothing when Sun doesn't know what to do.
"There's no *tying* with Solaris"
Oh yes there is. Once you start developing and using a platform in production, you are tied to it. Sure, you can always move off of it - at least in your utopia, not reality very easily.
Re:From an experienced Admin's perspective (Score:3, Informative)
Stay away?
What if you need real uptime with a load of 80 on a 32 cpu system? Can Linux handle the load and have years of uptime?
Yea, stay away. If you have a load of 80 on a 32 CPU system, you didn't design the hardware or software correctly.
Solaris just works and its made for servers. Linux seems always beta quality with its cutting edgness and is desktop oriented. I would not trust my job to it unless its Debian or RHES which costs $$$ as cutting edge features are not needed on a mission critical server. Solaris scales far better than any BSD or Linux distro out there.
Debian doesn't cost anything, and there's always CentOS if you want the RHEL reliability without the cost.
Ask any *real* Unix admin who uses both and more than likely they will say Linux is great for small jobs but Solaris is king for anything else.
Yea, these were the same Unix admins who used to ask me if I installed the latest kernel patch while they were still using sendmail (and patching it about as frequently). I didn't put a lot of faith in their opinion.
Oh and the article discussed a scarcity of third party apps. I found the opposite as most server ERP and database apps are on Solaris than Linux.
For the big big things, probably. Oracle? Works perfectly fine. I'm building a RAC now using commodity hardware that will probably be 1/3 the price of what it would cost to get something from Sun.
To be fair, I haven't used much Sun equipment (hardware or OS) in the past 6 years or so. There's a number of things they get right, like the Open Firmware. But from an OS and maintenance perspective, does Sun still have patch clusters? Do I have to head over to SunFreeware.com to get useful applications installed? I can provision a Linux server literally in a few minutes, but it would take the better part of a day to get Solaris set up (have to remember to disable telnet, find the latest patch cluster, reboot, install gcc and other apps). Bleah.
Re:From an experienced Admin's perspective (Score:5, Informative)
Having been a UNIX admin for 23 years and Solaris for 10 years, I'm not sure what you're drinking, but I'm staying away from it.
Solaris support has rocked. We've never had an issue that Sun hasn't been able to solve, and yes, we've thrown them some curves (and sliders for that matter). IBM's support has told us on multiple occasions to re-install the system as a fix for a problem. RedHat we've stumped more often than not. HP? Well - they still can't figure out how to handle more than 8 luns per target for scsi (as well as fibre)...
Solaris performance has been fantastic - outperforming Linux, AIX, HP-UX on modern equipment.
We've migrated workloads to and from Solaris - no big deal - as long as you know what you're doing.
(Our misguided DBA's started migrating from old SunOS 5.8 boxes to Linux - and are now migrating back.)
If you use tools that are available on multiple platforms, migrating isn't all that tough.
If you are developing native language apps, porting isn't terribly difficult although finding workarounds for pesky native quirks is troublesome at times.
So I guess it depends on what you call "experienced"...
Re:From an experienced Admin's perspective (Score:2, Informative)
I guess you have never worked in supercomputing....where Linux is just about what everything runs.
Re:From an experienced Admin's perspective (Score:1, Informative)
Linux seems always beta quality with its cutting edgness and is desktop oriented.... Ask any *real* Unix admin who uses both and more than likely they will say Linux is great for small jobs but Solaris is king for anything else.
Get your head outta your ass. When even Wall Street firms use Linux on their grids/farms to crunch numbers using complex modeling on very very critical jobs ($$$$), reliability, scalability along with cutting-edgeness - everything comes into picture. And they are replacing their Sun boxes for last 3 years.
I have worked on these (as a developer, mind you), so at least I can attest Linux's readiness with its 'beta' quality. So, take your bullshit somewhere else.
Re:Love that they open sourced it... but... (Score:3, Informative)
Because in order for it to operate optimally it must be part of the kernel, and Linus and crew refuse to put it in the kernel due to licensing issues.
It runs fine in userland with fuse, but it's slow.
Re:Nexenta (Score:3, Informative)
You might also want to try Zenwalk... it's an XFCE desktop out of the box, but there's KDE packages in the repository... I don't actually have kdebase installed on my system (the only things from KDE that I actually use are Konqueror and Kopete, which are in the kdenetwork package, and work without kdebase), but it's actually a stock, unmodified, compiled from source package that, if it's anything like every other package on the system, is about as close to what the KDE devs want it to behave/look like that you'll find.
I'm not saying that other systems, like arch, Sabayon, or openSUSE aren't great systems. But if you're interested in tinkering and trying things out, you may want to give Zen a try. :)
Re:On VirtualBox, it blows (Score:3, Informative)
The OS X version of Virtual box does not support (yet) any of the processor specific virtual machine extensions that speed things up considerably.
Re:The syntax for ipconfig is different? (Score:5, Informative)
Re:From an experienced Admin's perspective (Score:3, Informative)
A load of 80 on a 32 cpu system sounds like a poorly architected solution to me. :)
As for the rest, give me a break. One of the benefits Linux is that if you want cutting edgeness and desktop goodies you can have them, but if you're looking for stability and vendor support you can have that too. And it doesn't mean spending a ton of money either - RHEL is relatively cheap and Debian is free (no idea what you're smoking there), as are a number of other options (CentOS, Ubuntu LTS, etc).
We run hundreds of servers on Linux servicing millions of subscribers and have absolutely no stability issues whatsoever. Machines occasionally go down due to hardware faults, power incidents or kernel upgrades, but only a handful of kernel related failures over the years. We've actually had more failures with our Sparc/Solaris machines, generally exhibiting as spontaneous reboots.
Re:ZFS rocks (Score:3, Informative)
ZFS does have issues with NFS though. In particular NFS writes can lock up the client. Hopefully this issue will be fixed soon. it's not really ZFS's fault; it's NFS's fault. Yet no other FS has this issue, so I'm sure a workaround could be done. In the meantime there are times when the NFS clients on our 12 TB ZFS (Solaris 10) system are unusable.
ZFS will never come to Linux while the license remains incompatible with the GPL. I predict that one day Sun will relicense it, but not before they've really tried and failed with OpenSolaris. I think the FUSE idea is viable though. Just that the developer who was doing it is now working for Sun and has no time/inclination to do more with it. In theory all file systems should be implemented with a FUSE-type of separation from the kernel. This hearkens back to the Mach days where file systems were envisioned to just be user-space servers. Now it just might work. NTFS-3g has really good performance, really, and it is fuse. And when it crashes, it won't take down the OS.
So far the community or lack of, surrounding OpenSolaris is pretty disappointing. I think Sun just thought it would magically happen. Long term, I'd rather see the best technologies of Linux and Solaris merge (Linux massive hardware support, scheduler, dtrace, zones, zfs), rather than continuing in separate directions.
Re:maybe I should go and play around with this! (Score:5, Informative)
OpenSolaris works well as a server OS - that
The ZFS filesystem is interesting for desktop installations - it does allow seamless use of the 1-2 terabyte desktop disk configurations that are now possible. ZFS was designed for the datacenter - eliminating the need for the time-honored but fragile combination of journaling filesystem over software volume manager (usually over HW RAID). It's the first real innovation in filesystem architecture since journaling filesystems were developed.
Additional software packages are available from 3 well-known (in the Solaris community, at least) sites. Sun has it's own freeware site, blastwave.org and sunfreeware.com
http://www.sun.com/software/solaris/freeware/s10pkgs_download.xml [sun.com]
http://www.blastwave.org/ [blastwave.org]
http://sunfreeware.com/ [sunfreeware.com]
The package manager for blastwave.org is their own, the others use the standard Solaris pkgadd commands. The package naming convention is a long-standing convention - each vendor uses a different prefix, making it easy to differentiate the source of packages.
OpenSolaris commands, where Sun hasn't replaced stock UNIX commands with their own, reflect SVR5 standard rather than the more Linux-ish BSD syntax.
One of the places where Sun has replaced "normal" functionality is the init process. SMF is Sun's attempt at fixing the long-standing problems and in-efficiencies of the BSD or SVR5 init process. Apple has launchd, there's openrc and gentoo's baselayout that all have similar goals. SMF works well and there's a fair amount of support on the net for integrating non-distro apps.
One of the "why OpenSolaris" answers is that there is value in running the same OS on the desktop as on the server. For Solaris shops OpenSolaris on the x86* servers provides a common platform that enables system management efficiencies to be extended.
Re:ZFS rocks (Score:4, Informative)
To be fair, ZFS is a lot more than just a filesystem. It encompasses snapshotting, redundancy and volume management as well.
Re:maybe I should go and play around with this! (Score:3, Informative)
Re:From an experienced Admin's perspective (Score:3, Informative)
Funny you should mention the "desktop orientation" of Linux, considering that any recent Solaris puts on a shiny Gnome suit...
Solaris scales far better than any BSD or Linux distro out there.
Ask any *real* Unix admin who uses both and more than likely they will say Linux is great for small jobs but Solaris is king for anything else.
OK, I'll bite. I've been admin'ing Unix for 12 years, and a user for 20. I've maintained everything from an UltraSparc workstation to E10Ks. Lately (past five years) I've been doing AIX, maintaining over 150 OS instances.
For a test project I've been researching Linux and Solaris for an Oracle RAC installation. I will tell you that, hands down, the Linux network stack is faster. Raw i/o is close, but Linux wins small file writes and reads by a significant margin. BTW, the Sun.com site has a paper comparing Linux filesystems versus UFS, but take it with a grain of salt (look at the machine they use).
Now let's talk about scaling...
Up to the 8 processor machine that I tested (which is a *small* PC system), Linux continues to scale close to linearly for the Oracle/TomCat/Apache workload, as does Solaris. Beyond this I understand things can change, but so be it, that's not the platform I'm needing.
Now certainly you can build a workload requirement that will put Solaris on top, but in the vast majority of installations Linux will do just fine.
Of course there are other factors. RedHat doesn't have the most sterling support (compare it to IBM for example), but I can buy support through IBM (or Oracle or Sun for that matter) if I wanted. This may be the reason for a shop to go with Sun, but I wanted to correct the idea that Linux is not as technically valid as Solaris.
Re:From an experienced Admin's perspective (Score:3, Informative)
Absolute bullshit. I run Red Hat Enterprise Linux (64-bit) on a 4 node AMD Opteron cluster running Oracle RAC. Each server has 32 cores, 128GB of memory, and 8x 4gb HBAs, connected to an HP EVA 8100 SAN. I also tested Solaris 10 on this same hardware, and you know what? It outperforms Solaris 10 on the same hardware. I have the Oracle ORION benchmarks to prove it.
There was a time in the late 90s when Solaris was a far superior OS to Linux for use in the data center. I'm an SCSA, I know, because I started as a Solaris admin long before I worked on Linux. The reality is that now, Linux outperforms Solaris for I/O intensive applications like Oracle database. Why do you think Oracle themselves migrated to Linux a while back? Solaris and Sun have been losing their best and brightest engineers for a long time now, and the quality of their OS shows. It's getting dated. Sure, new features like ZFS are cool, but the core of the OS, where it really counts, hasn't been updated enough to take advantage of the large memory and CPU core footprint that new commodity servers have.
Re:What about hardware support? (Score:2, Informative)
I have had driver problems with Linux as well. Does it have such a site?
PS. You can also check the forums at http://opensolaris.org/os/ [opensolaris.org]
Re:maybe I should go and play around with this! (Score:4, Informative)
The ZFS filesystem is interesting for desktop installations - it does allow seamless use of the 1-2 terabyte desktop disk configurations that are now possible. ZFS was designed for the datacenter - eliminating the need for the time-honored but fragile combination of journaling filesystem over software volume manager (usually over HW RAID).
It's the first real innovation in filesystem architecture since journaling filesystems were developed.
just karma whoring here, but it's important to mention that pretty much everything ZFS has to offer was already available on tru64's advFS: http://advfs.sourceforge.net/ [sourceforge.net]
it's a shame HP killed this fine unix to keep that abominable HP-UX, so kudos to sun for bringing back the functionality of tru64 back to the datacentre AND the desktop.
hmmm, i wonder if my notebook (presario v6210) is compatible with opensolaris...
Re:From an experienced Admin's perspective (Score:3, Informative)
""it scales far better" in general is as absurd as it is patently untrue."
How so? With the exception of the SGI implementation, I don't don't of Linux distros out of the box that scale to 100+ processors well (indeed, the usual approach is to run multiple instances of the kernel on each processor or so on large ensembles). When discussing an OS, scalability is usually meant in terms of how the OS itself scales ... not how one's applications can be configured to scale.
So I think the poster's claim that Solaris scales far better is true FOR THE OS itself.
Now, whether THAT matters to your workload (or if your workload scales across many processors via multiple OS instances just fine) is another question entirely.
Re:From an experienced Admin's perspective (Score:3, Informative)
If 100% is the maximum load, how are you exceeding it? Unless you are using a higher value as the maximum percentage, your math troubles me.
The load average is the number of processes waiting to run, quite different from % CPU usage.
Re:maybe I should go and play around with this! (Score:3, Informative)
Irix was never meant to be more than a workstation OS...
Say what? SGI sold servers too. In the late 90s it was their main business. The email server at AOL used to be an SGI Origin — running IRIX.
SGI's business was an is (they still exist, albeit as a very tiny supercomputer/server vendor) high performance computing. Doing HPC with workstations and doing it with servers and supercomputers is not all that different. And until they shifted from MIPS to x64 and Itanium, all their systems ran IRIX.
Re:maybe I should go and play around with this! (Score:2, Informative)
It may happen some day, but until then its RAID5/6+LVM+ext3.
Right, and that day better come soon, at last before 2TB drives become mainstream in RAID5/6 setups. read http://blogs.zdnet.com/storage/?p=162 [zdnet.com] about the chance of double failure on RAID5/6 growing with the size of the disk. Happened twice on my servers, exactly with RAID5+LVM+ext3. We need ZFS (or similar) on linux servers right now.
Re:From an experienced Admin's perspective (Score:4, Informative)
One of our Linux servers regularly copes with a load in excess of 100. Things slow down, but nothing breaks.
Be careful with comparisons like these.
Linux lumps disk I/O into the load average, whereas most "other" Unixes don't. I've seen a Linux box with a load of 300+ and idle CPU, and a Sun with a load of 2 that was near unusable because the disks were being thrashed to death.
Comparing the two can be unfair to either side depending on the context. It's apples and laundry detergent.
Re:One significant difference... (Score:3, Informative)
I would not even begin to think how Linux could handle this
Evidently I am somewhat more adventurous, as I will dare to think. My thought is that linux could handle this by having some sort of kernel setting, called, say, "overcommit_memory", classed in the "sys/vm" part of the proc heirachy. I would think that one could alter the behaviour by echoing a setting into it, eg "echo 2 > /proc/sys/vm/overcommit_memory". Incidentally, I think it could work this way because it actually does work this way :P
A question though -- with overcommit disabled, things like java or wine (which allocate several hundred megs of ram to play with whether they actually use it or not) tend to start failing; how does solaris deal with this? I wonder if it's even noticed, as the last solaris box I saw had 8GB RAM and 32GB swap, whereas this problem is more apparent on my 64MB linux VMs...
Re:maybe I should go and play around with this! (Score:5, Informative)
I'd like to give Znork some credit here.
ZFS is really, really nice but it does have some warts and the biggest for many would be that arcane operating system that's dangling off its nutsack. Yes the solaris kernel is great, scales like a champ etc. but the userland and the lack of centralized package management (in 2008, no less!) are bad joke.
Re:md broken? (Score:3, Informative)
Only when it forgets where drives are, or wipes configuration during upgrades, or the fact that it's slower than software raid setups in FreeBSD or even Mac OS X.
md on a single box occasionally works. Managing 120 machines with md became a reason to never use md again.
Re:md broken? (Score:1, Informative)
OK, I'm not usually that bad, but I could be if everything wouldn't start crawling as soon as the second I/O operation started. Anyways, when I first started building my storage box last summer, I had heard that ZFS was the final word in storage. Period. The school has a SPARC Solaris box that the development students use, and I've never been a huge fan of it, but I had also never heard a file system described as sexy before ZFS. After fighting with OpenSolaris (funny how you get a GUI installer and on first boot, it tells you that your graphics card is unsupported!) for about a full day, setting up ZFS was the most pleasant experience I've ever had with a piece of software. I had a ZRAID pool up and running after about five commands and perhaps a small conf file. I could move storage around like it was water; it flowed however I wanted it to. And it was fast. I mean, FAST. It would take whatever I could throw at it while doing a snapshot and not break a sweat.
In the end, OpenSolaris just couldn't deliver; as great as ZFS was, it just wasn't worth babysitting OpenSolaris. I just couldn't help detesting the user land apps with such lacking command line options and a rather fugly directory structure. What, you want to use bash as root? Hah! You get sh - and a stern warning against changing it.
So, the box is running Slackware (it's rock solid stable, and I've yet to find software that won't compile on it... OpenSolaris blew up while patching twice in the few weeks I was running it, without allowing me to back out the patches, of course!) with md and multipath. Sometimes md decides to rebuild for no apparent reason, every time I reboot multipath renames my md devices so I have a script that figures which is which and mounts them accordingly in rc.local, and then starts the services that need the data contained within. MD is also very iffy when it comes to hot swapping in drives... it will refuse to acknowledge that the drive I just added is part of the device it degraded until I force it to check the metadata and then reinitialize (or was it rebuild or restart or reload the configuration file that only applies to devices with version 1 metadata, or the repair for version two metadata? It's not like it writes any of this to the disk... which is good if you're using RAID 10, as you have to start them in two steps!) If I wanted to do something for snapshotting my options limited as LVM only works for devices (I can't do my usual trick of dd'ing a 8 GB flat file and laying a file system over it so that I can gzip the sparse file and open it somewhere else or mount it on a loopback). Also, LVM is just slow. No matter how I align my PV stride to my RAID device, it just doesn't seem to fall in sync well.
Posting AC since I wrote this entire thing just to find out that I modded this article earlier today. Nice. Can we get some kind of reminder before we pen the entire War and Peace in a 2"x2" block, please?
Re:maybe I should go and play around with this! (Score:3, Informative)
yet you can't see the improvement on Linux's model?
Yah, the OOM crashes, flush hangs, dbf fragmentation and performance degradation issues really left a bad aftertaste. I didn't go into those details, as anyone else who'd run ZFS in production for some time would most likely know about them. Some are fixed now, but improvement... lets just say that any improvement I could see was rather outweighed by the problems.
Don't get me wrong, ZFS is nice if you use it where it has it's strengths. It's a perfect filesystem for hardware like thumpers serving out NFS. Unfortunately it seems to be designed with that, and only that, in mind; even worse, technical evangelism at Sun appears to have failed to mentioned that to their tech sales, getting ZFS installed in ways it's not at all suited for.
A more flexible architecture would have allowed ZFS to be great for many things where it can actually get in the way today.
You mean the fact that md is completely broken
So don't use md. Use lvm mirroring. Or hardware mirroring. Or drbd remote mirroring. See, flexibility. If a layer doesn't do what you want it to, change it.