ZFS For Mac OS X Source Code Available

nezmar writes "Noel Dellofano, who is part of the ZFS development team at Apple, has a post on Mac OS Forge announcing a late Christmas gift: he is making available binaries and source code, plus instructions, of the ZFS filesystem for Mac OS X."

The real questions are...

[slyn (1111419)]

How stable is it, and how soon till I can get it on my Mac by default?

Re:The real questions are...

[larry bagina (561269)]

a readonly version is included with leopard:

sh-3.2# zfs
Read-Only ZFS Implementation
missing command
usage: zfs command args ...

Re:The real questions are...

[Kremmy (793693)]

Not real sure, but you might want to ask the users of ISO9660 and UDF on optical media.

Re:The real questions are...

[wodgy7 (850851)]

It's so that you can bring a ZFS volume to any old Mac running Leopard, plug it in, and read data off it, without having to install any extra (currently beta) software.

This is also why, when you create a ZFS pool using the read/write drivers, it defaults to creating a pool with ZFS version 6 on disk, so that it's compatible with the version of ZFS shipping with Leopard. (You run "zfs update" to transform your pool to the most recent on disk version if this kind of compatibility isn't an issue for you.)

BTW, Leopard also reads from BSD and Solaris-created ZFS drives just fine.

Re:

[xeno (2667)]

"what good is a file system if you can't write to it?"

I could say the same of NTFS. After throwing in the towel with regard to Windows as a base OS, I have years of accumulated data on NTFS volumes spread across a small pile of drives. Linux support for NTFS is still a little shaky. But with read-only access to NTFS, I can throw those old desktop or laptop drives into an enclosure, connect it, and either pull all the data over to a writable volume for ongoing work (and perhaps dispose of the old drive), o

NTFS-3G on Linux is stable

[Cato (8296)]

Have you tried NTFS-3G? It really is very stable, no doubt due to the exhaustive testing regime on every release - see http://www.ntfs-3g.org/quality.html [ntfs-3g.org] - and is used by default in most Linux distros. It's a different codebase to the older Linux-NTFS and Captive NTFS projects, and has reasonably good performance.

Since ZFS is new, I don't think your scenario applies, and it's not intended for DVD/CD use.

Re:The real questions are...

[QuietLagoon (813062)]

It's already available on FreeBSD [freebsd.org] if you want to play.

Re:The real questions are...

[BrainInAJar (584756)]

Or more to the point, OpenSolaris... because that's where it came from

Re:The real questions are...

[wodgy7 (850851)]

I've been running ZFS on my home Mac server since the old developer seed. It's generally stable as long as you disable Spotlight indexing on the volume (it's not supported yet). Everything on the command line works, as does accessing the ZFS pool over AFS. It's *very* easy to set up btw, much easier than setting up a RAID in Linux. There were issues deleting files from the Finder in the last release; I haven't installed the 102A release yet. Still, if you're just using it for a server volume, you'll probably be happy with it.

Re:

[Eddi3 (1046882)]

"It's *very* easy to set up btw, much easier than setting up a RAID in Linux. "

I doubt that. Setting up a RAID array in Linux is about 4-5 lines in the CLI.

Re:The real questions are...

[wodgy7 (850851)]

It wasn't that easy to set up a RAID in Linux the last time I tried (admittedly long ago), but even in comparison, setting up a RAID-Z in ZFS is just a single line: "zpool create mypool raidz disk4s2 disk5s2 disk6s2"

Re:The real questions are...

[hjf (703092)]

then you need to mkfs, and if you run out of space you're screwed because you can't easily grow. also, you can't create a newer fs, nor you can have snapshots, send/receive snapshots, volumes, have on-the-fly checksumming and disks that don't drop off the array at the first read error, one-line CIFS/NFS/iSCSI sharing. Get over it... zfs is better than md+lvm+ext3+whatever.

I'm not trolling, it's just that ZFS has been developed without the traditional and orthodox methods of disk-partition-filesystem and put everything on a single "layer", and instead of losing flexibility, we gain more, just because zfs developers were thinking outside the box (the now "traditional" way of doing things is segregation: the OSI layers, etc, claim to be more flexible, efficient and manageable than throwing everything together). I know, I know, veritas had this for years, so we could say that it was stole^H^H^H^H^Hcopied from them -- just as gates copied jobs, and jobs copied xerox.

Imagine the possibilities of breaking traditionalisms (like linux does "socially" but not "technologically").

linux md is grow-able, as is xfs and ext3

[SuperBanana (662181)]

then you need to mkfs, and if you run out of space you're screwed because you can't easily grow.

All of Linux's md raid modes are grow-able.

LVM2, XFS, and ext3 are all capable of not just expansion, but *online* expansion. With xfs, it's one command- xfs_grow -d. It automatically senses the new block device size and presto, you've got a larger file system.

BTDT two weeks ago when I added a drive to my RAID5 array, expanded the LVM2 physical volume, grew the logical volume, and then grew the XFS volume (I make the choice to run LVM2 on top of the array- I could have just as easily put XFS directly on the array device itself.) The only caveat is that you won't see the extra space until the resilvering is done.

I'm not saying it's equal to ZFS, but Linux's filesystems and volume management are a lot more capable than you're claiming, and everyone needs to calm down and realize that RAID is not ZFS, ZFS is not RAID, etc.

Re:linux md is grow-able, as is xfs and ext3

[iPaqMan (230487)]

You should watch these. (thanks andrewg for links)

For anyone who has not seen the ZFS demonstration videos by Bill Moore you must watch the link.

High Bandwidth versions - http://www.sun.com/software/media/real/zfs_learningcenter/high_band [sun.com]...

Low Bandwidth versions - http://www.sun.com/software/media/real/zfs_learningcenter/low_bandw [sun.com]...

Also general info here:

- http://www.sun.com/software/solaris/ds/zfs.jsp [sun.com]
- http://www.sun.com/software/solaris/zfs_learning_center.jsp [sun.com]

Re:

[Solra Bizna (716281)]

zpool create pool raidz c5t6d0 c5t6d1 c5t6d2 c5t6d3

For those of you who have not used Solaris yet, or aren't sure whether ZFS is up to the hype; that notation is "disk n of target 6 of controller 5." Your home server has absolutely nothing on the dreadnoughts from Sun. They sell a box with 50+ hotswap drive bays, and the CPU power to back it up (and it's not even the top of their line).

-:sigma.SB

Re:

[baadger (764884)]

For LVM, one has to partition the disks first.

No you don't, LVM Physical Volumes can be initialised straight onto whole unpartitioned disks (/dev/sda).

Re:Linux md isn't rocket science...nor is ZFS raid

[wodgy7 (850851)]

You're mistaken. ZFS RAID-Z is definitely "raid" -- in fact it's RAID without the RAID-5 write hole on non-specialized (no NVRAM in the controller) hardware. Contrary to what you said, you *can* easily go from a single drive to a pair of mirrored drives (see ZFS admin guide, p. 59) or a RAID-Z (p. 60). The only real limitation is you cannot add an additional disk to an existing RAID-Z configuration, the idea right now being that you'll add another set of disks in RAID-Z as a top-level vdev. This is not optimal for a lot of scenarios but they're working on it. ZFS mirrored configurations are more flexible.

The data integrity advantages of ZFS over traditional RAID-4 and RAID-5 are hard to argue with... it validates the entire input-output path.

Not ready for prime time...

[maubp (303462)]

Reading their FAQ, it sounds like there are lot of niggles to fix yet - including assumptions in other parts of Mac OS. All in all it sounds like ZFS isn't ready for general use on the Mac just yet. Maybe Mac OS X 10.6 will ship with this by default?

Re:Not ready for prime time...

[iluvcapra (782887)]

I'll bet one of the reasons they're putting it out there is the hope that a few kind souls with some time on their hands will submit some patches and work out the kinks; given the amount of interest there is for this to be working on Mac OS X -- and there's a lot.

Maybe between Apple, some Sun devs on their breaks and Amit Singh they can have this all wrapped up in a few months :)

Academic question: What would have happened if MS had open sourced WinFS? Even under their PL, there would probably have been enough interest among enough dedicated nerds to... who knows.

Notes

[asparagus (29121)]

I installed this last week, got it working. It's still very early beta, managed to crash my machine half a dozen times before deciding to wait a little. Remember to do zpool exports before you eject external hard drives. But yes, very promising technology. OS X has gone from having a wonky 1/0 implementation to having one of the better software raid systems available. Back to scoping out four and eight drive usb sata enclosures and cheap 500gb hard drives. ;-)

Re:Notes

[by Anonymous Coward]

Well, if your looking for cheap HDs. Here is a GREAT script a guy wrote.

http://forre.st/storage [forre.st]

It works with newegg.com to find the best deals on HDs

Re:

[LKM (227954)]

Yeah, he's doing some absolutely awesome stuff. Makes me wish I could grow up now. On the other hand, I'll probably be dead once the world is flooded due to global warming and Mad Max meets Waterworld, so I shouldn't feel to bad for not having a computer when I was 14, I guess :-)

Re:Notes

[by Anonymous Coward]

Based on your limited experience with this filesystem, would you say that it would make sense to port the source code to Solaris? I'm sure there's a lot of Sun users who could use a shot in the arm like this right about now.

When do they say, "Just Kidding!"

[osgeek (239988)]

This reads like a nerd's unsubstantiated wet dream [macosforge.org].

An absolutely, positively, amazing feature set. I can't wait until it's stable enough for production use. After 7 years of staying away from Apple products, I'm going back to the Mac.

Re:When do they say, "Just Kidding!"

[larry bagina (561269)]

ZFS is also available in FreeBSD 7 and OpenSolaris (which should be the most stablest of all).

Re:

[osgeek (239988)]

Every time I have to mess with Solaris, I'm annoyed at how much dorking I have to do with it to get it to have a reasonably modern environment and set of tools on it like a fresh install of Ubuntu or Fedora Core.

FreeBSD... maybe... I kind of like the Apple hardware, though.

Great new filesystems

[PhotoGuy (189467)]

It's a shame that I'm gunshy with new (to the OS) filesystems. ZFS has so much to offer, but every time I try out a new filesystem, I end up with data loss, even ones that are supposedly new and wonderful and robust. (Even when ext3 was new but stable, I lost stuff on it.) I can't wait to hear lots of positive feedback on its stability and performance, so I can get up the nerve to try it.

Re:Great new filesystems

[mcowger (456754)]

Consider my comment some of that.

I've had no problems with 5T+ datasets, and we even get about a 10-20% performance boost out of it compared to UFS.

snapshotting & all those neat features work totally as expected.

Only minor issue I see is that a zfs send is single threaded, so you cant parralellize it over multiple processes easily.

Re:

[stinerman (812158)]

Only minor issue I see is that a zfs send is single threaded, so you cant parralellize it over multiple processes easily.

Well, a multithreaded filesystem is only a performance hack anyway. :-)

Re:Great new filesystems

[iluvcapra (782887)]

Here's a link explaining the parent [oreilly.com] for all you c|net "reporters" and NYT technology stringers who read slashdot. You know who you are.

Best ZFS Presentation

[this great guy (922511)]

I have been using ZFS (on Solaris) for more than a year, both at work and at home, and I am following closely the latest developments. IMHO the best intro on ZFS is the official ZFS slides (36 pages): http://opensolaris.org/os/community/zfs/docs/zfs_last.pdf [opensolaris.org]

That's nice.

[Jane Q. Public (1010737)]

Now, if we can only get it to talk to important things like NTFS, and Ext3, and Reiser...

Re:That's nice.

[Rebelgecko (893016)]

If you want to talk to Reiser, visiting hours are 9AM-5PM on weekends.

"he is making"

[by Anonymous Coward]

I know it may be unheard of to those reading /., but Noel is a girl.

Actually,

[antijava (128456)]

Noel is a she. I met her last year soon after Apple hired her away from Sun.

What if someone did port ZFS to Linux?

[halfdan the black (638018)]

Suppose I ported ZFS to Linux (not that I could, just suppose) as a native kernel module, and published the source code. If then I used ZFS on Linux, and some others also grabbed the 'Linux ZFS' code, built it and used it. What laws if any would I be breaking? Who and under what grounds could sue me / Linux ZFS users?

Re:

[corychristison (951993)]

Who and under what grounds could sue me / Linux ZFS users?

Short answer: nobody and nothing.

Long answer: The biggest issue (to my understanding) is that it will not be included in the official kernel. Google sponsored it to be included in FUSE to cover their butts because I suppose they just didn't want to get involved in the issues. I don't see why it couldn't be released as a patchset that someone would have to patch and install manually, at the very least.
But then again, this is my view and understa

Re:Hmm

[wootest (694923)]

Since Apple employs Noel Dellofano, hosts Mac OS Forge, has incorporated the stable read-only bits in the latest Mac OS X Server and makes a slightly older build of the same code as the Mac OS Forge read/write version available on their developer web site, I think they approve.

Re:

[leamanc (961376)]

Well then I wonder what Sun thinks of this.

Not that it really matters what Sun thinks about their F/OSS filesystem that anyone can download, modify or incorporate into their OS, but they are excited about Apple's adoption of ZFS, and have been contributing resources to the 'ZFS for OS X' project. It was widely rumored that ZFS would at least be an option in the shipping version of Leopard, if not the default filesystem. Someone over at Sun was even crowing about this a few months before Leopard was released.

I'd say Sun looks favorably upon thi

Sun CEO Encourages Apple to Use ZFS

[this great guy (922511)]

I'd say Sun looks favorably upon this.

Of course they do. Sun CEO Jonathan Schwartz encourages Apple to use ZFS [sun.com] (direct from his blog): "As an example, Apple is including ZFS is in their upcoming "Leopard" OS X release. This is happening without any payment to Sun (that's how truly free software works). Under the license, we've waived all rights to sue them for any of the patents or copyright associated with ZFS. We've let Apple know we will use our patent portfolio to protect them and the Mac ZFS community from Net App. With or without a commercial relationship to Sun."

Re:Hmm

[by Anonymous Coward]

Well then, what does Paris Hilton think of this?

Re:Hmm

[johnslater (61055)]

Paris Hilton? Think?

Re:Linux?

[larry bagina (561269)]

It's not a technical problem preventing linux usage so much as a political problem and a license problem. Unless this convinces those zealots that 1) FUSE isn't good enough and 2) CDDL is FREE, it won't do jack shit for linux.

Re:Linux?

[zsau (266209)]

It's not a question of whether people thing CDDL is Free or not. There are "zealots" like Stallman who think that both GPL v2 and GPL v3 are free. But he would be the first to say you can't include GPL v3 code, like a future relicensed version of the Solaris kernel, in GPL v2 code, like the Linux kernel.

And I think most people will agree with you that Fuse isn't good enough. But at the moment, there are only two options: complete reimplementation from the ground up, and Fuse. Fuse is easiest.

Re:Linux?

[lakeland (218447)]

Actually, I see no reason why FUSE wouldn't be perfectly acceptable for zfs - at least until people start seriously considering zfs for root filesystems. FUSE is fast enough, stable enough, and flexible enough. We don't have X in the kernel, I don't see that we _need_ to have the filesystem in the kernel either.

Re:Linux?

[lokedhs (672255)]

The sources has already been available [opensolaris.org] under an open source license since ZFS came out.

Re:Total garbage - has no error result codes!

[_merlin (160982)]

Total garbage - has no error result codes! Always assumes all writes have no failures, so unplugging a firewire drive or a USB drive or eSata drive guaranteed to either kernel panic or otherwise crash the OS.

ZFS is designed to perform writes asynchronously. If the write should be able to complete, it returns success and then goes off to do it. It's a different way of thinking about a filesystem. You need to do a "zpool export" or something before you can unplug a detachable disk to avoid the panic when you unplug it. That's not a bug. It's by design.

The Finder itself is lied to.

No it isn't. You're just misunderstanding the semantics of ZFS.

This is such an amateurish implementation, I am shocked that the source was even offered.

No it isn't. It's just not a filesystem that's suitable for the masses. Average users cannot understand or manage an advanced storage pool system like ZFS. They're better off with filesystems that make sense to them, like HFS+, ext2 or NTFS.

Shame on Apple for funding this quality of work.

Shame on all the geeks for telling everyone that ZFS will solve all their problems. ZFS is great under certain circumstances. It does what it does very well, but it isn't a filesystem for the masses.

I will admit, a few years ago, DURING BOOT, linux had a similar design bug and all IDE writes during boot had no error codes returned. But this is different. This is 2008.

Just plain not reporting errors is a bug. ZFS asynchronous write semantics is intentional, although counter-intuitive, behaviour.

Re:

[Nomen Publicus (1150725)]

The design of ZFS is intended to ensure that the data on the disk is _always_ a valid file system. If a system panics when a ZFS file system is unexpectedly removed, that is a different issue.

Then, of course, checksumming everything does wonders to protect against bit rot and flaky cables.

Re:Port it to Linux

[flyingfsck (986395)]

You should reformat with XFS.

Re:

[Hucko (998827)]

I'm considering it, but your answer just says there is two different methodologies while claiming that one is better than the other. I am more wary of CDDL just as I am wary of Lucent's licence for Plan 9 (for sheer clever thinking, it would be my prefered OS -- discounting I haven't learnt how to use it effectively. Not as clever as the OS).