Get To Know Mach, the Kernel of Mac OS X 413
An anonymous reader writes "Linux is a kernel, not an operating system. So what is Mac OS X's kernel? The Mach microkernel. The debate around Monolithic (Linux) and Micro (Mach) kernels continues, and there is a great chapter online about the Mach system from the very good book 'Operating System Concepts'. Which design is better? I report, you decide." Warning: link is to a PDF.
MirrorDot (Score:4, Informative)
Complete Book reference (Score:5, Informative)
There are also free online chapters for FreeBSD and Nachos.
Link to Wiley's purchase page (given that we are
mach inject (Score:5, Informative)
Re:Monolithic (Score:5, Informative)
I can't believe people are modding you up for this.
The Linux kernel is monolithic. Linux modules do not run in user-mode. They are loaded into the kernel proper.
mkLinux was an Apple-sponsored effort to run Linux on Mach. The Linux kernel was modified to run in user-mode; it basically became an executable. In fact, you could run multiple instances of the same kernel (or different kernels) simultaneously.
Re:Monolithic (Score:5, Informative)
Linux the OS that is not an OS? (Score:4, Informative)
Google's definition of Linux [google.com]
I think you have more of a chance to start a discussion on that statement then you do in regards to which kernel is "better".
Re:Monolithic (Score:2, Informative)
> that linux is NOT monolithic - its quite modular. Monolithic
> translates to no modules, correct?
No: Both the modules and the rest of the kernel run in the same address space, so Linux is monolithic.
A microkernel approach puts some (most, for second-generation microkernels like L4) traditional kernel features into user space, where they cannot hurt the kernel directly, by overwriting memory.
Xnu, not mach (Score:5, Informative)
stop spreading the myth that Xnu is a microkernel
Re:Monolithic (Score:5, Informative)
a) running in kernel space, not user space
b) communicated with by predefined hooks, rather than a generic message passing interfacing.
That's why linux modules, which are superficially like elements of a microkernel, are not really like them at all.
OT: PDF link clicking extension (Score:2, Informative)
PDF Download [mozilla.org]
Re:Monolithic (Score:4, Informative)
basically all the interfaces are defined as symbols to the linker, and all interfaces are defined c-native.
the micro-kernels are meant to use message passing and more abstracted interfaces, as well as separate address spaces to ensure a bad module does not take down the entire kernel. Think of it like the modules run as only semi-privileged applications, handling their hardware and then giving control back to the micro-kernel which does as little as possible to arbitrate control and schedule between the subsystems and user-mode applications. Drivers are no longer fully privileged, and the entire user-space can be considered a subsystem of the kernel.
it's different, and kinda hard to design for, but i can't wait for hurd to release a linux compat layer.
mach vs posix (Score:5, Informative)
Given this compatibility effort, mach is not a fair comparison, either in hurd or osx, for comparing the merits and performance between that of monolithic and microkernel achitectures because so much extra stuff was added to a design never intended for posix. Something like QNX4 and later, designed both as a microkernel and for posix, or perhaps a pure mach system running applications designed specifically for mach, might be a more fare basis to compare the value of microkernel vs monolithic architectures.
Mach on hurd is easier to grasp and test since many of the lower level mach kernel services are still represented and usable there. Apple seems to be trying to eliminate visibility of as many of the lower level mach services from application developers as possible. Yet, there are still many things that can only be done in the mach kernel on osx or darwin (such as threads that can be cancelled on socket operations or sleeps). If one wanted a bsd/posix compliant environment, I think Apple would have been far better off starting from PPC/xBSD or Linux kernels, rather than trying to rope and rebuild mach to fit into something it was never originally designed for.
Re:Monolithic (Score:4, Informative)
Re:Monolithic (Score:5, Informative)
No, you're mixing terms.
So, the linux kernel supports kernel modules, and its design is to some degree "modular" (as any project that size would have to be), but noone would claim it to be a microkernel.
--Bruce Fields
Re:Mac OS X is Mach, but it is not a Microkernel (Score:2, Informative)
It uses the FreeBSD userland utilities, but not the kernel. You can't just compile them together. I mean, it's possible to take parts related to one, like the XYZ bootloader and use it to start the ZYX kernel, you can't just plug in modules or code. The ABIs and APIs are different.
In this case, MacOS doesn't even do that. They just use a lot of the user utilities and such.
Re:qnx does just fine with a u-kernel and message (Score:3, Informative)
Re:design is better, performance is worse (Score:3, Informative)
No, which is why Apple's XNU runs in one address space for the most part (I don't even know whether there are parts which don't), and most message passing has been reduced to plain function calls. They still have the design advantages of something which is conceptually built from different subsystems with clean interfaces though.
Mono = one (Score:3, Informative)
Personally, I wouldn't consider a modular kernel to be monolithic, because it can take many forms. It may have a signle form at any given time, but over any period of time, it may vary that form. If we are going to give the Linux kernel a "technical" description, then "polymorphic" would seem to be more accurate.
However, if we now use "polymorphic" to mean "modular", then there is absolutely no point in having the extra term. We're adding vocabulary without adding any information. So, is it possible to have a kernel with many forms that is NOT modular? Arguably no, because it is only by being able to add/remove kernel code that the kernel becomes polymorphic.
What about the usage of "monolithic" for anything that is a single layer of code, as opposed to a microkernel which is multi-layered with lots of communication between layers? I'm not convinced this is a useful definition. It attempts to identify kernel types by a specific implementation detail, rather than by the design. It would be as useful as trying to classify cats by eye-colour. The relationship isn't necessarily as rigid as all that.
Re:Monolithic (Score:1, Informative)
This is all true, but it's also true that xnu (the os x kernel) is monolithic as well. They've taken mach and the bsd kernel and put them both in the kernel proper. It's not like freebsd is running as some userland process (a la lites) in a typical microkernel fashion.
Re:Mac OS X is Mach, but it is not a Microkernel (Score:5, Informative)
Re:Mac OS X is Mach, but it is not a Microkernel (Score:2, Informative)
Mach Sucks (Score:4, Informative)
Mach is painfully slow. It's an old microkernel and it uses async IPC (to allow for passing messages over the network). This is slow because you have to do a ton of context switches and copy the message between address spaces.
L4 [l4ka.org], on the other hand, uses sync IPC. It has a bunch of neat optimizations, but the main reason why it's faster than Mach is that it doesn't have to copy messages. You send an IPC and it goes into the part of your VM space that L4 sets aside for IPC and then L4 does a quick context switch to the target task, it processes the IPC, and then you get your data back. (Simplifyed a ton).
So, microkernels rock. Mach sucks.
Re:Not a "compatibility layer" (Score:2, Informative)
One example is the unified buffer cache, which ties directly into the mach vm layer. Mach is certainly used for more than bootstrapping the bsd subsystem.
OS X's kernel is not Mach. (Score:5, Informative)
Amit Singh has a well-written page about XNU: http://www.kernelthread.com/mac/osx/arch_xnu.html
NOT A MICROKERNEL (Score:5, Informative)
It uses Mach and BSD in THE SAME ADDRESS SPACE. As such, it's basically as monolithic as it gets. It just happens to incoporate Mach in the kernel space and uses it for threads and IPC.
Anyone who takes 10 minutes to look at the Darwin documentation would know this.
I wish
Re:mach vs posix (Score:3, Informative)
The level of Mach-iness of OS X is another good question, though. From what I gather, OS X looks more like a monolithic BSD kernel ported to a Mach/PPC architecture (where instead of targeting the PPC architecture, OS X's BSD kernel targets the higher level Mach abstractions), rather than a microkernel operating system of the type one would normally expect to find running on top of Mach.
Re:OT: PDF link clicking extension (Score:2, Informative)
* Tools -> Options
* Downloads -> Plug-ins
* Uncheck PDF Extension -> Click Ok
* Click [Ok] again to Exit Options Menu
* Find a link to a PDF and click on it
* A dialog box will pop-up asking what to do with it
* Verify that Open With reads "Acrobat Reader" (or "AcroExch" in the newer versions of Acrobat Reader)
* Check "Do this automatically..." -> Click OK
PDF files will now spawn Acrobat Reader out side of Firefox. This is also helpful on older PC's that slow down to a crawl when opening PDF's within the browser.
Apple does NOT use the MACH kernel. (Score:5, Informative)
It uses code FROM Mach, but it is not Mach and it is not a Microkernel. NT (NT 4.0, NT 5.0 (win2k), NT 5.1 (WinXP) does not use a Microkerenl either.
The only OS that I know of that actually uses a Microkernel is GNU/Hurd.
The OS X kernel, called XNU, is a mixture of *BSD kernel code and Mach kernel code.
Yes, yes, there was a ancient debate between Linus and that other guy about Micro- vs Macro-kernels, and guess what, Linus was right.
Apple does NOT use a microkernel. It does not use Mach. It is BASED on Mach. It would be considured a kludge compared to Linux or FreeBSD, but it works out fine.
Similar to how Mustangs are based on Ford Falcons and Granadas from the late 70's and early 80's. Those cars were as much as a failure as Mach, however the Mustang is flashy and many people desire it. So go figure.
Linux IS an OS, both historically and now (Score:5, Informative)
That is a true statement for the GNU project, but not for all of Linuxdom. Linux (the OS) was not started by the GNU folks. It was started as a separate project and incorporated items from the FSF (and BSD, etc.) into its release. From the beginning the whole OS has always been called "Linux" (search Google Groups for "linux 0.11 author:torvalds" and click on the "Linux information sheet" for an example of this).
Yes, RMS prefers to call the OS GNU/Linux, but that's because he's seeing things from the perspective of the GNU project incorporating the Linux kernel into their work. The rest of Linuxdom see Linux as the name of both the OS and the kernel, and qualify the name using the phrase "the Linux kernel" as an easy way to differentiate between the two.
So, the opening statement in the OS X story is false: Linux is an OS, and is used as such by folks every day. This is the reality of the situation, and it is, at best, wishful thinking on the part of folks who claim it is not to say otherwise.
Re:Mac OS X is Mach, but it is not a Microkernel (Score:5, Informative)
1. Mach is not a complete kernel. It requires someone to implement the areas which the Mach group were not researching. This has traditionally been done by compiling against BSD 4.3.
2. Mac OS X updated to the FreeBSD kernel instead of BSD 4.3 to gain a more modern kernel design with better hardware support.
3. OS 9 "Classic" is not a microkernel server, but rather a technology that Apple calls "Blue Box". Blue Box is a hardware virtualizer like VMWare that is capable of communicating directly with the OS X desktop. Using this communication, the OS 9 desktop is made to disappear, making the application appear to run on the OS X desktop.
4. The combination of Mach and FreeBSD is called "XNU" by Apple. The complete os is called Darwin, and the commercial variety with the Next and Mac APIs is called "Mac OS X".
More Info:
Mach Kernel [cmu.edu]
Wikipedia: Mach [wikipedia.org]
Wikipedia: XNU [wikipedia.org]
Blue Box info [kernelthread.com]
MacOS X is *not* a microkernel architecture (Score:4, Informative)
Just as an example, on the new MacMini hardware, sound level control is done in kernelspace (since HW doesn't support that anymore)! Whereas the LinuxPPC developers refuse to do things like this in kernelspace.
Actually in Linux many things are pushed out to userspace (think udev), making it much more microkernel-like than MacOSX.
(Not that Apple-Fanboys would understand anything of that)
Re:Monolithic (Score:5, Informative)
I can. Maybe you're too young to remember when the term monolithic was commonly used to describe a kernel which, instead of using loadable modules, was linked as a single binary image. This was, and is, a valid use of the word. Here's an example [linuxjournal.com].
The first time I heard someone say that Solaris is monolithic, I thought that they were saying that, like SysVR3, it didn't support loadable modules. I didn't realize that, with the development of microkernels, the term "monolithic kernel" had started to be used in a different context.
Re:Linux the OS that is not an OS? (Score:5, Informative)
Not so. Here a posting from Linux Torvalds about Linux -- from the beginning the term was used as both the name for the kernel and the whole OS:
LINUX INFORMATION SHEET
(last updated 13 Dec 1991)
1. WHAT IS LINUX 0.11
LINUX 0.11 is a freely distributable UNIX clone. It implements a
subset of System V and POSIX functionality. LINUX has been written
from scratch, and therefore does not contain any AT&T or MINIX
code--not in the kernel, the compiler, the utilities, or the libraries.
For this reason it can be made available with the complete source code
via anonymous FTP. LINUX runs only on 386/486 AT-bus machines; porting
to non-Intel architectures is likely to be difficult, as the kernel
makes extensive use of 386 memory management and task primitives.
[...]
2. LINUX features
- System call compatible with a subset of System V and POSIX
- Full multiprogramming (multiple programs can run at once)
- Memory paging with copy-on-write
- Demand loading of executables
- Page sharing of executables
- ANSI compliant C compiler (gcc)
- A complete set of compiler writing tools
(bison as yacc-replacement, flex as lex replacement)
- The GNU 'Bourne again' shell (bash)
- Micro emacs
- most utilities you need for development
(cat, cp, kermit, ls, make, etc.)
- Over 200 library procedures (atoi, fork, malloc, read, stdio, etc.)
- Currently 4 national keyboards: Finnish/US/German/French
- Full source code (in C) for the OS is freely distributable
- [...]
You are very confused. (Score:4, Informative)
And second, no BSD based kernel forces you to use modules. Have you actually tried any BSD? Modules are entirely optional, just like linux. In fact, openbsd's kernel only has support for modules, but nothing is actually compiled as a module, and using modules is unsupported.
Re:qnx does just fine with a u-kernel and message (Score:5, Informative)
So does Mach, and it's slow. I've never seen real-world measures to suggest that QnX is fast. All we know is that the performance of the OS itself is good, and that's a VERY DIFFERENT measure.
The slow performance is due to a number of problems:
1) not all MMU's are really suited to this task. Many are slower to set up than just copying the memory around. Sun found this to be at around 5k, below that, it was faster to just copy memory physically.
2) MMUs/VM are based on pages, 2 or 4k typically. Thus passing in a single 32-bit int parameter causes big page hits. You can tune this out, but it's still annoying.
3) Each copy takes TWO context switches - one to switch into the kernel to copy the memory across ports, another back out to the called program. This means that even the simplest "system calls" are twice as slow as under a monokernel, AT BEST.
4) Additionally the data has to be examined to see if it contains ports being passed around, and if so, they have to be translated because the port codes are private to a program (and thus different in the other one).
5) Using mapped memory ignores all the hardware specific solutions to these problems, many of which are built into modern processors.
It's exactly the sort of one-size-fits-all solution that you'd expect from a research project. One that doesn't work in the real world. One that should have been replaced, and was in L4, Spring, etc.
For instance, Spring included three different IPC systems, each tuned to certain types of data, each used in different ways on different CPUs. The "fast-path" used a half-switch into the kernel by mapping off registers, allowing IPC to degenerate into register passing largely identical to a procedure call. As long as the message fit within the limitations -- 8 registers, no port identifiers, etc. -- it was faster than a traditional Unix trap. These limitations seem serious, but were in fact used for 80% of calls and 60% of returns (you often say "getDiskSector(integer value)" which could fit into the fast-path, and get back 2k of data which wouldn't).
Maury
Re:Monolithic (Score:2, Informative)
The problem with a fully MicroKernel is that its very slow because of all the context switching that has to go on between userland/and kernel land to do what is essentially kernel functionality. Apple solved this by making XNU not act as a microkernel for things such as the BSD layer.
The result is a Kernel that is less prone to panics. In Linux a bad KLM would certainly panic the kernel because it runs in the same address space as the kernel. In OS X a bad Kext would just die like anyother user space program.
In xnu, KExt's are not userland processes. They are kernel extensions, just like on most other *nixes. I don't know where people get these patently false notions.
From the documentation...
Kernel Extension Overview
As discussed in the chapter "Kernel Architecture Overview", Mac OS X provides a kernel extension mechanism as a means of allowing dynamic loading of code into the kernel, without the need to recompile or relink. Because these kernel extensions (KEXTs) provide both modularity and dynamic loadability, they are a natural choice for any relatively self-contained service that requires access to internal kernel interfaces.
Because KEXTs run in supervisor mode in the kernel's address space, they are also harder to write and debug than user-level modules, and must conform to strict guidelines. Further, kernel resources are wired (permanently resident in memory) and are thus more costly to use than resources in a user-space task of equivalent functionality.
In addition, although memory protection keeps applications from crashing the system, no such safeguards are in place inside the kernel. A badly behaved kernel extension in Mac OS X can cause as much trouble as a badly behaved application or extension could in Mac OS 9.
Bugs in KEXTs can have far more severe consequences than bugs in user-level code. For example, a memory access error in a user application can, at worst, cause that application to crash. In contrast, a memory access error in a KEXT causes a kernel panic, crashing the operating system.
HURD on Mach is done. (Score:3, Informative)
While many devices are not supported, and the performance is not good, HURD/Mach is feature complete (and most of Debian runs on it, IIRC).
Because the performance was bad, the new HURD effort focuses on reimplementing on L4. Perhaps with a faster microkernel, Apple could have avoided the kludge of an in-kernel BSD peer.
If I am reading correctly, Mach is responsible for IPC in the Apple kernel. It would be interesting to see benchmarks of SYSV system calls to semaphores, queues, and shared memory (and perhaps even basic signal handling) under Linux and the Apple kernel on the same hardware if those are entirely handled by Mach.
The world is not i386. (Score:2, Informative)
Re:Apple does NOT use the MACH kernel. (Score:3, Informative)
As for winning or losing, you can see for yourself on Google Groups [google.ca] that it was not about winning, it was a discussion on the merits of both.
They have more in common than you may think... (Score:5, Informative)
XNU is essentially a monolithic kernel, much like Linux. The real differences, in my opinion, lie in the IOKit object oriented driver API, whereas Linux has no real driver API and drivers have complete access to all kernel functions as drivers are simply kernel modules.
Re:qnx does just fine with a u-kernel and message (Score:3, Informative)
The "at best" is assuming that the forgoing issues don't cause things like cache faults. Passing parameters in registers won't. Thus the performance really can be MUCH worse than twice even in the case of minimal calls.
But even in that case the real-world performance of Mach is, in fact, much worse that twice as slow. I believe it was Chan that did all the big measurement runs, and - going on memory here - BSD on a 50MHz 486DX took 21 usec per syscall, and Mach 3 on the same machine took 114. I may have the number for BSD wrong, that might be the L4 number, BSD might be 9 usec.
The other case is, in fact, even worse. This is the case where the system is constructed of a number of cleanly separated servers running as Mach tasks. In this case a system call may spawn off a series of calls, consider a page fault in the VM for instance. In this case you might end up with a chain of 5 IPC's causing traps. And each one of them is 5-15 times as slow. This is real-world noticable, even for I/O.
One of the papers I have on L4 shows theoretical peak and sustained network throughput for IP on Mach, L4 and BSD. Mach had about 1/10th the performance IIRC. So that "best case" is pretty rare.
Maury
Re:Monolithic (Score:3, Informative)
You can't download the binary of a driver, tell the kernel to load it, and expect it to work unless the person who compiled just so happened to have the exact same version info, and by some miracle the same compile options.
Yes, distros like RedHat and SuSE do have binary drivers for download, BUT ONLY if you stay with the stock kernel.
Just because you can "load modules" doesn't mean you are suddenly a microkernel. God it's like monolithic has become a swear word, it's a perfectly valid design.
What kind of crack have you been smoking... (Score:4, Informative)
To clear up any confusion: No *BSD uses a microkernel. (The only part of OS-X that is *BSD is the userland, which is derived from FreeBSD). The *BSD's are basically in the same classification as Linux/Solaris/HP-UX or any other UNIX or *NIX clone. Which means: all the *BSD's are monolithic in nature, with some modular abilities added on in recent years. Like Linux, the *BSD's can load a kernel-module upon request (either during boot, or upon superuser-request). These modules can also be compiled into the kernel itself (which is sometimes a good idea, as it saves a small amount of memory and improves performance).
Anyhoo, back to the original topic: The MACH microkernel. Apple's implementation is excellent these days, but it definitely went through its struggles (which is one reason why we continue to see major speed improvements with new versions of OSX, even on older hardware). Creating a monolithic kernel is difficult enough, but to create a micro like MACH, and do it properly... that takes serious skills. Mad props, Apple engineers.
Re:Linux the OS that is not an OS? (Score:1, Informative)
It wasn't until glibc 2.0 came out in the late 1990s that people started switching to GNU Libc. If you look at the glibc webpage, [gnu.org] it is copyright 1996 - 2004... What do you think Linux used before 1996?
So, to be a little bit technical, Linux used to be not just a kernel, but a kernel, C library, and boot loader. Since then 2 of those have been replaced by externally developed tools.
As an aside, there are a few more userland programs that might be considered part of "linux", including: module-init-tools, iptables, hotplugd, udev, etc. These are not GNU tools, and were explicitly written to extend the Linux kernel.
Re:Here's is why , Moron. (Score:1, Informative)
Actually, the Mac kernel is completely contained in Darwin and is therefore open source [apple.com]. You are more than welcome to download it, compile it (even on x86), modify it, etc. without any risk of getting sued. No, you can't resale it and the license is more restrictive than other OS licenses, but most of what you said is simply false and absurd.
Re:The problem with Mach (Score:3, Informative)
Actually, Apple's kernel is a collection of parts from BSD, Mach, and IOKit. [freebsd.org] It's a monolithic kernel like Mach 2.5, not a microkernel like Mach 3.0, although some parts from the Mach 3.0 code base were supposedly used.
IOkit is written in the "embedded subset" of C++, an idea from 1999 that never caught on. Drivers are loadable kernel modules, as with Linux, but the structure is quite different.
Any driver can crash the kernel. It's not a microkernel at all.
Re:design AND performance better with safe kernel (Score:3, Informative)
<URL:http://channel9.msdn.com/ShowPost.aspx