Torvalds Says It's No Picnic To Become Major Linux Coder

Jack Spine writes "Linus Torvalds has given an interview to ZDNet.co.uk about the trials and tribulations of becoming a Linux kernel developer. 'Torvalds said that, while it is relatively easy for coders and organisations to contribute small patches, the contribution of large patches, developed in isolation, could lead to both new and established contributors becoming frustrated. "It's definitely not easy to become a 'big contributor'," wrote Torvalds. "For one thing, the kernel is quite complex and big, and it inevitably simply takes time to learn all the rules — not just for the code, but for how the whole development environment works. Similarly, for a new developer, it will take time before people start recognising the name and start trusting the developer to do the right things.""

I gave up a few times

I considered, and once tried, pushing a file system into Linux. Unfortunately the fs does not have the right coding style and a few other things which make it hard to put into mainstream. Instead it just sits independently as a big patch which is pretty easy to apply by running a simple script.

This suites everyone that uses it pretty fine, except for the purist "it's got to be in the mainline" folks. Realists just pull it from a public cvs and apply it with minimal effort.

Although I might consider mainlining it again, for the moment the effort just is not worth it. The current model is workable for those that use it.

In other news...

Kernel development is hard! Film at 11.

This goes for every OS/FOSS project out there.

And considering the widespread usage plus amount of people that rely on the Linux kernel to be stable and not explode in a horrible firestorm I can certainly understand that Linux kernel development requires a Stalinist approach.

Re:This goes for every OS/FOSS project out there.

I am sure this goes for closed source as well. If the rookie comes in, he will also have to prove himself. The same goes for every other aspect in life.

If are new into a group, it is to be expected that you have to prove yourself. Whether this is with your local gang, a family you are married into or a bunch of coders is irrelevant.

Even Neo had to prove himself first. Each group will have its own rules and speed of how fast you are accepted. A group of drunk people in a bar might have a lower standard, but if you do not fit in, they will not accept you and will not listen to your sugestions, no matter how wise they are. (Stop drinking? Why? This is FREE BEER)

Learning the rules

"...and it inevitably simply takes time to learn all the rules..."

1. Linus is always right
2. If in doubt, see rule #1

Re:Learning the rules

Yes, but trick is that since you can't constantly be bugging Linus for all the answers, you have to know what his opinion is without asking him. That's the tough part.

Allegiances

What happens if:

1. Batman is more or less responsible for a big chunk of the kernel, e.g. the scheduler.

2. Torvalds knows Batman, and knows that Batman is employed by Redhat to work on the scheduler.

3. The Joker writes a new improved scheduler which has the potential to replace the old one.

Now, how does Torvalds react? It would be hard to tell Batman that he's no longer in charge of the scheduler. Batman's job might be on the line - why would Redhat keep paying Batman if he suddenly had a lot less work to do? Maybe Torvalds met Batman a few times and had a beer with him, making it even harder to replace his work because it becomes personal. Torvalds could harm Batman's career.

Surely this makes it hard to become a big new contributor? All the existing contributors already know eachother and they won't want to dump eachother's work.

Am I right or am I right?

Re:Allegiances

Am I right or am I right?

You're a tautology.

But let's take your unproven hypothetical as given for a second. If these sorts of decisions are being made, which provide technically inferior solutions for the Linux kernel.. then over time it will become obsolete.

But way before then we'll all be using the nuLinux kernel which has all of "The Joker"s fancy code.

In other words, F/OSS can take care of itself; we're just the dumb monkeys hitting random keys.

Re:Allegiances

3. The Joker writes a new improved scheduler which has the potential to replace the old one.

when (process.wantsToRun) {
retort(process, "Why so serious!?");
kill(process);
}

Very efficient. I like it.

Andy's revenge

For one thing, the kernel is quite complex and big

If that's the problem, wouldn't it be easier to work on it if it was a microkernel?

Re:Andy's revenge

Seriously though, the problem with microkernels is that they (in theory) help the system cope with mistakes but they don't help prevent mistakes in the first place. Each component in a microkernel is isolated from others using memory protection, but they can still corrupt themselves or crash themselves.

There's very few parts of the kernel that actually need pointer arithmetic, unsafe casts, or for that matter need to operate particularly quickly. You don't have to believe me just look at the code. Open up some random source files from the kernel and look for pointer math, unsafe casts. Figuring out what locks are held when is harder, but can be done (performance being more important when locks are held).

Microkernels are solving the wrong problem. They should be focussed on preventing the errors in the first place not on recovering from them. So, a 'safe kernel' that is mostly written in a language that prevents errors, such as Limbo/Dis or for that matter Java or C#. That would be much easier to work on and an improvement over Linux style kernels.

Re:Andy's revenge

Long answer: Fuck no.

    (From Stephen Fry)

Wisdumb

It's no picnic to become a major anything. Major people are people who have differentiated themselves from minor people. The means by which they've done that is to do something that's more difficult, which the other people cannot do. This is a tautology masquerading as wisdom.

Re:huh

Nope.
It is hard because.
1. Programming is hard.
2. System's programming is even harder.
3. Kernel code is mission critical code which is really hard.
4. When you are the new person it takes time before people trust that you know what you are doing.

In other words it is just like everything else. The difference is that if you want to make changes to your Kernel you can. If you want to put up a site with your patches you can.
If you want your code adopted in the "official" kernel you have to play by the rules and write good code.
So it works exactly as it should and really can not work any other way.

Re:huh

One additional point: don't try to take over a whole subsystem in one rewrite. Contribute small patches that are easily reviewed to get your feet wet and get noticed. Then, as you're better known and respected within the community, start scaling up your contributions.

It works the same way in any open source community. The new guy who rewrites half the code all at once isn't going to get a review of his code. Show that you can do the small changes right first.

Actually, it works this way in almost any cooperative group. You don't show up to your first meeting at Kiwanis, the Jaycees, or the Lions and start making resolutions. You don't sit in on drums once for a band and start telling them how to write songs. The US Senate even has a rotating term cycle so that there are always Senators with more experience to get the junior Senators acquainted with how things work.

People who think they should suddenly be in charge of a large portion of an established organization they've just joined are showing signs of detachment from society or megalomania. If you've never contributed anything worthwhile, you're nobody special compared to the people who have been doing the work. Don't expect to be a big part of a group without being a small part first. Some people move up the ranks faster than others through skill and hard work, but everyone pays some dues.

Re:huh

And the honest truth is that.
Unless you are some kind of super prodigy or have years of experience writing systems code odds are your complete subsystem rewrite is total junk.
It takes time to be good at anything.

Re:huh

I just started a new job 2 weeks ago. I haven't touched any code other than two trivial patches to some HTML. I expect it'll be another 2 weeks before I touch any actual code, and it'll be a few months before I'll touch anything that customers rely on. This is the same process that happens everywhere, the difference being that in the Linux kernel it's more open and ability based.

Re:huh

Mod parent down. I don't recognize him, therefore he can't know what he's talking about.

Still easier than coding the Windows Kernel

Qualify "luck". It seems to me that any large distributed development effort is going to require some sort of process - the anarchic development model isn't terribly successful.

With that in mind, developing the Windows kernel requires you to be employed by Microsoft etc, whereas developing for the Linux kernel just requires you to follow some established open processes.

What's the problem with that?

Re:Still easier than coding the Windows Kernel

At Microsoft, working on the kernel pays as a fulltime job

They pay monkeys to bash keys nowadays?

Re:huh

Bullshit. You can download the source code, make any changes you want and publish your own version without restrictions. That's the definition of open source, so the Linux kernel most definitely qualifies. The fact that it's hard for you to get your changes into Linus' kernel tree has nothing to do with it.

Re:huh

While I don't agree with your current flamebait mod (you bring up a very common criticism of open source) I do think you're wrong.

Nothing in open source has ever guaranteed that you get to contribute directly to a specific project. When a specific group of developers is maintaining a release (Linus et al, in this case) it's absolutely up to them what code gets in and what doesn't, and who they will accept contributions from. What open source guarantees is that when they make that release, you're free to take the source code they've created and modify it in any way you choose. You're free to fork the project and maintain releases just as strictly as they did, or open it up to all newcomers.

I think you'll find, though, that opening it up to any unknown person right off the bat will trash your project pretty quickly. The problem is, there are hugely varying ideas of what constitutes "correct" code and architecture, and it's just a fact that it takes time to prove that you understand what that means in terms of a large project such as this.

Re:huh

I disagree. I think you have the wrong idea about what "open source" means.

Open source pertains to the code base, not to any particular repository of the code. You are quite free to read, modify, and redistribute (with modifications) the code. You cannot compell Linus to incorporate your changes into his version, any more than he can compell you to revert a change from your version.

That Linus has a widely-respected "official" version is a moot point. It really just means he has an audience for his product (i.e. the version of the code he/his team host), and you may not have one for yours (a modified version you create but which isn't accepted as part of his product).

Much like free (as in speech) speech, open source doesn't guarantee you that anyone will listen (where "listen" in this context means "run your version of the code").

As for MS -- well, getting a job at MS isn't the same as getting a job that lets you make major modifications to the Windows kernel. I'd say the sitaution with regard to the high-profile products (i.e. Windows on oen hand, the "official" Linux kernel on the other) is about the same. The difference is that you can't modify Windows without being part of the official team. Not to distribute (even if you can find an audience for your work). Not even for your own personal use. That's the difference between open and closed source.

Re:It can't be THAT hard

he's heard of 'free'! quick! someone offer him a job!

Re:It can't be THAT hard

at mozilla