Are All Bugs Shallow? Questioning Linus's Law 596
root777 writes to point out a provocative blog piece by a Microsoft program manager, questioning one of the almost unquestioned tenets of open source development: that given enough eyeballs, all bugs are shallow. Are they? Shawn Hernan looks at DARPA's Sardonix experiment and the Coverity static-analysis bug discovery program in open source projects to conclude that perhaps not enough eyeballs are in evidence. Is he wrong? Why? "Most members of the periphery [those outside the core developer group] do not have the necessary debugging skills ... the vast numbers of 'eyeballs' apparently do not exist. ... [C]ode review is hardly all that makes software more secure. Getting software right is very, very difficult. ... Code review alone is not sufficient. Testing is not sufficient. Tools are not sufficient. Features are not sufficient. None of the things we do in isolation are sufficient. To get software truly correct, especially to get it secure, you have to address all phases of the software development lifecycle, and integrate security into the day-to-day activities."
Re:Bugs are an error in the... (Score:5, Informative)
Bugs are an error in the process, not the code. If you find a bug, you need to find the process error that allowed that bug to occur.
Agreed!
I read, with interest, the referenced article. I was expecting FUD - but I didn't find much, until I reached the Conclusion.
eg.
The many eyeballs argument is neat, tidy, compelling, and wrong.
The article starts with
Eric S. Raymond wrote [catb.org], “Given enough eyeballs, all bugs are shallow.” He calls this Linus’ law.
and then attempts to refute. Fair enough. Except - the link leads to The Cathedral And The Bazaar - where I cannot find the quote... Hmmm
Now this might be relevant if the "many eyes" routine was the only form of audit used in GNU/Linux - but [nsa.gov] is not [coverity.com] the only [google.com] form of review/audit used. I'm sure other, more knowledgable posters will be able to provide more evidence than I could find in a quick search.
I call FUD
Re:Bugs are an error in the... (Score:3, Informative)
* Ring 1 or higher code being able to write to Ring 0 locations.
* Administrative users necessary to run most things (MS software or otherwise).
* Proprietary networking.
* Lack of regression testing (LAND should just never have happened).
There's 5, who wants to take up the mantle from there.
Re:Bugs are an error in the... (Score:5, Informative)
A ridiculous amount of the linux kernel code is written by programmers paid by IBM, Intel, RedHat, etc.
Someone pays. I'm just glad it isn't me.
Re:Bugs are an error in the... (Score:2, Informative)
and then attempts to refute. Fair enough. Except - the link leads to The Cathedral And The Bazaar - where I cannot find the quote... Hmmm
It's on this page: http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html [catb.org]
Right after point #8, about halfway down.
Re:Bugs are an error in the... (Score:2, Informative)
http://marc.info/?l=openssl-dev&m=114651085826293&w=2
And so MD_Update(&m,buf,j); /* purify complains */ was commented out.
Re:Bugs are an error in the... (Score:1, Informative)
"File Locked rather than writeable by administrator for upgrade purposes"
Pure nonsense - unix behaves the same and is mearly semanticly different. Removing a file thats in use mearly unlinks it from the directory index its no different than renaming and 'deleting' later which can be done on NT based systems. Crap like this scares me anyway - I don't concider it to be a feature. The transactional kernel interface for file system and configuration modification in recent versions of windows is extremely cool - nothing like it is supported on other platforms of which I'm aware.
"Ring 1 or higher code being able to write to Ring 0 locations"
This is nonsense - Ring 1 does not exist on NT kernels.
"Proprietary networking."
Really? Last I looked MS has a standardized bsd style RFC defined socket interface for IPv4 and IPv6. You mean networking in terms of RPC, network file protocols? SMB? Can you be a little more vauge? What open "non propritary" protocols that didn't suck do you think existed when early versions of windows were being produced?
"Lack of regression testing (LAND should just never have happened)."
Agreed.
Competently done it'd just be propaganda (Score:4, Informative)
He reports Coverity's results on open source software
... but doesn't report Coverity's results on Microsoft's software.
He reports that Coverity scanned 280 open-source projects
...but doesn't report that only 180 of those have "active developer support".
He can't be bothered to present any data at all on the distribution of the reported or corrected defects — how many are in nethack or aalib or that long-abandoned "flash-based photo album generator"?
He doesn't, for instance, mention that Samba and several others have no defects Coverity can discover. None.
Vim has none. X.org has ... three. All of KDE, nearly five million lines of code, has ... ninety. glibc has none.
There have been MySQL and PostgreSQL and Berkeley DB versions with none. His bioblurb says he's "currently working to ensure that Microsoft SQL Server is secure". That's interesting. You mean it isn't, now? How many defects can Coverity find in SQL Server?
TFA is a nauseating pile of sneers and aspersions ("Hope is not a security strategy"?) built on a very carefully selected and very few facts. "No one is doing auditing" he quotes. "No one is doing auditing" and reporting it to some self-styled central authority almost no one ever heard of is what's true, but telling the truth isn't what he's doing here. He's a "Program Manager", and he works for Microsoft.
Re:Bugs are an error in the... (Score:3, Informative)
Creating and then enforcing standards and policies with respect
to source code and development process is not going to help if the whole thing is broken as
designed.
I was thinking of the irony of an MS project manager lecturing the Linux kernel devs on "bugginess."
Re:Bugs are an error in the... (Score:4, Informative)
To be fair to Microsoft this is no longer true. UAC asks the user if they wish to elevates privileges when an app does something unsafe. Vista took a lot of flak when UAC appeared (including from myself) but it did force user land applications to stop abusing the registry (e.g opening HKLM with read/write permissions), writing random files to random locations on disk and other unnecessary operations. The consequence is apps written / patched in the last 3 years run pretty cleanly and if they don't, you get the UAC popup. In practice it's little different from what happens in Ubuntu or OS X in similar circumstances.
Re:Bugs are an error in the... (Score:2, Informative)
Mr Graphic Designer Man: "Linux still doesn't do proper color management." Mr FOSS Man: "Writing unfree software is immoral!"
You can have more than decent color management on Linux and/or with FOSS software, actually. I bought a cheap calibration hardware that comes in Pro and non-Pro variants. I opted for non-Pro. The difference between the two is software. With ArgyllCMS, you can actually get better results using the non-Pro device than with whatever software comes with the Pro version. And that's without any compiling or patching or any kind of fiddling most 'normal' graphic designers find too difficult to even attempt. Not just that. Should I get a new device (at least one of the well-known brands), it would most likely work with the same software without any need for a change in work flow.
So, while I would maintain that it's not quite right to write software that you cannot share with your friends or modify if you know how, I would also like to point out how many anti-open-source arguments like the one above have been obsoleted by maturing open-source projects.
Re:Bugs are an error in the... (Score:4, Informative)
Plan9 is in the Unix family, one secuirty alert in 15 years
Re:Perfect code may not be perfect.. (Score:5, Informative)
Sorry, that's totally wrong. The Airbus FBW systems do allow reversion back to "just do what the damn human says". However, in the situation that aircraft was in, if it were a 1972 manufactured Boeing 747 with the same fault (no airspeed indication, inside a storm, in a flight regime where stall speed and maximum mach number are very close together) it is likely that the end result would have been the same.
Incidentally, how the A320 allows human handling of exception was very well demonstrated by the United flight that ended up in the Hudson - in which no lives were lost despite a very difficult situation.
Re:Most Difficult Bug for Me (Score:3, Informative)
Apparently some system that I didn't have source code to was interpreting files based on the prefix of their name, and I thought I was following convention. Turns out, if the file wasn't of a particular format, it would hang. I didn't realize that by following convention, i was causing a bug, so I had to rename the data object file to not include the special prefix.. with was do_ when normally all data objects would simply be prefixed with do .