Why Your Software Project Is Failing 119
An anonymous reader writes: At OSCON this year, Red Hat's Tom Callaway gave a talk entitled "This is Why You Fail: The Avoidable Mistakes Open Source Projects STILL Make." In 2009, Callaway was starting to work on the Chromium project—and to say it wasn't a pleasant experience was the biggest understatement Callaway made in his talk. Callaway said he likes challenges, but he felt buried by the project, and reached a point where he thought he should just quit his work. (Callaway said it's important to note that Chromium's code is not bad code; it's just a lot of code and a lot of code that Google didn't write.) This was making Callaway really frustrated, and people wanted to know what was upsetting him. Callaway wanted to be able to better explain his frustration, so he crafted this list which he called his "Points of Fail."
a little more proof reading please (Score:4, Funny)
"he should jut quite his work"
not a lot, just a little
Re: a little more proof reading please (Score:4, Insightful)
Re: (Score:3)
Re: a little more proof reading please (Score:5, Funny)
This whole situation makes me sic.
Re: (Score:2)
If I had a Bitcoin for every time I've seen the quiet/quite error, I'd be rich.
But quit/quite? This is the first time I see that. That's how bad the editors are on this website.
Re: a little more proof reading please (Score:5, Informative)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
i found it quite amusing, quit using the grammatical issue as a club.
Re: (Score:2)
If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?
Re: (Score:2)
Dear coward
Sometimes the latter, but sometimes, even if you were to follow all his points, there's just not enough interest in the project to make it sustainable. It all sounds so easy coming from a developer who works at one of the biggest open source companies there is that can devote enough resources to a project to make it successful. Try asking some independent developers of a successful project what they attribute their success to and they may have a different list altogether.
He hasn't always worked for Red Hat. They are a good list of rules to use if you want to see where your project could be improved. I suspect you have some obscure definition of success, and/or missed the implicit definition of FAIL.
Try asking independent developers for a list of general reasons why projects fail to get wider usage, grow, and last - I have (that's an old list), the resulting list is pretty much the same, only the scores seem to change.
Re:self-serving list (Score:5, Insightful)
Re: (Score:2)
>not because they want to be project managers. But if they don't manage the project it fails.
This. Creating something is fun. Managing is not. But, you HAVE to manage it, or it becomes some ridiculous unwieldy sword made of butter and you can't explain why it's made of butter.
Re: (Score:2)
If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?
Everything is obvious when you understand it?
If common sense was common... would people still say patently stupid things like "you can lead a horse to water but you can't make it drink" (want a bet?), "doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain - in some conditions - it's the broad brush of stupid, though some horses are smarter than others), "you'll catch more flies with honey than with vinegar" (try that with a fly trap).
I could go o
Re: (Score:2)
"you can lead a horse to water but you can't make it drink (want a bet?)"
Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.
"doesn't have the horse sense to stay out of the rain"
(clearly never owned horses, they will seek shelter from rain -[...]"
"Horse sense" is a synonym for "good sense" or "sound judgment". The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense. Y
Re: (Score:2)
"you can lead a horse to water but you can't make it drink (want a bet?)"
Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.
The limitations of your experience define the frame. Most animals like salt - but feel free to call PETA if I put a tiny dab inside a horse's lip.
If that horse has colic I will be cruel in order to get that horse to drink. The crueltry is relative. If you did think that unreasonably cruel to do to a horse with colic then you'd be deserving of a horse whipping.
"doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain -[...]"
Careful you don't hurt your back lugging those goal posts around.
"Horse sense" is a synonym for "good sense" or "sound judgment".
The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense.
[golf clap] Such a wit(ling).
...means exactly or nearly the same". In your desperate bounds from bank to bank you pass over the obvious. WILL != do.
I disagree with some of these points (Score:1)
What's wrong with open sourcing previously closed source projects?
I guess linux fails since Linus wrote his own source control for it.
Re: (Score:3)
It depends a lot on the codebase. Codebases tend to accumulate cruft. Having people refactor them because their requirements are different to yours can help, as can having a project developed without key product ship dates as the driving force. The bigger barrier is culture though. It's really hard to have a group of developers that have been working on a project for 10 years in private move to developing in public. In the list, he actually gives different numbers of fail points, more for projects that
Re: (Score:2)
Most of the version control commit messages I've seen do not describe at all the reason and purpose of the changes. No, VCS are no substitute for changelogs.
Re: (Score:2)
Re: (Score:2)
They won't. The release manager has to. Right now, at work, that role is mine, so I, I have to.
Re: (Score:2)
The one that I disagree with is 'Your source builds using something that isn't GNU Make [ +10 points of FAIL ]'. I disagree for two reasons. The first is that it implies using GNU make features, which likely means that you're conflating building and build configuration (which should gain some fail points). The projects that I most enjoy hacking on use CMake and Ninja for building by default (CMake can also emit POSIX Makefiles that GNU Make can use, but I take his point to mean that gmake is the only command you need to build, so the CMake dependency would be a problem). LLVM still more or less maintains two build systems, though the autoconf + gmake one is slowly being removed in favour of the CMake one. If I make a small change, it takes Ninja less time to rebuild it than it takes gmake to work out that it has nothing to do if I don't make any changes.
Agreed. Any cross-platform projects needs to be able to support multiple build systems, whether you like it or not. Tools like CMake/QMake/Qbs/Ninja/etc are just essential.
I'd also disagree with 'Your code doesn't have a changelog' - this is a GNU requirement, but one that dates back to before CVS was widely deployed. The revision control logs now fill the same requirement, though you should have something documenting large user-visible changes.
Agreed. You have version control. That's all you really need. ChangeLogs are generally outdated and for releases (where the VCS won't be available) a dump of the logs in some form should be sufficient, provided you are writing meaningful log messages. IOW, good VCS practices are a must.
Re: (Score:2)
I'd also disagree with 'Your code doesn't have a changelog' - this is a GNU requirement, but one that dates back to before CVS was widely deployed. The revision control logs now fill the same requirement, though you should have something documenting large user-visible changes.
He's referring to another criteria of success - user adoption. Revision control logs don't fill the same requirement if a user wants to know what's new. That plays some part in determining whether users will upgrade. If they don't, then additional development is rather pointless.
Note that he only gives not having GNU Make a FAIL score of 10. Which seems about right - it has a small effect on the entrance level for developers and wide spread adoption of the project. It's a critical factor only if your projec
Re: (Score:2)
Dear coward
What's wrong with open sourcing previously closed source projects?
I'd have thought that obvious. Licensing difficulties. The more difficulties a project has the greater the chances it won't succeed in the long term. (if only I could think of a simple analogy you could comprehend)
I guess linux fails since Linus wrote his own source control for it.
That's the problem with guessing. Git wasn't developed just for the kernel project. Fun fact: "own" has more than one meaning. Words can be tricky like that. e.g. linux, and kernel.
Even if the kernel (Linux) didn't use git - that's only 30 points of FAIL, just enough to make a baby cry.
Slashdot summary, as usual, misses the point (Score:5, Informative)
If we're going to talk about Callaway's Points of Fail, and create a link in the Slashdot summary that *looks* like it takes you to that list, then perhaps there should actually be a link to the list.
Callaway's original Points of Fail blog post [livejournal.com].
You know, instead of the usual Slashdot way of pointing to an article wrapper that talks briefly about some of the points and then eventually links to the real list.
Re: (Score:2)
That would require little timothy actually expend effort.
Re: (Score:2)
But it would spare us the badly written original article.
Re: (Score:2)
Re: (Score:2)
OTOH, my fascist firewall blocks blog posts such as Callaway's, so I really appreciate the hop through an unblocked source. I take it from context that article covers some stuff that isn't in the blog post, as well.
You're thinking of this as an either-or situation, when it really isn't. Hyperlinks are cheap. There's no reason for the summary not to clearly say, e.g."Here is the original blog post in its entirety, and here is an article which discusses some points from the blog post along with some other stuff." If they can't even manage that, then the link should at least clearly indicate that it isn't to the content described in the summary.
Instead, the Slashdot summary fails to link to the original blog post an
Correct link to TRA (Score:4, Informative)
His list [livejournal.com], instead of the link to a blog with an article about the list. That blog post is interesting - though the picture of the author scratching is just weird. Was that taken at a lice convention?
Re: (Score:2)
It was written in 2009
Re: (Score:2)
Thanks for pointing that out, I didn't even notice the date until you mentioned it.
Re: (Score:2)
Re:Correct link to TRA (Score:5, Funny)
Wrong. Wrong. Wrong! Google developers get paid buttloads of money, more than you or I could hope to make. These are the elites, the 1% of the 1%.
Because they are so highly paid the problem cannot lie with them since we are repeatedly told if you pay developers what they're worth you will get excellent results. Just like paying CEOs of companies who go running to Uncle Sam to protect them from their own incompetence.
The problem must lie elsewhere. Look harder.
Re: (Score:2)
Or UIs.
It's all halo effect. I think Google got search right (or less wrong than everyone else at the crucial time - just when AltaVista tried to turn into a portal and became utter poo) and have pretty much ridden that, with financial and reputational cross subsidies, ever since.
Sooner or later they'll run out of credit, one kind or the other.
Re: (Score:2)
I saw it on the Reg.
Here's the list (Score:3)
>> 1) If your codebase is too big, it's going to limit who's going to be able to download your code.
>> 2) There is no good reason in 2015 for a FOSS project to not have public source control. This helps people contribute and determine the health of your project based on the date of the last commit.
>> 3) If your source control has no web viewer and/or no documentation, these two are obvious things to have
>> 4) Code that doesn't build is worse than no code! You need documentation on how to build the project from the source.
>> 5) Use build tools
>> 6) Bundling is not going not be maintainable. Bundling leads to forking.
>> 7) Forcing people to install only in a specific directory
My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub, TFS online or one of the popular choices most FOSS projects already seem to use. (e.g. How would an experienced developer get these problems in the first place?) #4-6 are entry-level build issues. #7 refers to a best practice (let people pick their install directory) that's been commonplace in the industry for at least 15 years.
I see he's employed by Red Hat. Does this list as news suggest that Red Hat's internal development processes are immature too?
There are LOTS of projects with these problems (Score:3)
"How would an experienced developer get these problems in the first place?"
A lot of projects do not follow widely-accepted best practices... even if they are experienced... and that is a problem!
A remarkable number of OSS projects fail to have a public source control system (#2). That includes many established projects that everyone depends on. Actually, a number of OSS projects - and projects that people THINK are OSS but are not (because they have no license) - fail many of these points. It's not t
Re:Here's the list (Score:5, Insightful)
What depresses me bout software is how often we JUST DO NOT LEARN! Yes I am shouting. I am frustrated by the situation. Software development seems to be riddled with arrogant know nothings who think they can cut corners or reinvent the wheel because doing the right way isn't "7337".
Software Development is not an Engineering discipline by any means, at best it is a craft, because the hard lessons are not explicitly taught to newbies who are not evaluated on how well they follow those practices and tests them on them as part of a core knowledge base. Which is how real Engineering disciplines do it.
Mod parent up. (Score:4, Insightful)
And not just software. Look at security as well. And so many other computer-related areas.
For me it's more like ... someone "learned" one way of handing it when s/he was working ALONE.
Then that person never learned that the practices need to be changed when you are part of a TEAM.
And releasing your code to the public is being part of a team.
Re:Here's the list (Score:4, Insightful)
In the real world you can learn but you are most often prevented from putting the learning into practice. After 30 years programming, I still spend 95% of my time dealing with other people's code and "maintenance". The opportunity to do things the right way usually was passed years before I joined.
Actually real engineering disciplines screw up too and for some of the same reasons as software. Quick and dirty hacks in hardware driven by unrealistic deadlines from upper management, last minute bandaids applied because it's too expensive to redesign, confusing document control, lack of knowledge transfer if someone leaves the team, etc.
Also remember that the list is about open source software. Some of the things that are wrong there aren't wrong with proprietary software or internal tools, distributed teams have different requirements from teams that are all in the same aisle (ie, open source is greatly improved if there's a web based source code control browser, but that loses importance if everyone on the team already has a license of a the source code control system's GUI tool).
Open Source also means lots of college students or recent grads, full of excitement to get stuff done but without the real world experience to know how to go about it; and full of hobbyists who start off strong then slow down and stop; etc. For example, hobbyist has only a home PC with Windows, and for budget/time reasons does not want to bother with portability to hundreds of different types of systems; or the hobbyist is the sort who thinks Microsoft Visual Something is a really good state of the art tool and bases the entire project on that.
Humans (Score:2)
That's a problem with human nature, not just devs. We are not Vulcans. Humans are impatient, egotistical, fixate on the wrong factors, and often just plain random; and most don't know it or care.
I know some well-educated people who are complete idiots outside of their narrow specialty. I'm probably an idiot also in ways I don't even realize (please don't educate me in replies). My hea
Re:Here's the list (Score:5, Interesting)
Clearly you don't understand engineering. Engineering isn't just "model your entire design". Engineering is decomposing your problem into problems that are "spec-able". For example, build your bridge out of steel and bolts. You don't have a model of bolts in your design, you have a spec for bolts that you use in your design that is testable (performance and tolerance) and then you use parts hierarchically in your design. The bolt is designed separately and is made out of some alloy that has specs and is tested (performance and tolerance)...
The problem with most software isn't that it can't be modelling and rely on basic physical principles, it's that many projects fail to take specs and testing seriously, and the specs that exist don't address performance and tolerance (aka, error handling). If software did this, things would be more engineered.
Right now many software artifacts are similar to the prehistoric bridges that cross chasms in jungles in third world countries. They work, people cross them every day, but things were made empirically so nobody knows what might cause them to fail, so it's hard to rely on them.
It's not that bridges that were built 100 years ago were "better", but they were actually built to specs and of course survive to this day (which can't be said for the prehistorical variety). However, improved bridges are continually desired so we use better parts and build even better bridges today because modeling allows us to get tighter specs on the parts that make up bridges and the stresses that we are putting on those parts.
But doing all that requires better engineering discipline not dismissing it as a something that isn't applicable. Engineering is an useful approximation of the physics (an approximation which always gets improved over time), not a practice of physics.
Re: (Score:2)
Most requesters (users) don't really know what they want UNTIL they actually see something concrete, and then realize it didn't fit what they had in mind. We don't need engineering, we need mind-readers. If users had enough time to sit and be thoroughly interviewed about needs and preferences, they wouldn't need automation to begin with.
And fu
Re: (Score:2)
Re: (Score:1)
Not really. They often have to use it with actual data to relate to what it's doing with it. The data flow matters.
Re: (Score:2)
Mainly he's confusing a project which uses an open source licence for a project that wants a community based development model. Google doesn't release the Chromium source so that they can get contributions, they do it to be open so that nobody can claim they're creating another proprietary web like IE did with their closed source, non-standard implementation. And that is all. I mean, he's talking about the source code to the world's most popular browser so it's obviously a very narrow definition of "failure
Re: (Score:2)
My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub, TFS online or one of the popular choices most FOSS projects already seem to use. (e.g. How would an experienced developer get these problems in the first place?)....I see he's employed by Red Hat. Does this list as news suggest that Red Hat's internal development processes are immature too?
He wrote the list based on things he'd seen in Chromium, so it's Google's problems. Here is the full list [livejournal.com]. Not surprising, since they used to jam all their code into a single repository.
(It's hard to fault them for a 100+MB source code download though, unless there's a lot of redundancy in the code).
Re: (Score:2)
[...] (It's hard to fault them for a 100+MB source code download though, unless there's a lot of redundancy in the code).
That extracts to over 3GB, so yes, there's a hell of a lot of redundancy.
Re: (Score:2)
That's not the real list, follow the link to the link.
http://spot.livejournal.com/30... [livejournal.com]
Re: GitHub doesn't solve massive (Score:2)
My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub
I struggled to find what in the list actually applied to Chromium - However the one thing that definitely applies to Chromium is the massive code base... Chromium source really is fucking massive (even if you just clone with depth=1) and it takes a fuck load of time to do your first compile.
I really don't see how using one git host over another is going to solve that, once the host has reasonable resources (pretty easy in this day and age) then the users connection is the bottle neck, and it makes the build
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Should not be that challenging to compile any version of Python with newer VS. Just because the binary builds used 2008 doesn't mean you couldn't crank out a build with a newer version.
Can I go on the record as saying how SHITTY Python 3 is, btw? I mean wtf? If your language requires all kinds of weird distinctions between unicode and non unicode strings such that it takes nontrivial work to make that shit work between modules and different calls, your language is shit. I don't have to deal with that nonsen
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
meh (Score:1)
Re: (Score:1)
I'm curious; why do you think that? Is it just the slam against "Microsoft Visual Anything"?
Note that he gives points of FAIL for configuring source "with a handwritten shell script" (+10), "editing flat text config files" (+20), and "by editing code header files manually" (+30).
Re: (Score:3)
"editing flat text config files"
All while working for RedHat. RPM relies on shell scripts and doesn't have a reliable rollback/commit mechanism.
Is it just the slam against "Microsoft Visual Anything"?
But yeah, this obvious attempt at slamming business competition under the guise of technical know-how is oh, so 1995 (which was 20 years ago). But, in todays world, we have gotten to the point when it is not only easier, but more reliable to generate code than to write it by hand. And while they have some learning curve, visual code analysis tool are still better than text-only ones. Even the r
Re: (Score:2)
What is this code generator you talk of? The only thing I can think of is a compiler for a higher-level language. This may be extremely useful, but it has to take some input, and that input has to have an unambiguous meaning that describes a problem somehow, and therefore that input is a program that was written by a programmer.
Re: (Score:3)
Re: (Score:2)
What do you mean "locked to a single platform". I admit that I haven't tried it, but they give away the source code to VS 2015.
I don't think having access to the source code to VS 2015 is going to allow anyone to compile VS for any non-Windows platform. Not unless you have a few million man-hours available for porting and redesign (since much of the functionality present in VS wouldn't even make sense outside of Windows)
Re: (Score:2)
Re: (Score:2)
You are right. It's not an open source project. All it does is open its source.
I never said it wasn't open source. My comment was about whether or not it's "locked to a single platform". I assert that without a herculean porting effort, it effectively is.
C'mon. Bridging API frameworks is where "it's at" today.
I'll believe it when I see it. I guess getting it to run under WINE might be doable, but then again that was equally doable (at least in principle) when the source was closed.
Comment removed (Score:5, Insightful)
Re: (Score:1)
Re: (Score:2)
Standups, Kanban, Scrum, swim lanes, and other political middle management bullshit, but not for me. For all the _bad_ coders. Not me.
Re: (Score:2)
I don't consider myself a bad coder (and I don't think anybody else does, either), but I really liked Scrum when I had a chance to use it. It may be that we were attacking a reasonably small problem that was fairly well-understood up front, and it may be that we were actually doing a form of agile rather than just being told to write the code without any design.
Re:the usual suspects apply. (Score:4, Informative)
hmm? In one paragraph you say 'shut up and hack' and in the next you criticize Poettering for doing exactly that?
Specifically, he criticized Poettering for not taking care of features he'd created.
Not spongeworthy (Score:2)
I can't believe the billions of bits that died in the production and subsequent redistribution of this article. We're all dumber now for reading it.
I shall now drown my sorry in beer to see if I can't placate the inexorable struggle of those brain cells trying to make sense of this diarrhea and those others that are saying "WTF, Why did you read this?!? This is another let me share a bunch of drivel shit article. It's like Show and Tell in Kindergarten, just ignore it!"
Where's the video? (Score:2)
The summary starts with:
"At OSCON this year, Red Hat's Tom Callaway gave a talk [...]"
The summary has a link, which points to the article, which says:
"At OSCON this year, Red Hat's Tom Callaway gave a talk [...] My favorite part of this talk was Callaway's passion for the items on the list."
So where is the video. The list felt a bit bland so I also got the notion that the video would be more informative.
Project Management 101 (Score:1)
This is a checklist from any decent software project management book. Here's a good one written over 40 years ago:
Mythical Man Month [amazon.com]
And the issue of managing a large code base can be handled by a modern VCS system like Perforce that let's fetch and work with only the portion of the code base you're interested in.
Linux gets +30 Points of FAIL (Score:2)
"You've written your own source control for this code [ +30 points of FAIL ]"
Re: (Score:3)
You get -60 points of FAIL if you can convince the rest of the world to use your new and shiny version control system.
He just forgot that.
Link to video (Score:1)
What about market adoption aspects (Score:2)
I wanted to run my own social networking site just for me and my friends using a FOSS project, so I was excited about Diaspora, then I saw that it requires Node.js. I have no interest in setting my server up for that. I imagine this selection was made because developers think Ruby is cool and PHP is boring and lame. Unfortunately, whatever the justification was, to make Diaspora work you need to have, you know, Diasporas, but if the only people using the project are those that manage their own Node.js serve
Re: (Score:2)
Clickbait (Score:2)
Headline contains the word "your" and is therefore clickbait.
original list (Score:3)
this a slow news day? The list was crafted in 2009!
Linux kernel FAIL score. (Score:1)
== Size ==
* The source code is more than 100 MB. [ +5 points of FAIL ]
* If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ]
125372299 Jul 22 00:36 linux-4.1.3.tar.gz
== Source Control ==
* You've written your own source control for this code [ +30 points of FAIL ]
git was written originally for Linux kernel!
== Building From Source ==
* Your source is configured with a handwritten shell script [ +10 points of FAIL ]
Even worse: handwritten
Re:Luarvic FAIL score. (Score:2)
That list is well out of date. The most recent version is here [theopensourceway.org].
== Size == * If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ] 125372299 Jul 22 00:36 linux-4.1.3.tar.gz
FAIL. linux-4.1.3.tar.xz [kernel.org] 22-Jul-2015 00:36 79M [ -5 points of FAIL]
== Source Control == * You've written your own source control for this code [ +30 points of FAIL ]
FAIL Like quoting the largest archive choice for the source code, you choose the wrong meaning of "own". That's not the meaning implicit in Tom's list - nor does it fit his reasoning (using an obscure versioning system limits use and development).
"There is no publicly available source control (e.g. cvs, svn, bzr, git)" [theopensourceway.org]. [ -30 points of FAIL]
== Building From Source == * Your source is configured with a handwritten shell script [ +10 points of FAIL ]
FAIL. It's not handwritten. I
Re: (Score:1)
I should have put large <humour> ... </humour> tags around my comment.
I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail.
Re: (Score:2)
I should have put large <humour> ... </humour> tags around my comment.
I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail. If were to look at my posting history my reply was notably gentle.
I figured it twelve to a dozen on whether you were taking the piss (that's Oz-speak for good clean fun). Most of the fail points you gave had already been made in the comments on the old version of the list - including some truly brainless one's you missed (no documentation, no website, no bug tracker).
Humour tags would have just spoiled the fun. Gotta think of the larger audience. [smile]
For a good laugh at mega-TV-series FAILS - one of our consultants sent me a link to this [tumblr.com]. (I knew there was a reason we
What is OS X .zip format? (Score:1)
Pretty much off-topic: What is OSX .zip format, as opposed to .zip format? I don't think I have ever seen anything (software releases or other) that I'd know was a "special" OS X zip.
Major omission in list (Score:1)
Re: (Score:2)
mongodb? Use something that's been tried and true for at least 10 years. Go with MySql or PostGresql and screw the noSql toys until they mature and have decent docs.
Let pioneers take the arrows, the rest of us stay in proverbial Boston, which has infrastructure and seasoned specialists, and get shit done. And we have nice lawns to kick fanboys off of.