Computers Are Hard: Building Software With David Heinemeier Hansson (medium.com) 54
Wojtek Borowicz interviews David Heinemeier Hansson, the creator of the popular Ruby on Rails web development framework: Wojtek Borowicz: Software methodology is an industry of its own. There is Scrum, and Agile, and coaches, and books, and all of that. But you and your team at Basecamp don't follow these practices. Why?
DHH: First of all, our approach to software development is heavily inspired by the Agile Manifesto and the Agile values. It is not so much inspired by the Agile practices as they exist today. A lot of Agile software methodologies focus on areas of product development that are not where the hard bits lie. They are so much about the procedural structures. Software, in most cases, is inherently unpredictable, unknowable, and unshaped. It's almost like a gas. It can fit into all sorts of different openings from the same basic idea. The notion of trying to estimate how long a feature is going to take doesn't work because you don't know what you're building and because humans are terrible at estimating anything. The history of software development is one of late or cancelled projects. If you were to summarize the entire endeavor of software development, you'd say: 'The project ran late and it got canceled.' Planning work doesn't work, so to speak.
What we do at Basecamp we chose to label Shape Up, simply because that is where we find the hard work to be. We're trying to just accept the core constraint that it is impossible to accurately specify what software should do up front. You can only discover what software should do within constraints. But it's not like we follow the idea that it's done when it's done, either. That's an absolute abdication of product management thinking. What we say instead is: don't do estimates, do budgets. The core of Shape Up is about budgets. Not how long is something going to take but what is something worth. Because something could take a week or four months. What is it worth? [...]
Wojtek Borowicz: So the problem with those methodologies is they put too much focus on estimating, which is inherently impossible with software?
DHH: I'd go even further and say that estimation is bullshit. It's so imprecise as to be useless, even when you're dealing with fixed inputs. And you're not. No one is ever able to accurately describe what a piece of software should do before they see the piece of software. This idea that we can preemptively describe what something should do before we start working on it is bunk. Agile was sort of onto this idea that you need running software to get feedback but the modern implementations of Agile are not embracing the lesson they themselves taught.
DHH: First of all, our approach to software development is heavily inspired by the Agile Manifesto and the Agile values. It is not so much inspired by the Agile practices as they exist today. A lot of Agile software methodologies focus on areas of product development that are not where the hard bits lie. They are so much about the procedural structures. Software, in most cases, is inherently unpredictable, unknowable, and unshaped. It's almost like a gas. It can fit into all sorts of different openings from the same basic idea. The notion of trying to estimate how long a feature is going to take doesn't work because you don't know what you're building and because humans are terrible at estimating anything. The history of software development is one of late or cancelled projects. If you were to summarize the entire endeavor of software development, you'd say: 'The project ran late and it got canceled.' Planning work doesn't work, so to speak.
What we do at Basecamp we chose to label Shape Up, simply because that is where we find the hard work to be. We're trying to just accept the core constraint that it is impossible to accurately specify what software should do up front. You can only discover what software should do within constraints. But it's not like we follow the idea that it's done when it's done, either. That's an absolute abdication of product management thinking. What we say instead is: don't do estimates, do budgets. The core of Shape Up is about budgets. Not how long is something going to take but what is something worth. Because something could take a week or four months. What is it worth? [...]
Wojtek Borowicz: So the problem with those methodologies is they put too much focus on estimating, which is inherently impossible with software?
DHH: I'd go even further and say that estimation is bullshit. It's so imprecise as to be useless, even when you're dealing with fixed inputs. And you're not. No one is ever able to accurately describe what a piece of software should do before they see the piece of software. This idea that we can preemptively describe what something should do before we start working on it is bunk. Agile was sort of onto this idea that you need running software to get feedback but the modern implementations of Agile are not embracing the lesson they themselves taught.
Relevant (Score:5, Interesting)
http://programming-motherfucke... [programmin...fucker.com]
Re: (Score:2, Insightful)
When I first read Zed's rant about Rails [cat-v.org] I thought he was unreasonable. Then I had to work with a rails system briefly and realised that he's a mega tolerant person put upon by the industry who puts up with far too much shit. I only wished he had been clearer about how totally shit Rails can be.
Re: (Score:2)
You laugh, but if by some crazy happenstance that caught on as a 'methodology' (the use of 'mother fucker' greatly mitigates that chance), then you bet there would be consultants lining up to have 'programming motherfucker' themed terms for daily stand ups, estimations, and various other micromanagement and bureaucratic nonsense.
The problem is not so much with the 'theory' it is always that practice tends towards dysfunctional, and the sad reality is that despite trying your best, the dysfunction will alway
No such thing as C/C++ (Score:1)
Great until they said C/C++ in the Become a Programmer tab.
You can do C well. You can do C++ well. If you are very clever you might even be able to do both well. But you can't do C/C++. There is no such language.
Just ban this stupid woolly-headed C/C++ term from your vocabulary.
Re: (Score:2)
You can do C well. You can do C++ well. If you are very clever you might even be able to do both well.
This website doesn't state otherwise.
But you can't do C/C++. There is no such language.
Nowhere does this website claim that there is. It is clear that C/C++ is a grouping, especially when you see others such as "HTML / CSS" and "Delphi / Pascal". You make invalid assumptions far too easily and based upon this, you would be clearly incapable of translating written requirements into code that meets those requirements.
Computers aren't entirely hard. (Score:4, Insightful)
Re: (Score:2)
This is almost word-for-word what I was thinking when I read the headline.
Spooky.
Re: (Score:3)
That's because software, firmware, and hardware are still trumped by wetware. For now... ;-}
Must be nice (Score:4, Interesting)
Opposite, this is only approach that really works (Score:2)
So people lie, and when the deadline goes whizzing by the bean counters have spent x million and backing out now would be a waste of money
Do you really think they are so stupid?
Of course not, they now you are lying but don't care because used to the constant stream of lies from software developers, they simply budgeted way more that it should cost and determined they are OK with that amount...
Companies would be far better off if we all stopped pretending estimates meant anything, and followed the approach d
Re: (Score:3)
When you're developing code that your company is being contracted for, your customer wants an estimate, a due date, a schedule. I spent 37 years doing this kind of work before retiring last year, and for my $.02, it typically came down to this. Most program managers are trying to keep the engineering costs/schedule to a minimum, to win a contract, maximize profit, etc. They don't trust engineering estimates, and for good reason. Most experienced engineers know that program management isn't going to give
Re: (Score:1)
When you're developing code that your company is being contracted for, your customer wants an estimate, a due date, a schedule.
Yes I have been a contractor for over a decade...
They want those things, but the software industry will never become mature until everyone realizes they cannot have them. Not in the way they want anyway.
There certainly are things that can be developed with a realistic estimate, and plenty where you just won't know until you're half way into it.
Yes, but you do't know which is which
Re: (Score:3)
"They want those things, but the software industry will never become mature until everyone realizes they cannot have them. Not in the way they want anyway."
I was a DoD contractor for the last 37 years. They always got what they wanted, and until my old company was bought out by one of the giants, we always delivered on schedule or early. If you're saying that accurate estimation can't be done, I'll disagree, but as I said above, not in all cases.
"Yes, but you do't know which is which until well after you
Re: (Score:2)
Valid point, I have never worked for a contracting house, always for largish companies with their own internal IT divisions, so my viewpoint might be a bit lopsided.
Re: (Score:2)
The problem with estimation and projects is that it still tends to treat a software project as a 'you do it and then its done' rather than a malleable project.
Having a general order of preference and then slightly adjusting that order as things are easy/difficult and shipping what is ready as it is ready. Generally this is mentioned during an 'Agile' kool-aid session but is lost in the noise of backtracking back toward big monolithic deliverables and injecting massive micro-management which is what non-tech
Re: (Score:2)
The farther away the developers are from doing the task scheduling the faster the whole system falls apart. I think that this is the main reason agile has such a bad reputation with developers.
Once you get a few layers between the developers and the business leadership ("technical leads", product managers, project managers, dev managers) all you have is a system that tends toward failure. All of those positions advance their careers by promising things up the chain while the developers take the responsibili
Re: (Score:3)
Agile principles are good, when applied judiciously, by capable people, as intended by the Founders. Agile methodologies are pretty much all money-making / religious B.S., and at best dilute agile principles so much in the the interest of accommodating "less than ideal" workers that all net advantage is lost.
Re: (Score:2)
Re: (Score:2)
Yes, file away under "Things that should be good but don't work in practice".
Re: (Score:2)
All estimates tend to be rubbish to some extent. As an electronics design engineer, I know projects can vary between easy and very hard. An easy project goes straight through from initial specs, to working prototype, to production. A hard project goes through several defective prototypes, and may require revision of the initial spec, before finally getting into production. I can only guess whether a project will be easy or hard to bring to production status.
The thing is, management and sales need something
Re: (Score:2)
...backing out now would be a waste of money and bean counter heads would roll.
That's not a bean counter. A bean counter will shutdown your project the _minute_ it appears like you will be past the point of zero ROI. How much you already spent is what other successful projects have to carry; and only becomes an "investment" once your project delivers. If someone else leaves such burden on your shoulders, especially when you didn't want/support it in the first place, you too would feel someone's head should roll. And that feeling gets worse when that failing project isn't cancelle
What is a budget? (Score:2)
What is a budget if not an estimation for how long a resource will be needed?
Re: (Score:3)
And then there's this [joelonsoftware.com], which provides some better ways to define "project" and "late".
Re: (Score:2)
That is basically how you did tasks and estimates in "old school scrum", before they got infected by XP and created the modern mish mash :P
Re:What is a budget? (Score:5, Insightful)
The problem is overly detailed estimations getting in the way of productivity.
The ability to change course and re prioritize and deliver what actually works on time is better than a late delivery of a project as it was guessed to look like months ago. This was strangely largely baked into most 'Agile' training as a concept, but in practice gets thrown out.
Once upon a time I had an exec who would ask me for a sizing and was ecstatic because I said I would always deliver on time. When asked 'what would be delivered on time' it would be 'well, version x.y' and that was plenty good enough for that exec, though the managers between the exec and myself hated that vagueness to no end. To feel better we had a historical summary of *past* releases and that was plenty for the exec to feel good that real work was getting done without getting all screwed up over the particulars of what would come next.
Perfect description (Score:4, Informative)
No one is ever able to accurately describe what a piece of software should do before they see the piece of software.
This fully explains the software ServiceNow produces. They have no idea what their software is supposed to do so they can't even describe it to their own programmers. And thus, the steaming pile which is their supposed ticket tracking and asset management system.
I'm relatively certain they could screw up designing software to be a calculator.
Re: (Score:2)
Probably because service organizations have no idea how they're supposed to provide service. So it's a perfect impedance match, and ServiceNow can meet a given service organization's needs if/as they figure it out themselves.
Re: (Score:3)
I can assure you, ServiceNow cannot meet a service orgainization's needs. It is quite clear the people both designing and writing the software have no clue how people operate in the real world.
Re:Perfect description (Score:4, Insightful)
Ugh....they use ServiceNow where I work and I can't stand it. It is the most un-intuitive pile of crap I have ever worked with. Filling out what should be a simple service request ticket seems to take ages. Page after page of idiotic fields with zero descriptions. No pop up help. No mouse over descriptions.
Once you finally figure out how to navigate though it you join the tribal knowledge camp. A veritable badge of honor. Like some sort of Rubiks cube of end user software, it is a puzzle rather than a tool. All that come after you must endure the same torture you did. There is no training. There are no shortcuts.
Everyone that comes in contact with ServiceNow must make the same mistakes, box themselves into the same corners and pull out the same collective clumps of hair that you did. They too will learn to despise ServiceNow but only after they survive the initiation, like some twisted form of digital marine boot camp. Perhaps it builds character in some way that I fail to grasp.
But in the end, as a fully indoctrinated corporate dweeb, you will grudgingly accept ServiceNow and swear under your breath and quietly pound the keyboard as you churn out yet another service ticket that in the old days a simple email or phone call would accomplish. The ultimate irony of course is that ServiceNow is presented as a tool that will streamline operations and introduce previously undiscovered savings and efficiencies. While delivering exactly the opposite.
Software is hard because... (Score:3)
Quite simply the people who need it, cannot express what they need all that well. That is compounded by the fact that those making the software have even less idea (typically having around zero problem domain knowledge).
Or, it consists of the blind leading the blonde...
Re: (Score:2)
There are lots of software domains where that isn't true. I work on games (audio programming.) I know games very well as a user of games and as a maker of games. Our middleware guys know audio and audio design as well as the designers/users of the middleware. I think your generalization is more applicable to business/financial/marketing/commerce where it's uncommon that the programmer would have a personal interest in the domain in which it's used.
I do agree that people have a difficult time expressing what
i hope he's not in avionics (Score:1)
Re: i hope he's not in avionics (Score:1)
Having recently completed a job hunt (Score:5, Insightful)
Anyway, if you want to keep your skills current, encourage your team to use as many Agile processes as possible for your next project. You never know when you'll need to be job hunting next.
Re: (Score:3)
I find this quite a pain too. People want to know if you "know" Agile.... and I think its like asking if you know how to swim or bike. Does it really matter if the person doesn't know? Unless the job is a Coach, mentor, or such, there is no reason to have experience with Agile. Especially in a PM, programming, or analyst role. Because you are hiring for a multi-month project and this stuff literally takes under a month to pick up. The position should state their expectations and tools they use, they s
Airplanes (Score:1)
Re:Airplanes (Score:4, Interesting)
Planning in 2 week intervals is not good. You often need a lot more flexibility to get good productivity. Then Agile done the way an accountant thinks stands massively in your way. I am currently experiencing that some new person in upper management is introducing "Agile for Everything". Yes, the planning is possible, but I get maybe 50% of the actual productivity (estimation includes effects from lower quality) that I had before. I do not see it ever going above 75% or so of what I am used to. Accountant-Agile is _inflexible_. It stands in your way.
BS management is BS management and it does the least damage if it has the least control. Accountant-Agile gives it more control and hence management destroys more productivity. The most stupid aspect is perhaps that they want to make everything "measurable". That does maybe work on creating software on low difficulty level, but not beyond that, because you cannot actually measure difficulty of a task. It certainly does not work for security architecture, which is what I am currently doing there. My team is currently trying ways to work around the process to prevent productivity from dropping even further.
Designers in other fields already know this (Score:4, Insightful)
Design, and how it works, has been part of other fields which go back hundreds of years. The software industry is kinda new, and siloed. There's a lot of wisdom out there which the software industry is perhaps only just starting to discover for itself.
There are budgets and all manner of constraints. People don't know what they want until they see it. Your job is to give people what they never knew they wanted, and do it with whatever time and money you have. Basically, worry less about managing, and worry more about hiring smart creative people who are also good problem solvers, are very good at critique to throw out the good ideas and aim for the brilliant ones, and have a flair for pushing things forward.
Of course the vast majority of software is just bread and butter work, but if you want to understand design, then that's some of the things one needs to understand.
Re: (Score:2)
God save us from customers who do not know what they want.
Luckily, where I work now, there is a competent sales and marketing team, who seem to be able filter out the rubbish you might get when engineers have to deal directly with customers. There was a time when sales tended to over-promise, resulting in impossible deadlines. That does not happen so much now. I am told that our sales team is unusually good at turning customer inquiries into solid sales, so I rarely waste too much time on projects that go n
Re: (Score:3)
Design, and how it works, has been part of other fields which go back hundreds of years. The software industry is kinda new, and siloed. There's a lot of wisdom out there which the software industry is perhaps only just starting to discover for itself.
Any field that goes back hundreds of years has a giant set of constraints known as physics. Attempts to draw parallels to other fields invariably fails to grasp the fact that software is infinitely malleable. Software architecture is like building architecture if architects had to design a structure that could stand on a different planet every week. If they also have to design the planet. In a universe with a variable force of gravity. And have it done by Tuesday.
There's a lot of wisdom out there, sure
Wrong on so many accounts! (Score:3)
I'd go even further and say that estimation is bullshit.
If you never estimate, how will you learn to estimate? A self fulfilling prophecy.
It's so imprecise as to be useless, even when you're dealing with fixed inputs.
No it is not. If gives an idea if it is even worth to start the project or not. You tell me it is about $100,000 - I might consider it. Well knowing it might go up into the $400,000 range. If you tell me it is hard to do with $1,000,000 I just drop the project. Ooops, so easy.
And you're not. No one is ever able to accurately describe what a piece of software should do before they see the piece of software.
Completely bollocks. Minimum half of my projects I did in the last 30 years were _completely accurate defined_ before we even started. And that is often super easy: because they where rewrites of existing systems, and in some cases "product families" spun off from such a system.
Sorry, someone who "just makes web sites" most likely has no clue about "real software development".
Re: (Score:3)
I have watched people estimate using various methodologies in the games industry for getting close to 20 years now, and I've never seen an estimate worth anything at all. Experienced people, people that have shipped great games, people that do great work—none of them can estimate for shit.
At my first games company, one of the leads doubled everything that anyone told him. That kinda worked. (And I guarantee that if you were playing games 20 years ago, you heard about the game; it did very well. He was
Re: (Score:2)
The point with estimating is: you do it all over again and again. So you learn from your mistakes and have old mistakes and old good estimates as reference.
Obviously it is less important, if you know: we have the funding and we will deliver the next big block buster game. Does not matter if it is releases 12.2020 or 4.2021 ...
Interesting thoughts sough. Seems we think alike :D
Agile just means 'look busy' (Score:1)
I've been on a couple 'agile' projects and my main observation is that Agile make it look like you are busy while making less progress on what you are trying to do.
You have people running back and forth doing standups, etc all taking away from the real work. I would be interrupted an hour into my day with a standup and then have to jump into a weekly planning and don't forget the retrospective.
The only constant was that the projects would fail or be very late.