The Collapse of Complex Software 317
Nolan Lawson, writing in a blogpost: Anyone who's worked in the tech industry for long enough, especially at larger organizations, has seen it before. A legacy system exists: it's big, it's complex, and no one fully understands how it works. Architects are brought in to "fix" the system. They might wheel out a big whiteboard showing a lot of boxes and arrows pointing at other boxes, and inevitably, their solution is... to add more boxes and arrows. Nobody can subtract from the system; everyone just adds. This might go on for several years. At some point, though, an organizational shakeup probably occurs -- a merger, a reorg, the polite release of some senior executive to go focus on their painting hobby for a while. A new band of architects is brought in, and their solution to the "big diagram of boxes and arrows" problem is much simpler: draw a big red X through the whole thing. The old system is sunset or deprecated, the haggard veterans who worked on it either leave or are reshuffled to other projects, and a fresh-faced team is brought in to, blessedly, design a new system from scratch.
As disappointing as it may be for those of us who might aspire to write the kind of software that is timeless and enduring, you have to admit that this system works. For all its wastefulness, inefficiency, and pure mendacity ("The old code works fine!" "No wait, the old code is terrible!"), this is the model that has sustained a lot of software companies over the past few decades. Will this cycle go on forever, though? I'm not so sure. Right now, the software industry has been in a nearly two-decade economic boom (with some fits and starts), but the one sure thing in economics is that booms eventually turn to busts. During the boom, software companies can keep hiring new headcount to manage their existing software (i.e. more engineers to understand more boxes and arrows), but if their labor force is forced to contract, then that same system may become unmaintainable. A rapid and permanent reduction in complexity may be the only long-term solution.
One thing working in complexity's favor, though, is that engineers like complexity. Admit it: as much as we complain about other people's complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads -- it's only when these diagrams leave our heads, take shape in the real world, and outgrow the size of any one person's head that the problems begin. It takes a lot of discipline to resist complexity, to say "no" to new boxes and arrows. To say, "No, we won't solve that problem, because that will just introduce 10 new problems that we haven't imagined yet." Or to say, "Let's go with a much simpler design, even if it seems amateurish, because at least we can understand it." Or to just say, "Let's do less instead of more."
As disappointing as it may be for those of us who might aspire to write the kind of software that is timeless and enduring, you have to admit that this system works. For all its wastefulness, inefficiency, and pure mendacity ("The old code works fine!" "No wait, the old code is terrible!"), this is the model that has sustained a lot of software companies over the past few decades. Will this cycle go on forever, though? I'm not so sure. Right now, the software industry has been in a nearly two-decade economic boom (with some fits and starts), but the one sure thing in economics is that booms eventually turn to busts. During the boom, software companies can keep hiring new headcount to manage their existing software (i.e. more engineers to understand more boxes and arrows), but if their labor force is forced to contract, then that same system may become unmaintainable. A rapid and permanent reduction in complexity may be the only long-term solution.
One thing working in complexity's favor, though, is that engineers like complexity. Admit it: as much as we complain about other people's complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads -- it's only when these diagrams leave our heads, take shape in the real world, and outgrow the size of any one person's head that the problems begin. It takes a lot of discipline to resist complexity, to say "no" to new boxes and arrows. To say, "No, we won't solve that problem, because that will just introduce 10 new problems that we haven't imagined yet." Or to say, "Let's go with a much simpler design, even if it seems amateurish, because at least we can understand it." Or to just say, "Let's do less instead of more."
Complexity. It's a feature. (Score:5, Interesting)
It's how we maintain our employment. [zerobugsan...faster.net]
Re: (Score:3)
I disagree with the article where it says engineers love complexity. Most engineers hate it. Now on an economic side, engineers who are contractors love it as it's job security for them. But fundamentally the complex design is a broken design and most engineers have their internal idea of Occam's Razor that states that the simpler design is the better design.
My beef is the ever added features, and never once a removed feature. Even when there is literally no more memory available, people are still resista
Re:Complexity. It's a feature. (Score:4, Informative)
Programmers aren't engineers.
There are engineers that are also programmers, but programming is not engineering. It may even be illegal in your area to call yourself an engineer if you're not properly certified.
Re:Complexity. It's a feature. (Score:5, Informative)
Programming is engineering. Some legal fiction does not matter for that. Engineering is solving problems with technological means. That almost all programmers do not even remotely qualify as engineers is quite true though.
Re: (Score:3)
engineers give _guarantees_
coders give _liability waivers_
when an engineer designs a bridge, they say "if anything weighing under x tons goes on it and the bridge collapses, come sue me"
when a coder designs a program, they say "i am not responsible no matter what this piece of code does or does not do, under any conditions whatsoever"
Re: (Score:3)
Programming is engineering.
No it isn't, not necessarily at any rate. Constructing a house is not necessarily (and in all likelihood not) engineering. Just a bog standard building project.
Some legal fiction does not matter for that.
100% agree. Protect the term "chartered engineer" or whatever. By this stupid legal fiction, Brunel wasn't an engineer.
Engineering is solving problems with technological means.
Then all of the usual trades are engineering too.
That almost all programmers do not even remotely qualify
Re: (Score:3)
I agree with this, programming today is a continuous R&D process. Someday it will move on to the engineering phase but it is not there yet. Engineering is what comes after a solid R&D foundation has been built.
Another way to look at it -- software development is a immature engineering process, whereas building construction is a mature process. Construction went through the same phase - back around the turn of the century building and bridge failures were much more common.
Re: (Score:3)
Visit automotive or Nasa embedded programming. You are looking at this from the cavalier, care-free, free-wheelin', yee haaaa! programmers of desktop, Linux, and phone apps.
Having said that, I do not want to see "real engineering" applied as a mandate to such programmers, as they don't really need it. That world is fine for creative types who come up with clever algorithms or the next killer app.
Re: Complexity. It's a feature. (Score:3)
Engineer or Licensed Engineer.
Engineer - Having an engineering degree.
Licensed Engineer - An authority has given license to work as an engineer wirh responsibilities, rights and protections.
Re: (Score:2)
Re: (Score:2)
Sure, that's a gaol for some people. However I would hesitate to call this the "engineer's viewpoint". Everyone has multiple personalities inside them, so someone may have a business personality or a make-money personality that can override their engineer personality.
Re: (Score:2)
Buddy, you were on the wrong team.
Start your own software company, and compete against this team. From what you are saying, it will be an easy win.
Re:Complexity. It's a feature. (Score:4)
Now on an economic side, engineers who are contractors love it as it's job security for them.
Fuck off. As a contractor I ripped out a piece of shit overly complex system developed over years ruled over by some guy who was an employee and treated it as his empire. I replaced it with a far simpler reactive architecture in place of a Tibco fiasco that was brittle as fuck and when it invariably broke no one could figure out why except the little ruler, and even then it usually took a week to get it back on line. And if they wanted to add an extra data field to one of the upstream system's output, it would take 9 months or more to get it done. After my redesign it took one two week sprint to implement and one more for UAT sign off; and really it should only take one sprint, but 'procedures'. It was what architecture there wanted for years, but the empire owner employee had some pull because people believed he was a essential as he told them. Moral of the story: contractors like building cool stuff, and that means they like building stuff that works properly and is only as complex as needed. Employees are the ones who build shit. Them and vendors. Don't mistake vendors for independent contractors. Independent contractors have to rely on having a good reputation. Vendors have teams of marketers that can bullshit extremely well. Independent contractors don't.
Re: (Score:2)
There are bad and good contractors and employees, just because you can cherry pick an example of a bad employee, it proves nothing.
Re: (Score:3)
There are bad and good contractors and employees, just because you can cherry pick an example of a bad employee, it proves nothing.
It's actually a bit tricky to say that it proves nothing. Given that the point was to disprove the statement: "Now on an economic side, engineers who are contractors love it as it's job security for them." Then you could argue that it absolutely proves that ."... engineers who are contractors love it as it's job security for them." That's an absolute statement about all engineers who are contractors, therefore a single counterexample proves that it is a false statement. The tricky part is that qualifier: "N
Re: (Score:3)
The most memorable was at SW Bell, where they had one guy, Silver, who had staked out and camped out on one system long enough that no one else could understand it.
Done in COBOL, his functional main chunk (is about all you can call it) was FORTY green-bar pages long with NOT ONE PERFORM, only go to's. Standards insisted on descriptive
Re: (Score:2)
Mozilla constantly removes features from Firefox. Yet the browser is as complex and fragile as ever.
Gnome developers also have no qualms about removing large chunks of old code and configure-ability. Doesn't seem to help the complexity or usability either.
It's simple, except... (Score:5, Insightful)
Most systems grow in complexity because of all the nuances of handling inconsistent/dirty data, business rules that exist just because, etc., etc.
The new guys want to design a nice clean system that does only what's needed. Until they're told it doesn't do everything that's needed. So they start shoehorning in new features.
Re: (Score:2)
The new guys want to design a nice clean system that does only what's needed.
They won't succeed. IMO the best way to learn to design a nice clean system is to spend time cleaning up an old system. Then you'll have a good understanding of what works and what doesn't.
If they just build a clean system from scratch (without developing the skill), it will be readable but inflexible, and new features won't fit (as you pointed out).
Re: (Score:2)
Guess what, complex software is complex for a reason.
Security and Complexity are at odds (Score:5, Insightful)
Re: Security and Complexity are at odds (Score:5, Insightful)
Re: (Score:2)
I'd be keen to your exploits for Z/Os.
Re: (Score:2)
The question to ask is how systems became so complex. The answer is obvious, but very few are willing to accept it.
Bloggers Gotta Blog (Score:5, Insightful)
Anyone who's worked in the tech industry for long enough has been tech bloggers write blog posts that contain no real insight into anything.
Yeah, complexity exists because architects and engineers love it! Not because real world problems are complex, and require solutions that tend to be complex, especially as needs change over time.
A rapid and permanent reduction in complexity may be the only long-term solution.
Wow! He's done it! He's found the silver bullet! I am so relieved.
Re:Bloggers Gotta Blog (Score:5, Insightful)
Yeah. This blog post shows a lot of naivete. The word "haggard" was the first clue. For most engineers that have to hold down the fort for aging software:
- a sunset is welcome, engineers prefer to create new things. They are not mindlessly drawn to complexity the way the blogger insists.
- old software stays around because it's low maintenance. It might be crappy, but old bugs have been seasoned away. So there are few bugs. The difficulty comes in expanding the functionality for new use cases. That's when old software becomes maintenance Hell.
- Old systems hang around not because of some crazy gate-keeping mentality, but because of a lack of resources. Again. Most good engineers welcome the shake-up because it brings in new resources
Re:Bloggers Gotta Blog (Score:4, Funny)
1) Totally clueless - will never learn and maintain any system unless overseen by a senior,There are many of these
2) Inexperienced - Need a few years to learn the ideas and methods they have actually have been tried by another name and don't work. There are many of these
3)Genius - Will actually see whats good and how to rewrite and do a good job in a timeframe that is useful, These are extremely rare.
4) Contracted Consultant - appears to be a 3 but is actually a 1 and will cost the company a fortune before he goes on to the next company saying the team which has been hollowed of all real talent is told you can finish this now
please add varieties as you see fit
Re:Bloggers Gotta Blog (Score:5, Interesting)
Old systems also stay around as they make money. The bread and butter that keeps the company alive is the legacy product that used the lumpy burlap sack design pattern. All the new products that everyone inside the company loves don't sell so well, or not in a big enough volume to retire the legacy golden goose.
Re: (Score:3)
[Engineers] are not mindlessly drawn to complexity the way the blogger insists.
Agreed. The summary sounded to me like the author was working backwards from a premise, rather than making a convincing argument towards a conclusion. This line
\"Admit it: as much as we complain about other people's complexity, we love our own."
... seems like straight-up projection. Reminds me of one of the Simpsons Halloween episodes where Bart says to his monster doppelganger: "You're crazy!", to which monster-bart replies "Aren't we ALL a little crazy? I know *I* am."
Re: (Score:2)
A huge reason the complexity keeps increasing is back-compatibility. If the software has some kind of saved state or setup or scripts, it is almost impossible to rewrite it, as those old setups will not longer work.
I agree with most of the posters here that engineers actually very much dislike complexity. But they don't see any way to get rid of it without breaking compatibility.
Re: (Score:2)
Re: (Score:2)
Wait till he sees millions of lines of COBOL or millions of lines of PL/SQL for the first time. He'll age fast.
Re:Bloggers Gotta Blog (Score:5, Insightful)
On the one hand, I agree with you: the primary driver of complexity is the real-world needs that the code must solve. Time is also a factor, since the original system WAS simple, and solutions to more complex problems had to be tacked-on over time, and that tacking-on has made the code a lot more complex than it would have been if the features were part of the original design (but time travel is not an option so when that kind of complexity-sediment has accumulated the full re-write is often the only option).
BUT, on the other hand, there ARE software developers who have no tolerance for simple designs. They have got it in their head that inversion-of-control coupled with dependency injection is the one-size-fits-all solution to every single problem that exists, and that the flexibility that this pattern introduces means they will be able to solve any unknown future problem without very much effort (and without an increase in complexity), so they must design anything and everything they code that way, even when it is excessive overkill for the problem being solved.
I have met developers like this, and they are really adamant. They insist with religious zealotry that the upfront investment in introducing this more complicated pattern will save enormous costs in the long run, make everything better in every way, with no downsides. And, the vast majority of the time, they are dead wrong.
The actual changes that the real world throw at the project are 90% of the time "cross-cutting," requiring you to change your interfaces and carve a path of changes through all your layers, which in turn require you to make code changes to basically all of your unit tests. The dev costs of such changes increase exponentially, and this starts happening almost immediately after the coding is finished.
There ARE problem spaces where such patterns fit like a glove, but most aren't such a problem space, and forcing this kind of complexity into code that does not need it winds up causing far more problems than it solves. And there is a simple way to tell if this is happening to your code: if 90% or more of your interfaces have only one implementation, then your code is brimming with needless versatility (all of which has cost you a fortune and provided you with no benefit).
So, it can and does happen that a developer's love of complexity ruins or at least harms a product. But even if you have a team of developers that all love simplicity, the project will eventually attain a state of ruin (albeit much more slowly) as the real world's moving targets cause the layers of technical debt to accumulate.
Re: (Score:3)
Time is also a factor, since the original system WAS simple, and solutions to more complex problems had to be tacked-on over time, and that tacking-on has made the code a lot more complex than it would have been if the features were part of the original design
I think that this is a key factor in software complexity. Designing software / object models to deal with future features, many of which cannot even be guessed at today, is hard. Object orientation itself has given us a good tool to separate functionality and hide complexity in layers, making it fairly easy to add features... if we got our object model more or less right.
But even a good object model might need tweaks when you add things. How do you address such changes? All too often I've seen change
Re: (Score:2)
Object orientation itself has given us a good tool to separate functionality and hide complexity in layers, making it fairly easy to add features...
This is mostly a myth. It's what you've been taught, sure, but it was never really true. There are two basic sources of complexity in any software project: Flow control and dependency. OOP makes both things worse.
if we got our object model more or less right.
An important caveat. I'll assert that all non-trivial object models are wrong. I'll futher assert that building software around a model is doomed to fail. See, software actually needs to do things. When you start with an object model, the natural thing to do is adapt your processes to fit
Re:Bloggers Gotta Blog (Score:5, Insightful)
An important caveat. I'll assert that all non-trivial object models are wrong.
Counter example: I've coded a number of GUIs (mostly for internal tooling) in FLTK over the years. It's based around classic OO style ye olde C++. Basically... it works. While the code style is distinctly old fashioned, I never found the object model of that library to be a hindrance. It's pleasant and straightforward to use.
I'll futher assert that building software around a model is doomed to fail. See, software actually needs to do things. When you start with an object model, the natural thing to do is adapt your processes to fit that model. This necessarily leads to needlessly complex processes.
This rings true. But I think this is a case not of OO is bad and always a problem it's that OO is a tool or structure and misuse to inappropriate problems is bad. And if there's one thing programmers love beyond all else it's believing their shiny new toy solves ALL the things.
The design patterns book is one of the most widely misunderstood books ever written in our domain. It was meant to be a nomenclature giving a common naming scheme to a number of, well, patterns that crop up quite frequently. That's all. It was however taken as a mandate to use every single pattern listed in the book as much as possible where remotely conceivable.
Re: (Score:3)
Actually no. It makes both more easy.
Well, if all you know is OOP, then it probably does seem easy to you. You can dismiss or ignore all of the problems that crop up under the delusion that things would be worse with other approaches, when you don't know any better. OOP is known to complicate control flow. It's pretty much unavoidable. Adele Goldberg famously quipped "In Smalltalk, everything happens somewhere else." If that's describing complicated control flow, I don't know what is. Further, complicated dependencies are practically enc
Re: (Score:2)
They insist with religious zealotry that the upfront investment in introducing this more complicated pattern will save enormous costs in the long run, make everything better in every way, with no downsides. And, the vast majority of the time, they are dead wrong.
This is the most insightful thing this thread is likely to see. Also, good on you for slaughtering a sacred cow. That thing has been blocking the road for far too long.
Re: (Score:2)
Re: (Score:3)
Not because real world problems are complex, and require solutions that tend to be complex, especially as needs change over time.
Most software is dramatically over-complicated. Few problems are as complex as the software used to solve them.
By all means pretend things are more more complex than they are if it means a pay rise / job security, but at least be honest here.
Re:Bloggers Gotta Blog (Score:5, Interesting)
Yeah, so do it then. Dude works for Salesforce, enough said. He's showing his cluelessness.
I have the sneaking suspicion that he might be dishonest rather than merely clueless. If I were pushing a big SaaS offering encouraging people to throw away their internal systems and rebuild based on primitives they rent from me, while telling the CxOs and VPs that those primitives represent 'simplicity' when they are in fact merely opaque would be exactly what I would do if I were looking out for my interests.
Assuming my product isn't fractally broken there will be at least some part of it that cleanly abstracts something that used to require a couple of racks and a server jockey, so I won't quite be lying about the simplicity of building on my vendor primitives instead; but I'll know full well that implementing the giant pile of business logic and special cases that made the old system complex will, along with any devils in the details of my system, make the fancy cloud-replacement system as nightmarishly complex and expensive and risky to replace as the system it was built to replace was; but now I get paid for every minute the customer is unwilling or unable to move away from the new terrifyingly complex legacy system.
Guy is a moron if he thinks that "just do it simple, idiot" isn't the world's least helpful advice; outside of the realm of pure vanity projects things get complex because making them simple is hard, not because nobody has ever heard of the virtues of simplicity before; but one would be naive to assume that someone who sells a box of deceptively simple-looking primitives is advocating rapid complexity reductions because he's an idiot, rather than because he's selling something represented as a solution to the problem.
depends on your definition of "architecture" (Score:3)
For some, it's just "program structure". For others, it's a much more holistic discipline, trying to understand the system -in its context- and what's important to the full set of stakeholders (including the bill-payers, as well as developers, maintainers, users, corporate managers, IT managers, etc.) An "architect" who draws 'boxes and arrows' without understanding what the system is trying to do within its environment (technical, business, financial, etc) is NOT doing you any favors.
Now the decision to "replace, migrate, fix" is not simple. Often it's the non-functional requirements that are the hardest, such as availability ("we can't afford the downtime to replace the system")
A better sense of 'architecture' could help, but it will not fix problems with requirements instability, funding shortfalls or management impatience. But a better practice of 'architecture' would definitely help.
"What about DevOps or DevSecOps?" Well, is that an architectural approach, or just an approach to prioritize the band-aids? I'm not sure, I've heard examples where it was just 'software triage' and where it actually took an 'architectural approach' of "What are we trying to do here, and how can we do it better?"
The one thing I'm convinced, though, is that most of what gets called "Agile" is a box of band-aids.
Re: (Score:3)
Somewhere along the line the meaning of Enterprise Architect changed from “someone who understands OUR enterprise from a variety of perspectives and who can provide guidance to ensure that our systems can support our plans for the future” to “someone who understands tools and techniques that could possibly, if applied correctly, help you to understand YOUR enterprise from a variety of perspectives”. EAs have (d)evolved from being a knowledgable and responsible consultant to a tool pusher.
Re: (Score:3)
The specific notion of "architecture" I advocate and have used on multiple projects comes from ISO 42010-2011: "architecture: fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution" see http://www.iso-architecture.or... [iso-architecture.org]
I know some in the enterprise architecture community have used that definition, but a lot of what I've seen hasn't really aligned with how I've used 42010. And I can definitely understa
It takes a lot of discipline? (Score:2)
But then the PHB etc do not care because when those problems do happen, I'm the one that has to be on call to fix them.
Cue in the classic book (Score:5, Informative)
(Wikipedia)
https://en.wikipedia.org/wiki/... [wikipedia.org]
It is a management problem. (Score:5, Informative)
Re: (Score:3)
Re: (Score:2)
need to change a business rule? search and replace 27 different instances of the function.
You're being facetious, right?
Theoretically software will all become worthless (Score:2)
Once you write software, it's written and you can copy it as many times as you want. People keep re-writing it to stay relevant and to make money. But I don't go out and buy a new hammer every time I need to pound a nail, and I shouldn't have to do the same thing with software either. In the short term the money is slanted toward paid for software, but possibly in the future people will write most of the paid for software and maybe in the long term most software will be free. I think one thing that is prev
Joint effort, don't blame techie (Score:5, Insightful)
> One thing working in complexity's favor, though, is that engineers like complexity. Admit it...
No, many of us appreciate parsimony. The problem is that owners and bosses want something when they want something and don't care about long-term consequences. "I don't wanna hear flack, just make Addition X work!". If you want it badly enough, you can get it "badly".
A lot of people with power are petulant. It may work in marketing and public relations, but you can't spin to computers. Shortcuts will accumulate cruft over time regardless of sugar coating the justification. Lying only works on (some) humans, but the liars are so used to it that they want techies to somehow translate their lie into code. You're right, it doesn't make sense, but it's what the petulant do.
Most stake-holders have to care about parsimony to have a parsimonious system. It's not just the engineer's job.
Re: (Score:2)
Rewrite it in Javascript, please.
Re: (Score:2)
Exactly. The "holy" trinity of software complexity comes from:
* Feature Requests
* Security Fixes
* Performance Increases
These ranges from good ideas and mandatory requirements to total nonsense from users, product management, legal, or marketing. There are always edge cases that are never in the "spec".
Software, like Art, is never done. The trouble is most of the time management never wants to take responsibility for the Project Management Triangle [wikipedia.org] of time, resources, or features it will take.
Re: (Score:2)
A lot of people with power are petulant.
Spot on!
Over the years, I have found that for most managers, the higher up they are promoted, the more childish they became, simply because they can get away with it and it became a habit. The managers who grew more mature as they go up the ladder is not common.
Re: (Score:3)
No, many of us appreciate parsimony.
Do we? I mean I do, you do, and a few people I know do, but my experience is that we are the exception.
The problem is that owners and bosses want something when they want something and don't care about long-term consequences. "I don't wanna hear flack, just make Addition X work!". If you want it badly enough, you can get it "badly".
No this is wide of the mark. I've worked at places that don't do that. They suffer exactly the same problems. You end up with "core teams" who
Complexity is job security, is the old dev joke (Score:5, Insightful)
Re:Complexity is job security, is the old dev joke (Score:5, Insightful)
One main thing that adds complexity is features. Everyone always wants more features. Then developers want their work to be simpler and less boring, so they tack on libraries and frameworks which add complexity. Layers get added to layers, until the whole thing is beyond comprehension. The real cause of complexity is nature though. Developers are trying to map software on top of reality (e.g., a human workflow, a physical process, a desired human outcome, human ease of use) and reality is perverse, made of many interlocking different things, some of which only exist in the mind. The more accurate the mapping of the software onto reality the greater its complexity and the longer it takes to get it right. Changing it all to wedge in a new feature becomes more difficult with time. Finally someone new comes in and says this is way too complex! They simplify it and everyone loves how fast and simple it is, but they'd like one little feature added... I see this happening over and over with programming languages.
There is one additional and fixable source of complexity (that the blogger obviously missed). Obsolete technology.
You start building the solution on framework X v2, then framework X v3 comes out and in order to be secure you need to update the code to v3. Problem is it wasn't built with v3 in mind so some of those elegant designs aren't so elegant anymore.
And then framework Y emerges which turns out to be a better solution all around, but you're still on X v3.
I think the key to legacy systems is that once in a while you need to invest in them like a new project so that they're no longer legacy. If that requires a massive refactor then go for it. Sure you're going to spend forever wondering what the old code is doing with a bunch of weird edge cases, but those edge cases either represent real things you need to avoid or obsolete requirements that can be taken out behind the wood shed.
Re: (Score:2)
The problem there was building the project on the latest wiz-bang framework. A project shouldn't turn 'legacy' before it's old enough to start preschool, but that's the bizarre world we've found ourselves in.
The oft maligned "Not Invented Here" syndrome may cause a few problems, but far worse is the "Nothing Invented Here" syndrome that burdens so many...
Engineers love complexity? (Score:5, Insightful)
The best engineers I know of love the elegance of simplicity. They're the ones who figure out they can reduce overall complexity by splitting up that app into larger components using properly designed and well thought out interfaces. Then when requirements inevitably change, as they *always* do, those requirements hopefully aren't baked into unrelated parts of other systems.
Within a single library / module, it's far too easy to generate a nightmarish N-squared complexity matrix, as each component can be used by every other component in that same module. It's important to use a multiple levels of abstraction to limit this complexity, such that if you eventually need to, you can replace a higher-level module completely without rewriting the entire beast from scratch.
But of course, there's a fine line between building reusable modules and creating a ridiculously over-built / over-complex system that's 10x more code than needed to do something fairly simple. Moreover, it's also very difficult to design proper high-level interfaces between modules unless you have a firm handle on the problem domain, since you often don't recognize a bad interface until you're well into the project and realize "shit, this isn't working at all, because I didn't anticipate how x needs to interact with y". And your boss is not going to like hearing that you want to rewrite a chuck of code that's already working just fine...
No one ever said this was actually *easy*.
You missed one stage (Score:3)
.. where rewrites don't actually work, so then you get 2 big messes of code that you have to run forever.
Re: (Score:2)
Or the one where the rewrite fails, so they start another rewrite (Possibly after disposing of the team and getting another) which may fail, then possibly a third.
Or the one where the rewrite doesn't "fail", but takes 3-5x longer than planned for, double to 10x the resources, runs twice as slowly, and ends up reaching feature parity with the old one about the time a management reshuffle happens and the company begins sliding into obscurity.
In either case, the company's best bet is acquisition or merger... T
lol (Score:3)
Since when do engineers like complexity? (Score:2)
engineers like complexity. Admit it: as much as we complain about other people's complexity, we love our own. We love sitting around and dreaming up new architectural diagrams that can comfortably sit inside our own heads
I think you have this all wrong. What it seems like most engineers I've ever known really love is SIMPLICITY.
Because any real world program is dealing with complex problems you can't really ever have simplicity, what you strive for is minimum complexity. You do not love the complexity, bu
Re: (Score:2)
What a stupid concept and ill conceived blog post (Score:3)
Software developers love new technology to a fault. When it comes to line of business software, rarely does the latest new framework make for better applications. Newbs always think it is better to start from scratch, throw away a ton of money, and reinvent another wheel because they are too lazy to understand the original. Frankly I find the hubris of ignorant people without experience disgusting.
Build great maintainable software designed to last. Business is complex, and its support software is necessarily complex as well. If you don't understand large complex systems, maybe you need to consider a new career instead of making stupid decisions replacing functional legacy systems..
What a childish OP... (Score:2)
Complexity exists not only for the sake of complexity.
It is a MUST when you're looking at other variables (for example time).
Example: can you sort 3,000,000,000,000 items comparing one by one against the others (simple)? Yes. Should you? NO.
Complex Systems Replaced with Complex Systems (Score:2)
I work on a large complex monolith mostly written in C++. It's being replaced by an even more complex system with numerous Java micro services.
Why is a system complex? (Score:2)
Complexity is not an issue. Poor quality is. (Score:2)
If a system needs to be complex to do its job - so be it. Unfortunately, modern software development methods (cough, Agile, cough) actively promote poor code quality practices, lack of ownership, and the blatantly stupid idea that you can split a complex systems into infinitely small tickets, hand them out to anyone, and then put the programming humpty-dumpty back together again and end up with good quality.
Joel On Software talked about this 20 years ago (Score:5, Informative)
"[Netscape] did it by making the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.... It’s harder to read code than to write it."
https://www.joelonsoftware.com... [joelonsoftware.com]
Trying to handle everything is what ruins software (Score:2)
Re: (Score:2)
that won't work in the legacy 20+ year old code that handles then thousand to millions of transactions a day, there aren't enough humans to handle those 1 percent.
The development budget is nothing for big companies, really. A few million for a system that handles tens of millions or billions of dollars a year? pffft, chickenfeed.
Biggest problem - the lack of comments! (Score:5, Interesting)
The reason I would have total insight is because I comment the code in a meaningful, and complex manner, that outlines everything I'm thinking, while I'm writing it. Good code, should contain anywhere from 20% - 500% lines of comments against lines of code, and comments should be classified into different categories. If you wouldn't feel comfortable having someone look at your code in 10 years, having no documentation available, and no former knowledge of what the software was doing, it's not commented properly.
I've talked about this on my blog: https://andrewmurdoch.ca/ [andrewmurdoch.ca] under "Comment your code!", and I fully support the view that comments are what's missing from today, and yesterdays development. I shouldn't have to puzzle through your code, I shouldn't have to guess, wonder, run, debug, or look up document to understand it. I should be able to sit down, and get up to speed almost instantly, because no matter where I end up, I'll have enough comments to clarify everything!
Obviously a "Hello World" program, written in C, or JavaScript, probably doesn't need comments, but much beyond that, and it's commenting time. Another way to think about the problem, if you've been up for 72 hours, could you understand the code? If a new guy has been up for 72 hours, and has never seen the code, could they get up to speed quickly? If no, then the code isn't comment well enough (probably).
In my current project of nearly 100k lines of code, I have 56k lines of comments, bringing the total to 156k total lines. I understand what the code is doing without the comments, but if I leave will the next guy?
Re: (Score:2)
In my current project of nearly 100k lines of code, I have 56k lines of comments, bringing the total to 156k total lines. I understand what the code is doing without the comments, but if I leave will the next guy?
Have you tested all of the comments to make sure they (still) work?
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
REGEX_NOTE: This matches string condition such as '5e-6', etc...
REGEX_NOTE: This will be stored in match[2]
REGEX_EX: '5e-6' -> Match e-6, this will return -6
NOTE: ?:\[eE] -> Find an e or E character, ex: '5e-6'
NOTE: ([+-]?\d+) -> Find all digits with a +/-, keep looking until termination */
const regex =
Actually, no (Score:5, Insightful)
No, actually... we don't. What we like is not working our asses off to do stuff that we aren't actually being paid to do.
In my experience, the suits don't really give a shit if asking for features A,B, and C, will make the software more complex, or wasn't part of the original feature specification, they just want it to work, and they want it on schedule. This means that energies are focused on implementing those features, because the expenditure of energy required to offset the complexity they will introduce is not financially justifiable in the time frame permitted to implement the features.
In a nutshell complexity arises as software evolves simply by virtue of the 2nd law of thermodynamics, and requires a non-trivial expenditure of energy to avoid unless the new features are entirely orthogonal to existing system functionality Software engineers don't *like* complexity, but have usually become highly adapted to working with it. The additional effort to avoid complexity increase may be justifiable in the long run of a software lifecycle, but the people who pay our salaries often don't care about what the software will be like in 3 or 4 years time, let alone 10.
MediaWiki (Score:3)
MediaWiki [mediawiki.org], the software behind Wikipedia and thousands of other sites, seems like a great example of complex, robust software that's been well maintained for ~20 years. Kudos to Wikimedia Foundation.
Not modern (Score:2)
Re: (Score:2)
Legacy = anything I didn't write (Score:2)
The problem with these arguments is they are hopelessly abstract and non-specific. Complexity by itself is neither good nor bad. Whether simpler is actually better depends entirely on particulars of the endeavor. There is no universal truth to the assumption less complex solutions are magically better or ultimately the one that proves to result in cheaper to maintain solutions.
There are no universally correct approaches yet we have entire disciplines like Agile incentivizing piecemeal implementation of d
Bloat (Score:2)
What about complexity's cousin, bloat? That seems to be alive and well.
New isn't always better (Score:2)
It's a management problem (Score:2)
I've been attached to systems as a part-time administrator. Where local management gave me enough leeway to document the system, add comments and even do some software revision (bug fixes, adding new features, etc). But the scope of my job was poorly defined enough to allow me to refactor poorly written modules and port them from one system to another.
Eventually, we had a well documented, easy to maintain system that we were able to quickly add features when needed. And the company IT department got pissed
The rule of good old Mervyn Peake. (Score:2)
Till the last
Of the first
Depart,
And the least
Of the past
Is dust
And the dust
Is lost.
Hold fast!
Gormenghast.
It's not about code. (Score:2)
It's about knowledge.
Until the value in preserving the tribal institutional knowledge expressed in the complexity of code base "A" is recognized, we'll continue to build complex systems, and let them dead end.
People restart things because they're built to run, not to be learned, not to be shared. Companies will use one great coder to get a bunch of work done, but won't spend the time necessary to have that one great coder teach 20 other people what they did. And indeed, it might be difficult to find a cod
Re: (Score:3)
I know the past two years have been terrible, but I'm not sure if you can make the argument that 1950 is better than 2022. I mean, just from a technological point of view, you've got massive improvements in quality of life, with mobile phones, always-on entertainment, medical innovation, unleaded gasoline...now, you're right, before feminism corrupted the workplace and doubled the labor pool (and halved the wages), you could have single earner households much more often, but giving women the ability to cho
Re: (Score:3)
Okay: https://www.whitehouse.gov/bri... [whitehouse.gov]
Do you deny that has a net negative impact on gasoline prices?
Article really says a lot about nothing? (Score:2)
It seems to me that software typically gets more complex over time because of feature requests. It has nothing to do with engineers "liking complexity" or any of that. It's just that the more a company really uses an application and employees rely on it, the more they come up with wish-lists of additional things they'd like it to do.
I see developers, even at my current job, putting some effort into "optimization" of existing code. But that doesn't really mean they're trying to subtract features and reduce
SystemD (Score:2)
When will that join the list of complex software being removed for something like inet
Words of wisdom (Score:2)
Taleb must work in my department
Two things (Score:5, Insightful)
Two things annoy me in software:
1) lack of comments. Not useless comments that say "this adds one to x". Rather, comments that detail the rationale for this thing. What uses it.
I shouldn't have to use a tool (or a brute-force grep) and ten minutes just to get an appreciation of what I'm looking at. I want to know WHY this code exists, not how it works. Maybe give me an extra line of your thoughts at the top if there are any gotchas
2) the phrase "Code Smells". Whoever came up with that stupid condescending arrogant put-down had better be the smartest rockstar developer that ever lived. I've seen a lot of code. Good stuff, not-as-good stuff. I marvel at the clever stuff but I don't diss the imperfect stuff.
Everyone writes and thinks differently. Someone wrote that code and it solved a problem at that time. Maybe they weren't aware of this or that technique, or library. Or didn't handle an exception. So what.
Everything else I can live with.
Different kinds of complexity (Score:3)
There are two different kinds of complexity in software - architectural complexity and business rule complexity. I'm not sure the author distinguishes between the two.
One of the problems is that business rules are created in the software but not documented (the code is self-documenting, dontcha know!) so when the software needs to be replaced, nobody actually knows what the actual business rules, with all the edge cases and obscure requirements.
Blaming developers for business rule complexity (and for the business not knowing its own business rules) is just shooting the messenger.
Mostly agreed, but (Score:2)
FTA "Admit it: as much as we complain about other people's complexity, we love our own"
NO I DON"T! I hate the complexity that creeps into my code. I try hard to keep it simple, but it's a futile effort. It may turn out to be like the law of entropy. Software complexity increases even when the creator tries hard to avoid it
Its amazing how wrong many people are (Score:3)
Re:GUI is the key (Score:4)
Sufficiently complex software written that way becomes a problem in and of itself. I used to do Tivoli support. At the time it ran on Windows, OS/2, and something like 42 different flavors of Unix, with three different RDBMS backends to choose from, and everything down to the management GUI layout is dynamic (and object-oriented, and CORBA-based.) Most of the methods were scripts that had to utilize the different OSes to accomplish the same things. The system could fail in all kinds of interesting ways as a result...
Re: (Score:2)
You are talking about poor implementations, not an issue with the system.
The GUI does not need to know how to operate on multiple systems, it just needs a specified system to run select scripts on.
That is, you do not randomly select the server to run the script that the GUI button triggers. Instead it goes to server #35, via RDBM #2, runs a script that ONLY works on server #35, RDBM #2, then forwards results to a specific computer.
It does not matter where you happened to be when you trigger the GUI, you ca
Re: (Score:2)
In my experience developers only car about old code when someone raises an issue, typically a new requirement. Very few developers want to tinker with code that is working unless they get some sort of architectural itch.
They only complain about the old code when they are asked to implement something and it takes ages because of perceived problems with the old code base.