'How the Boeing 737 Max Disaster Looks to a Software Developer' (ieee.org) 388
Slashdot reader omfglearntoplay shared this article from IEEE's Spectrum. In "How the Boeing 737 Max Disaster Looks to a Software Developer," pilot (and software executive) Gregory Travis argues Boeing tried to avoid costly hardware changes to their 737s with a flawed software fix -- specifically, the Maneuvering Characteristics Augmentation System (or MCAS):
It is astounding that no one who wrote the MCAS software for the 737 Max seems even to have raised the possibility of using multiple inputs, including the opposite angle-of-attack sensor, in the computer's determination of an impending stall. As a lifetime member of the software development fraternity, I don't know what toxic combination of inexperience, hubris, or lack of cultural understanding led to this mistake. But I do know that it's indicative of a much deeper problem. The people who wrote the code for the original MCAS system were obviously terribly far out of their league and did not know it.
So Boeing produced a dynamically unstable airframe, the 737 Max. That is big strike No. 1. Boeing then tried to mask the 737's dynamic instability with a software system. Big strike No. 2. Finally, the software relied on systems known for their propensity to fail (angle-of-attack indicators) and did not appear to include even rudimentary provisions to cross-check the outputs of the angle-of-attack sensor against other sensors, or even the other angle-of-attack sensor. Big strike No. 3... None of the above should have passed muster. None of the above should have passed the "OK" pencil of the most junior engineering staff... That's not a big strike. That's a political, social, economic, and technical sin...
The 737 Max saga teaches us not only about the limits of technology and the risks of complexity, it teaches us about our real priorities. Today, safety doesn't come first -- money comes first, and safety's only utility in that regard is in helping to keep the money coming. The problem is getting worse because our devices are increasingly dominated by something that's all too easy to manipulate: software.... I believe the relative ease -- not to mention the lack of tangible cost -- of software updates has created a cultural laziness within the software engineering community. Moreover, because more and more of the hardware that we create is monitored and controlled by software, that cultural laziness is now creeping into hardware engineering -- like building airliners. Less thought is now given to getting a design correct and simple up front because it's so easy to fix what you didn't get right later.
The article also points out that "not letting the pilot regain control by pulling back on the column was an explicit design decision. Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all?
"MCAS is implemented in the flight management computer, even at times when the autopilot is turned off, when the pilots think they are flying the plane."
So Boeing produced a dynamically unstable airframe, the 737 Max. That is big strike No. 1. Boeing then tried to mask the 737's dynamic instability with a software system. Big strike No. 2. Finally, the software relied on systems known for their propensity to fail (angle-of-attack indicators) and did not appear to include even rudimentary provisions to cross-check the outputs of the angle-of-attack sensor against other sensors, or even the other angle-of-attack sensor. Big strike No. 3... None of the above should have passed muster. None of the above should have passed the "OK" pencil of the most junior engineering staff... That's not a big strike. That's a political, social, economic, and technical sin...
The 737 Max saga teaches us not only about the limits of technology and the risks of complexity, it teaches us about our real priorities. Today, safety doesn't come first -- money comes first, and safety's only utility in that regard is in helping to keep the money coming. The problem is getting worse because our devices are increasingly dominated by something that's all too easy to manipulate: software.... I believe the relative ease -- not to mention the lack of tangible cost -- of software updates has created a cultural laziness within the software engineering community. Moreover, because more and more of the hardware that we create is monitored and controlled by software, that cultural laziness is now creeping into hardware engineering -- like building airliners. Less thought is now given to getting a design correct and simple up front because it's so easy to fix what you didn't get right later.
The article also points out that "not letting the pilot regain control by pulling back on the column was an explicit design decision. Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all?
"MCAS is implemented in the flight management computer, even at times when the autopilot is turned off, when the pilots think they are flying the plane."
Really, you don't understand? (Score:2, Insightful)
I don't know what toxic combination of inexperience, hubris, or lack of cultural understanding led to this mistake.
Funny. Everyone else does. Outsource to cheapest overseas software firm available. You get what you pay for.
Re: Really, you don't understand? (Score:2, Funny)
Thanks Trump. We never saw any 737-Max airplanes plunging from the sky under Obama's watch.
Trump owns this.
Re: (Score:2)
under Obama's watch
Because the 737 Max wasn't in service during Obama's administration. It was certified during that time. But then the FAA's over dependence on Boeing's analysis got it's start during the Reagan administration.
Re: Really, you don't understand? (Score:5, Informative)
The 737MAX received its certification in March 2017, under Trumps administration. That doesn't change the fact that the certification program was conducted under Obamas administration, nor that there have been no significant changes in FAA certification policy under the past several administrations.
Re: Really, you don't understand? (Score:2, Insightful)
YEP!
I called this BS when I first saw it here on Slashdot. Boeing seriously shet the bed and took both its own people and regulators down the drain with it.
There were aviation "experts" saying Boeing didn't do anything wrong and that this was all down to pilot error. The amount of sychophantry going on was and still is INSANE!
Boeing needs a full, systematic, cleaning of its house or else forced to shut down completely. Too many livew were lost and too many people trying to deflect blame for it to ever be tr
Re: Really, you don't understand? (Score:5, Funny)
And furthermore, Notre Dame stood unburned during the whole Obama administration.
Carpenters can cut their fingers off with a saw (Score:5, Insightful)
And pilots can stall an airplane. But they generally don't, because they're trained to properly use the equipment.
Having the entire control envelope available is important, not so pilots can do stupid stuff, but so that pilots can respond properly in situations that egghead PhDs and/or software jocks can't predict. Including this one.
Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all?
Disturbing.
Re:Carpenters can cut their fingers off with a saw (Score:5, Insightful)
Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all?
Disturbing.
To put this in armchair quarterback terms, what would have happened if US Airways Flight 1549 had automatically pulled up because "ground collision" was imminent. I'll give you a hint, it would have turned a no-loss-of-life-event into a loss-of-life event.
Of course, had Air France Flight 447 forced the nose down against the pilot's wishes, that could have saved lives.
Historically, airplanes have deliberately erred on the side of giving control to the pilots. Until AI is smarter than a human, the pilot should be the final arbiter of decision. This means that some bad pilots will kill, and some good pilots will save, but all pilots will have the power to make the difference.
Re: (Score:2)
In normal control law, AF447 would have forced the nose down to prevent the stall - it's the very fact that AF447 was not in normal control law that meant that protection was not available to it.
Re:Carpenters can cut their fingers off with a saw (Score:4, Informative)
It isn't inherently unstable - the MAX has a particular characteristic which could trivially be handled by a 737NG pilot with the correct conversion training, but Boeing decided instead to pursue a common type rating with the earlier 737NG, which required all deviations from the 737NG handling characteristics to be mitigated, so they introduced the MCAS system to bring the MAX in line with the NG. That doesn't mean the MAX is inherently unstable, it simply means that the mitigation for the change in handling characteristics was not done properly.
The 737NG has its own similar characteristics which are similarly handled in software and automation systems to align it with the 737 Classic that came before it. And the 747-8 with the 747-400, and the -300, and the -200 and the -100...
Re:Carpenters can cut their fingers off with a saw (Score:4, Interesting)
Historically, airplanes have deliberately erred on the side of giving control to the pilots. Until AI is smarter than a human, the pilot should be the final arbiter of decision. This means that some bad pilots will kill, and some good pilots will save, but all pilots will have the power to make the difference.
Thank you thank you thank you.
And even if the AI is smarter than a human, it's all based in physical hardware, getting inputs from hardware sensors, going through wires, etc., and hardware can fail.
And the more I'm reading about these crashes, the more I see where the pilots tried to turn electric trim off, autopilot on and off, and still could not get control of the plane. Maybe some other thing was wrong, and everything, not just MCAS, needs to be examined.
Re: (Score:2)
This means that some bad pilots will kill, and some good pilots will save, but all pilots will have the power to make the difference.
Human caused deaths is preferable to AI caused deaths. Right.
Re: (Score:2)
Historically, airplanes have deliberately erred on the side of giving control to the pilots. Until AI is smarter than a human, the pilot should be the final arbiter of decision.
Except this had nothing to do with smartness, the system got bad data and was responding "correctly" to the faulty value. The problem was that it didn't check both sensors and disabled itself if they disagree. That would have been giving control to the pilots, as it were they had to manually disable a system that didn't realize it was faulty and undo the damage. To use a car analogy, a cruise control can't work with a faulty speed reading. It'll then either speed up or slow down suddenly and you have to fix
Re: (Score:2)
It can be fixed with AI smartness though. AoA sensor shows that there is supposed to be a stall, check other sensors if they confirm (if the airplane is climbing then it probably not stalled etc). After all, this is what the pilots do. But that would make the system extremely complex.
With your car analogy it's the same, you can make a smarter cruise control that uses the speed sensor, compares it with GPS, accelerometers and trying to visually calculate the speed of the car in order to function correctly ev
Re: Carpenters can cut their fingers off with a sa (Score:2)
Re: (Score:2)
Re: (Score:3)
The "design decision" was a necessity since the only time that MCAS should have been operating in the first place is when the pilot was already pulling the nose up.
Strangely, I thought the reason for MCAS was because the 737 MAX engines were larger and placed further forward, thus increasing the tendency of the aircraft to unexpectedly nose upward from what would otherwise be normal and reasonable throttle inputs.
"Pulling the nose up" through the yoke is a completely different situation, and one that software could have, should have, and apparently now does [boeing.com], readily discriminate.
Re: Carpenters can cut their fingers off with a s (Score:2)
Strangely, I thought the reason for MCAS was because the 737 MAX engines were larger and placed further forward, thus increasing the tendency of the aircraft to unexpectedly nose upward from what would otherwise be normal and reasonable throttle inputs.
It's not just throttle inputs; there are a number of factors that can cause the aircraft to "unexpectedly nose upward". But, as I said to the other guy, this in and of itself is not a problem, and just a nose-up condition does not trigger MCAS. The system was intended to only trigger during manual flight when the AOA approaches a stall. That pretty much guarantees the pilots will be pulling up on the yoke at the moment that the system needs to activate.
"Pulling the nose up" through the yoke is a completely different situation, and one that software could have, should have, and apparently now does, readily discriminate.
I'm not sure which part of that article you're refer
Re: Carpenters can cut their fingers off with a s (Score:2)
Re: Carpenters can cut their fingers off with a (Score:2)
The "one sensor" thing has absolutely nothing to do with what was being discussed. You can obviously go ahead and repurpose my analogy for that discussion too (as you did) but doing so just means that you've used my analogy to address a different problem. You haven't actually responded to my comment.
As such, I'm not sure why you prefaced your comment with "on the contrary".
Re: (Score:2)
The "design decision" was a necessity since the only time that MCAS should have been operating in the first place is when the pilot was already pulling the nose up.
Erm, nope. That was not the design decision. The design decision is: the engines are put "to far" forward. So if the pilot does not take care, the engines push the nose up. And the MACS systems was supposed to compensate for the push up effect of the engine. That is all. Has nothing to do with the pilots decision. However as the system failed it
Re: Carpenters can cut their fingers off with a s (Score:3)
I'm aware that you think you're an expert now because you've read a handful of newspaper articles, but the fact of the matter is that you still have absolutely no clue what you're talking about. No surprise there; you rarely do.
For those who are also under the impression that the aircraft is "unstable", but are actually willing to learn, check out discussions like this one for some pretty good info:
https://www.quora.com/Is-the-B... [quora.com]
Re: Carpenters can cut their fingers off with a s (Score:2)
The nose pitching up is not a problem in and of itself; it's only a problem when it causes the angle of attack to approach a stall condition. You're not going to get that just from increasing thrust during level flight.
Re: Carpenters can cut their fingers off with a s (Score:5, Insightful)
The nose pitching up is not a problem in and of itself; it's only a problem when it causes the angle of attack to approach a stall condition. You're not going to get that just from increasing thrust during level flight.
The story I've read from pilots, in many places including here, is in fact that is _exactly_ what happens- throttle up and the MAX plane climbs. And without compensation, it will nose up until it stalls.
My argument, and I absolutely disagree with analogies of ABS and others, is for example, most of us drive cars. A constant job when driving is steering- "keeping it between the ditches" as a friend of mine says. In a plane, you're less worried about left / right- you're more paying attention to altitude, vertical speed, artificial horizon. If the plane starts to climb or drop, you push or pull to compensate. It's what you do. Constantly. (well, you also use trim and autopilot, but beginners learn keeping it level).
I do not understand the (lacking) mentality of saying a system like MCAS _should_ override the pilots. Even if it was really well designed, based on lots of redundant sensors and GPS and whatever else, what it does is not that big a deal. And, the fact that MCAS had THAT much large / course control over the elevator is a HUGE design flaw.
And, the fact that if failed and killed almost 350 people, in spite of pilots' efforts to counteract MCAS's incorrect controls, proves that pilots need final control.
I don't blame engineers / programmers. Their job is to design a "black box". Someone is the overall program / product manager. There should have been umpteen engineering design review meetings. That all needs to be investigated. I'm SURE, just as with NASA's Challenger and Columbia, that there were engineers who said MCAS, as designed, could be a problem someday. And if that did not ever happen, then something is very wrong with Boeing and FAA.
Re: Carpenters can cut their fingers off with a s (Score:4, Interesting)
The story I've read from pilots, in many places including here, is in fact that is _exactly_ what happens- throttle up and the MAX plane climbs.
This is true to some extent for most modern airliners, and is not a problem.
And without compensation, it will nose up until it stalls.
This is not true. I mean possibly if the pilots have all gone to sleep and therefore don't realize that they're climbing like crazy, then eventually the plane might hit the ceiling and stall. But that's just a stupid scenario which has no bearing on any of the concerns surrounding the design of the MAX.
I do not understand the (lacking) mentality of saying a system like MCAS _should_ override the pilots.
It shouldn't, and wasn't designed to, any more than ABS overrides the driver. MCAS is designed to augment the pilot, not to overrule him. It's fair to point out that the implementation leaves much to be desired, but that doesn't change the intent.
And, the fact that MCAS had THAT much large / course control over the elevator is a HUGE design flaw.
It doesn't control the elevator at all, but yes, at this point everyone including Boeing agrees that the degree of movement which can be induced by MCAS is unacceptable, which is why it's being changed.
And, the fact that if failed and killed almost 350 people, in spite of pilots' efforts to counteract MCAS's incorrect controls, proves that pilots need final control.
The pilots had final control; they had the ability to switch off the electrical trim system. The Lion Air crew didn't seem to know how to do it (they should have). The Ethiopian Airlines crew did know, but they seem to have run into a different issue after doing so. This part is speculation based on incomplete data, but it seems that after they switched off the system they were unable to manually trim the stabilizer due to excessive aerodynamic loading. That's also an issue, but not something that's unique to MCAS equipped planes.
I don't blame engineers / programmers. Their job is to design a "black box". Someone is the overall program / product manager. There should have been umpteen engineering design review meetings. That all needs to be investigated. I'm SURE, just as with NASA's Challenger and Columbia, that there were engineers who said MCAS, as designed, could be a problem someday. And if that did not ever happen, then something is very wrong with Boeing and FAA.
On this part, at least, we are in agreement. There should be an investigation, and the FAA certification process could probably use some refining.
Re: (Score:2)
Thanks for an awesome reply. It's too late for me to write intelligently. I think we agree well, and where you have a difference, I'm just going by what I read, including some of the official investigations into the MAX crashes- specifically about the elevator (electric) jackscrew, but maybe I misunderstood or I'm using the wrong term- going from one extreme to the other, and the "g"-forces the people in the back of the plane must have experienced.
Actually one of my (older) vehicles has ABS that I don't t
Re: Carpenters can cut their fingers off with a s (Score:2)
No worries. The jackscrews you're thinking of don't drive the elevators, they drive the trim system. On the 737 that's the entire horizontal stabiliser (the thing that the elevators are attached to). You had the right idea in that MCAS does affect pitch, but it does it by adjusting trim rather than controlling the elevators.
I am not currently a pilot no. I do have experience with both operating and maintaining aircraft though.
Re: (Score:3)
What do you think about the decision to try to correct an unstable airframe with software? Most disasters occur for multiple reasons. This one had two main causes.
There are jets out there that have instability built in. As some wag noted, their cost of a failure is a pilot ejection, and a loss of the airframe.
MCAS allows both the loss of the airframe, and conveniently delivers all the passengers directly
Re: Carpenters can cut their fingers off with a s (Score:4, Insightful)
Odd, you are the only one claiming that the 737 Max is a stable airframe.
No, but I am apparently one of the few who understands what concepts like static and dynamic stability actually mean when it comes to aircraft, as well as how the MAX differs from earlier 737s. Calling the MAX "an unstable airframe" is a gross misuse of the phrase. If you applied the same criteria to other aircraft you would have to conclude that every single airliner in existence is likewise "unstable". Find me an airliner which don't have a tendency to undergo Dutch Roll, for instance.
The fact of the matter is that all airliners are positively stable, but can approach neutral stability under some specific conditions. The 737 MAX is no different in that regard. However every aircraft handles differently, which is why pilots require type certification; they have to be familiar with the quirks of the aircraft they're flying. MCAS was added to eliminate the need for specific type training for the MAX; it's intended to make it "feel" the same as as other 737 variants so that the pilot doesn't need to know to let off on the yoke under high AOA low speed manoeuvres. The aircraft will fly perfectly fine without it, but the pilots would need to be aware of the changed handling characteristics of the aircraft, just like they would need to be aware of the handling characteristics if they went from flying a 737 to flying an A320.
I understand that you're trying to find a simple equation to make yourself feel safer, but you're failing miserably at understanding the subject you're discussing. Aircraft stability is complicated; you can't boil it down to a boolean like you're trying to do.
Re: (Score:2)
As a software developer you could be so far out of the loop that you believe/assume there are three MACS systems installed with one sensor each ...
Re: (Score:2)
There is an elephant in the room that people are deliberately being diverted against. "Cultural problem" in software? What kind of double speak is that?
The reality is that putting engines in front changed the design of the airplane and Boeing didn't want to train pilots for that. Boeing decided - made a conscious decision - to convert pilot's input into something else and effectively deceive pilot. The fact that they told pilots about software doing something cannot be an excuse. They basically converted an
Re: Carpenters can cut their fingers off with a s (Score:4, Insightful)
And, the fact that if failed and killed almost 350 people, in spite of pilots' efforts to counteract MCAS's incorrect controls, proves that pilots need final control.
I narrow it down to 2 causes. Both pretty equal mortal sins. One is your noting above that if a pilot sees that the plan is doing something that will have a bad ending, the pilot can override the automation.
Second is the root cause of the problem. You don't make passenger jets inherently unstable. Unstable is for fighter jets. What is criminal ( and I mean exactly that) is that the design instability wasn't for anything other than trying to quickly make a jet to compete with Airbus, and then using software to attempt to make it handle like a normal and properly designed 737.
They failed rather miserably it would seem.
I don't blame engineers / programmers. Their job is to design a "black box". Someone is the overall program / product manager. There should have been umpteen engineering design review meetings. That all needs to be investigated. I'm SURE, just as with NASA's Challenger and Columbia, that there were engineers who said MCAS, as designed, could be a problem someday. And if that did not ever happen, then something is very wrong with Boeing and FAA.
That second problem - the unstable airframe - is where to start. That was a criminally negligent decision. Design and build a stable airframe, and you don't need software to make the plane fly straight. It flies straight by nature. Then the software can be designed to make it fly better.
No doubt the jerry-rigging of the basic 737 was considered to be a cost cutting measure. I wonder how much money they will have saved by the time this is all over.
Re: Carpenters can cut their fingers off with a s (Score:4, Informative)
It's also not the problem.
All aircraft with engines under the wings pitch up when you increase the throttle which is to say the angle of attack increases, not just the rate of climb. Also, all jet and turbofan engines generate some lift from their nacelles, which, if they are forward of the centre of mass, will also cause the plane to pitch up. This is true of all the modern airliners with engines under the wing. The opposite is true of aircraft with rear mounted engines like the MD-80, but I don't know if anybody still makes them.
The 737 MAX's engines are very big and very far forward, so their tendency to pitch the aircraft up when you increase thrust is higher than older 737's. In particular, for very high angles of attack, the 737 MAX engines generate so much lift, that, left to itself, the plane's angle of attack will increase until it stalls.
Re: (Score:2, Informative)
The nose pitching up is not a problem in and of itself; it's only a problem when it causes the angle of attack to approach a stall condition. You're not going to get that just from increasing thrust during level flight.
Actually, increasing thrust does cause a pitch up on this model of aircraft. Go to YouTube and look for Mentour Pilot. He has several videos covering a wide array of aircraft questions. He has a few about the 737 MAX, at least one about pitch changes caused by thrust on various aircraft, and why the MCAS system is considered necessary for this particular aircraft [youtube.com].
From my own other reading, it appears that the greatest contributor to the MCAS system "software failure" is that the system relies on the input f
Re: Carpenters can cut their fingers off with a s (Score:5, Informative)
There is just so much wrong with your entire comment ... I'll try to be brief:
Actually, increasing thrust does cause a pitch up on this model of aircraft.
You're not disagreeing with what I said. If you want more details scroll up slightly on the page and read my other, longer comment.
why the MCAS system is considered necessary for this particular aircraft.
Anyone who tells you that it is necessary has no clue what he's talking about. The system was added to minimize pilot retraining requirements. The plane flies perfectly fine without it.
it appears that the greatest contributor ... is that the system relies on the input from one sensor
That is a problem, but not the biggest one. There are other issues with the system which are more serious. You can read Boeing's updates on the fix for a better feel of the issues:
https://www.boeing.com/commerc... [boeing.com]
unless the aircraft purchaser paid for an upgrade that can include input from another sensor.
Total baloney as discussed further down on this page. There was no such upgrade option.
In the latest incident, the pilots had no training on how to override the MCAS system.
More nonsense. The Ethiopian Airlines crew (ie. latest incident) knew of the issue, and took appropriate action to disable MCAS input on the controls.
It was all about "saving money". (Score:2)
> The 737MAX could have passed certification and been safe to fly without
> MCAS, but Boeing ran the risk of having to classify the MAX as a new 'type'
> (meaning all pilots who fly it would have had to get and maintain a type
> rating for that new type which is expensive and time consuming for the
> airlines). MCAS makes the 737 MAX behave enough like the rest of the
> 737 models that the MAX didn't need to be assigned a new type.
Exactly, see https://www.youtube.com/watch?... [youtube.com] Note: Juan Brown
Re: (Score:3)
MCAS makes the 737 MAX behave enough like the rest of the 737 models that the MAX didn't need to be assigned a new type.
Except every once in a while when it dives the plane into land or sea.
Re: (Score:2)
You're not going to get that just from increasing thrust during level flight.
Yes you do. And that is the whole point of the MACS software existing. In a different airplane you would be right, but in this type you are wrong.
Re: (Score:2)
MCAS looks like it was written by an intern or by someone without experience.
If AoA is greater than set value, trim nose down for 5 seconds, wait 10 seconds and check again.
Oh, the airplane has two AoA sensors? Well, let's just alternate between them. What could possibly go wrong?
Quality assurance too costly? (Score:2)
Re: (Score:3)
Boeing may be fined and will most likely be sued by the families of the victims (and the airlines that are now losing money because the MAX is grounded). Calculate the total amount of money Boeing will pay because of this, add the cost of the fix and divide the total by the number of people who died.
That's how much the life of a passenger was worth in this case.
Re: (Score:2)
Just read up on the "Ferengi Rules of Acquisition".
The software takes multiple inputs (Score:2, Informative)
It can take inputs from multiple sensors, but that's an upgrade. Third world airlines can't afford it unlike the US and European carriers.
And unstable air-frames aren't anything new. They've been around since the 1970's when the F-16 and other now obsolete fighters were first developed.
Re: The software takes multiple inputs (Score:2, Funny)
It can take inputs from multiple sensors, but that's an upgrade
You are lying.
Re: The software takes multiple inputs (Score:2)
No, you just don't seem to understand that the article you've linked is talking about a different system.
Re: The software takes multiple inputs (Score:5, Informative)
Well you should read your own link.
"Boeing's optional safety features, in part, could have helped the pilots detect any erroneous readings. One of the optional upgrades, the angle of attack indicator, displays the readings of the two sensors. The other, called a disagree light, is activated if those sensors are at odds with one another."
The optional feature did NOT make the software look at both sensors. It just made it obvious to the pilot that the sensor was wrong without telling the system that was trying to kill them that it was wrong.
Feel more like marketing failure (Score:2)
It can take inputs from multiple sensors, but that's an upgrade. Third world airlines can't afford it unlike the US and European carriers.
This to me is key right here. Who was it that decided it was OK to make that feature optional? That feels like a mix of marketing and engineering, but why would engineering agree to go along with reducing the inputs to a dangerous level? Was that ever really reviewed?
I don't think this error is wholly on software, at all. Of course I would tend to defer to people who
Re: Feel more like marketing failure (Score:2, Informative)
It wasn't optional; it was nonexistent. He may be thinking of a different feature entirely, or he may just be making it up; either way there was never any option to "upgrade" the MCAS in any way.
Re: Feel more like marketing failure (Score:4, Interesting)
I assume this is what he's referring to:
The two safety features in question were an “angle of attack indicator” and an “angle of attack disagree light”, both of which were not included in the aircraft by Boeing as standard safety features
article [theguardian.com]
No idea whether it applies or not since I know nothing about plane systems.
Re: Feel more like marketing failure (Score:2)
It doesn't; both of those are display options in the cockpit and have nothing to do with MCAS.
They're also not safety features since there's nothing in the aircraft operating procedures which would change based on either of those indicators. Which is why they were optional, and why some airlines chose not to purchase them. They're a nice-to-have, nothing more.
Re: (Score:2)
Mr. c6gunner, stop it.
You are simply wrong. No idea why you think 20 people here who explained to you THE EXACT SAME THING now 20 times in the same words are all wrong???
Re: (Score:2)
No. What was optional was an attitude indicator. i.e. a display which said the angle of attack of the aircraft as read by the sensors.
Re: (Score:2)
Re: (Score:2)
The first point is unclear. There is an 'upgrade' that warns the pilot if the sensors disagree, and an upgrade that displays each sensor's readings, but I haven's seen anything that indicates an upgrade to get MCAS to actually look at both sensors (other than as a proposal for the fix after causing 2 fatal crashes).
As for the second point, neutral stability is for fighters and aerobatic planes, not cargo and passenger planes. Negative stability is for fighters only.
No it doesnt (Score:2)
The upgrade is just an informative upgrade that the AoA sensors are out of whack. it helps in that if the warning light is on you taxi back and do not takeoff. Once you takeoff with a bad AoA sensor the warning light cannot save you. If MCAS misbehaves you have 40 seconds to switch it off. The warning light may give you a headstart but if you miss the 40 second window you are dead - upgrade or no upgrade.
What Boeing is doing now is change MCAS software to switch off if the AoA sensors are disagreeing. This
Re: No it doesnt (Score:2)
If MCAS misbehaves you have 40 seconds to switch it off .... if you miss the 40 second window you are dead
Wow. That's a new one. How in the world did you come up with that fantasy?
Re: (Score:2)
Umm, the 737 Max has two sensors. Problem is: MCAS only makes use of one of them. There is an option for a sensor disagree warning. But then this would have required airlines to provide additional training on what to do should this warning appear. And that training would have put to a lie the design philosophy that the Max was similar enough to older 737 models so little additional training would be needed.
Re: (Score:3)
Which is why it seems to me like the MCAS was designed by someone without experience and not checked.
I can almost imagine the conversation:
1: Here, I finished the assignment, this was easy. Just look at the AoA sensor and trim the nose down if it indicates too high.
2: The airplane has two sensors, how do you deal with that?
1: Two sensors, I wonder why. Anyway, I only need one. Hmm, I'll just make it use one sensor for one flight and the switch to another sensor for the next flight. Maybe using the sensor we
Re: (Score:2)
Two sensors can be enough. If the values are too different, disable that particular system (autopilot, etc) and just give control to the pilots, telling them that you do not know which is the correct value for airspeed etc. The pilots have checklists for what to do when something fails (if you do not know the airspeed, set the throttle and pitch to a predetermined safe value).
Re: (Score:2)
You can't expect to dogfight in a 737 either.
Software is cheaper to design and harder to spot faults. With hardware you may be able to see that some part is cracked, bent etc, which would indicate that it is incorrectly designed. You can't easily see the equivalent of "200kg load held up by a single 2mm screw" in software.
Re: (Score:2)
Whatever they are, they will most likely be found now, because Boeing will be looking at those planes with a microscope, since if another MAX crashes for any reason other than a bomb, getting shot or a suicidal pilot it will be really bad for Boeing.
Gross oversimplification (Score:4, Interesting)
From TFA
"When MCAS senses that the angle of attack is too high, it commands the aircraft’s trim system (the system that makes the plane go up or down) to lower the nose. It also does something else: It pushes the pilot’s control columns (the things the pilots pull or push on to raise or lower the aircraft’s nose) downward"
The "trim system" is not the system that makes the plane go up and down. From "https://www.skybrary.aero/index.php/Trim_Systems"
"Trim Systems are considered to be a "secondary" flight control system. By definition, to "trim" an aircraft is to adjust the aerodynamic forces on the control surfaces so that the aircraft maintains the set attitude without any control input. "
So the pilots use the trim setting so they can stop pulling on the yoke. It's kind of like an attitude cruise control.
In this instance MCAS is auto-trimming the plane incorrectly due to a bad sensor reading. And then the pilots did not follow their memory procedures for a runaway trim, shut it off, and use the cranks to manually set the trim. It is possible they tried to use the cranks and could not due to the extremely high speed causing the jack screw to bind. In this instance they are supposed to go nose-down to relieve the pressure but either they were too low already or too freaked out trying to go nose-up to manually go nose-down.
Re: Gross oversimplification (Score:5, Insightful)
To be fair, the fact that the manual trim system can be rendered inoperative in certain flight conditions is in itself a rather large safety concern. It's also an issue which precedes the MAX variant; it has apparently been a known problem for many decades. The only reason it hasn't caused a crash previously is because runaway trim is pretty rare, and runaway trim occurring specifically during very low-level flying would be even more rare.
Yes, there were ways that both of the MAX aircrews could have recovered their aircraft but - at least in the case of the Ethiopean Airlines crash - I can't fault the aircrew much given what we know now. They seem to have done everything according to the book, but simply didn't have the altitude they needed to fix the problem. They could still likely have recovered the aircraft by going outside the manual and doing some very unorthodox things, but blaming them for not doing so would be foolish.
Re: (Score:2)
Shouldn't the autotrim move the trim wheels rather than the yokes?
Re: (Score:2)
Shouldn't the autotrim move the trim wheels rather than the yokes?
The autotrim doesn't move the yokes. It moves the trim wheels, same as when the pilot hits the trim switch on the yoke.
The pilot though will feel the difference in the yoke because as the trim is adjusted the pilot will no longer have to pull to maintain the same attitude.
Re: (Score:2)
No. They disabled the auto-pilot and attempted to use manual trim as described in the documentation. But they couldn't compensate for the MCAS changes in time because the manual trim was too slow. So they re-engaged the system and tried to use the automatic trim which was supposedly faster. But the automatic trim was also slower than the changes the MCAS made.
Re: (Score:2)
No. They disabled the auto-pilot and attempted to use manual trim as described in the documentation. But they couldn't compensate for the MCAS changes in time because the manual trim was too slow. So they re-engaged the system and tried to use the automatic trim which was supposedly faster. But the automatic trim was also slower than the changes the MCAS made.
This could be true, but I read a different possibility. The plane was flying at a very high speed. As the trim reached its extreme position the high speed would cause the stabilizer to impart a lot of force on the jack screw, making it very difficult to manually turn it. Pilots are trained to go nose-down to reduce the force, but maybe it still didn't help. It appears also that when they re-engaged the system the MCAS re-engaged a few seconds later.
This Is What Happens (Score:5, Insightful)
...When you have MBAs in charge cutting costs, like hiring software developers without sufficient relevant experience in flight control systems, limiting testing/simulation/crosschecking, etc etc.
This was entirely preventable and even predictable.
Boeing owns this. Hope they've saved plenty of cash for the lawsuits and other legal troubles that will be incoming.
Strat
Re: (Score:2)
Have to agree here. Blaming software developers is dumb, as they never have any autonomy except maybe at a startup. They don't design systems, and they rarely understand the full details of what they're designing anyway, they're given a task (process sensor input) and not asked "have you checked all the numbers for the aerodynamic design?" And engineering in general may have build the thing but under the direction of management. I suspect most engineers didn't notice any flaws because they're compartment
Re: (Score:2)
More properly: Software and Systems Engineering and Integration has been subverted and subsumed by accountants.
NOT "dynamically unstable" (Score:5, Informative)
"Big strike #1" is totally incorrect. "Boeing produced a dynamically unstable airframe" is not the case. Rather, the engine change slightly reduced the stability to less than the minimum required by FAA regulations, thus requiring a compensation system to artificially increase the stability back to the minimum required. It was never unstable, PERIOD. It's still quite stable even without MCAS - just not quite as stable as required by regulation.
I cannot disagree with the incredulity of designing this system with just one AOA sensor as an input. I also cannot fathom how they could possibly design it to NOT have a practical upper limit of its authority, or without an extremely visible notification of the action of the MCAS system. In the name of "we won't have to retrain the pilots" it violates a key tenet of automation: when you change the mode of operation, you notify the operator or user.
FWIW, I am an aircraft flight test engineer with a specialty in stability and control, with 29 years of experience in the field, and over 10 years of testing on Boeing-derived commercial-class aircraft and autopilot systems. I've flown simulator variants of the 737 and its autopilot, and know exactly how confusing automation can be, especially when it does something unexpected. From the cockpit in real flight, I've watched trained, highly experienced test pilots completely lose their ability to focus on where the airplane is headed because they're trying to troubleshoot a relatively unimportant system that just messed with their sense of expectation. I have a lot to say about this crash, and none of it is good for Boeing.
Re: (Score:2)
Re: NOT "dynamically unstable" (Score:2)
1-The designers could have auto-trimmed the aircraft to match engine thrust levels instead of angle of attack. That would make the aircraft fly like the previous design.
No, it wouldn't, since the issue isn't due strictly to "thrust levels" but also things like airspeed and angle of attack.
2-The trim toggles should work always, instead of forcing pilots to crank the trim up or down.
Yeah, the electrical trim system should work after you turn it off. Good plan.
3-Use both angle of attack sensors and compare, the data is already in the computer.
Only smart advice. Good news: it's part of the fix.
Re: (Score:2)
It's still quite stable even without MCAS - just not quite as stable as required by regulation.
A definition of unstable is not stable, if there is a requirement for for something to be considered stable and it isn't met then by definition it is unstable.
Re: (Score:2)
There are 3 big mistakes Boeing made, First was the decision that (in order to save money and to have something out fast to compete with the A320Neo) the new airplane had to have the same general size and shape (body, wings, tail, landing gear etc) as the old 737 rather than properly designing it so it could take the new engines without causing stability problems. (hence the decision to fix the stability with software rather than redesign the hardware to make it go away)
The second was (again to save money a
Re: (Score:2)
Its yet another example of regulatory capture in the USA.
Lets hope it doesn't take another pile of dead bodies and twisted metal before action is taken to stop manufacturers being able to declare their own products as safe to operate.
Re: NOT "dynamically unstable" (Score:5, Informative)
"After the test flights began in early 2016, Boeing pilots found that just before a stall at various speeds, the Max handled less predictably than they wanted. So they suggested using MCAS in those instances, too, according to one former employee..."
Source: NYT: Changes to Flight Software on 737 Max Escaped F.A.A. Scrutiny [nytimes.com]
Re: (Score:2)
You are incorrect as is OP (Score:2)
The Max is not unstable. It just flies differently than the NG. Which means pilots need to do difference training. But Boeing wanted to sell it as it flies the same as NG (and requires no new training) so they added MCAS to artificially make it seem to fly the same. MCAS was never needed from a safety perspective. For a marketing need an extraneous safety feature was added and that feature malfunctioned and crashed the plane.
Re: (Score:2)
If you sling them engines underneath, you will get a pitching moment on applying thrust, but that's not the problem here.
It's rather that as soon as you have something outside the center of gravity line it will impact the behavior of the aircraft. Low hanging engines are great from maintainability perspective, but they have a tendency to suck up crap from the ground and as noted also impact the behavior of the aircraft depending on thrust and airspeed.
The Max 8 and Max 9 of the 737 may very well be the end of the 737 line as they probably are a bit similar to what happened when the AC Ace got a 427 - only a few were able to c
Yesterday, today ... (Score:3)
It is astounding that no one who wrote the MCAS software for the 737 Max seems even to have raised the possibility of using multiple inputs, ...
It was on the list of things to do "tomorrow" but the scrums kept running long, so ...
No, not the programs fault (Score:2, Insightful)
The OP is out of his league. It sounds like slander to me.
I haven't heard anything about software being buggy.
If the engineers and analysts provided the wrong specs, that's not the programmers fault.
If the software tester didn't find any bugs, then it's not the programmers fault.
If management knew the software was buggy, and let the product ship, then that was managements fault.
None of this points to the programmer.
Pilot's view of this problem (Score:5, Informative)
Re: (Score:2)
Thanks for the link - very informative.
I've made aircraft avionics, and Boeing is wrong. (Score:5, Informative)
I've written software and been a systems engineer for aircraft instrumentation, and I'm very familiar with FAA standards at all levels, particularly at the certification level. I'm also familiar with the front-end of the process, the gathering and analysis/refinement of requirements.
Part of the Boeing problem has been assigned to the existence of DERs, independent consultants/contractors who are certified to act on the FAA's behalf.
Some seek "friendly" DERs willing to grease the certification path. My employer was different, instead pursuing good professionals who were total assholes when it came to FAA certification. We fired more DERs than we kept when they didn't know their shit. It's the difference between an accountant who will help you cheat on your taxes in ways the IRS won't see, and a CPA who's more ethical.
We actually hire two DERs: One very senior (and expensive) as an auditor, and a junior one he recommended who was willing to work in the trenches with us. She was a real trooper.
Our goal was to learn how to do FAA certification both faster and better (less wasted effort). Not faster and cheaper or easier or sleazier. We were a small company, and one mistake would be the end of us. Our DERs helped us completely redesign our internal certification system, costly the first time around, and a bargain thereafter. Lots of work, but great results. The FAA loved us.
Boeing views FAA certification as just another step in another process. It's shameful that people have to die for a company to change its status quo.
Re: (Score:2)
Agile! (Score:5, Insightful)
Do the first version fast to get it out the door making money.
Fix the bugs in the next version.
Creeping into every software project with the promise of faster to market, for more profits.
They made their release date. Now time for next version with fixes.
Multiple Inputs (Score:2)
It is astounding that no one who wrote the MCAS software for the 737 Max seems even to have raised the possibility of using multiple inputs, including the opposite angle-of-attack sensor, in the computer's determination of an impending stall. As a lifetime member of the software development fraternity, I don't know what toxic combination of inexperience, hubris, or lack of cultural understanding led to this mistake. But I do know that it's indicative of a much deeper problem. The people who wrote the code f
This is not a software issue (Score:2)
It's a engineering problem and one of simple greed. They stuck a engine under it that was two large for the air frame. As a result the moved it up where the intake was above the leading edge of the wings. This caused the plane to inherently pitch up under thrust. They should have never put that engine on that plane and it wouldn't have required so much tweaking of the software on the plane. All of this could have been overcome if they had not decided to make critical instrumentation a paid for upgrade.
Comment removed (Score:3)
Re: (Score:3)
Wanna hear from someone who wrote avionics software for Boeing?
When I was there and doing that, everything got cross-checked and single points of failure were unthinkable. Even the loudmouth job shopper who literally did not know the difference between a bit and a byte never made a blunder like using only one input.
Re: (Score:2)
So why, in your opiin, was MCAS designed for one angle of attack input, when a second one was available?
Re: (Score:2)
EDIT: Opinion...
Re: Let's take a tally... (Score:2)
I don't think anyone can tell you for sure why it was done that way, but I can give you the most likely scenario based on two known facts:
1. Checking 2 sensors doesn't tell you which one is good; only that one of them may be bad.
2. The initial design of MCAS was supposed to be far less aggressive, with smaller movements and a smaller total degree of travel.
Combine those two facts and it seems likely that the engineers would have assumed that, even with faulty sensor data, activating MCAS would not pose an
Re:Software developers are not experts in everythi (Score:5, Interesting)
Can't even call themselves a Software Engineer. In this case what we really want to hear from are the Systems Engineers who wrote the requirements, all of which were as far as we know met. Personally what I'd like to know is why the mcas system was changed so that it was impossible to have power assisted trim control while having autopilot/mcas off. This was a fundamental change added to the MAX.
Re:Software developers are not experts in everythi (Score:5, Interesting)
I used to work in aerospace, so I have a bit of insight into the culture that goes on internally. Here's what I see as likely to have happened:
The issue was likely related to "We gotta ship on time!! OMG!!" that happens when a large project starts to get behind schedule, or unexpected snafus come up.
In this case, it sounds like the first (and pivotal) failure was the development of an unstable airframe. Rather than admit that they could not meet their deadline (AND keep a reliable airframe), they decided to go ahead with a known problematic one, because "DEADLINES! OMG!!!".
SO, they had to implement a break/fix solution to the problem. However, they were likely over-cost budget already, since they probably spent a good deal of time and effort trying to fix the aeronautical issues with the airframe's design first. Management was likely breathing fire and brimstone, because their sales reps promised an unreasonable timetable on delivery, and GOD FORBID that they come clean about needing time to revise---- So they rush a software fix without implementing hardware changes. (Such as installing redundant sensors.)
Re: (Score:3)
The whole thing sounds like an issue with systems integration. MCAS is just a little system to provide a bit of assistance in a rare part of the flight envelope. It's classified as something that, if it fails, might cause a rough ride for the passengers, but nothing safety critical. The specs say that kind of system needs to have only one sensor and an off switch. Done.
Except the off switch also does something else, and interaction of the MCAS system with other systems ended up giving it more control than
Re: (Score:2)
Well, stop fucking disabling selinux then.
Re: (Score:2)
The Dunning-Kruger effect is probably even stronger for software developers. Even smart ones today rarely have a realistic appreciation for their limits.
Re: (Score:2)
Yes. They should have bought the seat belts. Surely.