Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Software Operating Systems Transportation United States

Boeing Unveils 737 Max Software Fixes (cnbc.com) 249

hcs_$reboot shares a report from CNBC: Boeing previewed its software fix, cockpit alerts and additional pilot training for its 737 Max planes on Wednesday, saying the changes improve the safety of the aircraft which has been involved in two deadly crashes since October. By the end of this week, Boeing plans to send the software updates and plan for enhanced pilot training to the FAA for certification approval. After the FAA approves the fix, Boeing said it will send the software update to customers. Among the notable changes to the MAX flight controls:
  • The plane's Maneuvering Characteristics Augmentation System, or MCAS, automated flight control system, will now receive data from both "angle of attack" sensors, instead of just one.
  • If those disagree by more than 5.5 degrees, the MCAS system will be disabled and will not push the nose of the plane lower.
  • Boeing will be adding an indicator to the flight control display so pilots are aware of when the angle of attack sensors disagree.
  • There will also be enhanced training required for all 737 pilots so they are more fully aware of how the MCAS system works and how to disable it if they encounter an issue.

This discussion has been archived. No new comments can be posted.

Boeing Unveils 737 Max Software Fixes

Comments Filter:
  • enhanced training (Score:3, Interesting)

    by zlives ( 2009072 ) on Wednesday March 27, 2019 @06:35PM (#58344644)

    so.. a youtube link?

    also these are workarounds, why not fix the actual problem of sensor reading incorrectly?

    • by Firethorn ( 177587 ) on Wednesday March 27, 2019 @06:43PM (#58344694) Homepage Journal

      Because the sensors are physical devices, and are this subject to all physical device problems. They can break, corrode, be bent by a physical impact, etc...

      They're regularly inspected, which is about the best you can do.

      • by zlives ( 2009072 ) on Wednesday March 27, 2019 @06:48PM (#58344706)

        which again goes to question the logic behind an automated system based on sensors that could be faulty forcing correction while on manual flight control... but i am sure i don't understand as I am not an industry insider.

        • by Firethorn ( 177587 ) on Wednesday March 27, 2019 @07:05PM (#58344834) Homepage Journal

          I was keeping my response simple, but for "flight critical" sensors the general idea is to have at least 3 and use a voting system. For sensors that are 99.X% reliable, the odds that two will be out such that they are throwing the same erroneous value(or at least within error margins) is quite low. Though there are differences between 'simple' sensors that report back a simple voltage or resistance where determining a fault can be difficult, and complex ones like radar, GPS, that are more likely to tell the system they have a problem. The vanes here are simple sensors.

          Though with the MCAS it was supposed to assist, not be critical, thus 1 vane being enough. Pilots were supposed to be able to override with just more stick application. That assessment is being challenged, and the 2 vane + alarm thing is Boeing hoping to avoid having to avoid installing another sensor for proper 3 sensor + voting reliability, as the extra sensor will be expensive.

          3 good sensors: all good
          2 good sensors: all good(less redundancy)
          2 good sensors, 1 whack - get fixed after landing
          1 good, 1 whack - system unreliable, turn off. Consider landing early.
          1 good - 2 whack(different values) - system unreliable, turn off, consider landing early
          1 good - 2 whack(same values) - hope you notice before crash/fire. Turn off system. Seriously consider landing early. Last good sensor may or may not be usable(does it have an output you can use?). Consider firing maintainers as it is likely at least one was whack when you took off.
          0 good - 2 whack(same values) - same as previous, really. Without minor hope of good sensor being useable.
          3 whack - same as previous. Consider firing maintenance department out of a cannon.

          • by Compuser ( 14899 )

            I am not sure why we do not do five sensors for critical stuff and three for less critical. This whole cost cutting business is shady as hell when lives are at stake.

            • Cost - which of course matters. There are always safety / cost tradeoffs. Overall commercial aviation is very safe, and very inexpensive per passenger-mile, so in general there seems to be a pretty good tradeoff. In this case they may have not gotten it right.

            • by jbengt ( 874751 ) on Thursday March 28, 2019 @08:31AM (#58347392)
              The more redundant devices you use, the more likely that there is a failure of at least one, which is not good, because now you have to decide what' going on. And if the failure modes are not different enough, it may be common that when one fails, many fail. You could be no better off with more and, depending on the math of the specifics, you might be actually worse off with more.
          • by rtb61 ( 674572 )

            Still the idiot version. You have total engine thrust at that time and measured airspeed (airspeed that can be defined by not just onboard instruments but by external data from the air' traffic control system). The design is inherently bad and unsafe, brought about by cheap shitty shortcuts and a corrupt approval system for US aircraft.

            Reality the only safe choice now, DO NOT BUY US AIRCRAFT, the approval system has been entirely corrupted, with the manufacturers self approving the aircraft and the FAA an

            • by Applehu Akbar ( 2968043 ) on Wednesday March 27, 2019 @08:19PM (#58345216)

              Reality the only safe choice now, DO NOT BUY US AIRCRAFT

              A whole set of EU pitot tubes would never ice over above a tropical storm, any more than an EU rudder would snap off in wake turbulence, would they now?

              • An EU rudder never snapped off in a wake turbulence. It was an American pilot using the rudder pedals like a dance dance revolution pad that broke it because he was so scared like a girl of wake turbulences.

                • Now I can't get the image of someone attempting PARANOiA Survivor MAX on ONI with airplane rudder petals out of my head... 0__o
              • by AmiMoJo ( 196126 )

                I hear the Russians make good planes.

              • by 6Yankee ( 597075 )

                Those EU pitot tubes were fitted to replace ones made in the good ol' USA by Goodrich - because of safety issues with the Goodrich ones So nobody exactly covered themselves in glory.

                But it's so cute how you pout and wrap yourself in that star-spangled blankie.

            • by brausch ( 51013 )

              Pretty much every industry worldwide is like this. Auditors check that various reviews and things have been done. The reviews etc. are done by the manufacturers. Take a look at the auto industry and the emissions issues the last few years. The government seldom does the testing, etc. They just set the standards and the manufacturers claim they meet them. Same with the drug manufacturers (see the recent worldwide recall of the blood pressure medicine irbesartan). There isn't enough government expertise or ma

          • by brausch ( 51013 )

            Another thought is that the system needs to have a sense of time as well when working with real sensors. There should be some time smoothing (exponential is simple to implement and usually pretty good at reducing noise in the signal) as well as some tracking of rate of change of the readings as a reality check.

          • I was keeping my response simple, but for "flight critical" sensors the general idea is to have at least 3 and use a voting system.

            Great. How do you determine whether the vote has a correct outcome?

            3 good sensors: all good

            /

            Unless the sensors have a design flaw and under certain weather conditions they all report an erroneous value. Like the Pitot tubes on the ill-famed Air France crash a few years back.

            1 good - 2 whack(different values) - system unreliable, turn off, consider landing early

            Unless the two different values are identical, in which case the system would think they are good and crash the plane. But let's see...

            1 good - 2 whack(same values) - hope you notice before crash/fire. Turn off system. Seriously consider landing early. Last good sensor may or may not be usable(does it have an output you can use?). Consider firing maintainers as it is likely at least one was whack when you took off.

            The idea is that crash tendency is noticed. Remember that both MAX 8 crashes happened very soon after take-off, when plane is

            • Great. How do you determine whether the vote has a correct outcome?

              Well, I'd start with reading my whole post before replying, because this is only like one of three questions you ask that are answered later in the same post. In some cases by the very next line.

              Why ask when the question is already answered?

              As for design flaw - that is a whacked sensor. I did mention firing people out of a cannon at that point...

              The idea is that crash tendency is noticed.

              Well, I said "hope" for a reason. It is a very scary situation to be minimized if possible.

              About the only defense against defective sensors that are all returnin

          • MCAS was not just assisting, it was flying. The first critical flaw was in the naming of the system. It's not augmentation. Once you name it wrong, you sneakily got away with doing things like using single sensor - not letting pilot know of its working / how to disable it.
            this fix is just a software fix and no hardware/maintenance fix is needed. So lot less expensive.
          • It should also check with other sensors. Like the plane would see the dodgy AoA data and think shit we're going to stall, except our speed is fine and out altitude isn't changing. Maybe our AoA isn't a problem after all and I shouldn't nose dive this thing to prevent a stall that doesn't seem to be happening
        • by Darinbob ( 1142669 ) on Wednesday March 27, 2019 @08:56PM (#58345348)

          Just your industry standard screwup. A better design is expensive, more testing is expensive, any delay is expensive. To the product managers will push and push and push for you to ship the product. The plan was not designed from scratch, it's an incremental modification of the 737 line and this feature was essentially a patch that was less expensive than a redesign.

        • by mjwx ( 966435 )

          which again goes to question the logic behind an automated system based on sensors that could be faulty forcing correction while on manual flight control... but i am sure i don't understand as I am not an industry insider.

          That is Airbuses model, if all 3 flight computers cant agree, they throw control back to the pilot and say "sorry, your plane now". A system that has been fantastically safe and Boeing has spend billions trying to rubbish.

          The system in the 737 MAX is there because they've changed the position of the engines from under the wing to in front of the wing which pushes the thrust directly under the surface of the wing. This has the nasty side effect of being able to increase the pitch of the aircraft without t

      • by PPH ( 736903 ) on Wednesday March 27, 2019 @06:55PM (#58344758)

        It might not be the physical sensor. Data from both the LION and Ethiopian flights shows an offset between the two AoA sensors of 22 degrees. Neither appear to be stuck, as they both track airplane movements. But with this offset. Same physical fault causing the exact same offset? Doubtful.

        One theory is that the 22 degree figure is pretty close to the value of one bit in the ARINC 429 word for AoA (22.5 degrees). So, software might be flipping a bit. This might be a tough bug to run down.

        • If the crashes were due to software bug, ouch. Didn't the LION flight take off with a known defective AoA sensor though?

        • by dgatwood ( 11270 ) on Wednesday March 27, 2019 @07:45PM (#58345056) Homepage Journal

          It might not be the physical sensor. Data from both the LION and Ethiopian flights shows an offset between the two AoA sensors of 22 degrees. Neither appear to be stuck, as they both track airplane movements. But with this offset. Same physical fault causing the exact same offset? Doubtful.

          One theory is that the 22 degree figure is pretty close to the value of one bit in the ARINC 429 word for AoA (22.5 degrees). So, software might be flipping a bit. This might be a tough bug to run down.

          It seems unlikely that software would suddenly start flipping a bit repeatedly. That usually implies faulty hardware. The real question is how two pieces of hardware could experience the exact same fault on exactly the same bit.

          My money is on thermal expansion of a BGA fastened with lead-free solder.

          • by PPH ( 736903 )

            That usually implies faulty hardware.

            It would seem so. Like an open/shorted lead on a parallel bus. Maybe a bad pin on an A/D chip. ARINC 429 is a serial protocol, so it's not likely something loose between boxes. What really rules the h/w angle out is the similar fault on (at least) two unrelated flights.

            • by dgatwood ( 11270 ) on Wednesday March 27, 2019 @08:18PM (#58345210) Homepage Journal

              What really rules the h/w angle out is the similar fault on (at least) two unrelated flights.

              It only rules out hardware if you assume that the failure is a random fluke. If it is the result of a mechanical design flaw or an under-specified simple component like a resistor, capacitor, or transistor, hardware failing in the same way isn't particularly rare. For example:

              GPU thermal failures often result in a small number of different sets of identical symptoms; the same solder balls break more frequently because of their location and the way that the chip expands.

              At one point, I was involved in a group buy of some preamplifier hardware from a manufacturer in China. There was something like a 40% failure rate, and it was caused by a single transistor being substituted with a lower-quality part that became unstable in the presence of too little capacitance. And they all failed with the exact same symptom, en masse.

              And a particular age range of certain models of TV failed en masse because of capacitor plague. In every case, the symptom was that they wouldn't turn on.

              Or consider the T-Con board that drives various LCD panels in TVs. They fail with alarming regularity, to such a degree that there's actually a third-party company manufacturing new replacement boards for old TVs. There are only a few different failure modes, usually involving one color channel stuck off or on, and statistically if you buy a used board, nearly 100% of the time you'll get a bad one, because it's the #1 cause of replacing TVs that contain certain models of T-Con board.

              And I can also recall a hard drive connector built by a major manufacturer that was attached by a screw on only one end, and repeatedly would work its way lose, requiring a complete redesign of the hardware in the next generation.

              You get the idea.

        • by giampy ( 592646 )

          In these cases when the sensors disagree for whatever reason, it looks like a light will turn on but essentially they will lose reliability of both sensor ans they won't know which one is faulty (assuming they won't fault at the same exact time, which i sa safe assumption).

          If so it's a little stupid, and sad, as there are plenty of techniques to decide which one is correct and which one is faulty based on the reading of the other sensors (and a small internal model of the aircraft). I hope they implement a

      • They can detect if a sensor is malfunctioning - atleast give a probablity. The recent crashes happened 'coz the sensor was constantly giving a high value.. likely say a high stuck value like 50 degrees. With some past values/ML/AI etc you can reasonably guess the instrument is stuck/malfunctioning. [because even after doing lot of trimming, why it still shows the same 50 degree]. The point is as a machine and something capable of analyzing lot of data in short time, it should provide as much insight to the
      • They're regularly inspected, which is about the best you can do.

        The best you (they) could do is to have implemented sensor cross-checking in the first place, not after people died. Our 2006 Sprinter has two pots on the accelerator pedal, and cross-checks them. WTF was Boeing thinking by not using both sensors? On what basis is that not criminal negligence? At very best, it's gross incompetence.

    • Because under the simulations, Boeing themselves found pilots might have under 40 seconds [cnn.com] to override the software or the plane might go into a unrecoverable dive. And that’s in a simulator where the pilots were expecting it.
  • patch (Score:4, Funny)

    by Anonymous Coward on Wednesday March 27, 2019 @06:38PM (#58344654)


    if (crashing() && uncrashFeatureEnabled()) {
      uncrash();
    }

  • by fahrbot-bot ( 874524 ) on Wednesday March 27, 2019 @06:39PM (#58344664)

    Before engaging MCAS the control software will display an animated dialog:

    Clippy: It looks like you're plane may stall. Would you like help?

    • by shanen ( 462549 )

      I'd give you the funny mod if I ever had one to give. (I think the comment that currently follows this one is also bidding for a funny mod, but I'm not getting the joke yet...) Anyway, I just wrote about a reincarnation of Clippy, though I wasn't joking.

      If I was a comedian, I'd try to come up with a funny expansion of MCAS. Something like Mud Capture Attack System.

      • I'd try to come up with a funny expansion of MCAS.

        May Cause Air Sickness

        • I'd try to come up with a funny expansion of MCAS.

          May Cause Air Sickness

          May Cure Air Sickness. It's hard to have air sickness when you're smeared across the runway as a chunky paste.

    • And In keeping with tradition Clippy's help will be totally useless.
  • So, the FAA previously left the MCAS certification (along with other systems) to Boeing engineers. Is this how the "fix" is going to go through again? Normally they should go back and have FAA engineers redo the certification of every 737 Max system that might affect safety.
    But that would take years and FAA/Boeing wouldn't like that, would they ;)

    And all the above is without talking about what is the major cause of concern: software trying to compensate for the hardware design shortcomings an airplane... We

    • by 0100010001010011 ( 652467 ) on Wednesday March 27, 2019 @07:08PM (#58344840)

      As someone that has worked in both functional safety and off-highway vehicles.

      How the fuck did this ever make it into production. Why is a 'second sensor' an upsell?

      When given the option to completely update the cockpit to the latest and greatest with digital displays.

      They chose to replicate the old mechanical dials so the pilots couldn't be retrained.

      The entire thing from start to finish was rushed. Mechanical design comes first. There is no 'try and develop software in parallel'. A clean software design depends on a good mechanical design.

      The plane should have been a white board redesign, it should have been balanced such that a pilot could fly it stable with no avionics. This isn't a jet fighter.

      But it was rushed because Europe invested in R&D and beat them to economy routes. How much money did Boeing C-suites make before 2011? During the 2009 crash there was a hiring spree by some companies because the market was flooded with cheap, good engineers that just got laid off. Companies invested in talent. Did Boeing?

      People died because... Boeing sat on R&D from post WWII while making a ton of money so when Airbus released a good plane they scrambled to retrofit an old design by putting huge engines on an airframe causing it to pitch up but to appease its clients it added software to mimic the old plane behavior and tested it themselves and told the FAA they promise they did it right.

      More or less.

      • B is a for-profit company in a capitalist economy. What do you expect? You need to question as a society how did you let B be the only player in the field? what happened to competition within the country? when someone claims B is greedy; no everyone in the society is greedy (likely your 401k is invested in B). You take shortcuts. and at times you pay the cost..collateral damage.
        If with good software you can reuse 80% plus of your previous design, why wouldn't you do it?
  • That new software needs to be audited, source code and all, by outside experts. The first thing that was drilled into me in basic instrument flight training was never to fixate on one gauge. Boeing seems to have committed a transport category aircraft to just that.

  • The problem isn't implementation bugs, it's the basic design that gives the autopilot control authority over the pilot. This exact sort of accident has been with us since the introduction of the first A320 (the first fly-by-wire aircraft where the autopilot could overrule the pilot's control inputs). The fix is in 2 parts:

    1. The flight control systems should always implement the pilot's inputs regardless of what the computers think, unless the pilot's actively told it to disengage his set of controls.
    2. Teach y
    • The first A320 accident showed that a fly by wire aircraft that overrides the pilot actually saves lifes.
      The pilot actively tried to stall the aircraft. Had he succeded, there likely would have been no survivors. Since the aircraft fought the pilot, it managed to decent much slower and onto the top of the trees cushioning the impact, only killing three people.
      This is why the flight control systems must disregard the pilot's inputs if they would put the aircraft outside of its flying envelope.

      https://www.fli [flightglobal.com]

  • Boeing screwed up ROYALLY and they'll pay for this, likely to the tune of hundreds of millions of dollars.

    This was an egregious engineering fuckup that was completely 100% avoidable. So many mistakes, it's horrendous and shameful for a company like Boeing to implement these insane design choices.

    Basic SOL and mission-critical applications are always always ALWAYS supposed to use a minimum of two sensors and in most cases they should use three (with an arbitrated voting system).

    In addition there was very lit

    • Boeing screwed up ROYALLY

      Certainly.

      and they'll pay for this, likely to the tune of hundreds of millions of dollars.

      Should be more in the billion(s), total including indirect costs (decline in orders, reputation, stock, ...). But we'll see, what actually happens. Boeing being a national treasure...

  • Passengers will keep debugging.
    • Passengers will keep debugging.

      This is the global trend. But unfortunately that pattern does not apply well for aviation (or medical)

  • The cost would be pretty high though :|
  • Proof of concept, Pay me bitch :P
  • by Wizardess ( 888790 ) on Thursday March 28, 2019 @04:30AM (#58346544)

    Can anybody imagine a 737 MAX pilot being anything less than viscerally aware of the problem and what must be done to fix it? Anything else being done is gilding the lily. Of course, turning off MCAS with an AoA sensor mismatch simply makes the job easier for the pilots. Now, why do they disagree? Are they really AoA indicators or something else entirely? Why aren't there three if you're going to use them in a flight safety critical manner?

    {^_^}

  • by PhotoGuy ( 189467 ) on Thursday March 28, 2019 @06:16AM (#58346776) Homepage

    The depressing (or incriminating?) part here is that the fix didn't require any hardware modifications, as I would have expected. I assumed that there was some cost/weight issue to having the MCAS have access to the left and right sensors. But nope, it could have compared both.
    If it can be fixed with a software fix, then it could have been done right from the start without any extra hardware costs of production.
    Very damning.
    I get so tired of the reports calling clear software/algorithm bugs "computer glitches."
    It's akin to blaming every pilot error situation on the plane.
    Just as with hardware design flaws, software design flaws should have repercussions for the manufacturer, and not written off as "oh, one of those computer glitches!" If your computers are glitchy, don't put them on my plane, thanks.

    • Similarly, I was shocked to see that the standard procedure for flight computers acting up was to simply re-power them. Computer hardware/software should be made reliable enough that you don't need to do the "Windows thing" of rebooting regularly to keep it operating. Circuit breaker instead of power switch, but the same deal, really.

  • ...when we have assholes like Boeing doing it to us anyway?

Ocean: A body of water occupying about two-thirds of a world made for man -- who has no gills. -- Ambrose Bierce

Working...