Forgot your password?
typodupeerror
Bug Transportation

FAA Computer Glitch Causes Widespread Airline Delays 133

Posted by Soulskill
from the reports-confirm-toothpaste-was-not-involved dept.
seven of five writes with this excerpt from an Associated Press report: "A problem with the FAA system that collects airlines' flight plans caused widespread flight cancellations and delays nationwide Thursday. It was the second time in 15 months that a glitch in the flight plan system caused delays. The FAA said in a statement that it is having a problem processing flight plan information. 'We are investigating the cause of the problem,' the agency said. 'We are processing flight plans manually and expect some delays. We have radar coverage and communications with planes.'"
This discussion has been archived. No new comments can be posted.

FAA Computer Glitch Causes Widespread Airline Delays

Comments Filter:
  • Here I sit... (Score:3, Interesting)

    by dtmos (447842) on Thursday November 19, 2009 @11:45AM (#30157226)

    ...stuck in Atlanta...

  • Sounds funny to me. Packet switch due to a database mismatch?
  • only the second glitch in fifteen months? One would assume that's far better than most commercially written software.
    • Re: (Score:3, Interesting)

      by Tellarin (444097)

      By "glitch" they mean "totally offline delaying and canceling flights".

      I'm pretty sure they had lots of other bugs and "regular glitches" in this time.

      On the other hand, I'm also pretty sure that this kind of software does indeed go through a much better development and verification process than most commercial software around.

      • by geekoid (135745)

        No, by glitch they mean glitch. "totally offline delaying and canceling flights" is the result the glitch.

          "On the other hand, I'm also pretty sure that this kind of software does indeed go through a much better development and verification process than most commercial software around."
        the scheduling software? probably not. It was done out of house for contract by people who don't give a damn the moment the software releases.

  • Damn Excel (Score:4, Funny)

    by headhot (137860) on Thursday November 19, 2009 @11:49AM (#30157320) Homepage

    Some one messed up their $2,000,000 excel macro that list morning.

  • Honest! (Score:5, Funny)

    by sycodon (149926) on Thursday November 19, 2009 @11:50AM (#30157332)

    I only changed one little line of code! It wasn't even important enough to test!

    • Re: (Score:3, Interesting)

      by Anon-Admin (443764)

      HAHAH, around my work you could add "or Document" to the end of that statement. I cant count the number of times people have changed things with no documentation and they did not consider it important.

    • No you see the problem was that someone tried to comment out a comment. However, their skewed vision on how comments work, and by some MIRACLE of programming and compilation, All the code ended up commented and the comment was compiled instead.

  • by Anonymous Coward
    The FAA and their omnipotent employees hate anything new/improved. I worked for a very large FAA contractor developing a modern system. We tried to use Solaris/C++ but they sandbagged it because we could not test each and every line of code in the OS and that even C++ stripped to essentially C may have hidden problems. They are comfortable running systems on ancient mainframes running ULTRA.
    • by eln (21727) on Thursday November 19, 2009 @12:04PM (#30157640) Homepage
      Given that the FAA is responsible for the lives of thousands of people who travel by air every day, not to mention cargo planes which represent billions of dollars in economic activity, I'd be extremely alarmed if they were constantly switching to the latest and greatest technology without testing each and every line of code in the entire software stack, up to and including the bare metal. Change is slow in organizations like that because it has to be. Improper testing can easily lead to lost lives, or at the very least huge delays like these ones that can cost millions in lost productivity.
      • We've heard about the antiquated ATC system for over a decade. Does anybody on the inside know if, and when they'll upgrade?

        • by horatio (127595)
          ADS-B, the next generation of ATC tracking/radar systems was brought online recently over Louisville, KY [faa.gov] and has been in testing in several other areas as well.

          They are also phasing in more GPS approaches (nav systems specifically designed for landing) and phasing out the NDBs (non-directional beacon) and taking failing VORs out of service that are often used at fields which have (ILS) instrument approach runways.

          That said, it is a very slow process. If your car's speedo goes out, who cares. If the
          • GPS only measures your speed across the ground, not your airspeed - which includes factors for wind.

            Assuming that most planes are flying where

                GPS speed - headwind > stall speed

            unless they're on take-off or approach, wouldn't a GPS plus a laser wind-speed detection system at airports solve that problem?

            • Wind varies tremendously by altitude, location, and time. Wind shear accidents occurred in the past because the resolution of the wind readings at airports compared to the rapid shear of the wind gave little to no warning of how rapidly the headwind component to an aircraft was about to change. In addition, indicated airspeed is actually measured by taking the ratio of ram pressure to static pressure, calibrated to read true airspeed at sea level on a standard day. This leads to the indicated airspeed be
              • It's an emergency when your airspeed indicator is malfunctioning, and it is the most direct indication of the amount of energy relative to the airstream your aircraft has. It's not something you want to muck with by putting multiple layers of indirection/abstraction (and thus additional things that can fail) between the sensors and the indicators.

                The GP was mentioning failures too - why not have a backup system available if this is something which happens enough to be concerned about?

                • Your idea of a backup system is an excellent one - and it's already been done, in a much more reliable way! The problem with the GPS + windspeed from some ground source is that:

                  a) It assumes the wind at the ground station is near the wind speed and direction of the aircraft. This is almost always a wrong assumption, as the wind speed rapidly increases with altitude, and tends to cant in quite a different direction as well. For a sensor that has to be accurate within a knot or two, this is completely in
                  • a) It assumes the wind at the ground station is near the wind speed and direction of the aircraft. This is almost always a wrong assumption, as the wind speed rapidly increases with altitude, and tends to cant in quite a different direction as well. For a sensor that has to be accurate within a knot or two, this is completely inadequate.

                    That's why I suggested the laser systems above - I don't understand exactly how they work, but by shining lasers through air masses they can determine wind speed, density, e

      • by DeadCatX2 (950953)

        This brings up the greater question of how old must a technology be before it is considered mature enough for purposes as mission-critical as air planes. 5 years? 10 years? 20?

        I think that reasonably, you ought to be able to trust technology that's been around for a decade.

        • by jklovanc (1603149)

          Not necessarily, if the technology was speced to handle 100,00 flights per day and they are trying to pump 200,00 flights per day through it, the software may not be able to handle it no matter how old it is.

    • by fbjon (692006)

      C++ stripped to essentially C may have hidden problems

      That sounds about right, yeah.

    • Re: (Score:2, Insightful)

      by sexconker (1179573)

      They are comfortable running systems on ancient mainframes running ULTRA.

      I would be comfortable doing the same thing.
      Why do people shit on mainframes? They may cost a lot of money and time, but they're orders of magnitude more dependable than any other server solution that exists.

      Yes, that includes the "cloud".

      • by Nerdfest (867930)
        A huge part of their reliability is due to restistance to, and lack of change. Almost every problem that is encountered is due to a programming error. The same hardware availability can be obtained on a 'cloud' using virtual machines, redundancy, automatic fallover, etc, many of the same technologies the mainframes use, with a huge performance increase and cost savings.
        • Uh, that's a very small part of the reliability.

          The hardware reliability comes from having fault-tolerant, hot-swappable, redundant, well-tested hardware. The "cloud" hopes to achieve this, yet doesn't.

          The software reliability comes from having fault-tolerant, well-designed, well-tested software. This takes time, sure, but it can just as easily be done with forward-thinking to allow for growth and change.

          The problem is not mainframes, mainframe programmers, or PHBs resistant to change. The problem is peo

      • by PitaBred (632671)
        Carrying things on my back and walking places is much more dependable than driving a vehicle which can break down. I'm still not gonna start hauling everything in a backpack.
    • The wallstreet journal article mentions the issue was with a new system, specifically noting that it was not related to their antiquated hardware that has historically been the cause of failures in the past. I think it's naive to suggest that old hardware or coding in C++ (over C, as if C's a completely dead language... they both have their places in complex engineering tasks) will eliminate failures in the system, because in this case it was newer hardware, which had a cascading effect, implying involvemen
  • Resolved... (Score:3, Informative)

    by dtmos (447842) on Thursday November 19, 2009 @11:55AM (#30157414)

    ...according to the Wall Street Journal. [wsj.com] Wonder if they'll give me a lift home?

  • they must have been transitioning over to Windows 7 this morning.. you know that can take days depending on what your current system is.
    • This is the government. They transitioned from Windows 2000 to XP only last year.
      • by vegiVamp (518171)
        So what you're saying is that this was the reboot halfway the upgrade process ?
      • Don't you see? The FAA was chosen as the Windows 7 upgrade test bed for the U.S. Govt Then if it doesn't work and people die, they can just blame Microsoft. Everyone hates them anyway right???
  • ...and Iran was publishing bad flight plans. You know, to block undesirable traffic.
  • by dkleinsc (563838) on Thursday November 19, 2009 @12:18PM (#30157898) Homepage

    "That's it, you're all grounded!"

  • by smooth wombat (796938) on Thursday November 19, 2009 @12:22PM (#30157986) Homepage Journal
    According to this article [marketwatch.com], the glitch is no longer.

    Apparently they fixed the glitch so the problem worked itself out naturally.
  • Foot TCP (Score:2, Insightful)

    by cosm (1072588)
    FTA: "The FAA said at that time the source of the computer software malfunction was a "packet switch" that "failed due to a database mismatch."

    We all know how large out of touch behemoths sometimes structure their IT. By 'packet switch' they mean 'guy who couriers hardcopy flight plans' and by 'database mistmatch' they mean their dewey-decimal-system was mixed-up by some jokesters.
  • Honest... (Score:5, Funny)

    by flogger (524072) <non@nonegiven> on Thursday November 19, 2009 @12:36PM (#30158258) Journal
    That bug just flew in under the radar...

    **ducks**
  • by round4 (1177033) on Thursday November 19, 2009 @02:04PM (#30159996)
    Yea, these are the guys (the feds) I want building cars, taking over health care...thanks god they are not building the planes. I'm just trying to think one government run organization that works as well as any private one. Any idea?
    • by DerekLyons (302214) <fairwater@gmail. c o m> on Thursday November 19, 2009 @04:20PM (#30162496) Homepage

      Yea, these are the guys (the feds) I want building cars, taking over health care...thanks god they are not building the planes. I'm just trying to think one government run organization that works as well as any private one. Any idea?

      Under their control, 87,000 flights a day cross the skies of the US. Despite incredible crowding at and in the airspace surrounding a large number of airports, collisions are one-in-billions events. The vast majority depart and arrive without undue delay. (And anyone expecting no delay in such a dynamic system with so many variable is smoking some good stuff.)
       
      In this incident, a problem was detected, backup procedures implemented, the problem was fixed, and full functionality was restored - all in a matter of hours without halting the system.
       
      I'd be the first to admit that there are a lot of badly broken government programs - but in this case, you're just blowing smoke.

      • by round4 (1177033)
        The radar technology used by air traffic controls is from the 50's. No GPS... Planes are still tracked by little "physical" slips of paper. The entire system has been in need of an overhaul since the 70's. Planes still use land based radio towers - waste tons of fuel due to indirect routes. The FAA is incapable of fixing this decrepit system. Please share with me your on-time stats. I know there few mid air collisions but there are pilots in those planes to stop them from crashing. "without halting the en
        • Repeating the usual party line like a parrot, all the while ignoring the points I made, very impressive.

          Not.

        • Re: (Score:3, Interesting)

          by Starker_Kull (896770)

          The radar technology used by air traffic controls is from the 50's. No GPS... Planes are still tracked by little "physical" slips of paper. The entire system has been in need of an overhaul since the 70's. Planes still use land based radio towers - waste tons of fuel due to indirect routes.

          That's true. The U.S. GOVERNMENT built the best air traffic control system in the world back in the 50's, one so robust, well designed and well managed that it has been the safety and operational count leader world-wide

        • by sahonen (680948)
          IANA Air Traffic Controller, but I am studying to be one.

          The radar technology used by air traffic controls is from the 50's.

          If by this you mean "the basic technology was invented in the 1950s," then sure. If you know of a better general-purpose way to find out where things are in the sky than to bounce radio waves off of them, I'd like to hear it. That said, the systems which process and display radar data however have received several updates since the 50s.

          No GPS...

          Are you referring to ADS-B? I
          • by RLaager (200280)

            The major concern of ADS-B is that is is susceptible to hacking (you can broadcast your own "phantom" aircraft if you know the standard) and reduces the anonymity of private and business aircraft users (every ADS-B transponder broadcasts a code that uniquely identifies an individual aircraft).

            Aren't the pilots already doing this with their radio (the code being the flight/tail number)?

            • by sahonen (680948)
              Non-commercial flights have the option of flying under Visual Flight Rules (aka VFR), in which case they are not required to be in radio contact with anybody except while flying in the airspace around towered airports. You can fly through 99% of the country under VFR, and you'll only ever have to key your radio at the start and end of your trip to either talk to a tower or on the traffic advisory frequency at an un-towered airport. Also with ADS-B anyone with a receiver can plug it into their computer and l
    • by Anonymous Coward on Thursday November 19, 2009 @04:41PM (#30162864)

      Yea, these are the guys (the feds) I want building cars, taking over health care...thanks god they are not building the planes. I'm just trying to think one government run organization that works as well as any private one. Any idea?

      This morning I was woken by my alarm clock powered by electricity generated by the public power monopoly regulated by the US Dept of Energy. I then took a shower in the clean water provided by the municipal water utility. After that, I turned on the TV to one of the FCC regulated channels to see what the national weather service of the national oceanographic and atmospheric administration determined the weather was going to be like using satellites designed, built and launched by the national aeronautics and space administration. I watched this while eating my breakfast of US Dept. of Agriculture inspected food and taking the drugs which have been determined as safe by the food and drug administration.

      At the appropriate time as regulated by the US congress and kept accurate by the national institute of standards and technology and the US naval observatory, I get into my national highway traffic safety administration approved automobile and set out to work on roads built by the local, state and federal departments of transportation, possibly stopping to purchase additional fuel of a quality determined by the environmental protection agency, using legal tender issued by the federal reserve bank. On the way out the door, I deposit any mail I have to be sent out via the US postal service and drop the kids off at the public school.

      After work, I drive my NHTSA car back home on the DOT roads, to a house which has not burned down in my absence because of the state and local building codes and the fire marshal's inspection, and which has not been plundered of all it's valuables thanks to the local police department.

      I then log onto the internet which was developed by the defense advanced research projects administration and post on freerepublic.com and fox news forums about how SOCIALISM in medicine is BAD because the government can't do anything right.

      • Bravo, bravo! A few minor nits to pick and some stylistic points, but against such a performance I'll let them pass. Again, bravo!

    • by MaWeiTao (908546)

      I have a lot of problems with government and many departments within that bureaucracy, but the FAA is not one of them. It's pretty amazing what they do and there probably aren't many other entities out there that could do what they do.

    • by dkleinsc (563838)

      A government-run organization that works as well as a private one: US Postal Service.

      Seriously. Those guys move tons of stuff around the US every day, and significant portions of daily business would grind to a halt if they stopped. Think about it: for around 50 cents, you can send something from Maine to California and expect it to arrive within a few days. They make extensive use of OCR, have an extremely sophisticated sorting and routing system, and are in many ways more efficient than FedEx or UPS (for

  • It was the second time in 15 months that a glitch in the flight plan system caused delays.

    Thanks for the arbitrary use of months. Is it a baby, or were you working from fortnights and thought you might as well round to the nearest lunar cycle and then convert to Gregorian? Do you also append "and a half" to your age, as appropriate?

  • by Bob_Who (926234) <Bob@[ ].net ['who' in gap]> on Thursday November 19, 2009 @02:49PM (#30160856) Homepage Journal
    Its true the flights are delayed. That's as much of the truth we get from this complex system of terribly overworked government and airline employees, national security, information systems, politics, FAA, corporate scum, secret clearance computer firms with bizarre contracts, and the 24/7 news spew with commercial airline as clients. They start by overbooking flights, while trying to account for each seat for security, and the bean counters, vips, and bad weather. To top it off information systems have to share some data, but not other data, while the people working have to comply with all sorts of rigid protocols and odd project management that controls everything except their overtime. I dunno, I have no reason to believe that any news we get about this type of event is necessarily true. It doesn't need to be. Its just the spin that reporters have to make in the absence of any real statement for the record. If they told you the truth you'd stop giving them your money. God forbid airlines make customers a top priority. Ha! Even if they were telling the truth, it would be a fluke, but just as vague and pointless. OK. We know the truth. Its aliens from Orion preparing for 2012. And they work for Delta.
  • ive had was a 7 hour standby at a connecting airport that was 45 minutes away from my house, only to be flown back to my original airport and directly flown to houston. downsides: i ate so much au bon pain at the airport i had these weird pastry farts the entire flight, and so much coffee i thought i was going to barf before we took off.

"You tweachewous miscweant!" -- Elmer Fudd

Working...