American Airlines Grounds Flights 91
Sez Zero writes "The Federal Aviation Administration said American Airlines requested a halt to hundreds of its U.S. flights on Tuesday as it works to resolve a reservation system problem. American Airlines explained on their Twitter feed they had a problem accessing their reservation system. Bad day to be on the AA ops team."
Re:Redundancy? (Score:5, Funny)
Don't you mean "RedundAAncy?"
It's a sAAd day for AA.
Re: (Score:2)
Don't you mean "RedundAAncy?"
It's a sAAd day for AA.
It's AA sAAd dAAy for AA
I hope you get modded 'redundant' to complete the joke.
Re: (Score:2)
Don't you mean "RedundAAncy?"
It's a sAAd day for AA.
It's AA sAAd dAAy for AA
I hope you get modded 'redundant' to complete the joke.
That was the idea, but you can never rely on mods
Re: (Score:2)
Only in this comment...
See now its that kinda thinking that leads to AA's entire fleet being grounded. I mean seriously you comment is a single point of failure. What if I'd scrolled past it? What is slashdot's web server failed to post your form post?
You single comment is just the sorta cowboy IT work that causes these disasters.
seems to happen now and then (Score:4, Informative)
From various airlines: 2004 #1 [slashdot.org], 2004 #2 [slashdot.org], 2011 #1 [slashdot.org], 2011 #2 [slashdot.org], and probably others I missed.
Re: (Score:1)
And this Alaska Airlines [katu.com] outage from just six months ago.
I was actually in the line pictured there at PDX. But, I left before the photographers arrived to re-book on another airline. It's really unfortunate I was able to re-book. That outage was an answer to my prayers on the way to PDX for something to happen to get me out of going on that business trip.
I've seen worse excuses for flight problems... (Score:2)
Re: (Score:2)
From various airlines: 2004 #1 [slashdot.org], 2004 #2 [slashdot.org], 2011 #1 [slashdot.org], 2011 #2 [slashdot.org], and probably others I missed.
OK, I'm travelling to the US later this year, I'm going to have to take some internal flights... Is there any airline there that I can reasonably count on not to screw up?
Re: (Score:1)
No. You're fucked mate.
Re:Are they on some older software that can't hand (Score:4, Insightful)
The software they run is Sabre, which was co-founded by American Airlines some decades ago. I have no particular knowledge of which software has undergone rewrites and which hasn't, but if you scan their own timeline [sabre.com], it's not hard to suspect that there are huge piles of ancient code still in there.
Re:Are they on some older software that can't hand (Score:4, Funny)
>> piles of ancient code still in there
ancient code ain't always slow code - remember what we had to write it for, you whippersnapper
Re: (Score:2)
True. A bigger problem is probably that it's 50 years of accumulated cruft that may or may not work together in any kind of maintainable way (probably "may not"). Stable legacy systems that are just maintained with minor bugfixes now and then can be perfectly reasonable, but systems that accrete code over decades tend not to be.
Re: (Score:2)
Re: (Score:3, Informative)
You see, no it isn't exactly like you say. I have written and have in production interfaces with SHARES and the problem isn't so much their infrastructure, as they are actively doing a lot more upgrades (I actually have had conference calls with their IT managers while migrating one of their systems) than anyone thinks. The problem is more operational than anything, but since I am actively under an NDA concerning things like that I can't bring up any specifics. Other than that, the other problem is too m
Re: (Score:1)
They are stable, I'll give you that, but I wish that I could get through a day without having to reset my terminal a few hundred times because I get stuck in a terminal with "X-Wait" and a locked term.
Re: (Score:2)
Re: (Score:1)
I'd rather type 4 lines of code to rebook your ticket and resync rather than have to wait for AirportApps or whatever GUI to refresh and do it. Six if you include the "ER, ER" to finish the job.
Now, would I want to do a new PNR in SHARES? absolutely not. but for day to day operations, SHARES isn't bad. If you want to see cruft, try working with Deltamatic. The same thing that takes one command in shares takes 3-4 in Deltamatic.
Re: (Score:3)
old code is plenty fast. It's just full of bugs, non-scalable, unmaintainable, and platform dependent. When we had slow computers speed was the only thing that mattered. Now we have fast computers and maintainability, scalability, and platform independence is worth a few lost clock cycles.
Old code is fast, but not if you count the time it takes to write, maintain, rewrite, and all the time spend debugging and rebooting computers and restarting services when things don't work.
Brian Kernighan put it eloq
Re: (Score:3)
Sabre was one of the original mainframe transaction processing systems. I can run OS/MVS under the Hercules System/370 emulator on one of the old junk Pentium 166 boxes sitting over in the corner and it would still out-perform the original Sabre computers by a considerable margin.
On the plus side, programs back then didn't have to deal with 16 different UI controls (menu, popup menu, toolbar, command keys, etc) so the source code base was much smaller. And you never rebooted a mainframe unless the world was
Re: (Score:2, Informative)
I'm a Sabre employee. The issue today was not related to any Sabre systems.
“American Airlines mistakenly reported they were having an issue with the Sabre reservations system, which they subsequently corrected. To clarify, all Sabre systems are operating as normal and have been all day. We see American Airlines is now up and running. We stand ready to help if needed.”
Dan
Re: (Score:2)
On the plus side, programs back then didn't have to deal with 16 different UI controls (menu, popup menu, toolbar, command keys, etc) so the source code base was much smaller.
But IBM terminals do support things like menus, toolbars, command keys, text input fields, and the like.
Re: (Score:2)
On the plus side, programs back then didn't have to deal with 16 different UI controls (menu, popup menu, toolbar, command keys, etc) so the source code base was much smaller.
But IBM terminals do support things like menus, toolbars, command keys, text input fields, and the like.
I'm not sure what IBM terminals you are talking about, but I'm referring to "green screen" terminals. Their idea of a menu/toolbar/commandkey UI was to have a row down at the bottom of the screen that said "F1 Help F3 END F7 FORWARD F8 BACK".
Re: (Score:2)
Their idea of a menu/toolbar/commandkey UI was to have a row down at the bottom of the screen that said "F1 Help F3 END F7 FORWARD F8 BACK".
There's incredible variation in "green screen" terminals. An adm3a doesn't even have cursor control. An IBMwhatever has fillable fields and limited form validation. A Tek terminal draws graphics. All of these are just a terminal, though the Tek may have a mouse it's still not an X terminal.
Re: (Score:2)
Tek, LSI, ADDS, and similar terminals aren't IBM terminals, by definition.
Back when Sabre was new, the IBM mainframe spoke strictly to IBM terminals - either the 3270 series or their predecessors (2660). I spent quite a few years working in that world. As befitted a mainframe, they transmitted data in blocks, not per-character like their ASCII cousins. They didn't have validation as such, just the ability to lock out non-numeric input to numeric fields and stuff like that. Definitely no GUI widgets except
Re: (Score:2)
Brian Kernighan put it eloquently:
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"
I'm going to guess that was said tongue-in-cheek. First, writing and debugging are two different, although related, skills. While writing (of code or anything else) does benefit from independent review, I doubt Mr. Kernighan really thinks there is nothing to be gained when someone reviews their own work.
Second, I can debug my own code because I'm smarter when debugging than I was when writing. I have the benefit of the experience of writing the code and observing the results of running in a test or sandb
Re: (Score:2)
The point of his statement was that software should be written in a way where it is easy to understand and maintain. It is much easier to make "hard to understand code" than it is to understand "hard to understand code".
If you are smarter when debugging, then you are already following his advice, because you are not "clever as can be" when writing it, because you are less clever writing it than when you are debugging it.
He is suggesting you *shouldn't* be "as clever as can be" when writing code, *because*
Re: (Score:2)
He is suggesting you *shouldn't* be "as clever as can be" when writing code, *because* it will be much harder to debug.
Oh. That I agree with 100%. I didn't really understand his point the first time I read the quote. Thank you.
Re: (Score:2)
[According to the article] there was nothing wrong with the software. It was that the network couldn't be accessed [was down]. This sounds more like a core router failure or a router that polluted the routing tables.
Airline reservations systems fall into the "mission critical" software development category. They tend to have fewer bugs because they get so much testing before deployment.
And one of the reasons AA spun off its [in-house] Sabre system was so that it could be kept modern. Another reason, IIR
Re: (Score:2)
Re: (Score:3)
I'm a [linux] kernel programmer. I write device drivers. I haven't used a goto in 30 years [not one!]. I've also written code to do realtime processing of broadcast quality HD video on a linux platform in conjunction with specialized hardware.
You're assuming that to make code fast, it's done [has to be done] by doing insane hacks [or the bad practices that you mentioned]. This [usually] only produces modest speedups on the order of a few percent. Most really fast code is highly modular and just uses a
Re: (Score:2)
You are saying that often a program *can* be made faster by also making it better. I am not disputing this. The most obvious example of this is choosing an algorithm that is asymptotically efficient.
But even once you get to the point where you wouldn't change a single character, there are usually still things to can do to increase speed but at the cost of modularity, scalability, maintainability.
Re:Are they on some older software that can't hand (Score:5, Informative)
Re: (Score:2)
The real issue is that these airlines are usually publicly owned. They care about profits and rewriting all the infrastructure code is expensive and might even stop normal operation (resulting in higher costs). When shareholders do the math for what makes most financial sense, it is usually to keep sticking on more duct tape until duct tape no longer works. At some point a full rewrite makes the most financial sense, and that's when it will happen.
Even when an airline is just stupid and only wants duct t
Re: (Score:1)
Since when does any shareholder ever decide what internal projects get worked on?
Since "the real issue is that these airlines are usually publicly owned" then the solution is to have government announce regulations that all airlines must be owned by a single person? Or maybe family owned or even some partnerships? That will solve the problem because you won't have a bunch of nameless, faceless people begging for a larger quarterly payout (dividend) but will instead have one person (or a handful of people) d
Re: (Score:2)
The shareholders are the collective owners of the company. They are the boss. They can have as much influence on what internal projects get worked on as they want. By deciding not to be involved, they are making an implicit decision to allow this to be handled by people lower in the totem pole. The owners typically decide on budgets which either allow for total rewrites or not. The engineers/engineering department can ask for more money to do a good job, and these requests are either approved or deni
Re: (Score:2)
The shareholders are the collective owners of the company.
You, with your 4 shares of stock don't get to tell anyone anything.
However, when you own enough shares that your vote matters to a large enough extent, you don't get to tell them exactly what to do ... but you sure as hell can make someones employment at the company difficult to impossible.
The reality of it is however, 99.99999% of the time, a total re-write is a retarded idea. You just get a whole new set of bugs and you've wasted all the time you put into the original system. There is nothing that actua
Re: (Score:3)
Owning 4 shares in a company gives me about as much say as a voter in an election. No I don;t get to decide who the senator of my state is, but I do have influence over it, even if that influence is small. All the voters added up completely determines the outcome. It's not important that the smallest shareholder always affects the outcome of every decision, it only matters that this is possible. Just like I don't need to cast the deciding vote in an election for my vote to count.
It depends on your defin
Re: (Score:2)
Owning 4 shares in a company gives me about as much say as a voter in an election. No I don;t get to decide who the senator of my state is, but I do have influence over it, even if that influence is small. All the voters added up completely determines the outcome. It's not important that the smallest shareholder always affects the outcome of every decision, it only matters that this is possible. Just like I don't need to cast the deciding vote in an election for my vote to count.
Have you ever actually voted your proxy? It does not work like a government election.
You can vote or withhold your vote for a select list of directors. They are more often than not the current board of directors. You do not get to choose between contenders for the same board seat. Since the directors on the ballot are nominated by shareholders with far more shares than you, your vote is mostly symbolic and meaningless.
Unless you are CALPERS, manage a large trust or investment fund, or are otherwise a .1%-er
Re: (Score:2)
Let's say a company has 1 million shares. I have one share. I don't have much influence. If I collectively work with half a million other people with once share, then we have the power of someone with a 50% stake in the company.
This is very similar to government elections except that some people get more than once vote. That doesn't change the fact that someone with 1 vote has only a small amount of influence.
No I have never owned stock. I spend all my money on my mortgage. I don't doubt that most inv
Re: (Score:1)
Re: (Score:3)
Re: (Score:2)
Perhaps you should log a query on expert sex change
Re: (Score:2)
What is missing from the timeline is the time they blew up a huge project. I remember reading about it in the early 90's but can't locate the source, so I give you Wikipedia: [wikipedia.org] basically they blew 125 million and 3.5 years of development work and AMR (American Airlines parent company) was sued by Marriot, Hilton and Budget (partners in the system) for the failure.
Then back in 2009, AMR hired HP to develop a new system [forrester.com] for them, which was seen as very risky.
Now, it seems that they have thrown in the towel [dallasnews.com], wha
Move the planes from the gates (Score:5, Informative)
"At American's hub in Miami, The Miami Herald reports that landing AA flights have run out of available gates since none of the airline's departures are taking off. A passenger on one of those flights -- 66-year-old Richard Bell -- tells the Herald he had been stuck on an AA flight arriving from Baltimore. He told the newspaper that the aircraft's engines were running and that the air conditioning was working. But he also said the flight's pilots come over the public address system to warn fliers that some other systems were not functioning. "He mentioned the toilet specifically as a problem,'' Bell tells the Herald."
This is total lack of human compassion that someone can't get in one of those tractors, push the plane at the gate out of the way to a spot off to the side and let the plane with the people unload. What kind of heartless ass is running American's operations at that airport? Oh, gee, that might inconvenience the airline personel because the first plane would then have to be trundled back over since it needs to leave first when things resume.
Re: (Score:3)
Re: (Score:2)
I agree, but I think the solution could be a lot easier. Most airports have stairs they can drive to the airplane door. Sure, that wouldn't help a disabled person, but there is no reason 200 people have to sit in a plane for hours when only 1 or 2 people can't walk down a flight of stairs.
In addition to the stairs, they also have elevators on trucks (this is how they get large items in and out of the cabin).
Re: (Score:2)
I thought there was some sort of protocol that says "when a passenger gets off the plain, their luggage must also get off the plane"? If so, that would mean they'd have to completely undo the flight and then redo it when it was ready to go...
Not sure about this.
Re: (Score:2)
If the passenger is still airside there is no reason to remove baggage, especially if it is an organised deplane to a holding area (because you aren't getting landside again until cleared) with an expectation of the flight taking place. Should it then be cancelled the usual routine of removing baggage would take place, just like with any arrival.
Re: (Score:3)
The stoppage in question was a "ground halt" meaning that once the planes get where they're normally going, they don't leave again. So the passengers are where they're supposed to be; OK, why not take their luggage off? Then shove the plane out of the way and get the next one into place that's otherwise idling full of people wasting their lives waiting. Maybe a handful of people are continuing on to that same plane's next destination but that's a really low percentage of the passengers.
Re:Move the planes from the gates (Score:4, Informative)
It's fucking laziness.
I've towed MANY aircraft and it isn't at all difficult to do.
Basically, back towbar up to nose gear, unpin steering links (your towbar does the "steering" when towing), connect towbar (aircrew remain in cockpit to apply aircraft brakes if towbar accidentally disconnects under tow), remove wheel chocks, tow aircraft to new spot, chock wheels, disconnect towbar, reconnect steering links, drive tug to next job.
It "ain't shit" be the aircraft large or small.
Re: (Score:2)
This is total lack of human compassion that someone can't get in one of those tractors, push the plane at the gate out of the way to a spot off to the side and let the plane with the people unload. What kind of heartless ass is running American's operations at that airport? Oh, gee, that might inconvenience the airline personel because the first plane would then have to be trundled back over since it needs to leave first when things resume.
Not quite that simple as you'll need an excavator to move all the support infrastructure (fuel, sewerage inlets and so forth).
What the airport should have is an over-flow area on the tarmac where passengers can be unloaded via those portable stairs you may have seen about and bused to the terminal. I've seen this setup in almost all SE Asian airports except for NAIA in Manila (and I'm not sure if that is because NAIA was designed by Americans or run by Filipinos).
This of course wont help with flights
Um...any TECHNICAL explanation? (Score:5, Insightful)
Kudos, SlashDot, for getting the story here on the same day as the rest of the media. Now how about some links that AREN'T ConsumerNews or USAToday or other crap. Does anyone know what the TECHNICAL reason for the failure is?
Re: (Score:3)
Technically, there was something that went wrong, probably something to do with a technical problem with some of the technology. Technically, that's just a theory, 'tho.
Re: (Score:2)
But technically correct.
The BEST KIND of Correct!!!!
Re: (Score:2)
I was literally trying not to speak figuratively
Re: (Score:2)
Re: (Score:2)
+2
That's so true it's funny.
Re: Um...any TECHNICAL explanation? (Score:2)
Actually, it is a very well documented system with strict change control and change management.
Disingenuous headline (Score:1)
You know very well what people's first reaction will be to that headline, coming a day after the marathon attacks.
How about this: "American Airlines IT Problem Grounds Flights". Still shorter than the average Slashdot headline.
Cover story? (Score:1)
Perhaps they had intelligence on a threat to an American Airlines flight and didn't want to alarm people...
al Qaeda usually does more than one attack at a time we've seen.
Re: (Score:1)
OR maybe itw as AQ hackers bringing SABER down.
Re: (Score:2)
Re:Cover story? (Score:5, Funny)
I think that a passenger saw a monster on the wing, like what happened in "Nightmare at 20,000 Feet".
Hey, if we're going to be making up random shit, it might as well be something cool.
Re: (Score:2)
I Think They Were Cracked... (Score:2)