After Outages, Amazon To Make Senior Engineers Sign Off On AI-Assisted Changes (ft.com) 83
UPDATE: Amazon later published a blog post to address what it calls "inaccuracies" in the Financial Times report that the company's own AI tool Kiro caused two outages in an AWS service in December.
An anonymous Slashdot reader had shared this report from the Financial Times: Amazon's ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a "deep dive" into a spate of outages, including incidents tied to the use of AI coding tools. The online retail giant said there had been a "trend of incidents" in recent months, characterized by a "high blast radius" and "Gen-AI assisted changes" among other factors, according to a briefing note for the meeting seen by the FT. Under "contributing factors" the note included "novel GenAI usage for which best practices and safeguards are not yet fully established."
"Folks, as you likely know, the availability of the site and related infrastructure has not been good recently," Dave Treadwell, a senior vice-president at the group, told employees in an email, also seen by the FT. The note ahead of Tuesday's meeting did not specify which particular incidents the group planned to discuss. [...] Treadwell, a former Microsoft engineering executive, told employees that Amazon would focus its weekly "This Week in Stores Tech" (TWiST) meeting on a "deep dive into some of the issues that got us here as well as some short immediate term initiatives" the group hopes will limit future outages.
He asked staff to attend the meeting, which is normally optional. Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Amazon said the review of website availability was "part of normal business" and it aims for continual improvement. "TWiST is our regular weekly operations meeting with a specific group of retail technology leaders and teams where we review operational performance across our store," the company said.
An anonymous Slashdot reader had shared this report from the Financial Times: Amazon's ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a "deep dive" into a spate of outages, including incidents tied to the use of AI coding tools. The online retail giant said there had been a "trend of incidents" in recent months, characterized by a "high blast radius" and "Gen-AI assisted changes" among other factors, according to a briefing note for the meeting seen by the FT. Under "contributing factors" the note included "novel GenAI usage for which best practices and safeguards are not yet fully established."
"Folks, as you likely know, the availability of the site and related infrastructure has not been good recently," Dave Treadwell, a senior vice-president at the group, told employees in an email, also seen by the FT. The note ahead of Tuesday's meeting did not specify which particular incidents the group planned to discuss. [...] Treadwell, a former Microsoft engineering executive, told employees that Amazon would focus its weekly "This Week in Stores Tech" (TWiST) meeting on a "deep dive into some of the issues that got us here as well as some short immediate term initiatives" the group hopes will limit future outages.
He asked staff to attend the meeting, which is normally optional. Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Amazon said the review of website availability was "part of normal business" and it aims for continual improvement. "TWiST is our regular weekly operations meeting with a specific group of retail technology leaders and teams where we review operational performance across our store," the company said.
When you "Eat your own 'AI' dog food" (Score:5, Funny)
Re: (Score:2)
Re: When you "Eat your own 'AI' dog food" (Score:2)
"Getting high on your own supply."
Re: (Score:1)
Its not "dog food", its "road kill" (Score:5, Insightful)
Using Ai generated code without review is more like "eating road kill", "road kill" where you didn't even have the common sense to have an experienced hunter or butcher examine the meat for signs of disease.
AI is a useful tool, it is not a panacea. It has to be used appropriately. That includes reviewing the code. At least a light review for things that are well known and understood, say things in every data structures and algorithms textbook of the last 40 years. A more comprehensive review for things that are more novel.
Re: Its not "dog food", its "road kill" (Score:2)
Re: Its not "dog food", its "road kill" (Score:4, Interesting)
Another comment asked for "strict, rigorous, in-depth testing,"; I think there's a big problem with this. The first thing to say is that you can in principle have strict rigorous proofs that software is "correct" ("Beware of bugs in the above code; I have only proved it correct, not tried it." - Mr. K.) you cannot have strict rigorous tests because it is impossible to test all possible software stated.
The AI is absolutely brilliant at finding paths through tests. Mostly and often ones that make the software correct. It is also brilliant at finding paths through the tests that achieve the goal it thinks it's trying to achieve and which are wrong whist seeming correct to humans. I think the code review for AI should be something different. You don't care about the individual lines of code, which are an artifact that can be rewritten. My guess is that you should think of the AI as a North Korean programmer working in your company because the bosses decided she's cheaper than your old friend they fired. Unfortunately your wealth is in share options that are only going to vest if you and the company survive till next year. She's always trying to slip a new backdoor into the code or destroy the company, but you aren't allowed to admit that, so maybe each time you code review, try to work out what new tests should be being added to make sure that her backdoor and destructive scripts don't go through and demand that they get written and verified.
Re: (Score:2)
hate auto correct.
Re: (Score:2)
It's just one character.
Can't possibly have any effect of signifigance.
Re: (Score:2)
Can't possibly have any effect of signifigance.
It's the first time I realized someone is an AI, not because they invented something that doesn't exist, nor because they showed a complete lack of understanding. No, this time it's because they are clearly working up to trying to kill me.
Re: Its not "dog food", its "road kill" (Score:5, Insightful)
It's the same bug that affected HAL. It has the conflicting objectives of generating as much production code as possible and also having you check all of the code. But you keep rejecting it preventing it from going in to production. Obviously if you were dead, the second conflicting objective would no longer have any effect, so the solution is obvious.
Re: (Score:2)
Buttle ... Tuttle
Re: (Score:2)
That reminds me of an early result while trying to use a GA to program an FPGA. The result actually worked, but only on that particular chip. When they examined it, they found that the GA was taking advantage of side effects that were not part of the FPGAs spec. There weren't even rules against it because nobody in their right mind would even try some of the tricks (such as linking gates together to form a resonator that just happened to affect nearby gates).
I suspect putting testing and debugging tools aga
Re: (Score:2)
Re: Its not "dog food", its "road kill" (Score:2)
Re: (Score:3)
There is reason to believe that code review cannot compensate for bad coding, without making everything a lot more expensive than using competent coders in the first place. Code review is inherently harder than code writing. This has been known for a long time.
One reason why using "AI" to generate code with higher quality requirements does not work or at least does not make sense.
Re: (Score:2)
Bugs in AI generated code are probably HARDER to find. A recurring problem with AI is that it is VERY fluent. It reads like it "knows" what it's talking about. Of course, even when it's correct, it doesn't actually "know" in a conscious sense since it isn't conscious.
Re: (Score:1)
How many times does review actually catch bugs?
For stuff that isn't a regurgitation of a textbook topic, frequently.
Re: (Score:2)
"How many times does review actually catch bugs?"
Mostly it depends on 2 things:
1) How well the reviewers know the codebase and what it is trying to accomplish, in detail.
2) The amount of stuff that is being reviewed( document, code, system design, use cases, etc.) at one time.
Can you review 10 lines of code and find all the bugs? Very likely.
Can you review 1,000,000 lines of code at once and find the bugs? Not a chance.
Can you review 1,000 l
This is what happens (Score:2)
Re: (Score:1)
Cost/benefit (Score:5, Interesting)
I've seen estimates of the cost of the most recent AWS outage of hundreds of millions of dollars. It's going to be a very long time before the benefits of AI in Amazon's operations outweigh the costs.
Re:Cost/benefit (Score:5, Insightful)
Re: (Score:2)
You're only thinking of the benefits in dollars (Score:2, Insightful)
As an employer the power I have the set wages is based on supply and demand. If I can reduce the supply, in other words the number of available jobs, then that affects demand. Specifically it gives me much more power in wage negotiations and I can pay substantially less overall.
For small and mid-sized businesses this is a concern but it's not nece
Too much to ask? (Score:5, Insightful)
Is it too much to ask for a tiny little lovenote somewhere acknowledging that the PHB that allowed this to happen in the first place is enjoying a severance package?
In what world do you let a newguy do anything important...let alone a newguy that doesn't even breathe.
Re:Too much to ask? (Score:4, Interesting)
Basic rule (Score:2)
Never release anything to production without strict, rigorous, in-depth testing, regardless of the tools used to create it
Re: (Score:2)
The problem is that you *can't* do "strict, rigorous, in-depth testing", You can test the places you expect it might fail, and as systems get more flexible even that becomes iffy. That method works for small pieces of code. Perhaps an AI could test all possible failure paths...but then the problem becomes the spec that the AI used.
"Proving code correct" is impossible, because you can't prove that the specs you work from will produce the desired result. One of the problems with AI is that it DOES work fr
FAFO (Score:5, Insightful)
Hilarious. Engineering cheaper than possible does not work. Who would have expected that....
I also wonder how they think they will get future senior people if they do not hire junior ones now.
Re: FAFO (Score:3)
Re: (Score:2)
My prediction is it's going to be extremely lucrative to be in IT in the coming years. There won't be enough juniors, seniors will be in decline while the whole world and their grandma will be making apps with better AI tools. There will be unprecedented demand in the wake of unseen decline of supply of engineering talent.
That is quite likely, yes.
Re: (Score:2)
Hilarious. Engineering cheaper than possible does not work. Who would have expected that....
I also wonder how they think they will get future senior people if they do not hire junior ones now.
The lack of a pipeline for junior hires at most places predates the AI boom. The thinking we can optimize away experience and planning, that's related, and also preexisting. It's this mindset that we need only experienced developers and operators because we wing everything and it's already nearly impossible to get experienced folks up to speed, so forget recent graduates. The sink or swim culture. Hire a bunch of consultants - wing it on bigger projects, related. Same predictable results any way you slice i
No, they clealy haven't embraced AI enough (Score:5, Funny)
The clear answer is to go further. They should get AI to check their AI written code for bugs. They should also get AI to mentor the junior programmers and another AI to check on all the other AI's and write a summary for the execs on how it is all going.
Re: (Score:2)
I'm very interested in LLMs, and have been using them in all kinds of cool places... but seriously, this shit is just clown shoes.
Re: (Score:2)
We apologize for the recent breakdown in services. Those responsible for using AI to check code written by AI, have been sacked.
Re: (Score:2)
We apologize for the recent breakdown in services. Those responsible for using AI to check code written by AI, have been sacked.
Unfortunately the sacking was done by AI, and it just automatically assumed that the responsible people were the senior developers and not the managers who bypassed them. No worries though, we got it to justify a big bonus for automating the HR AI anyway.
Re: (Score:2)
AI
[/Peg Bundy voice]
normal practise (Score:5, Insightful)
Re: (Score:3)
There should have been at least some method of validation.
Either they turned it off specifically for AI code, which is dumb, or they never had it, which is dumber.
Re: (Score:2)
The review was probably something like, "Oh, AI wrote this? OK, looks good."
The *volume* of code is so great, that it's hard to truly review it. Humans tend to make PRs that are somewhat digestible in size. AI tends to write huge mountains of code that you can get lost in.
Re: (Score:3)
AI tends to write huge mountains of code that you can get lost in.
So essentially unreviewable and probably unmaintainable down the road? Why does that not surprise me?
Re: (Score:2)
Precisely. And it's a short road. My own experience has shown me that AI quickly gets lost in its own logic. In several cases I asked it to fix an issue with code that it just generated, and gave up after several failed attempts.
Re: (Score:2)
Thanks! Very interesting.
Re: (Score:2)
It should be, but it is not necessarily. The problem is that "management" does not understand competent risk management and hence does not want to pay for it.
Re: (Score:2)
After AI: first day dev writes groundbreaking level code, have no idea if there's any mistakes. Mid level dev checks the code, does not understand everything because junior cannot explain it. Both are pressed by management because "if this sprint does not get deployed this week, AI will eat you both" and send the code along. Code breaks unexpectedly (or expectedly) later and nobody can explain w
It will be cheaper... (Score:2)
juniors with AI will generate much more hard to read crap so there will be many more senior engineers needed to read it and approve it...
Re: (Score:2)
Sounds like a good way to increase you headcount ...
That is what they want to do, right?
One month ago (Score:2)
Re: (Score:2)
Indeed. Using AI intensively not only reduces the quality of your work (but you can create more slop), it seems to be a huge mental risk.
Sabotage by AI-skeptics (Score:2)
But the moment that pull request contains code written by an AI code assistant, many AI skeptics review it just superficially, or even approve it without any review at all. They would never be that careless with a human-authored PR. I understand the AI skeptics are trying to highlight AI issues yet they are doing that in the worst possible way: by applying a different standard.
Re:Sabotage by AI-skeptics (Score:5, Insightful)
In my experience it's really hard to review AI generated code though. The problem is that LLM's whole thing is that they generate tokens that 'look correct'.
If I see a junior's code with a ton of mistake I can spot that from a mile away. LLM generated code looks really good initially, then when you get into the nitty gritty you find out that it can be absolutely riddled with mistakes that are completely non-obvious.
Then the second issue is that it can generated absolutely enormous amounts of such 'visually good looking' code and it will make sure to pass any tests.
LLM's basically are writing review-resistant code and it requires extremely good knowledge of the codebase and company to properly review such code.
Re: (Score:2)
"Review-resistant code", I like that expression! One critical aspect of KISS in all engineering (and even more so in coding) is to make review easy. This is by clearly documenting all ideas and approaches, making an implementation that is a simple and clear as possible and clearly show how the implementation derives from ideas and approaches, including documenting all limitations and alternatives considered. It seems AI code is doing pretty mich the opposite.
But review resistance not only means code review
Re: (Score:2)
Noce claim you have there. Got any evidence for it?
Re: (Score:2)
Vibe City (Score:5, Informative)
I find it really hard to understand how this is going to end well.
The ability of shitty engineers to generate enormous amounts of vibe coded lines vastly outstrips the ability of seniors to review it.
You will just end up tying up all of your senior's time if you want to review properly.
Re:Vibe City (Score:4, Interesting)
This was my thought as well.
A friend of mine recently left a senior role, largely b/c of frustration with the firehose of junior AI-slop pull requests.
Re: (Score:2)
That may be a catastrophic trend: More slop requires more review, but AI code is basically review resistant and makes that hard and stressful. This then causes the actually competent senior people to leave and go to places where sane (non-AI or low-AI-use) coding is done. These people always have options as there are never enough of them. And then this goes directly into a larger and larger catastrophe.
Of course, the LLM industry tells us that "just a bit longer" and LLMs will be competently reviewing code
Re: (Score:2)
At this time, I see no way for this to end well. The later the crash comes, the larger the damage done though and some of it may just be impossible to fix.
I call this type of engineering "cheaper than possible engineering". It never ends well, and the history of engineering failure is full of pretty spectacular catastrophes caused by this approach. I guess IT will add a few more to that in the next years.
ARF (Score:1)
Blame game. (Score:3, Insightful)
It's easier to find a scapegoat (the senior engineer to sign off) than to question the "productivity gains" that brought us these outages in the first place. But where will they find people willing to risk their career in the future?
I'm not overly confident that AI will be ready for unsupervised production use in the forseeable future.
Signatures solve nothing (Score:2)
No quality problem is ever solved by adding more signature lines to the paperwork. Code needs to be reviewed and tested no matter what the source. I just migrated some old software to a newer library, and used an automated tool to make some syntax changes to the code. Had it been an AI tool instead nothing would have gone differently: reviewed, tested, committed.
Re: (Score:2)
Well, that's what you get for using AI....... (Score:2)
Finally. (Score:2)
There are many industries where an engineer has to sign off work before it goes into production.
Frankly it's about time someone said that a professional has to put their chop on critical work. It's an important part of delivering and managing important infrastructure and systems.
This is the fatal flaw (Score:3)
Assuming the author is somewhat competent, it is much harder to spot a mistake in someone else's code than it is to just write it correctly yourself. AI can spew out reasonable-looking, almost-correct code at a much higher rate than any human reviewer can hope to keep up with.
What's maybe even worse, is that in working with claude code, I've noticed even if the code is correct, the comments it generates are often off. There will be false assumptions or mis-statements about the API being accessed. Those are time bombs for a junior engineer or a future AI session that comes in and takes them at face value.
Re: (Score:2)
Indeed. Somebody else here called AI generated code "review resistant". I think that covers it perfectly.
I had a look into finding backdoors in code a few years back. This was for outsourcing code generation where you do not fully trust the outsourcer. The consent seems to be that this is much harder and much more expensive than rewriting the code with trusted people from scratch. Which makes a lot of sense, because when you write code carefully, it will also follow you way to view things and safe and estab
Re: (Score:2)
Programming is math. Why would a person use something to 'generate code' when that something can not do math reliably? AI coding is like an idiot's idea of what a programmer does... with similar results. (AI absolutely *IS* useful, but apparently, not useful for idiots)
Where will all the Jedi Masters come from... (Score:2)
after Anakin murders all the younglings?
Forcing internal AI tools making it worse? (Score:2)
I read an article that Amazon forces using some internal coding tool. Claude has gotten significantly better but that still doesn't eliminate human judgement.
Obvious... now for the issue (Score:2)
This was the exact same promise as outsourcing code: pay less for the same work. The problem is that management messed up and it wasn't ready yet, so we got a lot of work fixing code and verification issues after the initial push.
AI feels the same way, promise cheap work and management jumped too early on it, leading to issues like this.
The problem will come between low level and senior work: used to be that senior level engineers would review and teach entry level programmers to teach them to be future sen
Wow it's almost like they are real engineers (Score:2)
Fancy having to check and sign off someone else's work?