Amazon Employees Are 'Tokenmaxxing' Due To Pressure To Use AI Tools (arstechnica.com) 50
An anonymous reader quotes a report from the Financial Times (via Ars Technica): Amazon employees are using an internal AI tool to automate non-essential tasks in a bid to show managers they are using the technology more frequently. The Seattle-based group has started to widely deploy its in-house "MeshClaw" product in recent weeks, allowing employees to create AI agents that can connect to workplace software and carry out tasks on a user's behalf, according to three people familiar with the matter. Some employees said colleagues were using the software to automate additional, unnecessary AI activity to increase their consumption of tokens -- units of data processed by models. They said the move reflected pressure to adopt the technology after Amazon introduced targets for more than 80 percent of developers to use AI each week, and earlier this year began tracking AI token consumption on internal leader boards.
"There is just so much pressure to use these tools," one Amazon employee told the FT. "Some people are just using MeshClaw to maximize their token usage." Amazon has told employees that the AI token statistics would not be used in performance evaluations. But several staff members said they believed managers were monitoring the data. "Managers are looking at it," said another current employee. "When they track usage it creates perverse incentives and some people are very competitive about it."
"There is just so much pressure to use these tools," one Amazon employee told the FT. "Some people are just using MeshClaw to maximize their token usage." Amazon has told employees that the AI token statistics would not be used in performance evaluations. But several staff members said they believed managers were monitoring the data. "Managers are looking at it," said another current employee. "When they track usage it creates perverse incentives and some people are very competitive about it."
So basically the AI equivalent... (Score:5, Insightful)
...of those apps that make it look like you are moving your mouse a lot?
Re: So basically the AI equivalent... (Score:5, Insightful)
Nah, this is more like working at a company that measures your productivity in LoC typed. People making shit long-winded just to game the numbers.
Can't wait to hear about some big incident because someone automated something unnecessarily to increase their token usage. It will be literally the fault of these policies regarding AI adoption.
Re: (Score:3)
Re: So basically the AI equivalent... (Score:2)
I haven't seen slop, but they already laid off all the other engineers. There is no one else left to create it except me and I refuse.
Re: So basically the AI equivalent... (Score:3)
Wait 'til they find out just how subsidized these tokens really are, mainly by circular financing. I'm not pooh-poohing AI, but I think there's a chasm between what executives expect it to provide, and what it actually provides.
Re: (Score:2)
One of the first managers I ever had told me something I have remembered for my entire career. "Be very careful about what you decide to measure, you are certain to get more of it."
Amazon has chosen to measure AI use. So the natural reaction of employees is going to be to put effort into finding was to use whatever AI tools they have been given. In the best case sure they will use them increase productivity, however after they exhaust the obvious use cases where there is real gains they are going to star
Re: (Score:2)
More like the equivalent of ballmaxxing [forbes.com], your balls look yuge, but they perform worse, it makes you sick, and your dick looks even smaller.
You know, a useful metaphor for many things these days.
Re: (Score:2)
Only in this case, managers at Amazon are congratulating and rewarding people who move their mouse a lot, and begging them to do it more!
Backwards from what they think? (Score:5, Insightful)
Given that big companies have already made it clear that they think AI will let them do the same work with fewer people, and given that using AI costs the company a lot in terms of compute resources, it seems intuitively obvious that the only reason execs would want to encourage more AI use is to find out what jobs can easily have their headcount reduced by more use of AI.
The people using the most tokens are the ones for whom more of their jobs can be most easily automated. This is not, IMO, a positive sign for the long-term survival of that particular job role. The only rational response is to use AI just enough to show a speed-up, assuming the speed-up actually happens at all, but not enough to be high up on the chart of AI users. Using it way more than that seems self-defeating.
Re: (Score:3)
"It seemed like such a _good idea_ at the time!"
And... thus... the Great Depression V.2 begins... companies still sell products, companies still make products, nobody can afford to buy the products.
Re: Backwards from what they think? (Score:1)
Not from AI. Governments create great depressions when they try to "fix" things. Namely Smoot-Hawley. If we get another depression, guarantee you'll be able to trace its origins to none other than Donald Trump. But it's unlikely due to the reversals outside of his control that have already happened. Or perhaps the Strait of Hormuz will force a global one anyway, which is also a "fix" that is now out of his control.
Re: (Score:2)
This costs money (Score:3, Interesting)
Re: (Score:3)
True costs are easily double that.
Yeah, I'm gonna guess it's a lot more than double. My company's enterprise agreement with MS has Claude Opus 4.7 now at 15x token consumption. I bet that's approaching what it actually costs but still not quite it.
That which is measured (Score:5, Insightful)
That which is measured is improved.
You want more token usage? You got it!
Re:That which is measured (Score:5, Insightful)
A million years ago when I got my first management job I had to attend a training session on 'Goals and Objectives', the current in vogue management tool. The instructor impressed on us that 'you get what you measure'. He used an example of police wanting to improve road safety by measuring the number of moving violation tickets given out. Ticket quantities went through the roof but there was no improvement in accident rates; go figure. What was true in the 70s is still true today.
Re: That which is measured (Score:4, Interesting)
Indeed. Here in California we are the poster children for this, because we have a 55 mph speed limit while towing which is NEVER enforced. We have a requirement that headlights be aimed correctly, same. We have a law saying that if there are five or more people behind you, you must pull over at the first safe opportunity to let them pass, same. Fender flares must project as far as tires, same. (Anyone who's ever had a rock break their windshield understands.)
CHP cares specifically and only about revenue generation, so they do nothing to improve safety except go after speeders. That's not nothing, but it's not enough.
Re: That which is measured (Score:3)
In many areas of California, pulling someone over actually increases the danger for everyone due to the heavy traffic and the dangerous impact of unexpected slowdowns on the expressway. They avoid enforcement in many cases because it's the correct public service behavior.
Re: (Score:2)
In many areas of California, pulling someone over actually increases the danger for everyone due to the heavy traffic and the dangerous impact of unexpected slowdowns on the expressway.
When you're driving, you should expect slowdowns. If you're not sufficiently aware to handle them, you're not sufficiently aware to drive. Also, if you don't know you can proceed to an exit or another safe place when being pulled over, same. This is what happens when we don't expect basic competence from drivers. Which, again, is because we want to maximize profit. In this case it's related to the war on public transportation still being waged by Big Oil, Big Auto, and Big Rubber. None of this has anything
Re: (Score:2)
Re: (Score:2)
A million years ago when I got my first management job I had to attend a training session on 'Goals and Objectives', the current in vogue management tool. The instructor impressed on us that 'you get what you measure'. He used an example of police wanting to improve road safety by measuring the number of moving violation tickets given out. Ticket quantities went through the roof but there was no improvement in accident rates; go figure. What was true in the 70s is still true today.
Which is why you have to ensure what you measure actually produces desired outcomes and not create perverse incentives. Reminds me of teh Dilbert cartoon where the PHB announces a bug bounty and Wally says “I’m writing myself a new minivan”
Re: (Score:1)
Goodhart's law in action.
You'd figure they'd hire managers who are good enough to understand this.
Re: That which is measured (Score:2)
This is Amazon, not ... you know ... a good company.
You get what you reward, not what is best (Score:2)
That which is measured is improved. You want more token usage? You got it!
I like a different spin on that. "You get what you reward, not what is best."
That was the phrase they used when they were training us that there is no magic bullet to measure performance. That judging performance is a hard job, stop reaching for this year's new snake oil. Do your job, know what's going on, who needs help with something. NEVER punish someone for admitting they need help with something, just do all you can for them.
Re: (Score:2)
Re: (Score:2)
It's not about the quality of the output, but someone, somewhere needs to see a nice chart showing an upward trend in token usage because in their mind, that somehow translates into "more productivity"
We have literally been told that AI usage is mandatory and that if we're not using AI for everything then we are dinosaurs. Don't get m
"internal leader boards" (Score:3)
Those kinds of shenanigans are a good example of why I remain self employed to this day. You can keep your Office Space style bullshit. Modern corporations, especially aggressive companies like Amazon, have gamified the workplace into just sucking every last ounce of energy out of their 'human resources'. You're more a slave and less an employee every year.
So, Amazon hasn't changed in 20 years... (Score:3, Interesting)
"Thank you for calling Amazon.com, may I help you?" Then - Click! You're cut off. That's annoying. You just waited 10 minutes to get through to a human and you mysteriously got disconnected right away.
Or is it mysterious? According to Mike Daisey, Amazon rated their customer service representatives based on the number of calls taken per hour. The best way to get your performance rating up was to hang up on customers, thus increasing the number of calls you can take every hour.
Re: (Score:2)
Amazon rated their customer service representatives based on the number of calls taken per hour. The best way to get your performance rating up was to hang up on customers, thus increasing the number of calls you can take every hour.
I wasn't at Amazon, but I can confirm. 20 years ago I worked in a call centre and they used a similar metric, which was basically "calls per hour." There was another perverse incentive too - if you didn't actually solve a customer's problem, they'd probably need to call back again soon, further inflating your stats. I remember listening to the guy who had the best stats, all he ever said was "fix your firewall."
As other posters have stated, the way to fix this has two parts. First, better metrics. Second, b
Yo dawg, I heard you like automation. (Score:3)
So, they're using an AI tool to automate "looking like" they're using AI? It's AI all the way down? AI driving AI? Maybe Amazon can luck out and just have a whole bunch of AI systems driving other AI systems without needing any employees? AND THE FANTASY CAN BECOME REAL!
Re: (Score:2)
Re: (Score:3)
It is called Model Collapse [wikipedia.org], and avoiding it is a hot research topic.
Re: (Score:2)
It is called Model Collapse [wikipedia.org], and avoiding it is a hot research topic.
What impresses me the most is that anyone would not understand what happens when AI starts referencing itself, because ot itself and other AI swamping everything else out. At that point, it becomes truth irrelevant and either worthless or a sort of religion.
Re: (Score:2)
That's not very hard (Score:4, Interesting)
Particularly if you have a system that supports multi-agent teams, you can just spin up a bunch of them, create a harness that makes them talk to each other in real-time and tell them what you want to do. It's not hard to do that.
For companies like Amazon that probably makes sense. They want to prolong the bubble of "Frontier model" companies. For the rest of the world, price hikes will eventually make them creative. My prediction is that we might see something a bit more clever than current coding agents that can deliver good, if not even great, results with comparatively tiny models. The smaller the model the easier it is to train, so maybe... future models will be so small you can train them for your project, on your own computers, for a fraction of the price of current solutions.
Nobody knows, but at our company, the next Anthropic price hike will mean very substantial costs for the company I'm working at. The kind of cost that gets you a substantial number of programmers.
Re: (Score:2)
You don't need to train a small model, you need to give them access to a vector database with a query tool and they do fine looking stuff up in pieces and using it.
Re: (Score:3)
Not sure if you have explored it very much but LM Studio and ComfyUI are huge ways you can run your own models. ComfyUI is built out a bit more for training and while it has more of focus on text-to-image workflows, you can configure it pretty well to work with LLMs.
Many of the models are quite light-weight and a decent gaming system can keep them in memory though quantization is generally needed a bit to help. Quanitization seems very necessary more with image models but in the case of code, you likely cou
Metrics create behavior. (Score:2)
Goodhearts law (Score:2)
at it's finest. When will people learn
Re: (Score:2)
Never. There's an (AI) sucker born every minute.
Re: (Score:2)
Transhumanists (Score:2)
The trouble with Transhumanists is that they all imagine there will eventually be only one human per company with an army of AI's to do the work.
But they all think they'll be that one person.
Also, they're insane.
Re: (Score:2)
The Midas Plague, software version (Score:2)
"Some employees said colleagues were using the software to automate additional, unnecessary AI activity to increase their consumption of tokens -- units of data processed by models. "
The AI blurb (oh the irony)
"Midas World is a 1983 science fiction novel by Frederik Pohl, a "fix-up" novel that expands on his classic short story, "The Midas Plague". It explores a future society with extreme abundance due to automation, where the poor are forced to consume a quota of goods to keep the economy running, while t
Re: (Score:3)
Re: (Score:2)
But at least they're not ... (Score:2)
Horrible (Score:2)
Amazon sounds like a truly horrible place to work. Meta too.