Amazon's AI Productivity Split: Why Half of Their Teams Got 4.5x Gains and Half Got Almost Nothing
Every vendor pitching an "agentic coding assistant" right now wants you to believe the tool is the story. A recent breakdown of Amazon's internal AI-driven engineering pilots says otherwise, and the data is uncomfortable enough that it's worth sitting with instead of skimming past.
What Amazon reportedly found
Amazon ran internal pilots across engineering teams, all with access to the same internal agentic coding assistant ("Curo," built on Claude). The tool was constant. The outcomes were not.
Roughly half the teams saw productivity gains under 3x. The other half saw a 4.5x median, with some teams reportedly hitting 10x or more. Same tool, same company, wildly different results. According to the reporting, the variable wasn't tool access — it was how teams changed their workflow around the tool.
Three case studies were cited, with very different evidentiary weight:
- Bedrock Mantle — a team of 6 engineers (2 of them distinguished engineers) reportedly built a new inference data plane in 76 days, against an original estimate of 30 engineers over 18 months — roughly a 20x reduction in effort-time. The speaker's own caveat: this was an elite team on a greenfield project. Treat it as a ceiling, not a baseline.
- Prime Video — a focused 10-day sprint (6 engineers, no on-call duties, pre-planned tasks) reportedly compressed a 90-week estimated delivery to a projected 24 weeks, based on commit velocity. Again, flagged by the speaker as an artificial, low-distraction setup — not representative of normal team conditions.
- Amazon Stores — the most credible evidence in the set. Fifty teams, normal distribution of seniority, existing/brownfield codebases, tracked over a full year, measured on deployment velocity to production (not commit counts, which are easy to game). This is the study that found the split described above, and it's the one worth taking seriously, because it looks like normal engineering conditions rather than a hand-picked demo.
Amazon's own speaker is explicit that even the Stores data is specific to Amazon's culture and internal tooling — it's not a universal law. Worth remembering before anyone tries to import "4.5x" as a KPI target.
The five habits Amazon says separated the high performers
According to the same source, these are the behaviors that distinguished the high-gain teams from the low-gain ones — not tool sophistication, but workflow discipline:
- Investing in agent context. High performers wrote down domain knowledge, architectural decisions, and team conventions into persistent "steering files" or "skills files" the agent reads before acting — and treated agent failures as a diagnostic ("what context was missing?") rather than a reason to intervene manually next time. One nuance worth noting: as models get stronger, teams reportedly needed less defensive over-specification than they did with older models — the context investment is not a fixed cost, it shifts with model capability.
- Slowing down to speed up. Productivity reportedly dropped at first for teams making this transition properly — before the payoff appeared. Teams improved error messages, built better internal tooling, and in some cases restructured codebases (even switching languages, e.g., Python/JS to TypeScript/Rust) specifically to give agents cleaner compiler and test feedback to work against. The gains showed up only after this foundational investment, not instead of it.
- Feeding agents, not babysitting them. The report draws a sharp line between "vibe coding" — constant back-and-forth chat, 30-second-to-1-minute cycles, human in the loop at every step, capped at one or two agents running at a time — and letting an agent run for hours against a task plus explicit acceptance criteria (tests, quality gates), self-correcting on its own and only interrupting a human once it actually clears the bar. This is reportedly the unlock that makes running multiple agents in parallel viable at all.
- Making intent explicit before coding. High performers wrote a spec or technical design document first and iterated on that with the agent, instead of iterating on scattered code changes after a vague prompt produced the wrong output. The logic: it's cheaper to correct a misunderstanding in a paragraph than across a diff.
- Shifting testing left. Linters and unit/integration/performance/security tests added early, plus mocked local services for fast, deterministic feedback (no cloud spin-up required). This is the mechanism that makes #3 possible — an agent can only self-correct for hours unsupervised if the feedback loop it's checking against is fast and trustworthy.
The report also names real friction that came with this, and it's worth repeating honestly rather than only citing the upside: burnout and FOMO from engineers staying up late to babysit overnight agent runs, cognitive load from managing multiple parallel agent sessions, junior engineers finding code review harder than code-writing (inverting the traditional learning curve), leadership impatience ("why aren't we going faster with these amazing new models"), and a new bottleneck — when code itself takes weeks instead of months, decision-making, approval, and launch-review processes become the actual long pole, not engineering.
Our take
Strip away the specific numbers and one claim in this report matters more than any of the multipliers: the tool was held constant and the outcome still split roughly 50/50. That's the finding that should reframe how any organization thinks about "adopting AI coding tools" — because it means adoption isn't a procurement decision, it's an operating-model decision, and most organizations are currently budgeting for the former while needing the latter.
A few things stand out to us as the more actionable parts of this:
The workflow habits map cleanly onto disciplines that predate AI tooling by a decade. Writing a spec before code, shifting tests left, investing in fast feedback loops — none of this is new advice. What's new is that an under-invested version of these practices used to just mean slower human development; now it caps how much value an agent can extract from a codebase at all. Weak engineering fundamentals don't get bypassed by agentic tooling — they get amplified as a ceiling on it. Organizations with messy test suites and undocumented architecture decisions should expect to sit in the "under 3x" half of this split regardless of which vendor's assistant they buy, because the bottleneck was never really the assistant.
The "slow down to speed up" dip is the part most likely to get an initiative killed early. If leadership approves an agentic AI pilot expecting a hockey stick from week one, and instead sees output dip while a team restructures tests and writes context documentation, the natural organizational reaction is to declare the pilot a failure and pull funding right before the payoff would have shown up. Any adoption plan that doesn't explicitly budget for a temporary productivity dip — and set that expectation with stakeholders in advance — is setting itself up to quit at exactly the wrong moment. This is arguably the single highest-leverage thing to get right before rollout, and it costs nothing but a conversation.
"Feeding, not babysitting" is a genuine management pattern shift, not just a UX preference. The distinction between constant-chat vibe coding and criteria-driven autonomous runs isn't about which feels more comfortable to the engineer — it's the difference between an approach that hard-caps parallelism at one or two agents and one that scales. Teams and leaders evaluating agentic AI capability should treat "can this team define crisp, testable acceptance criteria before work starts" as a more predictive readiness signal than "which model or vendor are we using." This mirrors something we've found true in client engagements well before agentic tooling existed: teams that can write down what "done" means before starting almost always outperform teams that discover it during review — AI agents just make that gap measurable in weeks instead of quarters.
The new bottleneck is a warning worth taking seriously now, not after it bites. If engineering cycle time compresses from months to weeks, and approval/launch-review processes stay on their old cadence, the organization hasn't actually gotten faster — it's just relocated the queue. Any group investing in agentic engineering capability should be looking one layer up the stack (decision rights, review cadence, launch gates) at the same time it invests in the engineering workflow, or it will hit a wall it didn't know it was building.
And a caution in the other direction: the two most dramatic numbers here (20x, ~4x compression) both come from conditions Amazon's own source flags as non-representative — elite engineers, greenfield work, artificially low distraction. The generalizable number is the 4.5x median from the Stores pilot, on real brownfield codebases with normal teams over a full year — and even that comes with the caveat that it reflects Amazon's specific culture and internal tools. Anyone repeating "10x with AI coding agents" as a planning assumption is quoting the ceiling, not the floor. Plan around the boring, brownfield, normal-team number, and treat anything above that as upside.
Bottom line
The headline number in the source report isn't 4.5x or 10x — it's that identical tooling produced a roughly even split between teams that transformed their workflow and teams that didn't bother, or didn't invest early enough to get past the dip. That's a more useful (if less exciting) finding than any specific multiplier, because it's the one piece of this that will hold regardless of which model or vendor is running underneath. The tool is rapidly becoming a commodity. The workflow discipline around it is not, and — per this evidence — it's most of the difference that actually shows up in production.
This post synthesizes third-party reporting on Amazon's internal engineering pilots, as summarized by Perfology Clips (source video, published 2026-09-12). Figures and case studies are Amazon's own reported claims via that source, not independently verified or reproduced by EffiBit. Commentary and analysis in the "Our take" section are EffiBit's own perspective.