How AI Redefined Engineering at Klaviyo
Generated by Klaviyo AI
Klaviyo made level 3 AI autonomy its engineering baseline, with agents implementing code while engineers define problems, set constraints, and remain accountable. The company measures impact through delivery, speed, quality, and hours saved while moving engineers closer to customer needs.
- Level 3 baseline: As of the end of July 2026, AI generated 85% of Klaviyo’s code under engineer-defined goals, constraints, and oversight.
- Judgment matters: Engineers increasingly focus on scoping problems, setting guardrails, and verifying whether AI-generated work should ship.
- Mixed experience: Teams combine junior engineers’ AI fluency with senior engineers’ experience in system design, decomposition, and verification.
On September 23, I joined Stephanie Cohen, Cloudflare’s chief strategy officer, and Owen Jennings, Block’s head of business, on a panel at The Information’s AI Agenda Live in San Francisco.
The event brought together leaders from OpenAI, Google DeepMind, NVIDIA, Atlassian, and EY to talk about where AI is gaining traction inside companies. On our panel, each of us described how our company has restructured around AI agents, and all of us concluded that AI works best when you redesign roles, decisions, and training around it.
At Klaviyo, that redesign started with how we trained our own engineers.
4 levels of AI autonomy for engineering teams
For us, the hardest problem was that the craft of engineering changed underneath people who’d spent a decade getting good at the old version of it.
We gave our engineering team a framework, borrowed from how the industry talks about self-driving cars:
- Level 1: AI handles boilerplate and autocomplete.
- Level 2: Engineers delegate whole discrete tasks.
- Level 3: Engineers set the goal, the constraints, and the context up front, letting an AI agent write the implementation but staying fully accountable for what ships.
- Level 4: Engineers direct agents across large cross-system builds. We haven’t seen anyone do this reliably yet, because it needs stronger testing, observability, and self-correction than exists today.
Level 3 is our baseline across Klaviyo today. As of the end of July 2026, 85% of our code was AI-generated. At level 3, engineers spend their time defining the problem, setting guardrails, and deciding whether what comes back is right. Andrew Bialecki, CEO of Klaviyo, discusses this shift in depth in his recent SaaStr interview.
That same shift in judgment also changed how we evaluate talent. Engineers interviewing at Klaviyo now show us how they’d work with AI agents once they’re on the team. It also changed how we think about junior engineers.
More than once on our panel, we heard that they still add value even though AI can generate so much code. Junior engineers are often AI native, so they start working at level 3 from day one. Senior engineers pick up these tools quickly too, and with their experience designing large systems and breaking work into the right pieces, they can become highly productive fast.
Teams do well when they mix both, pairing fresh fluency with agents and deep judgment in scoping and verifying the work.
How Klaviyo measures AI impact
Everyone on the panel had seen engineering velocity climb, including pull requests and feature delivery. But shipping faster only helps if the feature was worth building in the first place.
We track 4 numbers: how much we ship, how long it takes a change to reach customers, the quality of what ships, and how many hours we save across the teams we support. The fourth number comes from proximity. When engineers spend time alongside the teams they support, they find friction that no ticket captures, like hours lost to manual research before customer calls. Fixing that with AI has saved tens of thousands of hours across the organization.
That’s why we put engineers closer to customers. At a growing company, requirements often pass through layers of people before they reach an engineer, so someone ends up picking up work without knowing why it matters. Any engineer should be able to explain, in their own words, why what they’re building is worth their time and which customer pain point it addresses.
More of the building, by more builders
More people and more agents are doing the building at Klaviyo. Our autonomous coding agent opens fixes for production errors on its own, and engineers decide what merges. We also built K:Forge, an internal platform on Vercel, and opened it to all employees. Some ideas go live in under 3 minutes, and in the first two weeks we shipped 356 apps, 196 of them full-stack with their own databases.
With code this fast to write, the scarce skill is knowing which customer problem is worth solving. Engineers learn that by sitting close to customers and the teams that support them. If you’re redesigning your own engineering team, start there.
Related content
