Cat Wu leads product for Claude Code and Cowork at Anthropic, so she’s well-versed in constructing dependable, interpretable, and steerable AI programs. And since 90% of Anthropic’s code is now written by Claude Code, she’s additionally deeply aware of becoming them into routine day-to-day work. Final month, Cat joined Addy Osmani at AI Codecon for a hearth chat on the way forward for agentic coding and, equally essential, agentic code evaluate, how Anthropic truly makes use of the instruments they’re constructing, and what abilities matter now for builders.
The suggestions loop is itself a product
Boris Cherny initially constructed Claude Code as a aspect mission to check Anthropic’s APIs. Then he shared the instrument in a pocket book, and inside two months all the firm was utilizing it. That natural progress, Cat mentioned, was a part of what satisfied the crew it was price releasing externally.
However what actually made that inside adoption seen was the response on Anthropic’s inside “dog-fooding” Slack channel. The Claude Code channel will get a brand new message each 5 to 10 minutes across the clock, and this suggestions immediately and instantly informs the product expertise. Cat described it this fashion:
We rent for individuals who love sharpening the person expertise. And so plenty of our engineers truly dwell on this channel and discover when there’s points with new options that they’ve labored on they usually proactively lay out the fixes.
The crew ships new variations of Claude Code to inside customers many occasions a day. The suggestions loop is tight sufficient that it features as a steady integration system for product high quality, not simply code high quality.
Cat informed Addy how she as soon as by accident launched a small interplay bug between prompts and auto-suggestions. However by the point she began engaged on an answer, she discovered one other crew member had already overwhelmed her to it. It seems, he had arrange a scheduled job in Claude Code to scan the suggestions channel for something that hadn’t been responded to in 24 hours and open a PR for it. Since Cat hadn’t gotten to it but (whoops!), her teammate’s Claude noticed the unaddressed problem and glued it for her. And Cat solely came upon when “[her own] Claude seen that his Claude had already landed a change.”
The infrastructure for speedy enchancment, in different phrases, is now partly automated. The brokers are writing the code, then monitoring the suggestions and shutting the loop.
The bottleneck has shifted to evaluate
There’s no query that AI-assisted coding has created a increase in output. Anthropic engineers are producing roughly 200% extra code than they had been a 12 months in the past, Cat famous. As we speak the primary constraint is reviewing all that code to make sure it’s production-ready.
Cat’s crew concluded you can purchase plenty of further robustness for not that a lot additional value.
We opted for the heaviest, most strong model [of code review]. We truly plot what number of brokers and the way complete of a evaluate Claude does after which what number of bugs does it recall. And we picked a lot of very excessive recall and determined we should always ship this, as a result of for those who actually need AI code evaluate to be a load-bearing a part of your course of, you truly most likely simply need essentially the most complete attainable evaluate.
The evaluate agent doesn’t simply take a look at the diff. It traces code throughout a number of information and catches bugs in adjoining code that has nothing to do with the change in query. Cat gave two examples. One was a ZFS encryption refactor the place the agent discovered a key cache invalidation bug that wasn’t associated to the writer’s change in any respect however would have invalidated it. The opposite was a routine auth replace that turned out to have a nasty aspect impact, caught premerge. In each circumstances, engineers manually reviewing the code probably would have missed the bugs.
The human evaluate that continues to be is intentionally small in scope. For many PRs, the human reviewer skims for design precept violations and apparent issues and assumes practical correctness has been dealt with. 5 to 10 brokers run in parallel, every given barely completely different duties, returning independently after which deduplicating what they discovered.
The cultural shift that made this work, although, was possession. The crew moved to a mannequin the place the engineer who authors a PR owns it finish to finish, together with postdeploy bugs, and doesn’t lean on peer reviewers to catch errors. “In any other case,” as Cat identified, “you’ve conditions the place junior engineers put out a bunch of PRs after which your senior engineers are like drowning in AI-generated stuff the place they’re undecided how totally it’s been examined.”
Full possession meant the AI evaluate needed to truly be reliable, which drove the choice to go for prime recall moderately than a lighter contact. That mentioned, engineers are nonetheless anticipated to know each line of code an agent creates. . .for now. As Cat defined, it’s the one option to really forestall “unknown safety vulnerabilities and to have the ability to rapidly reply to incidents if they’re to occur.”
Everybody’s sort of an engineer now
Cowork, Anthropic’s agent instrument for nontechnical customers, is the corporate’s try to take what Claude Code does for engineers and convey it to data work extra broadly. Cat sketched an image of somebody 5 or 6 agent duties working concurrently in a aspect panel, managing a fleet of brokers the way in which a senior engineer manages a PR queue.
Within the nearer-term, she’s protecting tabs on the shift towards individuals utilizing Claude Code to construct issues for themselves, their groups, or their households that wouldn’t have justified skilled improvement effort or “in any other case been attainable.” The prototype is the storage mission, the household expense tracker, the instrument {that a} small crew truly wants however that no SaaS product fairly addresses. Cat’s purpose and hope is that Claude Code helps individuals “remedy their very own issues for themselves” and “stewards a brand new future of private software program.”
Product style as the brand new technical ability
Extra individuals constructing extra software program is unambiguously good. Boris Cherny has even floated the concept coding as we all know it’s “solved.” However what does that imply for the craft of software program engineering? Cat’s learn of the present second is extra nuanced:
I feel pre-AI, the talents that had been essential had been with the ability to take a spec and implement it effectively. And I feel now the actually essential ability is product style. Even for engineers. Can you employ code to ingest an enormous quantity of person suggestions? Do you’ve good instinct about which function to construct to deal with these wants, as a result of it’s typically completely different than precisely what customers are asking you for? After which, when Claude builds it, are you establishing the fitting bar in order that what you ship individuals truly love?
Cat’s not alone in highlighting the significance of style in a world the place code is a commodity. Steve Yegge, Wes McKinney, and lots of others, myself included, see style and judgment as a uniquely human worth. This has sensible implications for the way engineers ought to spend their time now, and for what the subsequent technology must be taught.
For junior engineers particularly, Cat described a development: Begin by utilizing Claude Code to know the codebase (ask all of the “dumb questions” with out embarrassment), take these solutions to a senior engineer for calibration, after which shut the loop by updating the CLAUDE.md with no matter was lacking.
Consider Claude Code as your intern that you simply’re attempting to stage up. Like, train it again to Claude. Add a
/confirmslash command. Put it within the CLAUDE.md or the agent README. Method this as senior engineers serving to you stage up, and you then serving to Claude and different brokers stage up.
The development course of, in different phrases, ought to be bidirectional. Engineers get higher at utilizing the instruments and the instruments get higher via the engineers’ gathered data. And considerably, this course of retains people firmly within the loop, taking part in a task that’s “energetic, steady, and expert.”
You may watch Cat and Addy’s full chat, plus every thing else from AI Codecon on the O’Reilly studying platform. Not a member? Join a free 10-day trial, no strings connected.
