Opus 4.6 Was the Last Coding Model That Changed How I Work
Models have kept getting better. My workflow has not changed nearly as much.
It seems like every month a new coding model lands with yet another chart showing how “groundbreaking” it is, and cue the fear mongering about how this new model is going to kill software engineering for good. Don’t get me wrong, I believe the improvements. I use these newer models myself daily. But the only time since the initial launch of copilots in 2023 I felt the ground “breaking” was when Opus 4.6 got released this past February. 1
I became comfortable delegating a large (yet scoped) task and letting the agent run. I could give a goal, let it explore the codebase, answer a couple of its questions, and then let it do its thing. Cursor’s Plan Mode better illustrates this loop; you ask, the agent formulates a plan, then executes it.
Did my productivity scale with Coding Benchmarks?
If you read the paragraph above, you know my hunch was no. But I wanted to check whether that was just a feeling, so I went back through my work from September 2025 through July 2026 and built a rough output measure. I assigned delivered work a simple weight based on how substantive it was: routine = 0.5, medium = 3, high = 8, flagship = 13. The inputs were JIRA tickets closed, PRs merged, incident follow-ups, design docs and larger projects shipped.
Substantive engineering output, normalized to the Sep–Jan average 2
My substantive output jumps from a Sep–Jan baseline of 100 to about 163 in February. March is 163, April 168 and July 163. The specific numbers are hand-wavy at best, but the shape is what interests me: spike when the actually workflow changed, then no obvious second step-change as newer models arrived month after month.
Correlation doesn’t always equal causation, but I’m trying to quantify how the job felt. To me using Opus 4.6 felt like a shift, and this corroborates that story for me.
As LLMs continue to make gains on coding benchmarks, the benchmark I care about (delivering shareholder value), hasn’t gone up. I suspect until we see a new paradigm shift in how we can interact with these agent teams, our productivity gains will continue to stagnate.
Coding Benchmarks aren’t the whole job
Coding benchmarks show real progress, sure. The models are better at solving some isolated software tasks, and those gains aren’t totally fake either. The scoring can be quite deceptive if you want to actually get real engineering output though. METR is unusually explicit about this: its time-horizon tasks are designed to be self-contained and well-specified, and it warns against reading it as a high-context engineer’s actual work. 3
HiL-Bench is a rare benchmark that gets closer to the failure mode I do care about. It starts with tasks a model can solve, removes information that is required to solve them, and gives the agent an ask_human() tool. Now the agent has to notice that something important is missing instead of confidently filling in the blank itself. 4
Scale AI HIL-BENCH judgment gap
Seeing this for the first time helped me back up what is was feeling. The model can be capable of implementing the right solution once it has the facts, while still being bad at recognizing that the facts it has are insufficient. What gets me is that the code often looks fine. It’s just based on the wrong assumption.
Caveat, newer models are improving here too. On Scale’s separate combined live leaderboard, Opus 5 is at 57% Pass@3 versus 38% for Opus 4.6. 5 That is real progress.
Yet it still only captures one piece of the puzzle. It can test whether an agent notices a missing piece of information. It doesn’t change the way we actually perform the work. Even if Opus 5 asks me more questions, I’m still there working with the agent in real time, and when it says it’s done you are most likely going to have to correct it anyways, just like you would have with Opus 4.6.
I tried giving the agents a “manager”
I’ve been pushing on this more aggressively using Gas Town 6, where a “Mayor” coordinates a crew of coding agents. My goal is to stay out of the low-level loop, I don’t have time to do all this micro-management of my agents. I give higher-level tasks to the Mayor, use longer planning sessions, keep persistent project directives, and let the crew distribute the work.
It works decently well. Individual tasks can disappear into the crew and come back mostly finished.
The problem appears over longer stretches of work: drift
An agent finds a niche engineering problem and starts solving it. The solution is often reasonable in isolation. But when I come back and use the product or look at the architecture as a whole, I realize we have spent too much effort on something that barely matters, added complexity I did not want, or slowly moved away from the original product vision.
None of the individual decisions have to be stupid. They are often locally sensible. That is exactly the problem.
Planning helps. Persistent memory helps. Reviewing a detailed implementation plan helps. I still check in every day, run parts of the project myself, and redirect the crew when the implementation starts drifting from the broader intent.
Yes, some of this is the peanut-butter-sandwich problem7: better instructions produce better results. But “write a perfect specification” is not a solution to autonomous engineering. Incomplete specifications are the normal state of software development. Requirements change as you build. You discover things. Users behave differently than expected. Two individually sensible decisions interact badly. Sometimes the correct solution to a difficult engineering problem is to avoid solving it at all.
So, what next?
AI slop image showing where we are at in the engineering timeline
The first jump was code generation: write this function, fix this error, build this component.
The second was individual task execution: here is the goal, explore the repo, figure out the implementation, ask when necessary, test it and bring me back a result. Opus 4.6 was the first model that made this reliable enough to change my default workflow.
The next jump is sustained project coherence: keep the product intent, architecture, priorities and unknowns in mind over days or weeks of work, and interrupt me when a decision actually requires judgment. Just like a team does with their manager in the real world.
That is the transition I am still waiting for. And I’ll share my brain dump on a post in the near future.
I write much less code myself than I did a year ago, if any at all. I can run most of the implementation in parallel and hand off all my work to an agent.
But since Opus 4.6, I’ve still been doing the same job every day, babysitting my agents while they work.
The models keep getting better. I’m still managing the team.
Anthropic, “Introducing Claude Opus 4.6,” Feb. 5, 2026. https://www.anthropic.com/news/claude-opus-4-6
May-June had ~25% of working days on PTO. Substantive units without PTO would be raised up by ~33% to meet at 150-160 range.
METR, “Task-Completion Time Horizons of Frontier AI Models.” https://metr.org/time-horizons/
Scale AI, “HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?” https://static.scale.com/uploads/67a153343e046988406ef320/HiL_Bench.pdf
Scale AI, HiL-Bench live leaderboard. https://labs.scale.com/leaderboard/hil
Gas Town: Multi-agent orchestration system for Claude Code, GitHub Copilot, and other AI agents with persistent work tracking. https://github.com/gastownhall/gastown
Classic teaching example that demonstrates the need for precise algorithms, showing how vague instructions lead to literal, chaotic failures. https://www.scientificamerican.com/article/robot-make-me-a-sandwich/