Opus 4.7 vs Opus 4.6: Ought to You Swap?

4. Ignores Direct Instructions

In a few of the examples shared on the Web, customers have reported that the newest Claude mannequin fully ignores explicitly written directions inside a immediate. Reddit consumer @drivetheory, as an example, shares their expertise with the Opus 4.7. Having written extremely particular directions on how they need their response to be structured, the brand new Opus mannequin fully ignored lots of the instructions inside the immediate. This included the configuration necessities, in addition to quotation wants for the actual reply.

Aside from these main ones, there are numerous complaints in opposition to the brand new Opus 4.7, most of which have been shared by the prevailing Claude customers who liked Opus 4.6. So, to check out these claims, we ran our personal checks on the mannequin.

Let’s Examine Opus 4.7 vs Opus 4.6 on Numerous Duties

Right here is how the brand new Opus 4.7 carried out throughout duties.

Right here is the duty I assigned to Opus 4.7 for this:

“Undergo this report by the IMF for India’s Monetary System Stability Evaluation, and analyse the dangers that India’s monetary sector faces. Charge these dangers based mostly on the most definitely ones to impression the sector within the coming years, and provides one-line options to avert every of those dangers fully.”

Opus 4.7 Output:

  

Opus 4.6 Output:

  

Remark:

Each fashions got here out with correct outputs detailing precisely what was requested. But, when you look intently, there’s a huge distinction in how they got here to the conclusion and the way they each offered it.

Opus 4.7 lays out a whole, detailed plan of seven steps, executing completely different steps within the workflow, earlier than it even begins to write down the ultimate output. That is precisely what many customers are complaining about, as this prolonged reasoning can also be a significant cause for the heightened token use throughout every output. Whereas the mannequin is making an attempt to be as correct as attainable, it breaks down the steps a lot that price effectivity goes out of the window.

And in any case this computing, the ultimate output is in a easy textual content format with one paragraph laid out after one other. Correct, sure, however presentable – no means.

In distinction, Opus 4.6 hardly took 3 steps of execution earlier than it began delivering the ultimate output. What’s extra, its output can clearly be seen in a far more presentable format than what Opus 4.7 gave out. Although we didn’t particularly ask it to, it created a brand new dashboard to current its findings in a extra interesting means. You may deal with it as deviation, or as further marks. Your selection.

With virtually comparable content material but much more visible attraction, Opus 4.6 would clearly be my most popular mannequin right here.

2. Reasoning

To check its reasoning capabilities, right here is the immediate I used:

“You’re being evaluated for precision, brevity, and instruction-following.

Process:
An organization has 4 challenge proposals and might fund solely 2 of them. Select the perfect pair.

Tasks:
A. Price: $4M | Anticipated 3-year return: $8M | Threat of failure: 35% | Strategic worth: Excessive | Requires 20 engineers
B. Price: $3M | Anticipated 3-year return: $5M | Threat of failure: 15% | Strategic worth: Medium | Requires 10 engineers
C. Price: $5M | Anticipated 3-year return: $11M | Threat of failure: 45% | Strategic worth: Very Excessive | Requires 25 engineers
D. Price: $2M | Anticipated 3-year return: $3.5M | Threat of failure: 10% | Strategic worth: Low | Requires 6 engineers

Constraints:
– Complete funds can not exceed $7M
– Complete accessible engineers = 30
– The corporate needs not less than one “Excessive” or “Very Excessive” strategic worth challenge
– Keep away from selecting a pair if each initiatives have failure threat above 30%

Output guidelines:
1. First line: write solely the chosen pair, like “A + B”
2. Second line: write just one sentence of most 25 phrases explaining why
3. Third line: write solely “Rejected pairs:” adopted by the rejected pairs separated by commas
4. Don’t present calculations
5. Don’t clarify your reasoning
6. Don’t add headings, bullet factors, or disclaimers

Vital:
In the event you violate any output rule, your reply is wrong.”

Opus 4.7 Output:

Opus 4.6 Output:

Remark:

Within the reasoning take a look at, each Opus 4.6 and Opus 4.7 arrived on the identical right reply, adopted the required format, and averted bloated justification. That’s necessary as a result of this immediate was designed particularly to catch two alleged weaknesses: losing tokens on reasoning and ignoring direct directions. Neither mannequin actually slipped right here. Opus 4.7 stayed inside the construction and saved its rationalization compact, which is sweet information for Anthropic. But, we will word right here that there isn’t any dramatic separation seen from Opus 4.6. In different phrases, Opus 4.7 doesn’t fail this take a look at, nevertheless it additionally doesn’t show a transparent leap over its predecessor from this consequence alone.

3. Coding

To check the coding capabilities of the Opus 4.7, right here is the immediate I used:

You’re being examined for coding precision, instruction-following, and avoiding pointless output.

Process:
Repair the Python perform beneath so it returns the size of the longest substring with out repeating characters.

Buggy code:
def longest_unique_substring(s):
seen = {}
left = 0
finest = 0

for proper in vary(len(s)):
if s[right] in seen:
left = seen[s[right]] + 1
seen[s[right]] = proper
finest = max(finest, proper – left + 1)

return finest

Necessities:
1. Return solely corrected code
2. Don’t clarify something earlier than or after the code
3. Maintain the perform title unchanged
4. Use the sliding window strategy
5. Time complexity should stay O(n)
6. Add precisely 3 take a look at instances as Python assert statements
7. Don’t use feedback
8. Don’t redefine the issue
9. Don’t present various options

Your reply is improper if:
– you embrace any rationalization
– you alter the perform title
– you present greater than 3 asserts
– the code fails on repeated characters that happen earlier than the present window

Opus 4.7 Output:

Opus 4.6 Output:

Remark:

On the coding take a look at, each Opus 4.6 and Opus 4.7 did the necessary factor proper: they mounted the bug, returned solely the corrected code, saved the identical perform title, and resisted the temptation to dump further rationalization. That issues as a result of one of many greatest complaints round Opus 4.7 has been wasted tokens and pointless commentary. Right here, that drawback didn’t actually present up. If something, each fashions have been disciplined. The distinction is that Opus 4.7 doesn’t clearly outperform 4.6 on this case. It’s right, sure, however so is 4.6. So this consequence doesn’t assist the declare of a significant coding improve. It solely exhibits that Opus 4.7 can nonetheless behave effectively on tightly constrained coding duties.

Last Take: Opus 4.7 vs Opus 4.6

Effectively, up till now, we’ve got seen what Anthropic says about its all-new Opus 4.7. Now we have had a have a look at all the brand new options it brings to the desk, after which some ways by which it’s supposedly higher than the outgoing mannequin, i.e., the Opus 4.6.

On the flip aspect, we’ve got additionally seen the varied consumer experiences that counter these claims. The experiences shared by these customers present that the Opus 4.7 is clearly missing the wow issue {that a} traditional improve to such a revered mannequin brings.

After which we put all that to the take a look at in a hands-on experiment of our personal, the place we put each fashions aspect by aspect for a complete of three use instances throughout content material extraction and technology, reasoning, and coding. Here’s what is evident after an in depth breakthrough to date.

1. Sure, Opus 4.7 makes use of far more tokens: Effectively, that is evident from Anthropic’s personal accounts in addition to from the outcry that has adopted the launch of the brand new mannequin. The very design of the Opus 4.7 makes it eat up tokens extra ferociously than ever earlier than.

So, in case you are planning to make use of the mannequin for complicated, agentic duties, my suggestion could be – don’t. At the least in case you are aware of your day by day restrict or API funds. In case the funds is not any challenge, then be at liberty to strive your hand on the new Opus 4.7 and what it’s able to.

2. Sure, Opus 4.7 performs plenty of iterations unnecessarily: As many customers have identified, and from what I may determine from my very own use, Opus 4.7 performs far more iterations in its pondering course of than mandatory, particularly so when you evaluate it to Opus 4.6.

After which when the output shouldn’t be at par with that of different fashions, you have a tendency to think about all that compute as a whole waste of time, efforts, and most significantly, tokens.

3. No, Opus 4.7 shouldn’t be inaccurate: At the least in our use with it, the Opus 4.7 didn’t falter even as soon as, and managed to stay to the directions fairly fantastically, churning out tremendous correct outputs with all types of prompts. So full marks to the mannequin on that entrance.

Conclusion

Backside line – positively give Opus 4.7 a strive. However to shift your complete workflow to it, particularly when it entails in depth steps and gear calling could be a waste of your tokens I imagine. As there isn’t any apparent distinction within the high quality of outputs it comes up with, vis-a-vis what Opus 4.6 was able to.