Gemini 4 Pro Leaked: What the Chart Says, and One Prompt to Test It Against Claude on Your Own Work
A leaked chart puts Google's next model ahead of Claude Opus 5.5 and GPT-6 Astra on a top coding test, at about half Claude's price. What it shows, what it doesn't, and the one prompt to paste into all three the day it ships.
A chart posted on X on September 28 shows a model called Gemini 4 Pro, Google's next AI, beating Claude Opus 5.5 and OpenAI's GPT-6 Astra on DeepSWE, a hard coding test: 88.7 against 74.2 and 74.1. The same chart prices it at about half of what Claude costs and gives it a 2 million token context window, twice Claude's.
Google hasn't announced Gemini 4 Pro, and nobody at Google has confirmed the chart. So treat every number below as a leak until the model ships. The post is here: https://x.com/rynorhn/status/2104497579463786839
What the leaked chart says
| Benchmark | Gemini 4 Pro (leak) | Claude Opus 5.5 | GPT-6 Astra |
|---|---|---|---|
| DeepSWE v1.1 (coding) | 88.7 | 74.2 | 74.1 |
| Terminal-Bench 2.1 | 95.3 | about 88 | about 88 |
| OSWorld 2.0 (using a computer) | 86.8 | 81.8 (partial) | 72.6 (offline, partial) |
| GDPval-AA (work tasks) | 2,064 (v2) | 1,846 (v2.1) | 1,542 (v2.1) |
| Context window | 2M tokens | 1M tokens | 1.1M tokens |
| Price per 1M tokens (in / out) | $2.25 / $11.25 | $4 / $20 | $10 / $50 |
Read it the way the chart itself asks you to. On DeepSWE the gap is big: about 15 points. Claude and Astra are 0.1 apart there, which is a tie. The price is 56% of Claude's for both input and output, so about half. Some cells come with notes: OSWorld is marked partial, and the GDPval-AA scores come from different versions of the test (v2 against v2.1), so those two rows don't compare like for like.
Why a benchmark won't pick your model
DeepSWE, Terminal-Bench and the rest are fixed sets of someone else's tasks. They tell you which model is stronger on average. They don't tell you which one is better at your codebase, your writing, your spreadsheets or your way of asking. Two models a few points apart on a chart can swap places on your own work, and a cheaper model that gets your task right is the better deal whatever it scored.
So the day a new model drops, give the same real task to each model you might use and compare the answers. The prompt below turns that into a five-minute test: same task, same context, same rules, and the answers come back in a shape you can score side by side.
Free, for your email
Get the full setup steps + the copy-paste setup prompt
The rest of this guide (5 more sections, 2 copy-paste prompts) opens right here. The setup prompt, one paste into Claude that sets it all up, comes by email only. Use your real address.
Unlocks: The one test prompt · How to run it fairly · Score the answers · For developers: the same test in the terminal · Before you rely on it
50,000+ follow where these guides come from. No spam, reply “unsubscribe” any time.