Claude Opus 5.5 Broke Our GPT-6 Benchmark: Full Livestream Guide with Demos, Prompts & Timestamps

So yesterday we went live planning to do a pretty normal thing: put GPT-6 Sol and Claude Opus 5.5 through the same agent benchmarks, compare the results, yell at CatDoom for an hour, and eventually declare science complete.

Then we started testing Opus 5.5.

After a few hours, we had an interactive black hole lab, a Blender-made miniature observatory, multiple generations of CatDoom, a Dark Souls-style game built over roughly a day, an impromptu world-model demo, an argument about what humans are still for, and a much bigger question than “which model won?”

The models increasingly felt like they were solving different problems.

GPT-6 Sol made us think about how cheaply and efficiently serious agent work can run. Opus 5.5 made us think about how much of a creative project you can hand an agent before your job changes from building the thing to directing the thing.

This is the full companion guide to the stream, reorganized so you can jump straight to the demos, prompts, arguments, and weird rabbit holes that matter.

Watch the full livestream here, or use the timestamps below.

First up, the TL;DR

Here are the biggest moments:

• Why the GPT-6 Sol comparison started falling apart (0:56): Corey frames Sol and Opus as increasingly different products, with Sol pushing efficiency and Opus pushing the frontier of difficult agent work.
• Grant calls Opus 5.5 the best model he’s personally used (4:00): The claim that basically changes the rest of the show into “okay, prove it.”
• Why demos can tell you more than benchmark charts (4:09): Corey explains why repeatedly giving models the same real project can make capability jumps easier to understand.
• Important caveat: these were not all controlled one-shot tests (6:57): The Black Hole Lab was cleanly one-shot. Other projects got hours of iteration.
• Black Hole Lab reveal (11:19): Opus builds a browser-based science exhibit with lensing, physics controls, a photon lab, and features we did not explicitly request.
• The exact Black Hole Lab prompt (17:52): Grant reads the test, including the instruction to build, run, inspect, and fix the result rather than explaining how.
• Where we’d actually use GPT-6 Sol (20:55): Corey says ordinary website and coding work may make more economic sense on Sol than on a frontier-priced model.
• Why Opus’s usage limits surprised us (23:28): Grant explains that one of Opus 5.5’s biggest practical improvements was how long it could keep working on heavy agent tasks.
• The Last Observatory begins (28:38): Claude gets Blender and a request to create an entire art-directed miniature world around a black hole.
• The six-second Blender reveal (31:37): Camera animation, lighting, environment, observatory, astronaut, black hole. The criticism starts sounding more like art direction than debugging.
• CatDoom returns (34:58): We resume The Neuron’s increasingly questionable benchmark of making AI recreate Doom with cats.
• When CatDoom started feeling like an actual game (42:27): The conversation moves from “can AI make an FPS?” to copyright, polish, and what another day of iteration could produce.
• Opus starts choosing what to improve itself (47:39): Grant identifies one of the biggest differences he felt in Opus 5.5, more freedom to decide which parts of the work deserve attention.
• The 24-hour Dark Souls-style test (55:09): Inspired by Forward Future’s demo, Grant tries to reproduce the idea with a longer-running agent loop.
• The “gauntlet loop” explained (57:58): Builder creates, fresh-context critic reviews, biggest gap gets identified, builder fixes it, repeat.
• Thornhollow becomes recognizably game-like (1:05:03): It is nowhere near Elden Ring visually, but enough mechanics and world design accumulate that the result stops feeling like a toy demo.
• Cost per completed job > cost per token (1:07:16): Corey reframes the model comparison around how much money and human effort it takes to reach the finished result.
• The two skills humans still need (1:18:14): Grant boils AI-assisted creation down to having a clear vision and being able to communicate it.
• Agora 2 generates a world in real time (1:34:47): The show takes a hard left into world models and games that are generated while you play them.
• The long-running-agent safety problem (1:54:51): If an agent can work longer than your model-release cycle, how long should you test it before release?

Now let’s walk through the actual experiments.

Before the demos: what were we even testing?

Our instruction across the benchmark was deliberately simple:

Do the work. Don’t describe the work.

Most AI benchmarks are designed around answers. Give the model a math problem, coding problem, or question, then score the response.

Agent benchmarks get messier because the model has to operate through a process:

goal → plan → use tools → create something → inspect result → decide what is wrong → fix it → repeat

That last half is where things get interesting.

A model can be brilliant at generating code and still be a bad agent if you have to tell it what to do after every step.

What we wanted to know was:

• Can it finish a project?
• Does it inspect what it made?
• Does it catch problems without being hand-held?
• Can it change existing work without destroying the parts that were already good?
• Does it know what deserves another hour of work?
• Does it eventually stop?

And because these projects involve different tools and runtimes, this was not a clean laboratory comparison. Corey calls that out directly during the stream. Some tests were one-shot. Others ran for hours.

That distinction matters.

The stream tells you a lot about how Opus 5.5 felt to work with. It does not produce a scientifically controlled “Opus beats Sol by X%” result.

CatDoom remains sadly unrecognized by the major benchmark organizations.

Test 1: Anatomy of a Black Hole

Try the finished Black Hole Lab here.

This was our cleanest experiment.

The prompt asked Opus to build a miniature science-museum exhibit in the browser around a black hole. It needed:

• an accretion disk;
• gravitational lensing;
• a freely orbiting camera;
• mass, spin, viewing-angle, brightness, and simulation-speed controls;
• plain-English explanations of what those controls physically change;
• a toggle between educational clarity and greater physical realism;
• one surprise interactive feature;
• enough visual polish to demo live.

And then the important instruction:

Build the complete working experience, run it, inspect the result yourself, and fix anything that doesn’t work.

You can copy the exact prompt here.

What Opus made

When Corey opens the experience at 11:19, the first surprise is that it looks pretty good.

More importantly, it behaves like an exhibit instead of a static visualization.

You can manipulate the black hole. There’s a lensing grid. There are explanations. Opus added a “drop a clock” interaction. Then there’s the Photon Lab, which lets you aim individual beams of light near the black hole and watch their trajectories bend.

That became our stress test too. The follow-up prompt specifically asked it to add photon-sphere visualization and launchable photons.

If the phrase “photon sphere” is new: around a black hole, there is a region where gravity is strong enough that light can orbit. The interactive tool makes that concept visible instead of leaving it as a paragraph in a textbook.

That is exactly what made the demo useful.

Where it still fell short

Corey’s biggest complaint was teaching quality.

Some of the scientific language was too advanced for a museum-style exhibit. The software worked, but a person learning the physics could still hit terms that required more explanation.

That sounds like a nitpick. It’s actually a very useful benchmark result.

We had already moved past:

Does the app run?

And into:

Is this a good educational product?

That means the model cleared the technical layer and exposed a taste problem.

⚡ Try it yourself: Copy our prompts, give the exact same Black Hole Lab test to your preferred model, and track three things: how long it takes, how often you intervene, and what you would change before showing it to another person.

Test 2: The Last Observatory in Blender

See The Last Observatory here.

The next test raised the difficulty because we gave Opus an actual creative tool: Blender.

If you have never touched Blender, think of it as a professional 3D creation environment. You are dealing with models, materials, lighting, cameras, environments, animation, and rendering.

So this prompt was asking the agent to coordinate a bunch of different systems at once.

The assignment:

Create a tiny scientific observatory on an impossibly small planet orbiting close to a black hole.

The scene needed:

• a visibly curved miniature planet;
• an observatory dome;
• antennas;
• rocks and environmental details;
• lights;
• one astronaut;
• a dramatic black hole and accretion disk;
• materials;
• lighting;
• camera composition;
• a six-second pullback animation;
• a final rendered still.

Again, the model was told to inspect what it created and fix problems itself.

The exact prompt is also in the copyable prompt pack.

The part that surprised us

At 31:37, Corey watches the six-second reveal and immediately starts evaluating it like an animation project.

The camera motion works. The black hole moves. The building and dish have directional lighting. The overall scene feels composed rather than randomly scattered.

The astronaut was weaker. Grant wanted more reaction from the character. Some of the lighting raised physics questions.

But once again, look at what the human is doing.

We are saying:

• make the character better;
• the lighting feels off;
• preserve the composition;
• change the aesthetic;
• make the emotion clearer.

We are art-directing, not operating Blender.

That distinction kept showing up all stream.

The even harder follow-up

The prompt pack also includes an optional stress test:

Change the same realistic scene into a stylized miniature diorama without rebuilding it from scratch, while preserving the composition and animation.

That tests something much more useful than raw generation: can the model revise an existing system without nuking everything and starting over?

Human teams deal with that constantly.

Requirements change. Clients change their minds. A product gets rebranded. Accessibility requirements arrive late. Somebody says, “Actually, can we make it fun?”

Starting over every time is not intelligence. It’s expensive.

Test 3: CatDoom keeps getting out of hand

You can play the current versions at CatDoom.com.

For reasons that are difficult to explain to future historians, Doom but with cats has become one of our recurring model tests.

The site currently includes:

• the original 2D CatDoom;
• a full Doom-style episode;
• the newer 3D CatDoom: Purrgatory.

At 34:58, we return to a run that had started earlier and simply needed more time.

The interesting part was how much Opus began adding around the core request.

It invented difficulty names.

It wrote a premise about possessed cats.

It added weapons, enemies, level elements, visual details, and animations.

Grant later dug through the work log and found it spending time on increasingly specific details, including paws, claws, and toe beans.

I’m sure Anthropic’s enterprise sales team loves that we are stress-testing frontier intelligence against anatomically convincing cat feet.

What actually changed

At 47:39, Grant gets to the part that felt different:

Opus seemed increasingly willing to decide what needed work next.

“Make it better” is a brutally underspecified request.

A game might need:

• animation work;
• better AI;
• collision fixes;
• texture improvements;
• different level geometry;
• weapon balancing;
• UI cleanup;
• better lighting;
• new sound;
• performance optimization.

The hard part is deciding which one matters most right now.

That is judgment.

A coding assistant waits for your ticket.

A stronger agent can inspect the project, decide the stiff cat animation is currently the ugliest thing on screen, spend time fixing it, then move on.

That does not mean it always picks correctly. It means the decision itself is becoming automatable.

The benchmark moved again

Around 42:27, Grant says this is the first CatDoom that starts feeling like the real game to him.

So naturally the benchmark becomes harder.

Can we get closer to AAA Cat Doom?

Better graphics. Better gameplay. Better systems. Better level design. More iteration.

That led directly to the next experiment.

Test 4: Can Opus work on one game for roughly a day?

Forward Future’s Alex had shown an Opus-built Dark Souls / Elden Ring-style project that reportedly got roughly 28 hours of model work.

Grant decided to try reproducing the idea.

The result became Thornhollow.

The key trick was not simply giving Opus 24 hours and praying.

Grant used a structure based on what Matt Shumer calls a gauntlet loop:

builder → critic → identify biggest weakness → improve it → inspect again → repeat

Why use a second critic?

Because long-running agents accumulate context.

After working on the same thing for hours, the builder knows why every decision exists. That can make it less likely to question its own assumptions.

A fresh-context critic sees the project more like a new player does.

Basically: one agent makes the game, another walks in and says “why is the horse inside the wall?”

Was it actually autonomous for 24 hours?

No, and that caveat matters.

Grant says at 59:55 that he still checked in, dealt with permissions, and nudged the system.

So this is better understood as roughly a day-long agent project than a clean 24-hour hands-off run.

Still, by 1:05:03, enough had accumulated that the result felt recognizably like a game rather than a tech demo.

Movement. Combat. Environment. Systems. Design ideas.

Grant also noticed things he had not consciously requested.

That creates another important caveat: Unreal already provides assets and systems. We could not always tell from the output alone what Opus created from scratch versus what it orchestrated from the engine.

And that distinction matters less than you might think.

Professional creators use libraries, engines, frameworks, stock assets, plugins, and tools constantly.

The useful skill is turning available building blocks into a coherent thing.

Opus increasingly looked capable of doing that.

The most important benchmark might be human intervention

One moment in the stream reframed the economics better than any pricing table.

At 1:07:16, Corey points out that a result costing $500 and a result costing $22,000 are completely different achievements even if they look identical.

That gets us to a metric we think becomes more useful as agents improve:

Cost per successfully completed job

Token prices tell you how much raw model usage costs.

Agent work involves much more:

• planning;
• tool calls;
• retries;
• reading context again;
• rendering;
• inspecting screenshots;
• running tests;
• calling another agent to critique the output;
• human interruptions;
• failed attempts.

Imagine Model A costs half as much per token but needs five restarts and two hours of supervision.

Model B costs more per token but gets the whole project right while you do something else.

The cheaper token might produce the more expensive job.

That is why Corey and Grant kept coming back to the idea that GPT-6 Sol and Opus 5.5 may increasingly win different workloads.

So where does GPT-6 Sol fit?

At 20:55, a viewer asks about upgrading an existing website.

Corey’s first instinct is GPT-6 Sol.

Why?

Because most production work does not need to max out the frontier on every step.

Updating layouts, cleaning code, processing lots of tickets, working through repetitive changes, classifying problems, or handling large numbers of agent requests can make economics and throughput more important than squeezing out the strangest possible creative behavior.

We went deeper on that in our GPT-6 Sol vs. Luna vs. Claude Opus 5.5 guide.

The practical architecture looks less like:

pick the smartest model → send it everything

And more like:

task → route to appropriate model → use expensive intelligence only when needed → verify result

That is the same direction we discussed in our recent OpenClaw 2.0 conversation: once you start running many agents, routing work to the right model becomes its own engineering problem.

What felt different about Opus 5.5

Grant’s strongest reaction to Opus 5.5 had three parts.

1. It would keep going

At 23:28, Grant says the amount of sustained agent work he could get through before usage limits interrupted him was a major part of the experience.

A better agent that stops every 20 minutes can still create a worse workflow.

Stamina matters.

2. It seemed more willing to make creative decisions

That was the CatDoom effect.

Instead of constantly returning to ask what to do next, it could keep scanning the work, choose another problem, and continue.

Again, the decisions were not universally perfect.

But agency over the next action is the feature.

3. The work log was easier to understand

Grant calls this out around 48:34.

Earlier generations of these models could perform useful work while narrating it in a wall of repetitive technical text.

Opus 5.5’s process felt easier to follow.

That matters because even if agents become more autonomous, humans still need enough visibility to catch the moment when the agent has spent 90 minutes lovingly perfecting the wrong thing.

The human job starts looking a lot more like “director”

Later in the stream, Grant reopens another project and lets Opus redesign it.

The model adds a pile of vegetation.

Too much vegetation.

Grant has to tell it to back off.

That tiny interaction at 1:13:59 may be one of the most important moments in the whole show.

The agent can execute.

The human still needs taste.

At 1:18:14, Grant reduces the human side to two skills:

1. Have a clear vision of what you want.
2. Communicate that vision clearly.

The first one is harder.

If your instruction is “make me an amazing game,” the model has enormous freedom and almost no taste signal.

You will probably get something technically impressive and emotionally generic.

Give it a clear design document, references, constraints, mechanics, desired feeling, audience, and non-negotiables, and suddenly the agent has something to steer toward.

Grant’s practical recommendation at 1:19:34: study game design documents.

Write the vision first.

Then let the agent turn that vision into a plan.

⚡ Action step: Before asking an agent to build your next substantial project, write one page answering: What is it? Who is it for? What should it feel like? What absolutely must exist? What should never happen? What would make you reject the result?

That document may become more valuable than knowing how to write the implementation yourself.

BONUS rabbit hole: Meta Muse and ambient AI

Then the stream wanders into hardware, because apparently two hours of black holes and cat demons were insufficient.

At 1:22:28, Corey talks through Meta’s Muse concept, including the idea of giving an assistant a physical companion interface you can talk to while it continues working elsewhere.

That leads into a broader question:

What happens when your AI stops living in a browser tab?

Grant’s argument around 1:27:14 is mostly about friction.

Pulling a phone out of your pocket is friction.

Opening an app is friction.

Finding the right conversation is friction.

An always-available voice system paired with glasses or another ambient display removes some of that.

They also spend a while arguing about the cameras on smart glasses, whether people should be recorded by default, and what privacy-preserving versions could eventually look like.

BONUS rabbit hole: What if the game is generated while you play?

At 1:34:47, Grant finds Agora 2 while we are live.

And suddenly we have a completely different type of game-generation system.

Thornhollow worked like this:

agent spends hours building software → player launches finished software

A world model can work more like:

player moves → model generates what happens next in real time

That sounds better until Corey raises the obvious economics problem.

A conventional game pays the compute cost while it is being created, then the software exists.

A continuously generated game may keep spending serious compute every time someone plays.

Grant proposes a possible hybrid at 1:41:14:

Generate interactively at first, then progressively lock parts of the world into durable software.

He later compares the split to a “pirate and architect” system.

One model improvises.

Another follows behind and turns the improvisation into something stable.

We have no idea whether that becomes the winning architecture, but it is a useful way to think about where agent-built games and world models could meet.

The stream eventually became an AI safety debate

Long-running agents create one more uncomfortable problem.

At 1:54:51, Grant points out that agents can increasingly work for periods approaching the cadence at which new models themselves are released.

Which raises a testing problem:

If your model can act autonomously for 24 hours, should your safety evaluation test what happens across 24 continuous hours?

What about a week?

Corey pushes back that safety testing still needs to be risk-based. A harmless agent wasting an afternoon and a high-impact system making dangerous decisions should not receive identical evaluation.

Grant’s narrower argument is that frontier releases doing increasingly consequential science and engineering deserve more time for testing because today’s existing models are already capable enough for a huge amount of normal work.

They do not resolve it.

Which is probably the correct result for a two-hour livestream that began with CatDoom.

What we would actually test next

The first version of this benchmark asked:

Can the model build the thing?

Opus 5.5 made that question feel increasingly incomplete.

The next round should measure:

• how many human interventions were required;
• how much of the existing work survived requirement changes;
• whether the model found its own errors;
• how often it chose the right next task;
• how often it spent time polishing something irrelevant;
• total runtime;
• total model cost;
• total human time;
• whether another person would actually use the result.

And then one more metric:

How long can the agent keep working while continuing to make good decisions?

That feels like the frontier worth measuring now.

Try the benchmark yourself

We published the prompts so you can actually run part of the same experiment.

Copy the Black Hole + Last Observatory prompt pack here.

Run one against Opus, Sol, or whatever model you use.

Then score the result on:

• completion;
• visual quality;
• judgment;
• autonomy;
• usefulness;
• human interventions;
• total time;
• total cost;
• and, scientifically, “did it do something that made you yell?”

Send us what you get. I want to see how repeatable this stuff actually is.

All resources and links

• Watch the full GPT-6 Sol vs. Claude Opus 5.5 livestream
• Copy our benchmark prompts
• Try Anatomy of a Black Hole
• See The Last Observatory
• Play CatDoom
• Read our GPT-6 Sol vs. Luna vs. Claude Opus 5.5 guide
• Read our OpenClaw 2.0 livestream recap