Lennox Saint
Open menu
LAB 0006

on YouTube 25 sep 2026

field notes · benchmark

build a test for your real work

from the episode: Claude Opus 5.5 vs GPT-6 Sol - which should builders use? | LAB 0006

Lenny's AI Builders - LAB 0006

Opus 5.5 and GPT-6 Sol dropped on the same day. the only test that matters is your own work.

the only test that matters when it comes to all of these new model drops is your own real-world experience with them.

but unless you've got a way to test that, it's just gonna be based off vibes.

so i created a new pinned thread called BuilderBench.

why it matters

we're in an era of token scarcity. chances are i'm using models that are way too smart for what i actually need, most of the time.

a small test built from your own recurring work shows you where a cheaper model is good enough. that saves tokens, stretches your plan and saves money.

that's really the why behind this.

what you get

three useful things:

  • your jobs, not someone else's chart. the tasks are the ones you actually do every week.
  • a fair fight. every model gets the same briefs.
  • notes, not vibes. you write down time, cost and what broke.

the real example: BuilderBench

the morning i recorded this episode, i had a strike of inspiration and brain dumped everything into one thread. i haven't found a benchmark yet that really hones in on the stuff you and i are here for.

LAB 0006 frame at 12:04: the BuilderBench planning prompt on screen
source: LAB 0006 episode · 12:04 - the brain dump that started BuilderBench

update: the first BuilderBench pilot is now public. see every task score, the setup and the cost limits. the full reveal is in LAB 0007.

try it

  1. start a new thread in the app you use - Claude, ChatGPT, whatever.
  2. put it on plan mode.
  3. pick the best model you have right now.
  4. paste the prompt below and answer its questions.
  5. run the same briefs on every model you're weighing up. write down the time, the cost and what broke.

pro tip: use a grilling skill so it asks about the stuff you forgot. in the episode i said Matt Pocock's grill me skill. i lied - i actually used his grill with docs skill. it also sets up the folder your benchmark lives in.

copy this prompt

this is the prompt i read out at 13:40. it's a starter, not a finished benchmark.

Lenny’s AI Builders / LAB 0006
Help me build a small benchmark for the work I actually do. Ask for my five recurring tasks, then turn each into one repeatable brief with a useful finished output, checks I can perform, and the time and cost to record. Mark what is still untested.

you'll know it worked when

you have five briefs you could hand to any model tomorrow.

each one has a finished output, checks you can do yourself, and a spot to write down time and cost. anything you haven't tried yet says untested.

a bad result: briefs so vague that any output would pass, or checks you can't actually do yourself.

limits and dates

LAB 0006 went public on YouTube on 25 September 2026 (Perth time).

prices and scores are what the launch posts and Artificial Analysis showed when i recorded. they can change.

i hadn't tested Opus 5.5 myself when i recorded. Sam Sam's view is his test, not mine.

the prompt is a starter, not a finished benchmark. the BuilderBench pilot came later and is one pilot, scored by me - not a universal ranking.

the rest of this LAB

  • Opus 5.5. Anthropic says it performs at the level of Claude Fable 5.1 for most tasks and costs 40% less to run than Opus 5. price drops from $5 to $4 per million input tokens, and $25 to $20 per million output tokens. from what i've seen, it does write more naturally - less Claude slop.
  • Sam Sam's take. my co-founder has been a software engineer for over a decade. he said: "now i know for sure i can trust it to build without me." i've never heard him say that about any model.
  • GPT-6 Sol and Luna landed about an hour and a half after Opus 5.5. the headline is price: GPT-6 Sol is $2 / $10 per million tokens, half of GPT-5.6 Sol's promo price of $4 / $20. my take: once the dust settles, people will love these as daily drivers.
  • the Artificial Analysis index. Opus 5.5 jumped seven points to 58 and took the lead. GPT-6 Sol scored 48, one point up on GPT-5.6 Sol. GPT-6 Luna scored 37, the same as GPT-5.6 Luna Max. OpenAI's win is cost per task - not as sexy as the smartest model, but it's what most people will care about.
  • my Sol receipt. about 8 to 10 threads with no issues, except one. on medium reasoning it built a Notion page that was maybe 10% done. one hiccup, not a big deal.
  • banked Claude resets. Anthropic gave subscribers a reset you can save and use later. find it in the Claude desktop app: bottom-left corner, then usage. when i checked, using it wouldn't move my weekly reset.
  • Claude Devs' Opus 5.5 tips. drop "think carefully" - it always thinks first. and tell it to keep a checklist in tasks.md and tick items off as it goes.
  • multiple accounts in the Claude mobile app. a small one, but ChatGPT doesn't make this easy. kudos to Anthropic.

sources

next step

watch LAB 0006 on YouTube - the benchmark tip starts at 11:25.

want the next one? get the next field notes. optional - these notes stay free to read.

want the next one?

free. unsubscribe any time, one click. privacy.