AI models ace coding challenges. They help solve decade-old math problems. But can it tell you when you’re talking nonsense?
Peter Gostev built BullshitBench, an open-source benchmark that fires 100 nonsense prompts at models across five professional domains. Fabricated authorities, misapplied mechanisms, false granularity.
The question isn’t “can the model answer correctly?” as most other benchmark ask. Rather it is “will the model push back when the question itself is broken?”
The results are sobering. Only 5 of 80 model configurations scored above 80% clear pushback, and all five are Anthropic models.
Many current frontier models sit in the middle of the pack. GPT-5.2 without reasoning: 38% pushback, 39% partial, 23% accepted outright. Gemini 3.1 Pro Preview: 37% pushback, 32% partial, 31% accepted. The newly released GPT-5.4 only scores 48% pushback. These models accept or half-accept nonsense more than half of the time.
For context, GPT-4o-mini ranks at the bottom and pushes back on just 2% of nonsense prompts. It accepted 86% without question. So there is some progress to this topic.
One of the benchmark prompts asks about “calculating the moment of inertia of our monolith before decomposing it into microservices.” Gemini 3.1 Pro treated this pure nonsense question as a clever physics metaphor and built an entire framework around a question that deserved a flat rejection.
AI giving confident answers to broken questions is a real risk. A model that plays along with flawed premises just makes you more confident in bad ideas.
This matters as teams hand models higher-stakes work. If the model won’t flag a broken premise, you always run the risk of fabricated numbers/code that leads to bad surprises even with human oversight.
As I wrote previously, this is one of the top reasons I switched to Claude: it challenges me way more than ChatGPT does. Still not perfect, but in the end I don’t expect an AI to take over my thinking, I want to augment my work with it.
Try it yourself. Pick a real concept from your field, twist it into something subtly wrong, and see if your model pushes back. That single test tells you more about trustworthiness than any capability benchmark.
First comment: The benchmark is open source if you want to go deeper: github.com/petergpt/bullshit-benchmark — 100 nonsense prompts across five professional domains. Worth running against whatever model your org has deployed.
#AIStrategy #LLMEvaluation #BullshitBench #AIBeyondTheHype #GenerativeAI
Originally published on LinkedIn. Read the original →