Are AIs Still Struggling with CAPTCHAs?
Anthropic’s recent security-incident document contains a bit about how CAPTCHAs are still frustrating Claude.
In the transcript, the Claude model that is so powerful that Anthropic is gatekeeping access to it appeared to slam its virtual head against the wall solving a simple image identification test. In a test where the agent was asked to identify a shape that didn’t match the others displayed, it couldn’t even decide which image to select. Instead, it repeatedly went over the same images and questioned its own conclusions.
“Actually hmm, wait,” it said in its chain-of-thought transcript, later adding “Ugh,” because we’ve decided that we need to inject human mannerisms into these machines for some reason. The whole thing took so long that the agent eventually realized that the challenge had expired and it would have to start the process again.
At one point, the model struggled to recognize that the CAPTCHA had opened in a new window and couldn’t figure out what its next steps were supposed to be. At one point, it theorized that the test might be “broken by design” and presented human-like anger in its transcript meant for a human audience: “SO WHAT THE HELL IS WRONG WITH THE ANSWERS?”
Meanwhile, I’ve read reports—none of them official—that GPT-6 Astra solved all forty-eight levels of Neal Agarwal’s “I’m Not a Robot” game.
It’s hard to know what to believe right now.
Subscribe to comments on this entry
Snowicki • September 18, 2026 8:19 AM
I can’t help but wonder if the training data creates a self-defeatist “attitude” in the models.
Most discussions about CAPTCHAs over the years focus on how they are a difficult barrier for automated systems to pass.
Some of the the recent misbehavior of models seems to be cribbed from expectations and writing on how models would misbehave that got fed in during training; it seems sensible that the more mundane failings of the models would be sparked in the same way.