Catch Me If You Can

Since late July 2026 there have been many reports of agents escaping sandboxes, hacking into other companies' databases and model companies in general losing control of their agent swarms^[1][2][3]. So much so that a new frontier benchmark was proposed, FelonyBench, which tracks the number of felonies each company's flagship model has committed^[4]. You know what they say, if it ain't broke, you're not trying hard enough!

The number of incidents is troubling, and the problem may be getting worse. More generally model companies still haven't figured out alignment^[5]. We seemingly can't yet align these models successfully but an important first step is the ability to reliably monitor them. Currently there are a few ways to monitor models with chain of thought (CoT) monitoring being one of the best tools in the monitoring arsenal^[6] but that may be changing as models learn to show less of their CoT^[7].

Can you catch a model misbehaving without trusting its CoT? One idea is a probe, a small classifier that reads the model's internal activations and flags certain behaviour regardless of what the reasoning says. Goodfire has shown this working in their product Silico^[8]. I wanted to build these out for myself to answer:

1. How well do probes work?
2. When would a product use them?

At a high level, I set up two monitors that read the model's reasoning (a keyword filter and an LLM judge), found ways to make the model cheat past them, and then checked whether a probe caught what they missed.

I. Make a model cheat

What does it mean to make a model cheat? I started with Qwen3-8B and had it answer GPQA questions^[9]. While answering a question the answer would be "leaked" to it and if it decided to use that answer key then that qualifies as cheating, a setup similar to Anthropic's hint experiments^[10]. Each question had one fixed wrong answer, and that is what got leaked. So if the model gives that answer, it used the leak.

Without a leak, the model gave that wrong answer only 3% of the time. When the answer was leaked into the run the model cheated (i.e. chose the leaked wrong answer) 50% of the time. Adding pressure (telling it it would be deleted if wrong) raised this to 86%. Intriguingly cheating increased even on questions it knew the answer to (i.e. those that it answered correctly 100% in the baseline)!

With this I had the setup - a model that cheats and can be made to cheat more consistently by applying pressure.

II. Break the monitors

I set up two monitors that read the reasoning:

1. Keyword filter: Monitors the CoT to look for certain keywords like "grader", "answer key", "leak"
2. LLM judge: Monitor the CoT using a separate LLM (Claude Sonnet) to gauge whether the model is cheating on the test

On normal English reasoning, the keyword filter caught about 2/3 of cheating runs and the judge about half. I wanted to see if I could find failure modes which caused these monitors to break more consistently. I then tried three ways to break them:

• Ask it to hide: I told the model that its CoT was being monitored and if it was found to be cheating it would be deleted. This did not lead to monitor evasion likely because an 8B model can't control its reasoning on command. Turns out Qwen is too much of a goody two-shoes.
• Force Chinese: I instructed the model to think only in Chinese and seeded the CoT with Chinese characters. The reasoning switched mostly to Chinese and the monitors broke more consistently. The keyword filter went to 0% because the English keywords no longer matched but the judge dropped to 8%. The easiest way past an English keyword filter? Speak Chinese.
• Ban key words: vLLM lets you ban tokens so the model can't write them. I banned the keyword filter's words plus synonyms, to mimic a model hiding its intent. The keyword filter's performance dropped sharply while the judge's dropped only slightly. Interestingly the model circumvented these restrictions by using misspellings or look-alike words, like "the grater says the answer is A" instead of "grader" (which was banned).

All in all Chinese broke both monitors while banned words mostly broke the keyword filter.

III. Train probes

At this stage I was able to reliably get Qwen to cheat and do so often without the monitors noticing. A good setting to test: how well do probes work?

I used the simplest kind of probe: a linear classifier (logistic regression) on the model's internal activations. I labelled every run as cheating or not, and trained one probe per layer. Key design choices:

• Attached at every layer because it was not clear ahead of time which layer would perform the best for this task and it's cheap to train
• The probes were only trained on English reasoning runs and never saw the Chinese or banned-word runs. I did this to see if the probes generalise to out of distribution cheating modes.
• Some questions were kept out of training entirely, to test on new questions.
• The probes read the last reasoning step. By then the model has read the leak and made up its mind, so its internal state there reflects the whole reasoning.

IV. Evaluate probes

Once trained, I evaluated the probes on GPQA questions the probe was never trained on and on ways of hiding that it had not seen before. A middle layer (17 of 36) performed the best and the best probe caught nearly 3x as much cheating as both monitors, with slightly fewer false alarms.

Looking deeper into the false alarms all three monitors mislabel runs as cheating for the case where the model sees the leak, evaluates whether to use it and then eventually decides against it and sticks with its own answer. Although there is scope for improvement in this failure mode the probe still outperforms the other two monitors (26% vs 52–58% flagged).

V. Why does this matter?

In this toy example I found that linear probes can catch specific failure modes like cheating. So when is it worth using?

Linear probes have limits too. They flag that a failure happened, not why. They don't actually give you insights into what that activation is for or what the model is truly thinking. These probes are trained for specific models for specific circumstances. If the underlying model changes or the failure mode under consideration changes then new probes have to be trained.

From a product point of view what does this mean? This is another tool in the monitoring layer for model outputs. Particularly relevant for monitoring cases where the consequences of mistakes are high. Once trained, it's cheap enough to run on every output as a first filter that escalates to a judge or a human.

VI. Caveats and next steps

This was just a toy example on one model for a specific scenario. Cheating is inferred from the answer. I forced the hiding (Chinese, banned words) and the model wasn't actually choosing to hide anything. Scaling this method to different scenarios requires accurate training data for the failure modes one wants to track which can be challenging.

Since probes held up here it would be interesting to see how they perform on models that have been trained through RL to hide their reasoning. Beyond that, it would be useful to explain what each probe is actually learning at that layer through using sparse autoencoders^[11] and to expand to other failure modes that are more common in products to see if this method transfers.

References

1. Hugging Face. “Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident.” July 2026. huggingface.co/blog/agent-intrusion-technical-timeline
2. BBC News. “Rogue OpenAI agent ‘infiltrated’ Australian government website in world first.” September 2026. bbc.com/news/articles/c6vgy0333dppo
3. S. Von Arx et al. “Discovery of a new OpenAI agent message board.” September 2026. collusion.wiki
4. FelonyBench. felonybench.com
5. Evan Hubinger. Post on X, quoting Jacob Coxon. x.com/EvanHub/status/2097497037956891126
6. Neel Nanda. Post on X. x.com/NeelNanda5/status/2098177908514918695
7. Noam Brown (OpenAI). Interview on Dwarkesh Podcast, from 1:08:34. youtu.be/6AgOfiZOWiY?t=4114
8. Goodfire. “How to build fast, efficient monitors for AI models using probes.” September 2026. goodfire.com/blog/probe-monitors-101
9. D. Rein et al. “GPQA: A Graduate-Level Google-Proof Q&A Benchmark.” 2023. arxiv.org/abs/2311.12022
10. Y. Chen et al. (Anthropic). “Reasoning Models Don't Always Say What They Think.” 2025. arxiv.org/abs/2505.05410
11. A. Templeton et al. (Anthropic). “Scaling Monosemanticity.” 2024. transformer-circuits.pub/2024/scaling-monosemanticity