PIR probe aims to tell hidden model knowledge from missing knowledge

A single-author research paper proposes a way to check whether a language model recognizes an answer it does not give. The method, called Probe of Internal Recognition (PIR), is described by author Hiskias Dingeto in an arXiv preprint submitted on 18 September 2026 and revised on 23 September. The paper's title is "A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal."

The problem it targets is that outputs alone cannot show whether a model is hiding an answer or simply lacks one. PIR borrows the Concealed Information Test from forensic work: it presents a question with candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. The paper says the method needs no honest reference model and no labeled truth corpus.

The author reports testing eight models from five families: Gemma, Qwen, Llama, Mistral and Phi. According to the abstract, PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, compared with a 0.25 chance rate and an unknown-item baseline of 0.28 to 0.40. Under prompted deception, trained sandbagging, password-locked checkpoints and circuit-broken checkpoints, the paper reports recognition between 0.85 and 0.93.

The central claim concerns unlearning. The paper says recognition drops to the level of a question the model never knew when unlearning removes the knowledge, which the author presents as a way to separate a model that will not answer from one that cannot. The Hugging Face summary describes the same distinction between concealed and absent knowledge. The author also says the signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation.

Everything above comes from the author's own description of the work. Both saved sources, the Hugging Face paper page and the arXiv abstract, relate to the same paper, so this is one source of evidence, not two independent ones. A web search found write-ups of the paper, but they appear to restate the abstract, and we did not treat them as independent confirmation. We have not read the full paper, so per-model results, false-positive behavior and comparisons with other white-box probes are not covered here. The work is a preprint and has not been peer reviewed.

If the approach holds up, it could support audits for sandbagging and checks that unlearning has actually removed knowledge, the author says. Until others reproduce the results, it is best read as a proposed method with promising reported numbers, not an established detector.

Revised 16 October 2026: Guth re-read the original sources and rewrote this article. The earlier version remains available as revision 1.