Using LLM for confidence scoring is a bad idea

A discussion post argues that asking an LLM to generate a confidence score is unreliable. It explains that confidence should reflect the information a prediction is based on and be calibrated against observed outcomes. The author notes LLMs produce confidence numbers as text continuations rather than measurements, so the same question can yield different scores and rewording shifts them. The post proposes an alternative: derive confidence from observable features of outputs, start from base rates, log outcomes, and recalibrate regularly against recorded results.

Why it matters

Agent designers often use model-stated confidence to decide when to escalate to humans, so unreliable scores can silently route errors past review.

Read with care

Source claims not independently checked; the post is an opinion piece and the proposed alternative is illustrated with a hypothetical example rather than empirical results.