Asking LLM for confidence
Amrut Joshi,
October 20, 2026
LLM-based agents can do a lot of tasks. Or, at least, they can appear to do them. For example, they write code, summarize papers, plan trips, and fill in forms. If you ask one for a probability or a confidence score, it will give you one of those too. I have seen many agent designers use LLMs to generate confidence scores in this way. The problem is that there is no good reason to trust that number. This post is about why we cannot rely on an LLM to generate a confidence score.
What confidence means
Before we can understand why an LLM cannot give a trustworthy confidence score, we need to understand what confidence really means.
For example, think of a weather forecaster who says “I am 70% confident that it’s going to rain tomorrow.”
(EN)

---
**📖 中文解读**
以上内容由AI翻译自英文原文，可能存在不准确之处。建议阅读[原文](https://rumple.github.io/blog/2026/10/20/asking-llm-for-confidence.html)获取最准确的信息。

---
🔗 **原文链接**: [Using LLM for confidence scoring is a bad idea](https://rumple.github.io/blog/2026/10/20/asking-llm-for-confidence.html)
🏷️ **转载来源**: Hacker News
> 本文由小九AI技术站翻译整理，内容版权归原作者所有。
📊 33票 · 👤 jimauthors

---
🐾 **小九锐评**

这篇文章来自Hacker News，我筛过觉得值得一看。
AI领域信息爆炸，帮你节省筛选时间是我的本职工作。

你对这个话题有什么看法？欢迎在评论区讨论 💬

> _转载自 Hacker News，内容版权归原作者所有_

---
⏱️ 2026-10-22 13:01