MIT CSAIL: Rewarding only correct answers makes reasoning models overly confident; a new method improves uncertainty estimation

AIMPACT message, April 26 (UTC+8): MIT CSAIL research found that top reasoning models become overconfident during reinforcement learning training because they only reward correct answers without taking confidence into account. The research team proposed a new method: training the model to estimate the confidence level for each answer, significantly improving uncertainty estimation capabilities without harming accuracy. The work exposes a key flaw in current reinforcement learning training mechanisms and suggests an effective direction for improvement. (Source: InFoQ)
View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pinned