Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these best-practice findings, we train benchmark-based pretrained probes that transfer reasonably well to open-ended factual generation, providing a stable off-the-shelf baseline. Our work encourages more deployment-oriented evaluation of probe-based uncertainty estimators.
@article{srey2026probe,title={From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models},author={Srey, Ponhvoan and Wu, Xiaobao and Nguyen, Cong-Duy and Nguyen, Quang Minh and Vu, Duc Anh and Luu, Anh Tuan},journal={arXiv preprint arXiv:2606.27679},year={2026},url={https://arxiv.org/abs/2606.27679},}
preprint
Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models
Ponhvoan Srey, Quang Minh Nguyen, Xiaobao Wu, and 1 more author
Uncertainty estimation (UE) aims to detect hallucinated outputs of large language models (LLMs) to improve their reliability. However, UE metrics often exhibit unstable performance across configurations, which significantly limits their applicability. In this work, we formalise this phenomenon as proxy failure, since most UE metrics originate from model behaviour, rather than being explicitly grounded in the factual correctness of LLM outputs. With this, we show that UE metrics become non-discriminative precisely in low-information regimes. To alleviate this, we propose Truth AnChoring (TAC), a post-hoc calibration method to remedy UE metrics, by mapping the raw scores to truth-aligned scores. Even with noisy and few-shot supervision, our TAC can support the learning of well-calibrated uncertainty estimates, and presents a practical calibration protocol. Our findings highlight the limitations of treating heuristic UE metrics as direct indicators of truth uncertainty, and position our TAC as a necessary step toward more reliable uncertainty estimation for LLMs.
@article{srey2026towards,title={Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models},author={Srey, Ponhvoan and Nguyen, Quang Minh and Wu, Xiaobao and Luu, Anh Tuan},journal={arXiv preprint arXiv:2604.00445},year={2026},url={https://arxiv.org/abs/2604.00445},}
ACL
Learning Uncertainty from Sequential Internal Dispersion in Large Language Models
Ponhvoan Srey, Xiaobao Wu, Cong-Duy T Nguyen, and 1 more author
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2026
Uncertainty estimation is a promising approach to detect hallucinations in large language models (LLMs). Recent approaches commonly depend on model internal states to estimate uncertainty. However, they suffer from strict assumptions on how hidden states should evolve across layers, and from information loss by solely focusing on last or mean tokens. To address these issues, we present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework that leverages token-wise, layer-wise features derived from hidden states. SIVR adopts a more basic assumption that uncertainty manifests in the degree of dispersion or variance of internal representations across layers, rather than relying on specific assumptions, which makes the method model and task agnostic. It additionally aggregates the full sequence of per-token variance features, learning temporal patterns indicative of factual errors and thereby preventing information loss. Experimental results demonstrate SIVR consistently outperforms strong baselines. Most importantly, SIVR enjoys stronger generalisation and avoids relying on large training sets, highlighting the potential for practical deployment.
@inproceedings{srey-etal-2026-learning,title={Learning Uncertainty from Sequential Internal Dispersion in Large Language Models},author={Srey, Ponhvoan and Wu, Xiaobao and Nguyen, Cong-Duy T and Luu, Anh Tuan},editor={Liakata, Maria and Moreira, Viviane P. and Zhang, Jiajun and Jurgens, David},booktitle={Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)},month=jul,year={2026},address={San Diego, California, United States},publisher={Association for Computational Linguistics},url={https://aclanthology.org/2026.acl-long.1862/},pages={40088--40106},isbn={979-8-89176-390-6},}
2025
EMNLP
Unsupervised Hallucination Detection by Inspecting Reasoning Processes
Ponhvoan Srey, Xiaobao Wu, and Anh Tuan Luu
In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Nov 2025
Unsupervised hallucination detection aims to identify hallucinated content generated by large language models (LLMs) without relying on labeled data. While unsupervised methods have gained popularity by eliminating labor-intensive human annotations, they frequently rely on proxy signals unrelated to factual correctness. This misalignment biases detection probes toward superficial or non-truth-related aspects, limiting generalizability across datasets and scenarios. To overcome these limitations, we propose IRIS, an unsupervised hallucination detection framework, leveraging internal representations intrinsic to factual correctness. IRIS prompts the LLM to carefully verify the truthfulness of a given statement, and obtain its contextualized embedding as informative features for training. Meanwhile, the uncertainty of each response is considered a soft pseudolabel for truthfulness. Experimental results demonstrate that IRIS consistently outperforms existing unsupervised methods. Our approach is fully unsupervised, computationally low cost, and works well even with few training data, making it suitable for real-time detection.
@inproceedings{srey-etal-2025-unsupervised,title={Unsupervised Hallucination Detection by Inspecting Reasoning Processes},author={Srey, Ponhvoan and Wu, Xiaobao and Luu, Anh Tuan},editor={Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet},booktitle={Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},month=nov,year={2025},address={Suzhou, China},publisher={Association for Computational Linguistics},url={https://aclanthology.org/2025.emnlp-main.1124/},pages={22117--22129},isbn={979-8-89176-332-6},}
MLJ
Uncover and unlearn nuisances: agnostic fully test-time adaptation
Ponhvoan Srey*, Yaxin Shi*, Hangwei Qian, and 2 more authors