Publications / Preprints
* denotes Equal Contribution
Please see my Google Scholar for the most up-to-date list of publications.
2026
- preprintTruth in Contrast: Calibrated Factual Evidence from Language Model LikelihoodsPonhvoan Srey, Jason Lucas, Pittawat Taveekitworachai, and 2 more authorsarXiv preprint arxiv:2610.23527, 2026
Large Language Models (LLMs) encode substantial factual knowledge. Yet, estimating confidence in factual claims reliably remains challenging. We study likelihood-based factual confidence from the complementary perspectives of discrimination and calibration across multilingual factual statements. Specifically, we instantiate contrastive generative likelihoods as Contrastive Factual Evidence (CFE), which compares the likelihood of the same claim under competing factual and non-factual conditions, and derive its probabilistic counterpart, the Contrastive Factual Posterior (CFP). We show analytically that CFP calibration depends on the differential error between the two surrogate conditionals, allowing errors shared across them to cancel and yield tighter calibration bounds. Empirically, CFP substantially improves calibration while maintaining factual discrimination comparable to its strongest competitor. Nonetheless, discrimination is uniformly weak across methods in lower-resource languages, which simultaneously undermines the utility of the measured calibration. To this end, we interpret a likelihood geometry, within which an evidence-bearing direction is induced, along with an orthogonal evidence-invariant direction. In our investigation, we selectively align target-language claims with stronger English evidence only along the factual-evidence direction, improving discrimination and open-ended factual generation without target-language labels.
@article{srey2026truth, title = {Truth in Contrast: Calibrated Factual Evidence from Language Model Likelihoods}, author = {Srey, Ponhvoan and Lucas, Jason and Taveekitworachai, Pittawat and Lee, Junyoung and Luu, Anh Tuan}, journal = {arXiv preprint arxiv:2610.23527}, url = {https://arxiv.org/abs/2610.23527}, year = {2026}, } - preprintReliable Memory from Unstable Growth: Curating Self-Written Agent Playbooks with Randomized ExperimentsHoang Pham, Cong-Duy Nguyen, Dong Le, and 2 more authorsarXiv preprint arXiv:2610.18427, 2026
Persistent textual memory lets language-model agents reuse lessons from past tasks to improve on future multi-step tasks without retraining. Systems such as ACE and Dynamic Cheatsheet keep it as a playbook: lessons the agent writes after each task and reads in its prompt. A key challenge is to decide which lessons improve later tasks. Current systems leave the decision to the agent itself, which makes performance unstable: the same procedure can write a playbook that helps or one that falls up to 69.6 points below no playbook. These findings motivate valuing lessons by their measured effect on task success rather than by the agent’s judgment. We propose Randomized Block Curation (RBC), which measures this effect with a randomized experiment on practice tasks. It groups related lessons into blocks large enough to move success measurably and, on each practice task, includes or leaves out every block at random. Random inclusion makes tasks with and without a block comparable, so the gap in their success estimates the block’s effect. RBC keeps the blocks that help and, since the same procedure can write a helpful or a harmful playbook, grows several and serves whichever curated, whole or no playbook scores best on fresh practice episodes. Across two memory systems, three models and three benchmarks, RBC raises average success to 66.9, against 55.5 with the whole playbook, 54.1 with no playbook and 53.8 when the agent’s own judgment picks the lessons to keep.
@article{pham2026reliable, title = {Reliable Memory from Unstable Growth: Curating Self-Written Agent Playbooks with Randomized Experiments}, author = {Pham, Hoang and Nguyen, Cong-Duy and Le, Dong and Srey, Ponhvoan and Luu, Anh Tuan}, journal = {arXiv preprint arXiv:2610.18427}, year = {2026}, url = {https://arxiv.org/abs/2610.18427}, } - preprintTIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative DecodingQuynh Vo, Cong-Duy Nguyen, Ponhvoan Srey, and 2 more authorsarXiv preprint arXiv:2607.11131, 2026
Speculative decoding accelerates autoregressive generation by letting a lightweight drafter propose multiple tokens that are verified by a larger target model. Although effective for text-only LLMs, speculative decoding yields limited gains in VLMs because drafters often diverge on vision-critical content, while existing multimodal acceleration methods do not directly address irrelevant visual evidence or optimize the verifier-accepted prefix length that governs speedup. We propose TIGER, a Text-conditioned vIsual GatEd Routing framework for multimodal speculative decoding. TIGER dynamically selects a sparse set of context-relevant visual tokens based on the drafter’s current textual state, rather than expose the full visual token set or a fixed compressed interface. To better align training with inference-time efficiency, we optimize the drafter with acceptance-aligned group-based policy training using verifier-derived rewards based on accepted prefix length, built on top of distillation warm start with KL anchoring. This encourages the drafter not only to imitate the target model, but also to produce speculative continuations that survive verification for longer prefixes. Experiments show that TIGER yields consistent gains in accepted prefix length and speculative speedup under exact verifier-side speculative decoding, while achieving favorable quality-latency trade-offs with comparable downstream accuracy in visual-routing analyses.
@article{vo2026tiger, title = {TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding}, author = {Vo, Quynh and Nguyen, Cong-Duy and Srey, Ponhvoan and Tuan, Luu Anh and Nguyen, Thong}, journal = {arXiv preprint arXiv:2607.11131}, year = {2026}, url = {https://arxiv.org/abs/2607.11131}, } - preprintFrom Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language ModelsPonhvoan Srey, Xiaobao Wu, Cong-Duy Nguyen, and 3 more authorsarXiv preprint arXiv:2606.27679, 2026
Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these best-practice findings, we train benchmark-based pretrained probes that transfer reasonably well to open-ended factual generation, providing a stable off-the-shelf baseline. Our work encourages more deployment-oriented evaluation of probe-based uncertainty estimators.
@article{srey2026probe, title = {From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models}, author = {Srey, Ponhvoan and Wu, Xiaobao and Nguyen, Cong-Duy and Nguyen, Quang Minh and Vu, Duc Anh and Luu, Anh Tuan}, journal = {arXiv preprint arXiv:2606.27679}, year = {2026}, url = {https://arxiv.org/abs/2606.27679}, } - EMNLP FindingsReinforcing Step-level Reasoning for Effective Self-Correction in LLMsVu Duc Anh, Nhat M Hoang, Do Xuan Long, and 3 more authorsIn Findings of the Association for Computational Linguistics: EMNLP 2026, 2026
Achieving effective self-correction, where models verify and correct their own mistakes, remains a fundamental challenge for large language models (LLMs). In this work, we propose Self-Fix Step-DPO (SFS-DPO), a reinforcement learning based, two-stage framework for step-level self-verification and self-correction. The first stage strengthens step-level reasoning via step-level preference optimization, while the second stage explicitly trains models to self-verify and self-correct. We further introduce a teacher-assisted variant, SFS-DPO-R, which incorporates explanatory rationales for error verification to provide stronger corrective signals. Comprehensive in-domain and out-of-domain evaluations across multiple LLMs demonstrate that SFS-DPO and SFS-DPO-R consistently outperform prior step-level training baselines. Our analysis further reveals improvements in self-correction frequency and effectiveness, highlighting the importance of strengthening step-level reasoning for robust performance.
@inproceedings{anh2026reinforcing, title = {Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs}, author = {Anh, Vu Duc and Hoang, Nhat M and Long, Do Xuan and Nguyen, Cong-Duy and Srey, Ponhvoan and Tuan, Luu Anh}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026}, year = {2026}, url = {https://arxiv.org/abs/2608.11573}, } - EMNLP FindingsTowards Reliable Truth-Aligned Uncertainty Estimation in Large Language ModelsPonhvoan Srey, Quang Minh Nguyen, Xiaobao Wu, and 1 more authorIn Findings of the Association for Computational Linguistics: EMNLP, 2026
Uncertainty estimation (UE) quantifies uncertainty in large language models (LLM) outputs to flag potential hallucinations and support reliable deployment. However, UE metrics often exhibit unstable performance across models and datasets, limiting their practical utility. We formalise this phenomenon as \emphproxy failure, arising because UE metrics are derived from model behaviour rather than being explicitly grounded in factual correctness. We show that low mutual information between a UE score and correctness necessarily constrains its discriminability to near chance. To address this, we propose \emphTruth AnChoring (TAC), a lightweight post-hoc calibration method that maps raw UE scores to truth-aligned correctness probabilities. Even with noisy and few-shot supervision, our TAC can support the learning of well-calibrated uncertainty estimates, and presents a practical calibration protocol. Our findings highlight the limitations of treating heuristic UE metrics as direct indicators of uncertainty about factual correctness, and position our TAC as a principled step toward more reliable uncertainty estimation for LLMs.
@inproceedings{srey2026towards, title = {Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models}, author = {Srey, Ponhvoan and Nguyen, Quang Minh and Wu, Xiaobao and Luu, Anh Tuan}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP}, year = {2026}, url = {https://arxiv.org/abs/2604.00445}, } - ACLLearning Uncertainty from Sequential Internal Dispersion in Large Language ModelsPonhvoan Srey, Xiaobao Wu, Cong-Duy T Nguyen, and 1 more authorIn Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2026
Uncertainty estimation is a promising approach to detect hallucinations in large language models (LLMs). Recent approaches commonly depend on model internal states to estimate uncertainty. However, they suffer from strict assumptions on how hidden states should evolve across layers, and from information loss by solely focusing on last or mean tokens. To address these issues, we present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework that leverages token-wise, layer-wise features derived from hidden states. SIVR adopts a more basic assumption that uncertainty manifests in the degree of dispersion or variance of internal representations across layers, rather than relying on specific assumptions, which makes the method model and task agnostic. It additionally aggregates the full sequence of per-token variance features, learning temporal patterns indicative of factual errors and thereby preventing information loss. Experimental results demonstrate SIVR consistently outperforms strong baselines. Most importantly, SIVR enjoys stronger generalisation and avoids relying on large training sets, highlighting the potential for practical deployment.
@inproceedings{srey-etal-2026-learning, title = {Learning Uncertainty from Sequential Internal Dispersion in Large Language Models}, author = {Srey, Ponhvoan and Wu, Xiaobao and Nguyen, Cong-Duy T and Luu, Anh Tuan}, editor = {Liakata, Maria and Moreira, Viviane P. and Zhang, Jiajun and Jurgens, David}, booktitle = {Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)}, month = jul, year = {2026}, address = {San Diego, California, United States}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2026.acl-long.1862/}, pages = {40088--40106}, isbn = {979-8-89176-390-6}, }
2025
- EMNLPUnsupervised Hallucination Detection by Inspecting Reasoning ProcessesPonhvoan Srey, Xiaobao Wu, and Anh Tuan LuuIn Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Nov 2025
Unsupervised hallucination detection aims to identify hallucinated content generated by large language models (LLMs) without relying on labeled data. While unsupervised methods have gained popularity by eliminating labor-intensive human annotations, they frequently rely on proxy signals unrelated to factual correctness. This misalignment biases detection probes toward superficial or non-truth-related aspects, limiting generalizability across datasets and scenarios. To overcome these limitations, we propose IRIS, an unsupervised hallucination detection framework, leveraging internal representations intrinsic to factual correctness. IRIS prompts the LLM to carefully verify the truthfulness of a given statement, and obtain its contextualized embedding as informative features for training. Meanwhile, the uncertainty of each response is considered a soft pseudolabel for truthfulness. Experimental results demonstrate that IRIS consistently outperforms existing unsupervised methods. Our approach is fully unsupervised, computationally low cost, and works well even with few training data, making it suitable for real-time detection.
@inproceedings{srey-etal-2025-unsupervised, title = {Unsupervised Hallucination Detection by Inspecting Reasoning Processes}, author = {Srey, Ponhvoan and Wu, Xiaobao and Luu, Anh Tuan}, editor = {Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet}, booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing}, month = nov, year = {2025}, address = {Suzhou, China}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2025.emnlp-main.1124/}, pages = {22117--22129}, isbn = {979-8-89176-332-6}, }