The missing piece in AI transparency: Reasoning humans can verify

Original paper: Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces

Authors: Runtao Zhou, Giang Nguyen, Nikita Kharya, Anh Nguyen, Chirag Agarwal

Read the paper: https://dl.acm.org/doi/10.1145/3742413.3789134 

About the researcher

Nikita Kharya, Vice President, Morgan Stanley, US. Find out more here: https://www.linkedin.com/in/nikita-kharya-14a83670/

What problem does this paper address, and why does it matter?

LLMs are now being trusted to reason through problems in education, healthcare, law, and other high-stakes domains. But as these models generate increasingly long reasoning chains, a dangerous gap has emerged: humans are expected to verify AI outputs they can barely read, let alone critically evaluate. A wall of thousands of tokens of chain-of-thought reasoning is cognitively overwhelming, and research shows people default to just trusting the AI. This means errors and hallucinations pass undetected precisely where the cost of getting things wrong is highest. If we cannot solve the human verification problem, deploying LLMs in critical applications is fundamentally unsafe.

What did this research discover/create?

We built three interactive reasoning interfaces called iCoT, iPoT, and iGraph, where the LLM itself generates an interactive web interface around its own reasoning. In a study of 125 participants, all three outperformed standard chain-of-thought in error detection. iGraph led at 85.6%, followed by iPoT at 82.5% and iCoT at 80.6%, compared to 73.5% for standard CoT. Users were also faster, with iGraph users averaging 57.9 seconds per question versus 64.7 seconds for standard CoT.

How could this research impact real-world applications?

Any system where a human must review AI-generated reasoning can directly benefit from these designs, including tutoring platforms, medical decision support tools, legal research assistants, and coding tools. The approach is also practically lightweight because the LLM generates the interactive interface itself, with no additional infrastructure required.

Who should care about this work? 

HCI and AI researchers, EdTech developers, UX designers building AI-powered tools, practitioners in education and other high-stakes fields, and policymakers focused on ensuring meaningful human oversight of AI systems.

What is noteworthy about this research?

We tackle a largely overlooked question in AI safety and human-AI interaction: how does the presentation format of AI reasoning affect whether humans catch mistakes? The finding that visual graph-based reasoning (iGraph) produced the biggest gains in both accuracy and speed was surprising. It also turns out the LLM can generate these interfaces itself, which makes the whole approach immediately deployable.

What's the ONE key takeaway you want people to remember? 

How you show LLM reasoning matters just as much as the reasoning itself. Interactive design can make AI explanations easier to follow, verify, and trust.

Read More

Access the full paper: https://dl.acm.org/doi/10.1145/3742413.3789134 

Have research to share or recommend? https://forms.gle/kaTJEd3gfJV8vtDx8