Paper Overview

Does Theory of Mind Improvement Really Benefit Human-AI Interactions?

This page presents the core structure of the paper in the same progression as the manuscript: motivation, interactive paradigm, and main result block with Figure 3, Table 1, and Table 2.

Nanxu Gong, Zixin Chen, Haotian Li, Zishu Zhao, Jianxun Lian, Huamin Qu, Yanjie Fu, Xing Xie

Go to Figure 1 Go to Figure 2 Go to Results View Code on GitHub
Interactive ToM Evaluation Goal-Oriented + Experience-Oriented Figure 1/2 + Figure 3 + Table 1/2

Abstract

Improving Theory of Mind (ToM) in large language models is widely considered important for socially effective Human-AI interaction. This work tests whether gains on static ToM benchmarks transfer to dynamic interactive tasks. The paper introduces an interactive evaluation paradigm, compares representative ToM enhancement methods across goal-oriented and experience-oriented settings, and reports benchmark plus user-facing findings.

Evaluation Gap Static benchmark improvements do not always map to stronger interactive behavior.
Generalization Gap Performance shifts differ between goal-oriented and experience-oriented tasks.
Perception Gap Measured gains may remain below user-perceived quality thresholds.

Introduction Framing (Figure 1)

The paper motivates a shift from static, third-person story-question-option evaluation to dynamic, first-person Human-AI symbiosis settings. This transition grounds ToM assessment in realistic interaction demands.

4Benchmarks
9Domains
2Task Families
Figure 1 from the paper: shift from story-question-option paradigm to Human-AI symbiosis paradigm
Figure 1. Shift from static story-question-option paradigm to Human-AI symbiosis paradigm.
Figure 2 from the paper: overview of interactive ToM evaluation paradigm
Figure 2. Overview of the interactive ToM evaluation paradigm.

Method Structure (Figure 2)

  • Perspective Shift: third-person reasoning to first-person interaction.
  • Metric Shift: single accuracy to task-aligned evaluation metrics.
  • Setting Shift: fixed prompts to multi-turn dynamic interactions.

Results Summary

This section follows the paper order exactly: Figure 3 first, then Table 1, then Table 2.

Figure 3: CollabLLM

Model variants' performance comparison on CollabLLM.

Figure 3 from the paper showing model variants' performance on CollabLLM
Figure 3. Model variants' performance on CollabLLM.

Table 1: ChatBench

Performance of model variations on ChatBench.

Table 1 from the paper: ChatBench performance comparison
Table 1. Performance of model variations on ChatBench.

Table 2: MentalChat16K + ESC

Performance on MentalChat16K and Emotional-Support-Conversation.

Table 2 from the paper: MentalChat16K and Emotional-Support-Conversation performance
Table 2. Performance of model variations on MentalChat16K and Emotional-Support-Conversation.

Contributions

  • Introduces an interactive evaluation paradigm for ToM in Human-AI settings.
  • Compares representative ToM enhancement methods across goal-oriented and experience-oriented scenarios.
  • Reports key gaps between benchmark improvements and practical interaction outcomes.