Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?

* Equal contribution.

[Paper] [Project Page]

Abstract (Brief Summary)

Deep research agents increasingly automate complex information-seeking tasks, producing evidence-grounded reports via multi-step reasoning, tool use, and synthesis. Their growing role demands scalable, reliable evaluation, positioning LLM-as-judge as a natural supervision paradigm. Yet the reliability of these judges remains poorly understood — posing a critical meta-evaluation problem: before deploying LLM judges to supervise research agents, we must first evaluate the judges themselves.

REFLECT (REliable Fine-grained LLM judge Evaluation via Controlled inTervention) addresses this gap with a meta-evaluation benchmark targeting fine-grained failure detection in agentic environments. It defines a detailed taxonomy of process- and outcome-level failure modes, instantiated via controlled and localized interventions on quality-screened agent execution traces. Experiments show that current LLM judges remain unreliable: even the best-performing models achieve overall accuracies below 55% across reasoning, tool-use, and report-quality failures, with especially poor performance on evidence verification.

Main Contributions

• REFLECT Benchmark: Meta-evaluation framework with verifiable, fine-grained instances for validating LLM judges on agentic tasks — beyond coarse human-preference agreement.
• Failure Mode Taxonomy: A structured taxonomy of process- and outcome-level failures in deep research agent evaluation, instantiated via controlled interventions.
• Systematic Judge Analysis: Reveals that state-of-the-art LLM judges are unreliable for agentic evaluation, with accuracy below 55% and critical weaknesses in evidence verification.