๐Ÿ“–
AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification
Qizhou Wang, Jiawen Zhu, Guansong Pang, Christopher Leckie

AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

Recent multimodal LLM (MLLM)-based anomaly detection (AD) methods can generate textual explanations that help interpret their predictions. However, reliably and efficiently evaluating these explanations remains challenging. While LLM judges can assess explanations through structured rubrics, they often require substantial token generation, making evaluation costly and difficult to scale. We instead propose to anchor evaluation in the available ground truth by extracting verifiable claims from each explanation and verifying them against the corresponding annotations. We instantiate this idea as AnomalyJev, a lightweight framework that extracts verifiable claims from contextualised representations using ground-truth-defined queries and pointer based heads, without token generation. The claims are then verified against the ground truth to score faithfulness and completeness. For evaluation, we construct a human-annotated set of 640 explanations with over 1,600 expert ratings across in-domain and cross-dataset settings. Despite its compact design, AnomalyJev achieves stronger agreement with human annotators than most open MLLM judges and approaches that of flagship MLLM judges, at a fraction of their runtime and resources.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification
Author / Organization Qizhou Wang, Jiawen Zhu, Guansong Pang, Christopher Leckie
Subject & Discipline cs.CV
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-271343
Original Source Link Visit Source Page on arxiv.org »