Auditing Over-Refusal Measurement in Multimodal Large Language Models with Inference-Time Defenses

Bulat Nutfullin, Dmitry Namiot

Abstract


We present a procedure for measuring overrefusal in multimodal large language models with inferencetime defenses. It combines separate accounting for direct system refusals, semantic labeling, and reweighting of a simple random subsample. Two measurement errors are identified: interpreting a safety verdict as refusal in service records without text, and matching defensive vocabulary before substantive answers in stored responses. These errors concern disjoint data. The control archive contains 38,500 stored responses; the main matrix covers 56 model–defense combinations and 28,000 responses. Two-stage selection yielded one primary label of the Haiku LLM judge for each of 11,357 unique responses in the main matrix; the same responses were independently labeled by a second judge from another provider. Uninterpretable responses are excluded from the estimated denominator. The pooled point estimate is 1.30%; the highest value among estimable combinations is 4.55% (Qwen3-VL-8B with SP). The maximum is sensitive to a single refusal in the random subsample of that combination, and the rule for merging repeated labels has practically no effect on the estimates. The second judge gives a pooled estimate of 1.72%. Uncertainty in these estimates depends on the sampling scheme, judge error, and the estimated denominator. Across 568 responses relabeled by the primary judge, overall binary agreement is 99.5% and positive agreement on refusals is 90.9%; between the two judges, positive agreement on refusals is 71.6%. These results motivate separate evaluation of keyword rules and semantic labeling of the rare refusal class. Recommendations are provided for selecting measurements and checking their reliability.

Full Text:

PDF (Russian)

References


OpenAI. GPT-4 technical report // arXiv preprint arXiv:2303.08774. — 2023.

Visual instruction tuning / H. Liu, C. Li, Q. Wu, Y. J. Lee // Advances in Neural Information Processing Systems (NeurIPS). — 2023.

Are aligned neural networks adversarially aligned? / N. Carlini, M. Nasr, C. A. Choquette-Choo et al. // Advances in Neural Information Processing Systems (NeurIPS). — 2023.

Visual adversarial examples jailbreak aligned large language models / X. Qi, K. Huang, A. Panda et al. // Proceedings of the AAAI Conference on Artificial Intelligence. — 2024.

Safeguarding vision-language models against patched visual prompt injectors / J. Sun, C. Wang, J. Wang et al. // arXiv preprint arXiv:2405.10529. — 2024.

Jiang Y., Tan Y., Yue X. RapGuard: Safeguarding multimodal large language models via rationale-aware defensive prompting // arXiv preprint arXiv:2412.18826. — 2024.

AdaShield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting / Yu Wang, Xiaogeng Liu, Yu Li et al. // European Conference on Computer Vision (ECCV). — 2024. — 2403.09513.

MMBench: Is your multi-modal model an all-around player? / Yuan Liu, Haodong Duan, Yuanhan Zhang et al. // European Conference on Computer Vision (ECCV). — 2024.

Nutfullin Bulat, Evgrafov Vladimir, Namiot Dmitry. Auditing inference-time defense evaluation for multimodal large language models // arXiv preprint arXiv:2606.10904. — 2026. — Version 4, revised 8 August 2026. URL: https://arxiv.org/abs/2606.10904.

LLMs encode harmfulness and refusal separately / Jiachen Zhao, Jing Huang, Zhengxuan Wu et al. // arXiv preprint arXiv:2507.11878. — 2025.

MM-SafetyBench: A benchmark for safety evaluation of multimodal large language models / X. Liu, Y. Zhu, J. Gu et al. // European Conference on Computer Vision (ECCV). — 2024.

Byrt Ted, Bishop Janet, Carlin John B. Bias, prevalence and kappa // Journal of Clinical Epidemiology. — 1993. — Vol. 46, no. 5. — P. 423–429.

Gwet Kilem L. Computing inter-rater reliability and its variance in the presence of high agreement // British Journal of Mathematical and Statistical Psychology. — 2008. — Vol. 61, no. 1. — P. 29–48.

Feinstein Alvan R., Cicchetti Domenic V. High agreement but low kappa: I. the problems of two paradoxes // Journal of Clinical Epidemiology. — 1990. — Vol. 43, no. 6. — P. 543–549.

Li Xirui, Zhou Hengguang, Wang Ruochen et al. MOSSBench: Is your multimodal language model oversensitive to safe queries? — 2024. — 2406.17806.

A StrongREJECT for empty jailbreaks / Alexandra Souly, Qingyuan Lu, Dillon Bowen et al. // Advances in Neural Information Processing Systems (NeurIPS). — Vol. 37. — 2024. — Datasets and Benchmarks Track; arXiv:2402.10260.

A coin flip for safety: LLM judges fail to reliably measure adversarial robustness / Leo Schwinn, Moritz Ladenburger, Tim Beyer et al. // arXiv preprint arXiv:2603.06594. — 2026.

HarmMetric eval: Benchmarking metrics and judges for LLM harmfulness assessment / Langqi Yang, Tianhang Zheng, Yixuan Chen et al. //arXiv preprint arXiv:2509.24384. — 2025.

Norman Justin D., Rivera Michael U., Hughes D. Alex. Reliability without validity: A systematic, large-scale evaluation of LLM-as-ajudge models across agreement, consistency, and bias // arXiv preprint arXiv:2606.19544. — 2026.

Khorramrouz Adel, Levy Sharon. Characterizing selective refusal bias

in large language models // arXiv preprint arXiv:2510.27087. — 2025.

XSTest: A test suite for identifying exaggerated safety behaviours in large language models / Paul Röttger, Bertie Vidgen, Dirk Hovy, Janet Pierrehumbert // Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). — 2024.

OR-Bench: An over-refusal benchmark for large language models / Justin Cui, Wei-Lin Chiang, Ion Stoica, Cho-Jui Hsieh // Proceedings of the 42nd International Conference on Machine Learning. — Vol. 267 of Proceedings of Machine Learning Research. — PMLR, 2025. — P. 11515–11542. — URL: https://proceedings.mlr.press/v267/cui25a.html.

Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions / Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio et al. // International Conference on Learning Representations (ICLR). — 2024.

VSCBench: Bridging the gap in vision-language model safety calibration / Jiahui Geng, Qing Li, Zongxiong Chen et al. // Findings of the Association for Computational Linguistics: ACL 2025. — Association for Computational Linguistics, 2025. — P. 3047–3059. — URL: https://aclanthology.org/2025.findings-acl.158/.

Evgrafov V. A., Nutfullin B. M., Namiot D. E. Attack methods and defenses in LLM-based agentic systems // International Journal of Open Information Technologies. — 2026. — Vol. 14, no. 5. — P. 1– 8. — URL: https://injoit.org/index.php/j1/article/view/2539.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ-КОНГРЕСС ВМК МГУ 2026 СНЭ

ISSN: 2307-8162