VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

Anonymous Submission; Anonymous Submission

VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

Florian Seligmann Emiliyan Gospodinov Enes Ulas Dincer Gerhard Neumann

Autonomous Learning Robots Lab, Karlsruhe Institute of Technology (KIT)

CoRL 2026

Paper arXiv

Abstract

Vision-language-action models (VLAs) achieve state-of-the-art performance on many robotic manipulation tasks, yet they can still behave unpredictably in out-of-distribution scenarios. Runtime failure detection is therefore essential for the safe real-world deployment of VLAs. However, existing task failure detectors require computationally expensive action sampling, are based on architectural assumptions that limit their applicability to VLAs, or need access to failure rollouts. We propose VLA-FAIL, a lightweight and broadly applicable failure detection framework for VLAs that combines two novel failure detectors with minimal overhead, without requiring failure data. The first, last-layer Mahalanobis distance (LLMD), detects out-of-distribution states by measuring token-wise deviations in last-layer features relative to the training data. The second, action chunk consistency (ACC), exploits the temporal overlap induced by receding-horizon control and detects failures when consecutive action chunks become inconsistent. To capture the trade-off between detection accuracy and detection latency, we introduce AUCPDT, a threshold-independent metric that jointly evaluates precision, recall, and detection time. Through extensive real-world and simulation experiments, we demonstrate that LLMD and ACC capture complementary failure modes whose combination enables reliable and early failure detection across diverse tasks, frequently outperforming significantly more expensive baseline methods.

Framework Overview

VLA-FAIL performs runtime failure detection for vision-language-action models without requiring failure rollouts. The framework combines last-layer Mahalanobis distance (LLMD), which detects out-of-distribution states from model features, with action chunk consistency (ACC), which detects failures from inconsistent receding-horizon action predictions. These complementary signals provide early and reliable failure detection while adding minimal computational overhead.

Inference Latency

Inference latency is a major concern for the real-time deployment of large VLAs. Failure detectors must therefore be computationally efficient to be practical in real-world applications. This graph shows the time in milliseconds to sample an action chunk for X-VLA on an NVIDIA RTX 5090 GPU when using various failure detectors. 'Raw' corresponds to the inference latency without failure detection. The numbers behind ACE and Diff indicate the number of action samples. These baseline methods incur significant computational overhead and are therefore not real-time capable. The time for the VLM forward pass (17 ms) is excluded as it is identical across methods.

Real-World Failure Detection

We introduce two complementary failure detectors for VLAs: LLMD (feature-based) and ACC (action-based). Both can reliably detect task failure across diverse real-world tasks.

The red line shows the detector score for either method. As soon as it crosses the white dashed threshold line, a likely failure is detected.