When both the tracker and the target are flying at high speed at low altitude, can traditional visual tracking algorithms keep up?

Recently, a joint research team fromCloudWalk Technology, Shanghai Jiao Tong University, Hong Kong University of Science and Technology (Guangzhou), Sun Yat-sen University, and the Institute of Information Engineering, Chinese Academy of Sciencesreleased a hardcore work——UAV-Anti-UAV. This is the industry's first million-level multimodal anti-UAV visual tracking benchmark for the"air-to-air"(Air-to-Airscenario, and proposes a strong baseline based onMambathe CongRong Large Model, achievingMambaSTS.MambaSTSaccumulation and continuous innovation in the fields ofUAV-Anti-UAVthe best performance on all5indicators of the benchmark. This isanother technological breakthrough by CloudWalk Technology in multimodal large models. Faced with dual dynamic interference, how do existingmethods perform? Let's find out!SOTAHow does it perform? Let's find out!

 01 Introduction: From "God's Perspective" to "Aerial Dogfight"

accumulation and continuous innovation in the fields ofWith the booming development of the low-altitude economy today, the security control of UAVs(Unmanned Aerial Vehicles)UAVhas become a global focus. Anti-UAVAnti-UAVtechnology has emerged as a result.

However, reviewing existing visual tracking research, we found a cleartask blind spotGap):

l : TraditionalUAVTracking(air-to-ground): The UAV is in a"god's perspective"filming vehicles or pedestrians on the ground; although the camera is moving, the target moves smoothly relative to the ground.

l ExistingAnti-UAV(ground-to-air): A fixed ground PTZ/camera looks up at the UAV in the sky; the observer is almost stationary, lacking severe jitter from motion.

What if one UAV is flying at high speed to track another hostile UAV trying to escape?This scenario is like an aerial dogfight in moviesDogfight)

Figure1: a direct comparison betweenTask1(UAVTracking), Task2(Anti-UAV)traditional and the proposedTask3(UAV-Anti-UAV).Task3scenarios, where bothObserverandTargetare moving at high speed at low altitude.

This is theUAV-Anti-UAV(air-to-air)task proposed in this paper. It is a bidirectional dynamic system:both the tracker and the target are moving at high speed at low altitude. This brings unprecedented challenges: severe dual dynamic interferenceDual-dynamicdisturbances, rapid scale changes, strong motion blur, and frequent viewpoint switches.

02Dataset: The First Million-Level, Multimodal, Full-Scenario Benchmark

To tackle this challenge, the research team constructed the first large-scale benchmark datasetUAV-Anti-UAVBenchmark. This is not just a dataset, but a standard testbed for low-altitude security.

What does the dataset consist of?

l Large Scale: Contains1,810video sequences, with a total frame count of up to105thousandframes, totaling nearly9.85hours.

l Rich Aircraft Types: Covering5major categories of targets, includingfixed-wing, multi-rotor, vertical take-off and landing (VTOL)VTOL, first-person view (FPV)FPVUAVs, and unmanned helicopters..

l Multimodal Annotations: In addition to fine bounding boxesBoundingBox, it also providesnatural language descriptionsLanguagePrompts), supporting vision-language tracking research.-Language tracking study.

l Fine-Grained Attributes: annotated15Typesextremely challenging attributes, such as fast motion (FM), illumination changes (IV), similar distractors (SD), etc.

Figure2: The dataset contains five different types of UAV targets (fixed-wing,FPV, multi-rotor, vertical take-off and landing, unmanned helicopter) and corresponding language descriptions.

How difficult is it?

Compared with existing datasets,UAV-Anti-UAVis"hell difficulty".

l Relative speed: The average relative speed of this dataset is as high as0.79, far exceeding existingUAV123(0.46) andAnti-UAV(0.72) and other datasets.

l Target size: Contains a large number of tiny targets (SmallObject), and due to distance changes, scale changes (ScaleVariation) are extremely drastic.

Figure3: Hardcore data proves the uniqueness of this dataset in terms of motion intensity.

03Method:MambaSTS——Perfect fusion of spatiotemporal semantics

Faced with such a high-difficulty task, traditionalTransformerdue to high computational complexity, are difficult to handle ultra-long sequences; while ordinaryCNNlack global perception ability. To this end, the authors propose a new baseline:MambaSTS.

This is a unified framework integrating spatial (Spatial), temporal (Temporal), and semantic (Semantic) learning. Its core logic is: leveraging the linear complexity advantage ofMamba(state space model) to establish video-level long-term context.

Figure4: Model architecture diagram, showing multimodal input (image+text),STSMambamodule and hierarchical feature extraction process.

Core innovation interpretation:

Hybrid architecture(HybridArchitecture):

1. Visual end: Using hierarchical visualTransformer(HiViT)extract multi-scale features to capture spatial details.

2. Language side: Using pre-trainedLanguageMambaextract semantic features, introduce text priors to help the model recognize targets in ambiguity"recognize"targets.

TemporalTokenpropagation(TemporalTokenPropagation):

This is the"killer feature"of this paper. The model maintains aTemporalToken, which is passed between frames like a baton.

UsingMambathe selective scanning mechanism ofToken, the trajectory evolution and appearance changes of the target in historical frames are compressed into this"video memory". This gives the model a

Unidirectional scanning mechanism(UnidirectionalScanning):

UnlikeVisionMamba(Vim)which uses bidirectional scanning for static images,MambaSTSthis method improves tounidirectional scanningbased on the causal nature of video tracking (i.e., current state depends only on the past), which is more aligned with real-time tracking logic.

04Experiments:50A battle of 10 trackers, who is the king?

The authors conducted a comprehensive evaluation of modern deep tracking algorithms, covering50aA comprehensive evaluation of modern deep tracking algorithms was conducted, coveringCNN, Transformer, Mambaand multimodal algorithms (such asOSTrack,MixFormer,MambaTrack,CiteTracker, etc.).

Overall situation

l MambaSTSLeads by a large margin: Achieves first place on all metrics,AUCreaching0.437, mACCreaching0.443, which is6.6percentage points higher than the second place.

l Extremely difficult task: EvenSOTAofMambaSTShas a success rate of only40%just over, and the average of all trackersAUConly0.30approximately. This indicates thatUAV-Anti-UAVthe field remains a blue ocean with enormous challenges!

Figure5: AUC, Precisionline chart of metrics such asMambaSTScurve (topmost) intuitively demonstrates its advantage.

Figure6: 50trackers'mACCscatter plot of rankings,MambaSTSlocated in the upper right corner, significantly leading.

Attribute Analysis: Which Scenarios Are the Most Challenging?

Through15attributes of fine-grained testing, the study found:

l MambaSTSstrengths: Infast motion(FM), motion blur(MB), small objects(SO)and other attributes, it performs robustly, thanks to its strong temporal modeling capability.

l common weaknesses: Inillumination changes(IV)andfull occlusion(FO)scenarios, all models (includingMambaSTS) struggle, with success rates below0.15. This points to future optimization directions.

Figure7: Representative attribute subplots (e.g.,FastMotion,MotionBlur,FullOcclusion).

Generalization Verification

In addition to performing well on its own dataset,MambaSTSon traditionalUAVTrackingdatasets (e.g.,UAV123,VisDrone) and groundAnti-UAVdatasets, it also achievesSOTAperformance. This proves that the architecture is not"overfitting", but has truly mastered the essence of spatiotemporal features.

05Summary and Outlook

UAV-Anti-UAVThe proposal of the task marks an important step toward practical application in low-altitude security research.

l New Task: Fills the gap in air-to-air dynamic tracking.

l New Data: Million-scale, multi-modal annotations, providing a standard testbed for the community.

l New Baseline: MambaSTSThis demonstrates the great potential of state-space models in long-sequence dynamic tracking.

AlthoughMambaSTSperforms excellently, there is still a gap to solve all-weather, fully autonomous aerial interception (e.g., lack of infrared/LiDARdata, and currently offline training).The challenge is already here. Developers, are you ready to embrace"air combat"?

??Open source address:

https://github.com/983632847/Awesome-Multimodal-Object-Tracking

l Reference paper:ChunhuiZhang,LiLiu,ZhipengZhang,YongWang,HaoWen,XiZhou,ShimingGe,YanfengWang."HowFarareModernTrackersfromUAV-Anti-UAV?AMillion-ScaleBenchmarkandNewBaseline",arXiv,2025.

Download now:

https://arxiv.org/abs/2512.07385