中文 |

Mamba Tracker Uses Earlier Frames More Selectively

Author: FENG Jiahao |

A recent study titled Efficient temporal-aware RGBT tracking network based on Mamba has been published in Infrared Physics and Technology. The work was carried out by researchers at the Changchun Institute of Optics, Fine Mechanics and Physics of the Chinese Academy of Sciences. The team developed ETATrack, a single-stream network that combines visible-light and thermal-infrared images with selected historical frames. On LasHeR, the largest benchmark used in the study, the tracker reached a success rate of 58.2 percent while retaining real-time inference.

Following the same object through a video becomes difficult when illumination changes, motion blurs an image or another object briefly blocks the view. Visible images provide detailed texture and color, while thermal images record heat patterns that remain informative under poor lighting. Using both types of images can therefore make tracking more dependable. Earlier frames offer another source of evidence when the object’s appearance changes, but they are not always helpful. A stored frame may contain blur, obstruction or outdated appearance, and processing every frame increases the computational load. The central challenge is to preserve useful history without allowing unreliable observations to mislead the tracker.

ETATrack addresses this challenge in one shared processing stream. The network converts visible-light and thermal-infrared templates, several earlier templates and the current search region into a sequence of visual tokens. Temporal embeddings mark when each template was recorded, helping the network distinguish recent information from older observations. A bidirectional Mamba encoder then extracts and fuses the features together. Because this sequence model scales linearly with sequence length, it can handle a longer temporal context with less growth in computation than attention-based designs.

Two mechanisms control how the history is used. The first maintains a compact guidance token that represents the target object. At selected stages of the encoder, information from the current search region updates this token, and the revised token is returned to the feature sequence. This repeated exchange helps preserve the object identity as the video changes. The second mechanism compares each historical template with different positions in the current search region. For each position, it draws on the template that provides the most relevant response, so a useful earlier view can contribute without letting a weak template dominate the entire prediction.

The researchers evaluated the network on four public visible and thermal tracking benchmarks and compared it with representative methods. The results showed a consistent balance between accuracy, model size and processing speed. Ablation experiments also clarified the role of the design choices. Removing temporal cues, the guidance process or the complementary use of templates reduced performance. The tests further showed that adding more historical templates beyond the best tested setting did not improve the result and could introduce redundant or outdated information. This finding supports the study’s emphasis on selecting useful history rather than accumulating it indiscriminately.

The study provides a practical way to bring temporal evidence into multimodal object tracking while keeping the model compact. It also identifies limits that remain for future work. The current method uses a fixed number of stored templates, so it cannot yet adjust the history length to the difficulty of each sequence. Deployment on devices with restricted computing resources will also require further simplification. The researchers plan to explore adaptive template selection and lighter network designs, with the aim of retaining useful temporal information while reducing unnecessary computation.


Contact

YU Yi

Changchun Institute of Optics, Fine Mechanics and Physics

E-mail:

cas_logo.png



       Copyright @ 吉ICP备06002510号 2007 CIOMP130033