中文 |

Dual Input Fusion Improves Visible Infrared Object Detection

Author: FENG Jiahao |

A recent study titled “Dual-input remote sensing image fusion for object detection based on hybrid-level Mamba fusion” has been published in Pattern Recognition. The work was conducted by researchers from the Changchun Institute of Optics, Fine Mechanics and Physics of the Chinese Academy of Sciences. The team developed a hybrid-level dual-input Mamba framework that links feature fusion with final decisions. On the primary public benchmark, it reached 81.74 percent mean average precision at the standard overlap threshold and 56.62 percent across the full threshold range. Tests on two other public benchmark datasets further indicated that the approach also works across widely different viewpoints, object types, and imaging conditions.

One input stream carries texture, edges, and scene detail, while the other preserves object contours when illumination is weak. Together they offer fuller evidence, but fusion does not automatically improve detection. Existing methods often blend the streams before state space processing, so complementary cues, repeated information, and conflicting responses enter the same representation. Once their sources are blurred, the model cannot easily decide which branch to trust. This matters when the quality of the two inputs can differ sharply.

The framework organizes fusion in three connected stages. First, a dual-branch wavelet attention module compares feature maps and identifies areas where one source can add useful information to the other. It enhances these areas while limiting repeated background responses. Second, a binary fusion Mamba module sends features from the first input, features from the second input, and their interactions into state equations as separate inputs. Their origins therefore remain distinguishable during long-range modeling. Third, a three-branch decision head reviews predictions from the two input streams and the fused stream. It adjusts their influence according to scene conditions and the agreement among branches before producing the final boxes and class labels.

The evaluation included repeated trials. The researchers compared the complete system with representative single-source and multimodal detectors, repeated the component tests under different random initializations, and examined different subsets separately. Removing individual modules reduced detection accuracy, while combining feature selection with dual-input modeling produced the main improvement. The decision stage added a further gain and had its clearest effect in challenging subsets, when one branch retained stronger object responses. This result supports the design choice to connect feature fusion with a later check on prediction consistency.

Cross-dataset experiments used the same overall network design on two other public benchmark datasets, changing only the number of classes and output heads. The datasets differ from the primary benchmark in viewpoint and scene content, so they provide a practical test of whether the fusion mechanism depends on one benchmark. Quantitative comparisons placed the framework among the leading methods, and the visual examples showed fewer missed detections and false positives in challenging scenes. These observations give the numerical results a concrete interpretation without implying that performance will transfer unchanged to every data source or environment.

The study also identifies limits. The current dual-backbone network remains too large and slow for strict real-time use on resource-constrained platforms. Results vary by object class, with the less represented Freight category remaining more difficult than common classes. The transfer tests cover three datasets rather than unrestricted operating conditions, and they use registered input image pairs. Future work will therefore focus on lighter dual-input modules and continuous image sequences, where temporal information and agreement between adjacent frames may help stabilize predictions. Broader tests will be needed before the approach is used with new data sources, scene types, or computing hardware.

Contact

WANG Yongcheng

Changchun Institute of Optics, Fine Mechanics and Physics

E-mail:




       Copyright @ 吉ICP备06002510号 2007 CIOMP130033