A Method that Utilizes Negative Queries in Object Detection

DEtection TRansformer DETR is an end-to-end object detection model based on transformers that generates multiple object queries per ground truth box and selects the best prediction through matching. However, studies have shown that many object queries rejected by the Hungarian matching algorithm still focus on foreground elements. Leveraging these potentially useful negative queries in DETR has thus emerged as a promising research direction.In this paper, I propose FUSION-DETR, a novel model that introduces a one-to-many matching strategy by grouping queries while preserving DETR's traditional one-to-one matching. This approach utilizes the foreground information embedded in queries rejected by Hungarian matching as negative samples. Furthermore, during training, the model dynamically assigns weights to the one-to-many loss using a clustering-based method, enhancing its robustness.Experiments demonstrate that the FUSION-DETR approach improves Deformable-DETR by 3.0 mAP50 on the BDD-100K datasets and achieves 70.5 mAP50 on COCO2017, outperforming existing DETR-based models incorporating one-to-many assignment.