Hierarchical Label Auto-Labeling and Relationship Constraints for Multi-Granularity Image Classification

Hierarchical multi-Granularity classification HMC aims to assign each object a label with multiple granularities from coarse to fine, focusing on the hierarchical structure of the label encoding. However, obtaining multi-granularity image labels through extensive manual labeling by experts is both costly and impractical for large-scale Fine-grained visual classification FGVC datasets and new scenarios. In this paper, we propose a hierarchical label auto-labeling clustering algorithm HLA to automatically generate hierarchical multi-granularity image labels. Additionally, we introduce a hierarchical constraint loss HCL and propose a hierarchical prediction constraint loss HPCL to constrain the relationship between different hierarchies. Extensive experiments on three commonly used FGVC datasets demonstrate that the proposed HLA can obtain similar performance with manual label method on CUB-200-2011, FGVC-Aircraft and Stanford Cars datasets. The introduced HCL and HPCL achieves promising performance on multi-granularity image classification datasets. Meanwhile, the consistent improvement on all object Re-identification tasks demonstrates the effectiveness of our method.