Feature estimations based correlation distillation for incremental image retrieval
Chen, Wei; Liu, Yu; Pu, Nan; Wang, Weiping; Liu, Li; Lew, Michael S. (2021-04-21)
W. Chen, Y. Liu, N. Pu, W. Wang, L. Liu and M. S. Lew, "Feature Estimations Based Correlation Distillation for Incremental Image Retrieval," in IEEE Transactions on Multimedia, vol. 24, pp. 1844-1856, 2022, doi: 10.1109/TMM.2021.3073279.
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Deep learning for fine-grained image retrieval in an incremental context is less investigated. In this paper, we explore this task to realize the model’s continuous retrieval ability. That means, the model enables to perform well on new incoming data and reduce forgetting of the knowledge learned on preceding old tasks. For this purpose, we distill semantic correlations knowledge among the representations extracted from the new data only so as to regularize the parameters updates using the teacher-student framework. In particular, for the case of learning multiple tasks sequentially, aside from the correlations distilled from the penultimate model, we estimate the representations for all prior models and further their semantic correlations by using the representations extracted from the new data. To this end, the estimated correlations are used as an additional regularization and further prevent catastrophic forgetting over all previous tasks, and it is unnecessary to save the stream of models trained on these tasks. Extensive experiments demonstrate that the proposed method performs favorably for retaining performance on the already-trained old tasks and achieving good accuracy on the current task when new data are added at once or sequentially.
- Avoin saatavuus