Abstract
Recently, deep hashing methods play a pivotal role in image retrieval tasks by combining advanced convolutional neural networks (CNNs) with efficient hashing. Meanwhile, second-order representations of deep convolutional activations have been established to effectively improve network performance in various computer vision applications. In this work, to obtain more compact hash codes, we propose a supervised deep second-order covariance hashing (SDSoCH) method by combining deep hashing with second-order statistic model. SDSoCH utilizes a powerful covariance pooling to model the second-order statistics of convolutional features, which is naturally integrated into the existing point-wise hashing network in an end-to-end manner. The embedded covariance pooling operation well captures the interaction of convolutional features and produces global feature representations with more discriminant capability, leading to the more informative hash codes. Extensive experiments conducted on two benchmarks demonstrate that the proposed SDSoCH outperforms its first-order counterparts and achieves superior retrieval performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gong, Y., Lazebnik, S., Gordo, A., et al.: Iterative quantization: a procrustean approach to learning binary codes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 817–824 (2011)
Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Advances in Neural Information Processing Systems (NIPS), pp. 1753–1760 (2009)
Shen, F., Shen, C., Liu, W., et al.: Supervised discrete hashing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 37–45 (2015)
Xia, R., Pan, Y., Lai, H., et al.: Supervised hashing for image retrieval via image representation learning. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), pp. 2156–2162 (2014)
Lin, K., Yang, H.F., Hsiao, J.H., et al.: Deep learning of binary hash codes for fast image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 27–35 (2015)
Li, W.J., Wang, S., Kang W.C.: Feature learning based deep supervised hashing with pairwise labels. In: The Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), pp. 1711–1717 (2016)
Wang, X.F., Shi, Y., Kitani, K.M.: Deep supervised hashing with triplet labels. In: Asian Conference on Computer Vision (ACCV-16), pp. 70–84 (2016)
Lai, H., Pan, Y., Liu, Y., et al.: Simultaneous feature learning and hash coding with deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3270–3278 (2015)
Liu, H.M., Wang, R.P., Shan, S.G., et al.: Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2064–2072 (2016)
Zhu, H., Long, M.S., Wang, J.M., et al.: Deep hashing network for efficient similarity retrieval. In: The Thirtieth AAAI Conference on Artificial Intelligence (AAAI), pp. 2415–2421 (2016)
Cao, Y., Long, M.S., Wang, J.M., Wen, Q., et al.: Deep Quantization Network for efficient image retrieval. In: Thirtieth AAAI Conference on Artificial Intelligence (AAAI), pp. 3457–3463 (2016)
Jiang, Q.Y., Li, W.J.: Asymmetric deep supervised hashing. In: Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), pp. 3342–3349 (2018)
Zhou, M., Zeng, X.H., Chen, A.Z.: Deep forest hashing for image retrieval. Pattern Recogn. 95, 114–127 (2019)
Cao, Y., Liu, B., Long, M., et al.: HashGAN: deep learning to hash with pair conditional Wasserstein GAN. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1287–1296 (2018)
Ionescu, C., Vantzos, O., Sminchisescu, C.: Matrix backpropagation for deep networks with structured layers. In: IEEE International Conference on Computer Vision (ICCV), pp. 2965–2973 (2015)
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: IEEE international Conference on Computer Vision (ICCV), pp. 1449–1457 (2015)
Valmadre, J., Bertinetto, L., Henriques, J., et al.: End-to-end representation learning for correlation filter based tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2805–2813 (2017)
Arnab, A., Jayasumana, S., Zheng, S., Torr, P.H.S.: Higher order conditional random fields in deep neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 524–540. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_33
Sun, Q.L., Wang, Q.L., Zhang, J.X., et al.: Hyperlayer bilinear pooling with application to fine-grained categorization and image retrieval. Neurocomputing 282, 174–183 (2018)
Wu, Y., Sun, Q.L., Hou, Y.Q., et al.: Deep covariance estimation hashing. IEEE Access 7, 113223–113234 (2019)
Li, P.H., Xie, J.T., Wang, Q.L., et al.: Is second-order information helpful for large-scale visual recognition? In: IEEE International Conference on Computer Vision (ICCV), pp. 2089–2097 (2017)
Chatfield, K., Simonyan, K., Vedaldi, A., et al.: Return of the devil in the details: delving deep into convolutional net. In: British Machine Vision Conference (BMVC) (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)
He, K.M., Zhang, X.Y., Ren, Q.S., et al. Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Yang, H.F., Lin, K., Chen, C.S.: Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 437–451 (2018)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, May 2009
Chua, T.-S., Tang, J., Hong, R., et al.: NUS-WIDE: a real-world web image database from National University of Singapore. In: International Conference on Image and Video Retrieval (CIVR), pp. 368–375 (2009)
Acknowledgements
This work was partially supported by the National Key R&D Program of China (2018YFC0910506), the National Natural Science Foundation of China (61972062), the Natural Science Foundation of Liaoning Province (2019-MS-011), the Key R&D Program of Liaoning Province (2019 JH2/10100030) and the Liaoning BaiQianWan Talents Program.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, Q., Wu, Y., Zhang, J., Zhang, H., Che, C., Shan, L. (2020). Supervised Deep Second-Order Covariance Hashing for Image Retrieval. In: Zeng, J., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2020. Communications in Computer and Information Science, vol 1257. Springer, Singapore. https://doi.org/10.1007/978-981-15-7981-3_35
Download citation
DOI: https://doi.org/10.1007/978-981-15-7981-3_35
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7980-6
Online ISBN: 978-981-15-7981-3
eBook Packages: Computer ScienceComputer Science (R0)