这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

A Fast and Lightweight 3D Keypoint Detector

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Keypoint detection is crucial in many visual tasks, such as object recognition, shape retrieval, and 3D reconstruction, as labeling point data is labor-intensive or sometimes implausible. Nevertheless, it is challenging to quickly and accurately locate keypoints unsupervised from point clouds. This work proposes a fast and lightweight 3D keypoint detector that can efficiently and accurately detect keypoints from point clouds. Our method does not require a complex model learning process and generalizes well to new scenes. Specifically, we consider detecting keypoints a saliency detection problem for a point cloud. First, we propose a simple and effective distance measure to characterize the saliency of points in a point cloud. This distance describes geometrically essential points in the point cloud. Next, we present a regional saliency based on relative centroid distance representation that can globally characterize keypoints with regional visual information. Third, we combine geometric and semantic cues to generate a saliency map of the point cloud for determining stable 3D keypoints. We evaluate our method against existing approaches on four benchmark keypoint datasets to demonstrate its state-of-the-art performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  • Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., & Tai, C. L. (2020). D3Feat: Joint learning of dense detection and description of 3D local features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6359–6367).

  • Bai, Y., Wang, A., Kortylewski, A., & Yuille, A. (2023). CoKe: Contrastive learning for robust keypoint detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp. 65–74).

  • Barroso-Laguna, A., & Mikolajczyk, K. (2022). Key.Net: Keypoint detection by handcrafted and learned cnn filters revisited. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 698–711.

    Article  Google Scholar 

  • Boroson, E. R., & Ayanian, N. (2019). 3D keypoint repeatability for heterogeneous multi-robot SLAM. In International conference on robotics and automation (ICRA) (pp. 6337–6343). IEEE.

  • Castellani, U., Cristani, M., Fantoni, S., & Murino, V. (2008). Sparse points matching by combining 3d mesh saliency with statistical descriptors. Computer Graphics Forum, 27, 643–652.

    Article  Google Scholar 

  • Chen, H., & Bhanu, B. (2007). 3D free-form object recognition in range images using local surface patches. Pattern Recognition Letters, 28(10), 1252–1262.

    Article  Google Scholar 

  • Choi, S., Zhou, Q. Y., & Koltun, V. (2015). Robust reconstruction of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5556–5565).

  • Deng, X., Zuo, D., Zhang, Y., Cui, Z., Cheng, J., Tan, P., Chang, L., Pollefeys, M., Fanello, S., & Wang, H. (2022). Recurrent 3D hand pose estimation using cascaded pose-guided 3D alignments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 932–945.

    Article  Google Scholar 

  • Gao, Y., He, J., Zhang, T., Zhang, Z., & Zhang, Y. (2023). Dynamic keypoint detection network for image matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 14404–14419.

    Article  Google Scholar 

  • Geng, Z., Sun, K., Xiao, B., Zhang, Z., & Wang, J. (2021). Bottom-up human pose estimation via disentangled keypoint regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14676–14686).

  • Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649–677.

    Article  Google Scholar 

  • Hosmer, D. W., Hosmer, T., Le Cessie, S., & Lemeshow, S. (1997). A comparison of goodness-of-fit tests for the logistic regression model. Statistics inMmedicine, 16(9), 965–980.

    Google Scholar 

  • Hu, J., Mao, M., Bao, H., Zhang, G., & Cui, Z. (2024). CP-SLAM: Collaborative neural point-based SLAM system. Advances in Neural Information Processing Systems, 36.

  • Jelavic, E., Nubert, J., & Hutter, M. (2022). Open3D SLAM: Point cloud based mapping and localization for education. In Robotic perception and mapping: Emerging techniques, ICRA Workshop (p. 24). ETH Zurich, Robotic Systems Lab.

  • Lafferty, J., McCallum, A., & Pereira, F. C. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C]// The Eighteenth International Conference on Machine Learning(ICML). 1(2),1–8.

  • Lee, C. H., Varshney, A., & Jacobs, D. W. (2005). Mesh saliency. In ACM SIGGRAPH Papers (pp. 659–666).

  • Li, J., & Lee, G. H. (2019). USIP: Unsupervised stable interest point detection from 3D point clouds. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 361–370).

  • Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2015). SMPL: A skinned multi-person linear model. ACM Transactions on Graphics, 34(6), 248.

    Article  Google Scholar 

  • Lu, C., & Koniusz, P. (2022). Few-shot keypoint detection with uncertainty learning for unseen species. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 19416–19426).

  • Lu, F., Chen, G., Liu, Y., Qu, Z., & Knoll, A. (2020). RSKDD-Net: Random sample-based keypoint detector and descriptor. Advances in Neural Information Processing Systems, 33, 21297–21308.

    Google Scholar 

  • Luo, Z., Xue, W., Chae, J., & Fu, G. (2022). SKP: Semantic 3D keypoint detection for category-level robotic manipulation. IEEE Robotics and Automation Letters, 7(2), 5437–5444.

    Article  Google Scholar 

  • Pomerleau, F., Liu, M., Colas, F., & Siegwart, R. (2012). Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research, 31(14), 1705–1711.

    Article  Google Scholar 

  • Prakhya, S. M., Liu, B., & Lin, W. (2016). Detecting keypoint sets on 3d point clouds via histogram of normal orientations. Pattern Recognition Letters, 83, 42–48.

    Article  Google Scholar 

  • Rister, B., Horowitz, M. A., & Rubin, D. L. (2017). Volumetric image registration from invariant keypoints. IEEE Transactions on Image Processing, 26(10), 4900–4910.

    Article  MathSciNet  Google Scholar 

  • Shi, C., Chen, X., Huang, K., Xiao, J., Lu, H., & Stachniss, C. (2021). Keypoint matching for point cloud registration using multiplex dynamic graph attention networks. IEEE Robotics and Automation Letters, 6(4), 8221–8228.

    Article  Google Scholar 

  • Sipiran, I., & Bustos, B. (2011). Harris 3D: A robust extension of the Harris operator for interest point detection on 3d meshes. The Visual Computer, 27, 963–976.

    Article  Google Scholar 

  • Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. Computer Graphics Forum, 28, 1383–1392.

    Article  Google Scholar 

  • Taylor, Z. (2023). Find 3D normals and curvature. https://ww2.mathworks.cn/matlabcentral/fileexchange/48111-find-3d-normals-and-curvature

  • Teng, H., Chatziparaschis, D., Kan, X., Roy-Chowdhury, A. K., & Karydis, K. (2023). Centroid distance keypoint detector for colored point clouds. In Proceedings of the IEEE/CVF Winter conference on applications of computer vision (pp. 1196–1205).

  • Tinchev, G., Penate-Sanchez, A., & Fallon, M. (2021). Skd: Keypoint detection for point clouds using saliency estimation. IEEE Robotics and Automation Letters, 6(2), 3785–3792.

    Article  Google Scholar 

  • Tombari, F., Salti, S., & Di Stefano, L. (2013). Performance evaluation of 3d keypoint detectors. International Journal of Computer Vision, 102(1–3), 198–220.

    Article  Google Scholar 

  • Unnikrishnan, R., & Hebert, M. (2008). Multi-scale interest regions from unorganized point clouds. In IEEE Computer Society Conference on computer vision and pattern recognition workshops (pp. 1–8). IEEE.

  • Uy, M. A., & Lee, G. H. (2018). PointNetVLAD: Deep point cloud based retrieval for large-scale place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4470–4479).

  • Wang, H., Guo, J., Yan, D. M., Quan, W., & Zhang, X. (2018). Learning 3D keypoint descriptors for non-rigid shape matching. In Proceedings of the European conference on computer vision (ECCV) (pp. 3–19).

  • Wang, Y., Yan, C., Feng, Y., Du, S., Dai, Q., & Gao, Y. (2022). STORM: Structure-based overlap matching for partial point cloud registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 1135–1149.

    Article  Google Scholar 

  • Wimmer, T., Wonka, P., & Ovsjanikov, M. (2024). Back to 3D: Few-shot 3d keypoint detection with back-projected 2d features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4154–4164).

  • Yang, H., & Pavone, M. (2023). Object pose estimation with statistical guarantees: Conformal keypoint detection and geometric uncertainty propagation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8947–8958).

  • Yang, J., Xian, K., Xiao, Y., & Cao, Z. (2017). Performance evaluation of 3D correspondence grouping algorithms. In 2017 international conference on 3D vision (3DV) (pp. 467–476). IEEE.

  • Yew, Z. J., & Lee, G. H. (2018). 3DFeat-Net: Weakly supervised local 3D features for point cloud registration. In Proceedings of the European conference on computer vision (ECCV) (pp. 607–623).

  • Yi, L., Su, H., Guo, X., & Guibas, L. J. (2017). SyncspecCNN: Synchronized spectral CNN for 3D shape segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2282–2290).

  • You, Y., Liu, W., Ze, Y., Li, Y. L., Wang, W., & Lu, C. (2022). UKPGAN: A general self-supervised keypoint detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 17042–17051).

  • You, Y., Lou, Y., Li, C., Cheng, Z., Li, L., Ma, L., Lu, C., & Wang, W. (2020). KeypointNet: A large-scale 3D keypoint dataset aggregated from numerous human annotations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13647–13656).

  • Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In IEEE conference on computer vision and pattern recognition (pp. 373–380). IEEE.

  • Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017). 3DMatch: Learning local geometric descriptors from RGB-D reconstructions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1802–1811).

  • Zhang, J., Chen, Z., & Tao, D. (2021). Towards high performance human keypoint detection. International Journal of Computer Vision, 129(9), 2639–2662.

    Article  Google Scholar 

  • Zhang, R., Zhang, C., Di, Y., Manhardt, F., Liu, X., Tombari, F., & Ji, X. (2024). Kp-red: Exploiting semantic keypoints for joint 3d shape retrieval and deformation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 20540–20550).

  • Zheng, Q., Gong, M., You, X., & Tao, D. (2022). A unified B-Spline framework for scale-invariant keypoint detection. International Journal of Computer Vision, 130(3), 777–799.

  • Zheng, T., Chen, C., Yuan, J., Li, B., & Ren, K. (2019). Pointcloud saliency maps. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 1598–1606).

  • Zhong, Y. (2009). Intrinsic shape signatures: A shape descriptor for 3D object recognition. In IEEE 12th international conference on computer vision workshops, ICCV Workshops (pp. 689–696). IEEE.

  • Zhong, C., You, P., Chen, X., Zhao, H., Sun, F., Zhou, G., Mu, X., Gan, C., & Huang, W. (2022). SNAKE: Shape-aware neural 3d keypoint field. Advances in Neural Information Processing Systems, 35, 7052–7064.

  • Zhong, C., Zheng, Y., Zheng, Y., Zhao, H., Yi, L., Mu, X., Wang, L., Li, P., Zhou, G., Yang, C., Zhang, X., & Zhao, J. (2023). 3D implicit transporter for temporally consistent keypoint discovery. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3869–3880).

  • Zohaib, M., & Del Bue, A. (2023). SC3K: Self-supervised and coherent 3D keypoints estimation from rotated, noisy, and decimated point cloud data. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 22509–22519).

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62106227, 62272419, 62402449, 61902159). the Teacher Professional Development Project for Domestic Visiting Scholars in 2023 (Project No. FX2023007), the National Natural Science Foundation of China Joint Fund for Regional Innovation and Development Key Support Projects (Project No. U22A20102), the Zhejiang Province Vanguard Leading Goose R&D Key Project No. 2023C01150), the Zhejiang Provincial Natural Science Foundation of China (Project No. LZ22F020010), the Open Project Program of the State Key Laboratory of CAD&CG (Grant No. A2413), Zhejiang University, and the China Postdoctoral Science Foundation (Project No. 2023M743132). We thank LetPub (www.letpub.com) and professor Daniel Morris from Michigan State University for their linguistic assistance while preparing this manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Zhejiang Normal University, Jinhua, China

    Chengzhuan Yang, Yunliang Jiang & Zhonglong Zheng

  2. School of Computer Engineering, Jiangsu University of Technology, Changzhou, China

    Qian Yu

  3. Laboratory of Cognitive Algorithm and Model, School of Computer Science, Fudan University, Shanghai, China

    Hui Wei

  4. School of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Fei Wu

  5. University of California at Merced, Merced, CA, USA

    Ming-Hsuan Yang

Authors
  1. Chengzhuan Yang
    View author publications

    Search author on:PubMed Google Scholar

  2. Qian Yu
    View author publications

    Search author on:PubMed Google Scholar

  3. Hui Wei
    View author publications

    Search author on:PubMed Google Scholar

  4. Fei Wu
    View author publications

    Search author on:PubMed Google Scholar

  5. Yunliang Jiang
    View author publications

    Search author on:PubMed Google Scholar

  6. Zhonglong Zheng
    View author publications

    Search author on:PubMed Google Scholar

  7. Ming-Hsuan Yang
    View author publications

    Search author on:PubMed Google Scholar

Corresponding authors

Correspondence to Zhonglong Zheng or Ming-Hsuan Yang.

Additional information

Communicated by Kwang Moo Yi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, C., Yu, Q., Wei, H. et al. A Fast and Lightweight 3D Keypoint Detector. Int J Comput Vis 133, 5216–5237 (2025). https://doi.org/10.1007/s11263-025-02425-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-025-02425-3

Keywords