Hostname: page-component-7f64f4797f-d7bbv Total loading time: 0 Render date: 2025-11-10T22:08:49.382Z Has data issue: false hasContentIssue false
  • English
  • Français

Monocular Vision-based Sense and Avoid of UAV Using Nonlinear Model Predictive Control

Published online by Cambridge University Press:  06 March 2019

Yizhai Zhang ,
Wenhui Wang ,
Panfeng Huang  and
Zainan Jiang
Yizhai Zhang
Affiliation:
Research Center of Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, Shaanxi, 710072, China. E-mails: zhangyizhai@nwpu.edu.cn, wenhuiwang024@mail.nwpu.edu.cn
Wenhui Wang
Affiliation:
Research Center of Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, Shaanxi, 710072, China. E-mails: zhangyizhai@nwpu.edu.cn, wenhuiwang024@mail.nwpu.edu.cn
Panfeng Huang*
Affiliation:
Research Center of Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an, Shaanxi, 710072, China. E-mails: zhangyizhai@nwpu.edu.cn, wenhuiwang024@mail.nwpu.edu.cn
Zainan Jiang*
Affiliation:
State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, 150080, China
*
*Panfeng Huang and Zainan Jiang are both corresponding authors. E-mails: pfhuang@nwpu.edu.cn, jiangzainan@hit.edu.cn
*Panfeng Huang and Zainan Jiang are both corresponding authors. E-mails: pfhuang@nwpu.edu.cn, jiangzainan@hit.edu.cn
Get access Rights & Permissions [Opens in a new window]

Summary

The potential use of onboard vision sensors (e.g., cameras) has long been recognized for the Sense and Avoid (SAA) of unmanned aerial vehicles (UAVs), especially for micro UAVs with limited payload capacity. However, vision-based SAA for UAVs is extremely challenging because vision sensors usually have limitations on accurate distance information measuring. In this paper, we propose a monocular vision-based UAV SAA approach. Within the approach, the host UAV can accurately and efficiently avoid a noncooperative intruder only through angle measurements and perform maneuvers for optimal tradeoff among target motion estimation, intruder avoidance, and trajectory tracking. We realize this feature by explicitly integrating a target tracking filter into a nonlinear model predictive controller. The effectiveness of the proposed approach is verified through extensive simulations.

Keywords

Aerial roboticsNavigationControl of robotic systemsSense and avoidObstacle avoidance

Information

Type
Articles
Information
Robotica , Volume 37 , Issue 9 , September 2019 , pp. 1582 - 1594
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Oren, G. and Yerach, D., “Fast and efficient visible trajectories planning for the Dubins UAV model in 3D built-up environments,” Robotica 32(1), 143–163 (2014).Google Scholar
Prats, X., Delgado, L., Ramirez, J., Royo, P. and Pastor, E., “Requirements, issues, and challenges for sense and avoid in unmanned aircraft systems,” J. Aircr. 49(3), 677–687 (2012).CrossRefGoogle Scholar
Zeitlin, A. D., “Sense and avoid capability development challenges,” IEEE Aerosp. Electron. Syst. Mag. 25(10), 27–32 (2010).CrossRefGoogle Scholar
Melega, M., Lazarus, S., Savvaris, A. and Tsourdos, A., “Multiple threats sense and avoid algorithm for static and dynamic obstacles,” J. Intell. Robot. Syst. 77(1), 215–228 (2015).CrossRefGoogle Scholar
Wilson, M., Ryan, D., Bratanov, D., Wainwright, A., Ford, J., Cork, L. and Brouckaert, M., “Flight test and evaluation of a prototype sense and avoid system onboard a scaneagle unmanned aircraft,” IEEE Aerosp. Electron. Syst. Mag. 31(9), 6–15 (2016).CrossRefGoogle Scholar
Yu, X. and Zhang, Y., “Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects,” Prog. Aerosp. Sci. 74, 152-166 (2015).CrossRefGoogle Scholar
Fasano, G., Accado, D., Moccia, A. and Moroney, D., “Sense and avoid for unmanned aircraft systems,” IEEE Aerosp. Electron. Syst. Mag. 31(11), 82–110 (2016).CrossRefGoogle Scholar
Lee, H. C., “Implementation of collision avoidance system using TCAS II to UAVs,” IEEE Aerosp. Electron. Syst. Mag. 21(7), 8–13 (2006).Google Scholar
Stark, B., Stevenson, B. and Chen, Y. Q., “ADS-B for small unmanned aerial systems: Case study and regulatory practices,” In: International Conference on Unmanned Aircraft Systems (2013), pp. 152–159.Google Scholar
Accardo, D., Fasano, G., Forlenza, L. and Moccia, A., “Flight test of a radar-based tracking system for UAS sense and avoid,” IEEE Trans. Aerosp. Electron. Syst. 49(2), 1139–1160 (2013).CrossRefGoogle Scholar
Allistair, M., Rutherford, M. J., Michail, K. and Valavanis, K. P., “UAV-borne X-band radar for collision avoidance,” Robotica 32(1), 97–114 (2013).Google Scholar
Fasano, G., Accardo, D., Tirri, A. E., Moccia, A. and Lellis, E. D., “Radar/electro-optical data fusion for non-cooperative UAS sense and avoid,” Aerosp. Sci. Technol. 46(2), 436–450 (2015).CrossRefGoogle Scholar
Chamberlain, L., Scherer, S. and Singh, S., “Self-aware helicopters: Full-scale automated landing and obstacle avoidance in unmapped environments,” Ahs Forum (2011).Google Scholar
Kownacki, C., “A concept of laser scanner designed to realize 3D obstacle avoidance for a fixed-wing UAV,” Robotica 34(2), 243–257 (2016).CrossRefGoogle Scholar
Ramasamy, S., Sabatini, R., Gardi, A. and Liu, J., “LIDAR obstacle warning and avoidance system for unmanned aerial vehicle sense-and-avoid,” Aerosp. Sci. Technol. 55, 344–358 (2016).CrossRefGoogle Scholar
Mejias, L., Lai, J., Ford, J. J. and O’shea, P., “Demonstration of closed-loop airborne sense-and-avoid using machine vision,” IEEE Aerosp. Electron. Syst. Mag. 27(4), 4–7 (2012).CrossRefGoogle Scholar
Hugo, R., Sergio, S. and Rogelio, L., “Visual servoing applied to real-time stabilization of a multi-rotor UAV,” Robotica 30(7), 1203–1212 (2012).Google Scholar
Wang, H., Guo, D., Liang, X., Chen, W., Hu, G. and Leang, K. K., “Adaptive vision-based leader-follower formation control of mobile robots,” IEEE Trans. Ind. Electron. 64(4), 2893–2902 (2017).CrossRefGoogle Scholar
Wang, H., Liu, Y., Chen, W. and Wang, Z., “A new approach to dynamic eye-in-hand visual tracking using nonlinear observers,” IEEE/ASME Trans. Mechatron. 16(2), 387–394 (2011).CrossRefGoogle Scholar
Mejias, L., Mcfadyen, A. and Ford, J. J., “Sense and avoid technology developments at Queensland University of Technology,” IEEE Aerosp. Electron. Syst. Mag. 31(7), 28–37 (2016).CrossRefGoogle Scholar
Min, Y. and Min, Z. Z, “Unmanned aerial vehicle dynamic path planning in an uncertain environment,” Robotica 33(3), 611–621 (2015).Google Scholar
Suzuki, S., Ishii, T., Aida, Y., Fujisawa, Y., Iizuka, K. and Kawamura, T., “Collision-free guidance control of small unmanned helicopter using nonlinear model predictive control,” Sice J. Control Meas. Syst. Integr. 7(6), 347–355 (2014).CrossRefGoogle Scholar
Yang, X., Alvarez, L. M. and Bruggemann, T., “A 3D collision avoidance strategy for UAVs in a non-cooperative environment,” J. Intell. Robot. Syst. 70(1–4), 315–327 (2013).CrossRefGoogle Scholar
Yang, L., Pan, Q., Zhao, C. and Zhang, Y., “Vision-based UAV collision avoidance with 2D dynamic safety envelope,” IEEE Aerosp. Electron. Syst. Mag. 31(7), 16–26 (2016).Google Scholar
Huh, S., Cho, S., Jung, Y. and Shim, D. H., “Vision-based sense-and-avoid framework for unmanned aerial vehicles,” IEEE Trans. Aerosp. Electron. Syst. 51(4), 3427–3439 (2015).CrossRefGoogle Scholar
Nordlund, P. J. and Gustafsson, F., “Probabilistic noncooperative near mid-air collision avoidance,” IEEE Trans. Aerosp. Electron. Syst. 47(2), 1265–1276 (2011).CrossRefGoogle Scholar
Yu, H. and Beard, R.W., “Vision-based local-level frame mapping and planning in spherical coordinates for miniature air vehicles,” IEEE Trans. Control Syst. Technol. 21(3), 695–703 (2013).CrossRefGoogle Scholar
Nardone, S. C. and Aidala, V. J., “Observability criteria for bearings-only target motion analysis,” IEEE Trans. Aerosp. Electron. Syst. AES 17(2), 162–166 (1981).CrossRefGoogle Scholar
Aidala, V. J., “Kalman filter behavior in bearings-only tracking applications,” IEEE Trans. Aerosp. Electron. Syst. 15(1), 29–39 (1979).CrossRefGoogle Scholar
Aidala, V. J. and Hammel, S. E., “Utilization of modified polar coordinates for bearings-only tracking,” IEEE Trans. Autom. Control 28(3), 283–294 (1983).CrossRefGoogle Scholar
Passerieux, J. M. and Cappel, D. V., “Optimal observer maneuver for bearings-only tracking,” IEEE Trans. Aerosp. Electron. Syst. 34(3), 777–788 (1998).CrossRefGoogle Scholar
DiCairano, S., Park, H. and Kolmanovsky, I., “Model predictive control approach for guidance of spacecraft rendezvous and proximity maneuvering,” Int. J. Robust Nonlinear Control 22(12), 1398–1427 (2012).CrossRefGoogle Scholar
Weiss, A., Baldwin, M., Erwin, R. S. and Kolmanovsky, I., “Model predictive control for spacecraft rendezvous and docking: Strategies for handling constraints and case studies,” IEEE Trans. Control Syst. Technol. 23(4), 1638–1647 (2015).CrossRefGoogle Scholar
Ponda, S., Kolacinski, R. and Frazzoli, E., “Trajectory optimization for target localization using small unmanned aerial vehicles,” In: AIAA Guidance, Navigation, and Control Conference (2009).Google Scholar