Abstract
The vast majority of Shape-from-Polarization (SfP) methods work under the oversimplified assumption of using orthographic cameras. Indeed, it is still unclear how Stokes vector projection behaves when the incoming rays are not orthogonal to the image plane. In this paper, we try to answer this question with a new geometric model describing how a general projective camera captures the light polarization state. Based on the optical properties of a tilted polarizer, our model is implemented as a pre-processing operation acting on raw images, and a scene-independent rotation of the reconstructed normal field. Moreover, our model is consistent with state-of-the-art forward and inverse renderers (as Mitsuba3 and ART), intrinsically enforces physical constraints among the captured channels, and handles the demosaicing of DoFP sensors. Experiments on existing and new datasets demonstrate the accuracy of the model when applied to commercially available polarimetric cameras.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.
Notes
Eq. (4) holds since \(S_0^2=S_1^2+S_2^2+S_3^2\) for completely polarized light.
Circular polarization is relatively rare in nature (see Cronin and Marshall (2011)) and therefore it is usually not accounted for.
To be precise, \(-\vec {r}_{z_j}\) is the true direction but changing the sign will not affect the orientation and Mueller calculus still applies.
Since our dataset uses a different plane material than the one used for PPA, we estimated a different set of parameters for each dataset.
References
Atkinson, G. A., & Hancock, E. R. (2005). Multi-view surface reconstruction using polarization. In: ICCV, vol. 2, p. 3
Atkinson, G. A., & Hancock, E. R. (2006). Recovery of surface orientation from diffuse polarization. IEEE transactions on image processing, 15(6), 1653–1664.
Ba, Y., Gilbert, A., Wang, F., Yang, J., Chen, R., Wang, Y., Yan, L., Shi, B., & Kadambi, A. (2020). Deep shape from polarization. In: European Conference on Computer Vision, pp. 554–571 . Springer
Baek, S.-H., Jeon, D. S., Tong, X., & Kim, M. H. (2018). Simultaneous acquisition of polarimetric svbrdf and normals. ACM Trans. Graph., 37(6), 268–1.
Bass, M., DeCusatis, C., Enoch, J. M., Lakshminarayanan, V., Li, G., MacDonald, C., Mahajan, V. N., & Van Stryland, E. (2009). Handbook of Optics, Third Edition Volume I: Geometrical and Physical Optics, Polarized Light, Components and Instruments(set). Handbook of Optics,
Chen, G., He, L., Guan, Y., & Zhang, H. (2022). Perspective phase angle model for polarimetric 3d reconstruction. In: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part II, pp. 398–414 Springer
Chen, L., Zheng, Y., Subpa-Asa, A., & Sato, I. (2018). Polarimetric three-view geometry. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 20–36
Chen, H., & Wolff, L. B. (1998). Polarization phase-based method for material classification in computer vision. International Journal of Computer Vision, 28(1), 73–83.
Collett, E. (2005). Field Guide to Polarization. Field Guide Series,
Cronin, T. W., & Marshall, J. (2011). Patterns and properties of polarized light in air and water. Philosophical Transactions of the Royal Society B: Biological Sciences, 366(1565), 619–626.
Cui, Z., Gu, J., Shi, B., Tan, P., & Kautz, J. (2017). Polarimetric multi-view stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1558–1567
Cui, Z., Larsson, V., & Pollefeys, M. (2019). Polarimetric relative pose estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2671–2680
Fatima, T., Pistellato, M., Torsello, A., & Bergamasco, F. (2022). One-shot hdr imaging via stereo pfa cameras. In: International Conference on Image Analysis and Processing, pp. 467–478 Springer
Fukao, Y., Kawahara, R., Nobuhara, S., & Nishino, K. (2021). Polarimetric normal stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 682–690
Goldstein, D. H. (2017). Polarized Light,
Huynh, C. P., Robles-Kelly, A., & Hancock, E. (2010). Shape and refractive index recovery from single-view polarisation images. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1229–1236 IEEE
Huynh, C. P., Robles-Kelly, A., & Hancock, E. R. (2013). Shape and refractive index from single-view spectro-polarimetric images. International journal of computer vision, 101(1), 64–94.
Ichikawa, T., Purri, M., Kawahara, R., Nobuhara, S., Dana, K., & Nishino, K. (2021). Shape from sky: Polarimetric normal recovery under the sky. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14832–14841
Jakob, W., Speierer, S., Roussel, N., Nimier-David, M., Vicini, D., Zeltner, T., Nicolet, B., Crespo, M., Leroy, V., & Zhang, Z. (2022). Mitsuba 3 Renderer. https://mitsuba-renderer.org
Kadambi, A., Taamazyan, V., Shi, B., & Raskar, R. (2017). Depth sensing using geometrically constrained polarization normals. International Journal of Computer Vision, 125(1–3), 34–51.
Korger, J., Kolb, T., Banzer, P., Aiello, A., Wittmann, C., Marquardt, C., & Leuchs, G. (2013). The polarization properties of a tilted polarizer. Optics express, 21(22), 27032–27042.
Kupinski, M. K., Bradley, C. L., Diner, D. J., Xu, F., & Chipman, R. A. (2019). Angle of linear polarization images of outdoor scenes. Optical Engineering, 58(8), 082419.
Lei, C., Qi, C., Xie, J., Fan, N., Koltun, V., & Chen, Q. (2022). Shape from polarization for complex scenes in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12632–12641
Lu, J., Ji, Y., Yu, J., & Ye, J. (2019). Mirror surface reconstruction using polarization field. In: 2019 IEEE International Conference on Computational Photography (ICCP), pp. 1–9 IEEE
Mahmoud, A. H., El-Melegy, M. T., & Farag, A. A. (2012). Direct method for shape recovery from polarization and shading. In: 2012 19th IEEE International Conference on Image Processing, pp. 1769–1772 IEEE
Meriaudeau, F., Ferraton, M., Stolz, C., Morel, O., & Bigué, L. (2008). Polarization imaging for industrial inspection. In: Image Processing: Machine Vision Applications, vol. 6813, pp. 72–81 SPIE
Miyazaki, D., Tan, R. T., Hara, K., & Ikeuchi, K. (2003). Polarization-based inverse rendering from a single view. In: Computer Vision, IEEE International Conference On, vol. 3, pp. 982–982 IEEE Computer Society
Miyazaki, D., & Ikeuchi, K. (2007). Shape estimation of transparent objects by using inverse polarization ray tracing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 2018–2030.
Morel, O., & Gorria, P. (2006). Polarization imaging for 3d inspection of highly reflective metallic objects. Optics and Spectroscopy, 101, 11–17.
Nayar, S. K., Fang, X.-S., & Boult, T. (1997). Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163–186.
Ngo Thanh, T., Nagahara, H., & Taniguchi, R. -i. (2015). Shape and light directions from shading and polarization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2310–2318
Pistellato, M., Fatima, T., & Wimmer, M. (2023). Exploiting light polarization for deep hdr imaging from a single exposure. Sensors, 23(12), 5370.
Rahmann, S. (2000). Polarization images: a geometric interpretation for shape analysis. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 3, pp. 538–542 IEEE
Rahmann, S., & Canterakis, N. (2001). Reconstruction of specular surfaces using polarization imaging. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, p. IEEE
Shakeri, M., Loo, S. Y., Zhang, H., & Hu, K. (2021). Polarimetric monocular dense mapping using relative deep depth prior. IEEE Robotics and Automation Letters, 6(3), 4512–4519.
Smith, W. A., Ramamoorthi, R., & Tozza, S. (2018). Height-from-polarisation with unknown lighting or albedo. IEEE transactions on pattern analysis and machine intelligence, 41(12), 2875–2888.
Strutt, J. W. (1871). Xv. on the light from the sky, its polarization and colour. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 41(271), 107–120.
Taamazyan, V., Kadambi, A., & Raskar, R. (2016). Shape from mixed polarization. arXiv preprint arXiv:1605.02066
Tominaga, S., & Kimachi, A. (2008). Polarization imaging for material classification. Optical Engineering, 47(12), 123201.
Umow, V. N. (1905). Chromatische depolarisation durch lichtzerstreuung. Phys. Z, 6, 674–676.
Wilkie, A. The Advanced Rendering Toolkit. http://cgg.mff.cuni.cz/ART
Wolff, L. B. (1990). Polarization-based material classification from specular reflection. IEEE transactions on pattern analysis and machine intelligence, 12(11), 1059–1071.
Wolff, L. B., & Boult, T. E. (1993). Constraining object features using a polarization reflectance model. Phys. Based Vis. Princ. Pract. Radiom, 1, 167.
Yang, L., Tan, F., Li, A., Cui, Z., Furukawa, Y., & Tan, P. (2018). Polarimetric dense monocular slam. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3857–3866
Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(11), 1330–1334.
Zhang, J., Luo, H., Hui, B., & Chang, Z. (2016). Image interpolation for division of focal plane polarimeters with intensity correlation. Optics express, 24(18), 20799–20807.
Zhao, J., Monno, Y., & Okutomi, M. (2020). Polarimetric multi-view inverse rendering. In: European Conference on Computer Vision, pp. 85–102 . Springer
Acknowledgements
This work was partially supported by DAIS - Ca’ Foscari University of Venice within the IRIDE program.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Svetlana Lazebnik.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pistellato, M., Bergamasco, F. A Geometric Model for Polarization Imaging on Projective Cameras. Int J Comput Vis 132, 4688–4702 (2024). https://doi.org/10.1007/s11263-024-02119-2
Received:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s11263-024-02119-2