鱼眼相机内参模型fisheye详解

本博客主要从fisheye论文角度，详细介绍该内参模型。

参考论文：A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses

1. 论文总述

本论文介绍了一种可以建模传统相机和鱼眼相机的通用的内参模型（radially symmetric projection model）：Kannala Brandt 模型 / opencv中的fisheye / kalibr中的 pinhole + equidistant 都是指该模型。
成像示意如下：

In this paper, we concentrate on accurate geometric modeling of real cameras. We propose a novel calibration method for fish-eye lenses that requires that the camera observes a planar calibration pattern. The calibration method is based on a generic camera model that will be shown to be suitable for different kind of omnidirectional cameras as well as for conventional
cameras.

2. pinhole+radtan无法建模鱼眼相机

（1）透视映射无法将fisheye映射到有限的平面上

The pinhole camera model accompanied with lens distortion models is a fair approximation for most conventional cameras with narrow-angle or even wide-angle lenses [1], [6], [7]. But it is still not suitable for fish-eye lens cameras. Fish-eye lenses are designed to cover the whole hemispherical field in front of the camera and the angle of view is very large, about 180度. Moreover, it is impossible to project the hemispherical field of view on a finite image plane by a perspective projection so fish-eye lenses are designed to obey some other projection model.
This is the reason why the inherent distortion of a fish-eye lens should not be considered only as a deviation from the pinhole model [14].

3. 内参模型

3.1 Radially Symmetric Model

• r(θ)是一种投影模型，只不过建模的时候重点考虑fisheye的特性，建模了径向畸变？
• θ表示入射光线和Z轴的夹角，θ = acos(Zc / norm)，norm = sqrt(Xc^2 + Yc^2 + Xc^2)
• r(θ)表示映射到图像坐标系上的点到中心点的距离，r = sqrt(x^2 + y^2)
• φ表示相机坐标系下的入射光线和X轴的夹角，φ = atan(Yc / Xc)
• 根据（7）式，这个模型有个假设：就是相机坐标系下的φ和图像坐标系下的φ是相等的
• （2）-（5）已经建模了焦距f，但在（6）式时又将其去掉了，还是将其建模到了内参fx中？
• 根据鱼眼相机的特性，是用（2）-（5）来建模的，但作者为了通用性，统一用（6）式来建模，多项式近似，一般都是用前5项

3.2 Full Model

• 由于镜头的制造工艺等，并不是所有的鱼眼相机都是Radially Symmetric，因此作者又额外建模了径向畸变和切向畸变（14个参数），p9 + 14 = p23
• 但在各大开源工具对该模型的复现中，一般只用p9模型，并不用p23，可能是因为如今的镜头工艺比较成熟了。
  

3.3 affine transformation

• 式（10）添加了3.2的畸变模型
• 式（11）从图像坐标系变化到像素坐标系，这里的mu 并不是fx , 是dx ?
• 式（12）表示整体的内参模型
  
  Pc = A * D * F

3.4 Backward Model

• 一般来说：projection是指将相机坐标系的一个3D point映射到2D像素坐标系中
• 一个内参模型的精确度还要看它的back-projection，即从2D 映射到 3D。

4. 标定

• 估计模型的参数
• 本文提供的内参估计算法如果用圆形标靶，精度更高
• 具体的算法还是去paper里具体看

Next we describe a procedure for estimating the parameters of the camera model. The calibration method is based on viewing a planar object which contains control points in known positions. The advantage over the previous approaches is that also fish-eye lenses, possibly having a field of view larger than 180度, can be calibrated by simply viewing a planar pattern. In addition, a good accuracy can be achieved if circular control points are used, asdescribed in Section IV-B.

4.1 ORIFL190-3 lens 可以被标定

• 标定靶

It was originally obtained by capturing a single image of a calibration object consisting of two orthogonal planes, each with 256 circular control points. The camera.

• 实验结果

• 190度的fisheye可以被标定的原因：（1）标靶不一样，采集的数据可能没有畸变特别严重的（2）标定算法的影响？ 可能不太大

5. 内参结果验证

• 利用额外的test images，根据train里的images估计内参之后，用该内参来计算test images的外参，然后算重投影误差rms
  

参考文献

[1] 鱼眼镜头是怎么「鱼眼」的
[2] opencv fisheye示例
[3] camodocal: fisheye内参估计算法复现（和本paper提供的算法不一致）
[4] kalibr: pinhole+equi表示fisheye