相机内参模型Mei/omni-directional详解

本博客主要是从Mei的论文出发，详细介绍该模型。

参考论文：Single View Point Omnidirectional Camera Calibration from Planar Grids

1. 论文总述

本论文主要是介绍了一种实用的针对全向相机的实用的内参模型Mei，并不完全等于UCM+radtan，因为除了radtan畸变模型，作者还定义了一个更加泛化的内参矩阵K，fx对应fx*η=γ（同fy）。

Mei模型的提出主要是建模catadioptric相机（即camera+mirror），但作者也证明了可以近似dioptric鱼眼相机（fisheye），而且笔者做实验也发现针对鱼眼相机，Mei模型的标定结果确实也很稳定，如果标定数据采集的规范，重投影误差基本都在0.5以下。

本论文的重点不在于内参模型（基本都是建立在前人的工作），而是基于此模型提出一个方便、稳定的标定算法，只用棋盘格就可以完成标定。

注：Mei模型的复现请看参考文献小节给出的链接。

This paper presents a flexible approach for calibrating
omnidirectional single viewpoint sensors from planar
grids. Current approaches in the field are either based on theoretical properties and do not take into account important factors such as misalignment
or camera-lens distortion or over-parametrised which leads to
minimisation problems that are difficult to solve. Recent techniques
based on polynomial approximations lead to impractical
calibration methods. Our model is based on an exact theoretical
projection function to which we add well identified parameters
to model real-world errors. This leads to a full methodology
from the initialisation of the intrinsic parameters to the general
calibration. We also discuss the validity of the approach for fisheye
and spherical models.

注：

• Mei模型的参数都有其物理意义
• 提出Scaramuzza模型不是很实用

2. Scaramuzza模型的不足

• 映射模型的参数初始值难以获得
• 在边界处，映射不准

In [10], the authors propose a method relying on a polynomial
approximation of the projection function. With this model,
initial values of the projection function are difficult to obtain
so the user has to select each point of the calibration grid
independently for the calibration.
We will show that by using
an exact model to which we add small errors, only four points need to be selected for each calibration grid. The parameters that appear in the proposed model can also be easily interpreted in terms of the optical quality of the sensor.

We may note that polynomial approximations are often
valid only locally and badly approximate the projection
around the edges. This bias will have a negative impact for
example when estimating the motion of the camera using a
maximum likelihood estimation under the assumption of a
Gaussian distribution of the error.

3. Mei畸变模型参数的选择

• 太多的参数容易有过多的局部最优，最小化困难
• Mei模型只建模Fig2所示的畸变
  

Figure 1 presents the different parameters that could be
taken into account for example in the case of a parabolic mirror
with a telecentric lens. Gonzalez-Barbosa [5] describes
a calibration method to estimate all of these parameters.
However too many parameters make the equations difficult
to minimise because of the numerous local minima, the need
for a lot of data and the numerical instability introduced into
the Jacobian. We decided to reduce the number of parameters
by making the assumption that the errors due to the assembly
of the system are small (Fig. 2).

4. 改进的Unified Projection Model

4.1 3D -> 2D（0<ξ<1）

• Paper中的Fig4图示有点小问题，归一化的图像平面πmu 到Cp 的距离为1，这个球是单位球，半径为1，所以πmu应该在球里
• 通过引入ξ，将原点拉到Cp，入射光线与Z轴的夹角变小，从而可以映射更大的fov
• Catadioptric相机模型等效为UCM的证明可以去看Paper：A Unifying Theory for Central Panoramic Systems and Practical Implications
• Catadioptric的ξ一般小于1
• Zs + ξ 需要大于零，所以Zs < - ξ的不能被映射到πmu
• 提出了更加通用的内参K矩阵，所以这里的fx(即：γ)和pinhole模型的fx并不对应
  
  

4.2 3D -> 2D（1<ξ）

• 笔者实验时用Mei模型标定fisheye（纯折射dioptric）或者广角相机时，算出来的ξ一般大于1
• Q为Cp出发射线与单位球的切点
• 由相似三角形可以证明：Q在Cm下的Z坐标为-1 / ξ，所以Zs坐标小于-1 / ξ都不能被映射到归一化图像平面πmu上，不过对于190度的相机已经够用了
  

4.3 2D -> 3D

• 这里的（x,y）为归一化图像坐标系下的，并不是像素坐标系下的，h-1 表示映射到单位球面
• camodocal就是这么复现的
  

5. ξ和η的物理含义以及mirror equations

6. Validity for fish-eye lens

• 证明：Mei对鱼眼相机也是有效的
• 文献[13]已经从理论上证明了UCM模型也可以建模fish-eye lens的相机，然后作者在这里又推导了当ξ = 1时，得出的md 可以 转化为ρd，即文献[2]的division模型
• 笔者在做实验时，用Mei模型来建模鱼眼相机得出的重投影误差是小于0.5的，是可以实际应用的
• [13]文献：X. Ying and Z. Hu. Can we consider central catadioptric cameras and fisheye cameras within a unified imaging model ? In ECCV, 2004.
  

7. 完整的Mei模型

• 在第4节的基础上加入了radtan（径向、切向畸变）
• Paper第四节重点讲了这些参数的初始化，这部分建议去看paper

8. Mei用于wide-angle sensor的实验结果

• fov约为70度
• 这里的ξ 没办法简单的解释，因为它并没有mirror

The calibration was also tested on a wide-angle sensor
(∼ 70度) on 21 images of resolution 320×240 . The grid used
was the same as in the hyperbolic case. For the wide-angle
sensor, there is no border so the center of the image was
taken to initialise the principal point. Table VI summarises
the results. As before, we can see a very slight bias towards
the edges in Figure 12.
The strong change in γ after minimisation is probably due
to the radial distortion and the change in ξ. The value of ξ
does not have a simple interpretation for wide-angle sensors.

参考文献

[1] OpenCV: Omnidirectional Camera Calibration
[2] kalibr
[3] camodocal
[4] Omnidirectional Calibration Matlab Toolbox