Survey summary for 3D pose estimatation

Survey summary for 3D pose estimatation
[Survey paper] Deep Learning Methods for 3D Human Pose Estimation under Different Supervision Paradigms: A Survey

我们关注的是 Multi-Person这一栏。：

[mvp] Direct Multi-view Multi-person 3D Pose Estimation (NeurlPS)
code

[voxelpose] VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment
[基于graph的商汤的那篇] Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images
[*可借鉴第一步] Learnable Triangulation of Human Pose(iccv2019 oral)
可借鉴第一步
第二步也是3d cnn做的，cuboid(64x64x64xk).

[*可借鉴第二阶段]FLEX: Parameter-free Multi-view 3D Human Motion Reconstruction (2021)
可用
主页
Train a neural network to predict joint angles and bone lengths without using any of the camera parameters.
[*可借鉴]Generative Multi-View Based 3D Human Pose Estimation(ACM2021)
[GAN 可能解决数据问题]无代码
[*feedback可借鉴]Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors(ICMLSC 2019)
[带feedback，to 嵌入式器件]无代码

[无监督的方法]Unsupervised Domain Adaptation for 3D Human Pose Estimation(ACM2019)
无监督的方法，无代码

[*可借鉴]Cross View Fusion for 3D Human Pose Estimation(ICCV2019)
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Ok58r1ed-1655955445780)(https://arxiv.org/pdf/1909.01203v1.pdf)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-gQBISzZQ-1655955445782)(https://github.com/microsoft/multiview-human-pose-estimation-pytorch)]
[使用Recursive Pictorial Structure Model来做的]
亮点：

1. 在2D pose estimation阶段做了融合。
2. 在第二阶段PSM模型中，改进为RPSM模型，将cuboid切分step从16开始往下迭代获取more finer-grained grid.
   PSM: model comes from 3D Pictorial Structures for Multiple Human Pose Estimation-2014

Z(F) is the partition function and E are the graph edges.
φ conf i (Ji , F) are computed based on the previously estimated multi-view 2D pose heatmaps F.
We compute the average confidence over all camera views as the unary potential for the hypothesis.

[plane sweep stereo可借鉴]Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo(cvpr2021)
通过plane sweep stereo的方法做的
[缺点：至少需要两个camera覆盖]
[地面热力图的方法][可借鉴，作为第一步来用]Cross-View Cross-Scene Multi-View Crowd Counting(cvpr2021)
在做第一阶段时可以参考这种方法
亮点：该方法train/test采用不用场景不同camera布局，对摄像头的的多少、位置、顺序具有较强的泛化性。
View-weight + view-pooling 解决摄像头位置变化和数量问题。
可借鉴做第一阶段，地面热力图的方法
[from survey paper]Light3DPose: Real-time Multi-Person 3D Pose Estimation from Multiple Views. In Proceedings of the 25th International Conference on Pattern Recognition (ICPR)
[亮点]：bottom-up, 拥挤场景下有较好的 accuracy results and computational time。
它的Volumetric Processing也是用的3Dcnn.
[Light3DPose中Volumetric Processing用到的方法]V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map (cvpr2018)

[100FPS] Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS (2020)
速度快，是因为它采用迭代计算各个view的方式，各个view不是同步的，每次计算仅仅加入一个view。
保留historical 检测到的joints的3D位置以及2D位置。
每次的Detections 集合D 与tracked targets 集合 T 做match。分别从2d和3d上计算：

2d 上的match affinity:

3d上的match affinity:

是通过计算3d坐标点到camera C的极线距离实现的。
根据计算得到的Affinity来实现每个view的Detections和已知的Targets的association. 然后做update。
[Direct Regression, Bone-based but not Joints-based may be referred] Compositional Human Pose Regression (2017)
[reason]:

• Joints- global coordinate system
• Bone - local coordinate systems
  Using bones brings several benefits: Stability, Geometric convenience, Application convenience(许多姿势驱动的应用程序只需要局部骨骼，而不需要全局关节).

[做渲染的，如果做重投影的话可借鉴]https://openaccess.thecvf.com/content/CVPR2021/papers/Wang_IBRNet_Learning_Multi-View_Image-Based_Rendering_CVPR_2021_paper.pdf
[增加一个虚拟的view]
[缺点：需要很多个view，不能用！]

[PAF(Part Affinity Fields)] OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
Introduced a representation of pairwise scores via the so-called Part Affinity Fields (PAF)
[借鉴] 可以做3d PAF.

[Dong 浙大] Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views （2019）
[top-down]
[做匹配的论文] 提出了多路匹配算法，优点是：快速，循环一致性，减少fp。
[4D graph 做匹配, 清华] 4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras (2020)
[Code] (https://github.com/zhangyux15/4d_association)
[这篇文章与上面那篇(Dong 浙大)的做匹配的文章做了对比，效果要比上篇好]
4D (2D spatial, 1D viewpoint and 1D temporal), Bottom-Up
[主要思想] By regarding the detected 2D skeleton joint candidates in the current frame and the 3D skeleton joints in the former frame as graph nodes, we construct edges by calculating confidence weights between nodes. Such calculation jointly takes advantage of feature confidences in each individual image, epipolar constraints and reconstructed skeletons in the temporal precedent frame

[Compress, 单目多人3D姿态估计中heatmaps压缩方法，提高计算速度，3D cnn来编码3d feature的论文] Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation
[压缩] 这里所谓的压缩不过是将heatmap的大小变小为原来的1/8，然后用3d cnn 后编码，再解码出来，这种方法适用于单目的情况。
[多个cameras fuse] MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation
[no code]