深度估计 双目深度估计+单目深度估计 ONNX运行程序

双目深度估计ONNX Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation https://github.com/ibaiGorordo/ONNX-CREStereo-Depth-Estimation

• 双目深度估计需要从dataset读取左右两个view的图片。
  
  
  

• 使用模型进行深度图的估计

model_path = f'models/crestereo_{version}_iter{iters}_{shape[0]}x{shape[1]}.onnx'
depth_estimator = CREStereo(model_path)
disparity_map = depth_estimator(left_img, right_img)
1
2
3

CREStereo model: https://github.com/megvii-research/CREStereo
CREStereo - Pytorch: https://github.com/ibaiGorordo/CREStereo-Pytorch
PINTO0309's model zoo: https://github.com/PINTO0309/PINTO_model_zoo
PINTO0309's model conversion tool: https://github.com/PINTO0309/openvino2tensorflow
Driving Stereo dataset: https://drivingstereo-dataset.github.io/
Depthai library: https://pypi.org/project/depthai/
Original paper: https://arxiv.org/abs/2203.11483
1
2
3
4
5
6
7

单目深度估计monocular

GLPDepth

• 下载github代码(稍微修改一下，用本地读取替换在线读取)
  

• 去PINTO0309’s model zoo: https://github.com/PINTO0309/PINTO_model_zoo下载模型

• 运行bash
  
  

monodepth2 https://github.com/nianticlabs/monodepth2

python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192
1

    with torch.no_grad():
        for idx, image_path in enumerate(paths):

            if image_path.endswith("_disp.jpg"):
                print('don t try to predict disparity for a disparity image!')
                continue

            # Load image and preprocess
            input_image = pil.open(image_path).convert('RGB')
            original_width, original_height = input_image.size
            input_image = input_image.resize((feed_width, feed_height), pil.LANCZOS)
            input_image = transforms.ToTensor()(input_image).unsqueeze(0)

            # PREDICTION
            input_image = input_image.to(device)
            features = encoder(input_image)
            outputs = depth_decoder(features)

            disp = outputs[("disp", 0)]
            # 还原大小  torch.Size([1, 1, 235, 638])
            disp_resized = torch.nn.functional.interpolate(
                disp, (original_height, original_width), mode="bilinear", align_corners=False)

            # Saving numpy file
            output_name = os.path.splitext(os.path.basename(image_path))[0]
            scaled_disp, depth = disp_to_depth(disp, 0.1, 100)
            if args.pred_metric_depth:
                name_dest_npy = os.path.join(output_directory, "{}_depth.npy".format(output_name))
                metric_depth = STEREO_SCALE_FACTOR * depth.cpu().numpy()
                np.save(name_dest_npy, metric_depth)
            else:
                name_dest_npy = os.path.join(output_directory, "{}_disp.npy".format(output_name))
                np.save(name_dest_npy, scaled_disp.cpu().numpy())

            # Saving colormapped depth image
            disp_resized_np = disp_resized.squeeze().cpu().numpy()
            vmax = np.percentile(disp_resized_np, 95)
            normalizer = mpl.colors.Normalize(vmin=disp_resized_np.min(), vmax=vmax)
            mapper = cm.ScalarMappable(norm=normalizer, cmap='magma')
            colormapped_im = (mapper.to_rgba(disp_resized_np)[:, :, :3] * 255).astype(np.uint8)
            im = pil.fromarray(colormapped_im)

            name_dest_im = os.path.join(output_directory, "{}_disp.jpeg".format(output_name))
            im.save(name_dest_im)

            print("   Processed {:d} of {:d} images - saved predictions to:".format(
                idx + 1, len(paths)))
            print("   - {}".format(name_dest_im))
            print("   - {}".format(name_dest_npy))

    print('-> Done!')

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

有时没有网络的情况下运行报错urllib.error.URLError:

# download_model_if_doesnt_exist(args.model_name)
1

参考与更多

单目深度估计总结
深入研究自监督单目深度估计：Monodepth2
consistent-video-deprh-estimation：github上的一个有趣的应用
效果惊艳！ONNX-CREStereo深度估计！

monodepth https://github.com/mrharicot/monodepth

https://github.com/nianticlabs/monodepth2
Digging Into Self-Supervised Monocular Depth Estimation
https://arxiv.org/abs/1806.01260

• https://www.bilibili.com/video/BV1HZ4y1c7nC