毫米波成像 论文阅读笔记 | HawkEye, CVPR 2020

原文链接: https://mp.weixin.qq.com/s/KyMyyZert7ZYltfZaYrBWA

论文阅读笔记：[1] J. Guan, S. Madani, S. Jog, S. Gupta, and H. Hassanieh, “Through Fog High-Resolution Imaging Using Millimeter Wave Radar,” in 2020 CVPR

Abstract

• mmWave high-resolution imaging: in dense fog
• Motivation

  • mmWave signals:

    ✅ have favorable propagation characteristics in low visibility conditions

    ❌ suffer from very low resolution, specularity, and noise artifacts.

  • 光学传感器则反之

• 本文：HawkEye
  • leverage cGAN to recover shapes from low-resolution mmWave heatmaps
  • 基于mmWave signal 的structure和nature 设计
  • 实现了a data synthesizer：合成训练数据集
  • implement our system on a custom-built mmWave radar platform + demonstrate performance improvement

1 Introduction

mmWave Radar high-resolution imaging的意义

• Significance : severe weather conditions (dense fog, smog, snowstorms, and sandstorms)、low light(无LiDAR时) 下的自动驾驶需要毫米波雷达
• Several challenges : However, mmWave wave radar is still difficult to use for imaging (如下图d,e)
  • 1 Low resolution
  • 2 highly specular reflection
  • 3 multipath propagation： creating shadow reflections and artifacts in various locations

• Existing Solutions :
  • 1 improve resolution: use human-sized mechanically steerable arrays
  • 2 eliminate multipath reflections : isolate the object being imaged in the near field
  • 3 address specularity: rotate the arrays around the object
  • 显然，上述设计对于自动驾驶而言 extremely bulky and not practical

本文工作 HawkEye 概述

• Key idea :

  • cast the problem of predicting high-frequency shape from raw mmWave heatmaps as a learning problem

• Advantages :

  • Use of learning provides robustness

  • effectively leverage priors on shapes of cars to make reasonable predictions

• Innovations

  • the design of the Neural Network : map 3D input heatmaps to 2D depth maps

  • Loss function : combination of perceptual, L1, and adversarial loss

  • Training Data :

    ✅ Realistic radar data synthesizer that captures unique characteristics of radar +

    ✅ real-world data collection platform to collect real data for fine-tuning and benchmarking

• System Overview (四个模块,如下图)

  • Module 1 : custom-built mmWave imaging module $\Rightarrow$ capture radar data

  • Module 2 : A wide baseline stereo camera system $\Rightarrow$ capture GT (high-resolution 2D depth maps)

  • Module 3 : Synthesizer $\Rightarrow$ 合成数据 from 3D CAD models of cars and mmWave ray tracing algorithms

  • Module 4 : GAN $\Rightarrow$ generate high-resolution depth maps + reconstruct the car in the real scene in fog (from raw 3D mmWave heatmaps)

    

2 Related Work

2.1 Super-Resolution

• 图像超分辨

  • rely on the correspondence of image patches between low and high resolution images

• 点云超分辨 (closest to this work)

  • upsamplng sparse 3D LiDAR data to create dense 2D depth maps

• 上述工作的特点(优势) ：

  • 或 同时使用相机 + LiDAR
  • 或 rely on high frequency visual features like edges to cluster and upsample objects

• 高分辨毫米波超分辨的困难

  • Significantly lower spatial resolution
  • High frequency visual features (like boundaries and edges) are not apparent
  • 传统上采样/超分辨技术无法解决的artifacts and specularities

2.2 LiDAR in Fog

现有工作的limitations

• 1 require knowing a depth map of the scene as priori
• 2 require static object (estimating the statistical distribution of the photon reflected off the object)
• 3 limited detection depth or resolution or FOV

但毫米波没有这些限制

2.3 Radar Imaging Systems