编辑丨3D视觉开发者社区
将激光雷达传感器的前视图展平为2D图像，必须是3D将空间中的点投影到可扩展的圆柱形表面，形成平面。
本文代码参考自论文《Vehicle Detection from 3D Lidar Using Fully Convolutional Network》
论文链接：https://arxiv.org/pdf/1608.07916.pdf
# h_res = horizontal resolution of the lidar sensor # v_res = vertical resolution of the lidar sensor x_img = arctan2(y_lidar, x_lidar)/ h_res y_img=np.arctan2(z_lidar,np.sqrt(x_lidar**2 y_lidar**2))/v_res 问题是，图像的接缝将直接放置在汽车的右侧。将接缝定位在汽车的最后部更有意义，因此前部和侧部更重要的区域是不间断的。让这些重要区域不间断，使卷积神经网络更容易识别这些重要区域的整个对象。以下代码解决了这个问题。
# h_res = horizontal resolution of the lidar sensor # v_res = vertical resolution of the lidar sensor x_img = np.arctan2(-y_lidar, x_lidar)/ h_res # seam in the back y_img=np.arctan2(z_lidar,np.sqrt(x_lidar**2 y_lidar**2))/v_res 变量h r e s h_{res}和v r e s v_{res}非常依赖于使用LIDAR传感器。在KTTI数据集中，使用的传感器是Velodyne HDL 64E。根据Velodyne HDL 64E的规格表，它具有以下重要特征：
垂直视野为26.9度，分辨率为0.垂直视野分为传感器上方4度 2度，传感器下-24度.9度
水平视野360度，分辨率0.08-0.35(取决于旋转速度)
旋转速率可选择5-20Hz之间
代码可以通过以下方式更新：
# Resolution and Field of View of LIDAR sensor h_res = 0.35 # horizontal resolution, assuming rate of 20Hz is used v_res = 0.4 # vertical res v_fov = (-24.9, 2.0) # Field of view (-ve, ve) along vertical axis v_fov_total = -v_fov[0] v_fov[1] # Convert to Radians v_res_rad = v_res * (np.pi/180) h_res_rad = h_res * (np.pi/180) # Project into image coordinates x_img = np.arctan2(-y_lidar, x_lidar)/ h_res_rad y_img=np.arctan2(z_lidar,d_lidar)/v_res_rad 然而，这导致大约一半的点在x轴负方向上，而且大多数点在y轴负方向上。投影到2D图像需要将最小值设置为(0，0)，因此需要做出一些改变:
# SHIFT COORDINATES TO MAKE 0,0 THE MINIMUM x_min = -360.0/h_res/2 # Theoretical min x value based on specs of sensor x_img = x_img - x_min # Shift x_max = 360.0/h_res # Theoretical max x value after shifting y_min = v_fov[0]/v_res # theoretical min y value based on specs of sensor y_img = y_img - y_min # Shift y_max = v_fov_total/v_res # Theoretical max x value after shifting y_max = y_max 5 # UGLY: Fudge factor because the calculations based on # spec sheet do not seem to match the range of angles #collectedbysensorinthedata. 将3D点投影到2D最小值为(0，0)的坐标点可绘制成2D图像。
pixel_values = -d_lidar # Use depth data to encode the value for each pixel cmap = "jet" # Color map to use dpi = 100 # Image resolution fig, ax = plt.subplots(figsize=(x_max/dpi, y_max/dpi), dpi=dpi) ax.scatter(x_img,y_img, s=1, c=pixel_values, linewidths=0, alpha=1, cmap=cmap) ax.set_axis_bgcolor((0, 0, 0)) # Set regions with no points to black ax.axis('scaled') # {equal, scaled} ax.xaxis.set_visible(False) # Do not draw axis tick marks ax.yaxis.set_visible(False) # Do not draw axis tick marks plt.xlim([0, x_max]) # prevent drawing empty space outside of horizontal FOV plt.ylim([0, y_max]) # prevent drawing empty space outside of vertical FOV fig.savefig("/tmp/depth.png",dpi=dpi,bbox_inches='tight',pad_inches=0.0) 在函数中放置上述所有代码。
def lidar_to_2d_front_view(points, v_res, h_res, v_fov, val="depth", cmap="jet", saveto=None, y_fudge=0.0 ): """ Takes points in 3D space from LIDAR data and projects them to a 2D "front view" image, and saves that image. Args: points: (np array) The numpy array contaning the lidar points.
The shape should be Nx4
- Where N is the number of points, and
- each point is specified by 4 values (x, y, z, reflectance)
v_res: (float)
vertical resolution of the lidar sensor used.
h_res: (float)
horizontal resolution of the lidar sensor used.
v_fov: (tuple of two floats)
(minimum_negative_angle, max_positive_angle)
val: (str)
What value to use to encode the points that get plotted.
One of {"depth", "height", "reflectance"}
cmap: (str)
Color map to use to color code the `val` values.
NOTE: Must be a value accepted by matplotlib's scatter function
Examples: "jet", "gray"
saveto: (str or None)
If a string is provided, it saves the image as this filename.
If None, then it just shows the image.
y_fudge: (float)
A hacky fudge factor to use if the theoretical calculations of
vertical range do not match the actual data.
For a Velodyne HDL 64E, set this value to 5.
"""
# DUMMY PROOFING
assert len(v_fov) ==2, "v_fov must be list/tuple of length 2"
assert v_fov[0] <= 0, "first element in v_fov must be 0 or negative"
assert val in {"depth", "height", "reflectance"}, \
'val must be one of {"depth", "height", "reflectance"}'
x_lidar = points[:, 0]
y_lidar = points[:, 1]
z_lidar = points[:, 2]
r_lidar = points[:, 3] # Reflectance
# Distance relative to origin when looked from top
d_lidar = np.sqrt(x_lidar ** 2 + y_lidar ** 2)
# Absolute distance relative to origin
# d_lidar = np.sqrt(x_lidar ** 2 + y_lidar ** 2, z_lidar ** 2)
v_fov_total = -v_fov[0] + v_fov[1]
# Convert to Radians
v_res_rad = v_res * (np.pi/180)
h_res_rad = h_res * (np.pi/180)
# PROJECT INTO IMAGE COORDINATES
x_img = np.arctan2(-y_lidar, x_lidar)/ h_res_rad
y_img = np.arctan2(z_lidar, d_lidar)/ v_res_rad
# SHIFT COORDINATES TO MAKE 0,0 THE MINIMUM
x_min = -360.0 / h_res / 2 # Theoretical min x value based on sensor specs
x_img -= x_min # Shift
x_max = 360.0 / h_res # Theoretical max x value after shifting
y_min = v_fov[0] / v_res # theoretical min y value based on sensor specs
y_img -= y_min # Shift
y_max = v_fov_total / v_res # Theoretical max x value after shifting
y_max += y_fudge # Fudge factor if the calculations based on
# spec sheet do not match the range of
# angles collected by in the data.
# WHAT DATA TO USE TO ENCODE THE VALUE FOR EACH PIXEL
if val == "reflectance":
pixel_values = r_lidar
elif val == "height":
pixel_values = z_lidar
else:
pixel_values = -d_lidar
# PLOT THE IMAGE
cmap = "jet" # Color map to use
dpi = 100 # Image resolution
fig, ax = plt.subplots(figsize=(x_max/dpi, y_max/dpi), dpi=dpi)
ax.scatter(x_img,y_img, s=1, c=pixel_values, linewidths=0, alpha=1, cmap=cmap)
ax.set_axis_bgcolor((0, 0, 0)) # Set regions with no points to black
ax.axis('scaled') # {equal, scaled}
ax.xaxis.set_visible(False) # Do not draw axis tick marks
ax.yaxis.set_visible(False) # Do not draw axis tick marks
plt.xlim([0, x_max]) # prevent drawing empty space outside of horizontal FOV
plt.ylim([0, y_max]) # prevent drawing empty space outside of vertical FOV
if saveto is not None:
fig.savefig(saveto, dpi=dpi, bbox_inches='tight', pad_inches=0.0)
else:
fig.show()
以下是一些用例：
import matplotlib.pyplot as plt
import numpy as np
HRES = 0.35 # horizontal resolution (assuming 20Hz setting)
VRES = 0.4 # vertical res
VFOV = (-24.9, 2.0) # Field of view (-ve, +ve) along vertical axis
Y_FUDGE = 5 # y fudge factor for velodyne HDL 64E
lidar_to_2d_front_view(lidar, v_res=VRES, h_res=HRES, v_fov=VFOV, val="depth",
saveto="/tmp/lidar_depth.png", y_fudge=Y_FUDGE)
lidar_to_2d_front_view(lidar, v_res=VRES, h_res=HRES, v_fov=VFOV, val="height",
saveto="/tmp/lidar_height.png", y_fudge=Y_FUDGE)
lidar_to_2d_front_view(lidar, v_res=VRES, h_res=HRES, v_fov=VFOV,
val="reflectance", saveto="/tmp/lidar_reflectance.png",
y_fudge=Y_FUDGE)
产生以下三个图像：
Depth
Height
Reflectance
目前创建每个图像非常慢，可能是因为matplotlib，它不能很好地处理大量的散点。因此需要创建一个使用numpy或PIL的实现。
测试
需要安装python-pcl，加载PCD文件。
sudo apt-get install python-pip
sudo apt-get install python-dev
sudo pip install Cython==0.25.2
sudo pip install numpy
sudo apt-get install git
git clone https://github.com/strawlab/python-pcl.git
cd python-pcl/
python setup.py build_ext -i
python setup.py install
可惜，sudo pip install Cython==0.25.2这步报错：
“Cannot uninstall ‘Cython’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.”
换个方法，安装pypcd：
pip install pypcd
查看 https://pypi.org/project/pypcd/ ，用例如下：
Example
-------
.. code:: python
import pypcd
# also can read from file handles.
pc = pypcd.PointCloud.from_path('foo.pcd')
# pc.pc_data has the data as a structured array
# pc.fields, pc.count, etc have the metadata
# center the x field
pc.pc_data['x'] -= pc.pc_data['x'].mean()
# save as binary compressed
pc.save_pcd('bar.pcd', compression='binary_compressed')
测试数据结构：
“ >>> lidar = pypcd.PointCloud.from_path(‘～/pointcloud-processing/000000.pcd’)
>>> lidar.pc_data
array([(18.323999404907227, 0.04899999871850014, 0.8289999961853027, 0.0),
(18.3439998626709, 0.10599999874830246, 0.8289999961853027, 0.0),
(51.29899978637695, 0.5049999952316284, 1.944000005722046, 0.0),
…,
(3.7139999866485596, -1.3910000324249268, -1.7330000400543213, 0.4099999964237213),
(3.9670000076293945, -1.4739999771118164, -1.8569999933242798, 0.0),
(0.0, 0.0, 0.0, 0.0)],
dtype=[(‘x’, ‘<f4’), (‘y’, ‘<f4’), (‘z’, ‘<f4’), (‘intensity’, ‘<f4’)])
>>> lidar.pc_data[‘x’]
array([ 18.3239994 , 18.34399986, 51.29899979, …, 3.71399999,
3.96700001, 0. ], dtype=float32) ”
加载PCD：
import pypcd
lidar = pypcd.PointCloud.from_path('000000.pcd')
x_lidar：
x_lidar = points[’x‘]
结果：
Depth
Reflectance
本文为CSDN博主「W_Tortoise」的原创文章，点击“阅读原文”可查看原文出处。
本文仅做学术分享，如有侵权，请联系删文。
在「计算机视觉工坊」公众号后台回复：，即可下载深度学习算法、3D深度学习、深度学习框架、目标检测、GAN等相关内容近30本pdf书籍。
在「计算机视觉工坊」公众号后台回复：，即可下载计算机视觉相关17本pdf书籍，包含计算机视觉算法、Python视觉实战、Opencv3.0学习等。
在「计算机视觉工坊」公众号后台回复：，即可下载独家SLAM相关视频课程，包含视觉SLAM、激光SLAM精品课程。
扫码添加小助手微信，可申请加入3D视觉工坊-学术论文写作与投稿 微信交流群，旨在
也可申请加入我们的细分方向交流群，目前主要有、、、、、等微信群，请扫描下面微信号加群，备注：”研究方向+学校/公司+昵称“，例如：”3D视觉 + 上海交大 + 静静“。请按照格式备注，否则不予通过。添加成功后会根据研究方向邀请进去相关微信群。也请联系。
▲长按加微信群或投稿
▲长按关注公众号
：针对3D视觉领域的四个方面进行深耕，更有各类大厂的算法工程人员进行技术指导。与此同时，星球将联合知名企业发布3D视觉相关算法开发岗位以及项目对接信息，打造成集技术与就业为一体的铁杆粉丝聚集区，近3000星球成员为创造更好的AI世界共同进步，知识星球入口：
学习3D视觉核心技术，扫描查看介绍，3天内无条件退款
圈里有高质量教程资料、可答疑解惑、助你高效解决问题