Explore LiDAR Sensors to Improve the Peripheral Perception

Introduction to LiDAR Technology

Lidar is subdivided into mechanical Lidar and solid Lidar. Traditional LiDAR uses mechanical rotation to achieve 360-degree detection of the surrounding environment. The mechanically rotating LiDAR structure is quite complicated and heavy, which is one of the reasons for the high cost of LiDAR. Solid-state LiDAR, on the other hand, uses optical phase control. The array technology replaces the rotating mechanism to realize the wide-angle scanning function, so that the rotating element is no longer needed in the lidar, and the total volume of the system can be minimized.

The basic components of LiDAR are laser light source, light sensor, and imaging mechanism. The laser light source is generally a semiconductor laser, and the light sensor is generally Photodiode (PD) or Avalanche photodiode (APD). The imaging mechanism refers to a scanning or non-scanning imaging mechanism. According to the survey, the automotive LiDAR market is increasing at a compound annual growth rate of 20%.

With the widespread application of ADAS systems to high-end vehicles, it brings people different experiences in driving safety, comfort, convenience, and energy saving. Based on existing technology, the exploration of multiple sensor applications has become mainstream. For application to existing automatic driving technology, the LiDAR sensor is the most important feedback unit currently being developed in automatic driving technology. Compared with other sensors, LiDAR can accurately determine the position of the target object and manage the basic components required for Society of Automotive Engineers (SAE) Level 5 autonomous driving. Through the processing of advanced LiDAR sensors and machine learning software algorithms, vehicles can simulate various perceptions of humans and are superior to the accuracy of humans in object recognition. It is reliable and stable and can be real-time and all-around. Peripheral perception ability becomes the basis for decision-making and judgment of vehicle behavior control.

The research on the object recognition stability of LiDAR sensors is one of the key projects of the current automatic driving technology. The calculation speed, the ability to resist environmental interference, and the recognition accuracy are three important indicators for the current development of the technology.

LiDAR technology applied to self-driving

1. Introduction to LiDAR for Vehicles
   LiDAR is an active optical sensor. The flight time of the beam reflected to the sensor after the laser beam hits the point to be measured can be calculated to obtain the relative distance to the object to be measured. The amount can be converted into 2D or 3D physical coordinates. The common laser beams include ultraviolet light, visible light, and infrared light. Among them, the wavelength of 600-1000nm is the most used. At present, 2D and 3D sensor products are more common on the market, and they are classified according to the movement state of the laser transceiver. Solid-state type and rotary type 3D sensors are generally divided into 1, 8, 16, 32, 64, 128 beams according to the number of laser beams. 360-degree, high-frequency scanning is a common solution performed for automatic driving. At present, various products are often used in the automotive field. These products currently require complete anti-hacking verification to ensure the security of sensing information to avoid malicious fake optical attacks.
   Compared with other types of sensors, LiDAR is currently the most expensive in terms of price. However, the accuracy of distance measurement for objects is maintained to within 3cm, which is more advantageous than other types of sensors. factor. However, for multiple different types of sensing specification requirements, such as the distance between the sensor and the object to be measured, the relative movement relationship, the classification degree of the object to be measured, the moving speed, the future trajectory, the semantic analysis, and the detection time, etc., only LiDAR has the capability.  For adverse environmental factors, such as raindrops, smog, and dust, it will be necessary to make good use of the advantages of different sensors to meet the basic requirements of autonomous driving. If the detection capabilities of all the specifications listed above are required, it is more suitable for the integration of multiple sensing information (Sensor fusion), to achieve the integration of these various sensors.
   There will be huge cost differences in the choice of LiDAR according to the requirements of the resolution used. High resolution means that objects can be classified and identified. Low resolution means that only object detection can be performed. However, distance detection does not require additional processing to be more accurate than cameras. In terms of cost, cameras are much smaller and less expensive than LiDAR, but the processing of distance detection output is currently a huge challenge. Therefore, some camera products start from the hardware and use multi-sensor fusion such as multiple cameras or infrared. (IR) Some cameras use only a single camera and calculate in software. Method have improved to produce more accurate distance information.
2. Deep learning object detection (DNN-based)
   At present, the literature on the development of LiDAR 3D deep learning is quite scarce compared to the traditional Rule-based. Since Apple released the 3D spatial grid Voxel Net at the end of 2016, its accuracy has been comparable to other deep learning. The network is high, but the fatal disadvantage is that it needs to mesh the raw data of the overall point cloud, which results in the number of units and channels input to the network and the amount of calculation to be extremely high compared to other existing algorithms. As a result, the cost of hardware has risen a lot to achieve the frames per second (FPS) specified by the car specifications. Therefore, in the following years, a deep learning recognition method of 3D conversion to 2D image has been evolved and based on the existing mature 2D theory, it can be applied to LiDAR recognition. For example, Squeeze Seg Net adopts the existing 2D network architecture and is used in the network. The number of input layers can be fixed, so the amount of calculation is greatly reduced and the calculation speed is increased. This is the method adopted by most subsequent research and development units.
   The Industrial Technology Research Institute analyzed a variety of existing deep learning algorithms, compared the specifications required for self-driving cars, and finally adopted Squeeze Seg Net, which can maintain a balance between FPS and accuracy. This method is to project 3D point cloud data to 2D images, and then use After recognition by the deep learning network. The point cloud on the 2D image is converted back to the 3D point cloud, and the result of 3D point cloud segmentation is produced.
   In most self-driving car applications, in order to ensure the stability of data transmission, down sampling will be performed after feature extraction of the data to simplify the amount of transmission. This method can avoid the occurrence of multiple surrounding objects generating an excess amount of data. Excess data will cause serious transmission delay, and the data simplification process will be simplified by using different algorithms according to the needs of the control end. This article describes the method of using the Orient Bounding Box (OBB). This method wraps the grouped objects in a rectangular parallelepiped of Axis-aligned Bounding Box (ABB). The method is to calculate the maximum and minimum of the point cloud of a specific object based on the existing coordinate axes. After the value, 8 square points are formed, and then Principal Component Analysis (PCA) is performed to generate the transition matrix or quaternion of rotation and translation, and then apply to the non-directional directional box to generate the result of the directional box.
3. Deep learning data labeling method
   The existing 3D point cloud marking tools are mainly manual marking. The manual part is operated using a specific algorithm to reduce the operation procedure. Semi-automatic development is still at the development stage, and with of publication of papers in progress, we can see automation is very important for marking tool developers. The next milestone is the development of semi-automated functions, and the degree of semi-automation will also vary depending on the accuracy of the existing modules. In this research, the manual data marking will correspond to the deep learning algorithm used, and the point cloud division marking will be carried out. The type marking for a single point, and the definition of the type of marking will also vary, and correspond to the current LiDAR analysis degree.
   Based on semi-automatic marking software, existing object detection software can be extended, and basic grouping algorithms can be used to mark possible key objects in advance. So manual marking only needs to deal with redundant or wrong marking points. There is no need to mark the 3D point cloud after the image is judged. This method can also effectively reduce the missing mark rate on the manual mark.
   In order to improve the performance of LiDAR computing, the Industrial Technology Research Institute inputs the relevant parameters of the self-driving vehicle into the simulation environment and performs the corresponding sensor settings. The virtual marker data generation tool is used in addition to the LiDAR sensor. It can also be used in the environment of cameras or other sensors. In the LiDAR environment, the corresponding marking data can be synchronized to output the original data and the marking data of LiDAR in a game method, so that the cost of data marking can be greatly reduced.
   In addition to the above-mentioned methods for obtaining marked data, there are also related organizations that collect and mark key point cloud data, and then open them to related research fields. The deep learning algorithm is susceptible to different recognition results due to the different installation positions of the LiDAR sensor. Therefore, the use of the existing LiDAR database for training can only be used as a part of the reference basis, or it can only be carried out according to the vehicle used for collection and labeling of sensing data.

In view of the development trend of international self-driving cars and the technological advantages and gaps in my country's telecommunications industry, the development of vehicle sensor technology and the establishment of high-reliability surrounding object perception capabilities of vehicle regulations are the primary tasks of autonomous driving technology. Through the existing machine learning technology, and the limited sensing devices and information, and the changeable driving environment to provide reliable vision recognition capabilities for self-driving cars, LiDAR sensing technology plays a key role. The Industrial Technology Research Institute is committed to developing LiDAR deep learning algorithms and exploring how to improve the performance of LiDAR marking data. Therefore, unlike the manual marking of traditional images, LiDAR point cloud marking is more difficult. The Industrial Research Institute has observed Semi-automatically generate labeled data to verify the feasibility and accuracy of the deep learning algorithm; at the same time, reduces the cost of labeling. During the development process, it is found that using virtual labeled data for model training is likely to cause overfitting. In the future, based on the advancement of Lidar's deep learning algorithm, the simulated mark data and real mark data will be mixed to train a module, which can improve the object detection accuracy of the mark data obtained by using the simulated environment or real data alone. With lower problems, self-driving cars have modules trained by combining simulated environment and real-world mark data to provide more accurate environmental sensing and positioning.