Smart IoV System with Seamless Experience - Smart Cockpit

Development of the Smart Car Market:

Smart vehicles have become an important direction for the development of the global automobile industry, and there are also relevant incentive policies and standards. For example, the Road Map 2025 issued by Euro NCAP requires that starting from 2020, vehicles that want to obtain the EU five-star safety certification must with driving monitoring. In addition, since 2023, Euro NCAP will use the function of child detection in the car as one of the scoring criteria, and C-NCAP will follow up in 2025. This will drive market demand, and the warning function of the driving monitoring system will expand to the monitoring system of people in the car, and extend to the detection of children left behind.

Therefore, the monitoring system based on image recognition will use wide-angle cameras as sensors to detect the entire cockpit. However, due to factors such as viewing angle, installation location, and resolution, the accuracy of the legacy detection system in the cabin may be reduced. The image recognition and detection rate of the entire cockpit are developed by the development of deep learning algorithms and millimeter-wave radar-assisted image recognition. Through information conversion and analysis, the movement characteristics of objects in the cabin are extracted as reference information for passenger legacy judgments.

Due to the promotion of Euro NCAP and C-NCAP and market demand, the development of smart cockpit systems will be accelerated. In terms of safety, DMS has expanded from driving status detection to entire cockpit monitoring and has the function of detecting children left behind in the rear seat. To this end, the vehicle center has established image recognition AI algorithm technology and will integrate millimeter-wave radar sensing in the future to improve the accuracy of overall cockpit monitoring and develop line-of-sight detection technology to strengthen the accuracy of driving line-of-sight during Level 3 automatic lane maintenance driving. Need to correctly judge whether the driver can take over, and ensure the safety of himself and other passers-by during automatic driving.

What is a Smart Cockpit?

The smart cockpit is a product with convenient and diversified human-computer interaction/personal interaction, more in-cabin sensors, and an app that is more suitable for automotive application scenarios. At the same time, the smart cockpit is an evolving concept, because the future car cockpit will have more and different application scenarios with the improvement of the degree of automatic driving. For example, long-distance driving and urban commuting will lead to different smart cockpit designs. The smart cockpit covers voice interaction, real-world navigation, and driving status detection. This application highlights the practice of wisdom.

It also includes personalized experience and situational awareness safety is inseparable from the human-computer interaction between the user and the vehicle. The digital cockpit can recognize driving gesture commands, and the digital cockpit supports natural language processing and can realize precise voice control. This makes human-computer interaction more convenient and intuitive, improves efficiency, and can complete various tasks while driving safely.

When comparing smart cockpit concepts to cockpit concepts with rich display environments, one needs to consider how the display environment interacts with the user. He also pointed out that simply having multiple monitors with standard features doesn't make the experience smart. Rather, what is shown on these displays and how they make driving easier is what defines the smart cockpit. Just as more and more artificial intelligence assistants are being used in the cockpit, they configure the user experience by obtaining consumer preferences and behavioral information. They can play their favorite music or plan driving routes autonomously. At the same time, drivers will obtain the latest car services, functional features, and maintenance information through software updates; for car safety upgrades, they will see a new concept that connects drivers and dealers. At the same time, the car can provide payment services, smoothly supporting toll payments or other types of transactions. And the auto insurance industry will also benefit from the use of driving behavior data, which is an important demand for consumers and commercial fleets.

Smart Cockpit Monitoring Technology:

Prepared for a seamless travel experience, the smart cockpit focuses on user needs. Based on the merger of software and hardware, it integrates onboard electronics, dashboards, and smart onboard systems, and connects multiple screens and mobile terminal devices in the cabin. Using the AI intelligence engine and deep learning as a bridge, it actively recommends the services that users need, and at the same time provides the driving behavior monitoring required for safety. Based on professional technical capabilities in various aspects and a variety of third-party software services in series with smart terminals, the user experience is continuously optimized and updated to create an integrated and personalized smart cockpit.

The smart cockpit monitoring technology currently developed by the Vehicle Center (ARTC) mainly installs a wide-angle camera under the interior rearview mirror to capture images of the front seat and some rear seats. And uses the AI image deep learning method to perform face detection and head predict the yaw angle of the face, and then calculate the line-of-sight angle according to the recognized face position. Judging by the position of the cockpit, it recognizes the driver and combines the head tilt angle and line of sight angle to judge whether the driver is distracted, and immediately sends out a warning to remind the driver that there is an abnormal state. In addition to monitoring the driving status while driving, when not driving, the cockpit position is used to determine whether there is a human face in the back seat. To detect whether there are passengers or children left in the cabin, and to provide necessary warnings to avoid and reduce parental abuse or if an adult negligently leaves a child in the car, resulting in injury or death of the child. Or a sleeping passenger is locked in the car, causing an accident.

Characteristics of the Smart Cockpit: Personalization, Digitalization, and Immersive Experience

• Content digitization, including maps, contacts, media, etc.
• Personalize the driver and user and migrate the home environment into the vehicle.
• An immersive cockpit experience, including using applications such as noise reduction to improve the audio and voice experience in the cockpit.

After you enter the driving position, the camera of the driver monitoring system can perform facial recognition to determine who is in control of the vehicle. Elsewhere in the cabin, other separate methods of user identification may be used, such as voice or fingerprint recognition. After recognition, your personalized comfort settings are activated, including seat position and cabin temperature. A vehicle is a connected digital device that seamlessly syncs your content and information.

Gesture-sensing controls allow drivers to more safely interact with the car's audio, ventilation, and other systems. Achieving such an immersive digital experience in the car requires a combination of advanced technologies and knowledge of the car's operating environment, especially in terms of safety, reliability, and efficiency.

ADI has already started collaborating with automakers to help them deploy MEMS microphones with beamforming capabilities for voice control and AI-based software technology for local voice recognition. ADI's advanced noise reduction system can mask road noise to form a personal audio environment while allowing drivers to hear sounds related to driving safety. By deploying microphones, digital audio systems, and advanced DSP, supported by sensors such as accelerometers, the performance of ADI's new noise reduction technology has been greatly improved to provide a quiet cockpit environment and bring a new level of immersive digital experience.

The scope of in-vehicle software control functions continues to expand, leading to a surge in the number of in-vehicle network nodes and the number of signals transmitted over the network. Traditional networking technologies deployed in vehicles require additional cables to cover each additional node and connection. Advanced busbar network technology can shorten the length of cables in the car, help reduce weight, and thus greatly reduce energy consumption. New wireless networks for data or entertainment system components in vehicles can eliminate the weight and cost of cables. ADI's GMSL SerDes technology provides the high bandwidth needed to transport multiple types of data that are increasing in capacity, successfully deploying complex advanced driver assistance systems (ADAS) and infotainment functions in vehicles. Advantages of GMSL include faster aggregation of data, the ability to maintain data integrity for safety applications, and support for systems to display different content on multiple screens in the vehicle.

Uninterrupted high-speed network connectivity is the key to achieving a high degree of digitalization in the car of the future. ADI's new Ethernet-to-the-Edge bus (E2B) uses the new automotive Ethernet 10BASE-T1S technology to support sensors and actuators for Ethernet-to-edge device connectivity. This facilitates the transition to Zonal-based architectures, provides significant savings in cable costs, significantly reduces the number of ECUs, and enables the deployment of new features and enhancements through over-the-air updates. The continued rapid pace of development is helping automakers realize their vision for a new in-cabin experience.

Major Development Trends of the Smart Cockpit:

1. Panel makers are expected to become Tier1 OEM suppliers
   In terms of smart cockpit hardware, the Micro LED immersive cabin display solution is launched, which can be matched with the cockpit interior panels to display textures in line with the overall style, so that the cockpit design has a consistent aesthetic. At the same time, the high brightness and high contrast of Micro LED Outstanding features, high-penetration textured optical film, etc., meet the various application requirements of smart cockpits.
   Especially in the field of smart cockpits, future automakers are likely to skip Tier 1 and directly purchase software and hardware integration solutions from panel manufacturers. The reason is that when the importance of smart cockpits becomes more and more obvious, automakers will inevitably make differentiated content in terms of user experience. Therefore, customization needs based on car series will also occur frequently. For automakers, if panel makers can meet their needs in a one-stop manner, and will be able to achieve their goals more effectively.
   Technically, it will take some time for panel makers to integrate software, hardware (ECU), and panels, and the testing and running-in with car makers is still going on. But what is certain is that panel factories will play an important role in the field of smart cockpits in the future. It is expected that panels will account for about 50% of the cost of smart cockpits.
2. Software-defined hardware becomes the key
   In the field of smart cockpits, software-defined hardware will emphasize how to create a better interactive experience. The real smart cockpit interactive experience should be based on the following conditions:
   • Personalization is the most important thing.
   • Humans are visual animals, so how to create an immersive experience is one of the key points.
   • Human-computer interaction will be the ultimate goal of the smart cockpit.
   The so-called smart cockpit must be a cockpit that can learn and grow together with driving. In other words, it is possible to optimize the driving experience through user behavior and preferences; and to achieve this state, it is necessary to rely on self-learning technology. This self-learning technology does not go to the cloud and does not return data, that is to say, we will be able to ensure that these personal data will not be used by third parties (depots). In the past, car manufacturers focused on hardware rather than personalization in the cockpit. To achieve personalization, more attention should be paid to the cockpit software.
3. The status of edge AI is greatly improved
   The type of traditional cockpit products is relatively simple, mostly dominated by hardware components, controlled by a simple automotive microcontroller (ECU), and the instrument and the vehicle infotainment system (IVI) system are independent and unable to interact with each other. There are fewer internal screens and a single function. In the era of smart cockpits, the screen in the car has become the carrier for displaying services and content, which directly drives the demand for AI and high-performance computing in the cockpit.
   In terms of user experience in the cockpit, in the past, IVI, head-up display, center console, and instrument panel all required individual control by the user, and other common configurations such as seats and rear-view mirrors required manual adjustments. Conversely, the smart cockpit can bring users more immersive experiences through AI and high-performance computing. Such as AI virtual assistance, in-car games, 4K multi-screen display, audio entertainment equipment, multi-screen assistance, and human-machine co-driving wait. This is the main reason why edge computing AI chips have been widely discussed in the automotive market in recent years.
   Why is edge computing so important for vehicles? The car of the future will be composed of many AI sensors. These small heads with AI functions surround the car body and cover different sensing ranges to cope with multiple functions such as self-driving cars and smart cockpits. When the signal paths connecting each sensor to the central processing unit are different and of different lengths, the time of message transmission will also be inconsistent, which may easily cause security concerns. Therefore, for the vehicle, a better architecture would be that the small heads on each edge are connected to the calculation function. And the result signals that have been calculated are transmitted to each other, to reduce the time spent on transmitting images, thereby reducing information transmission time error.
   The advantages of innovations lie in fast and high flexibility, and the AI chip has a Transformer computing model and reconfigurable architecture technology. Under the premise of small size and low energy consumption, the same group of chips can support gesture recognition and face recognition. Expression recognition, voice interaction, driving fatigue detection, and other intelligent functions.

The Smart Car of the Future will Connect Your World:

The smart cockpit is a systematic concept, which can be implemented in a wide variety of forms, and the market penetration rate can vary greatly depending on the definition. Its rapid development covers two points: technology and ecology.

The smart cockpit system integrates driving and the car and controls the driving environment and in-car information through intuitive and visual modes. Driving information is conveyed by vehicle sensors and multimedia channels. While enjoying various entertainments, it can also stay connected at any time to ensure the safety of drivers and passengers.