3D DASHBOARDS, SMART HEADLIGHTS AND MIXED REALITY DRIVING
The 2020s will be the decade of the autonomous automobile. As cars become Internet-connected machines, constantly streaming data from 4G and 5G networks, their use of new display technology will rapidly evolve—not only to deliver an incredible new range of data to drivers, but also to create new entertainment options for everyone in the car. A trend toward larger screens is already apparent, from the 17-inch touch screens of the Tesla Model S and X to the even larger prototypes seen in concept cars like the Sony Vision-S and the Byton M-Byte—big enough to encompass entertainment applications as well as the more traditional instrument panels and navigation—with holographic and other stereo 3D display techniques coming on strong. No wonder that, at its CES 2020 press conference, automotive component supplier Bosch cited estimates (by consultancy Global Market Insights) that the market for vehicle displays will more than double by 2025, reaching $30 billion worldwide.
HoloWire talked to some of the leading innovators in the automotive market about the latest in automotive display technology and the new applications they expect to see in the years to come.
3D Displays: More Data for Driving
Manufacturers are experimenting with new ways to display data, including bringing depth cues into play. Bosch earned a CES 2020 Innovation Award for its passive 3D technology, which can be added to LCD, OLED or microLED screens. The stereo effect is generated by an optical stack that refracts light to create different views of the image for stereo perception. The depth effect can be seen by the front passenger and others in the vehicle and may be tuned on a per-user basis. “There is a high degree of design freedom for the car manufacturers and hardly any additional space required compared to a standard display,” Bosch’s display product team tells us.
Stereoscopic technology can enhance the rear camera and nav displays, as well. “When parking, the rear-view camera image is more realistic, allowing detection of obstacles earlier,” Bosch says. “Drivers can get an even better idea of how much space they have left between the rear fender and, say, a parking garage wall. When navigating street canyons, this 3D effect also plays a decisive role as the spatial depth of the map display makes it immediately clear which building marks the next turn.”
Visteon provides an instrument cluster for the Peugeot 208 that takes a different approach to depth, using a dual-display system. An image is projected from one seven-inch LCD display onto a semi-transparent layer positioned at an angle so that the image, though flat, seems to be floating about 15mm in front of the 10.25-inch background LCD screen. According to Elijah Auger, Visteon’s senior manager of display products, its advantage over competing autostereoscopic solutions is that the second image really exists in space, rather than forcing the brain to reassemble left-eye and right-eye views into a single binocular image.
Both Bosch and Visteon agree that bringing certain warnings or notifications forward in a stereo display can make drivers notice them more quickly. Auger says that’s part of the appeal for auto makers as well as drivers. “Aside from the simple wow factor of having a 3D display, the [safety] benefit is what really makes the value proposition for this technology,” he says. “Even if you don’t have your eyes on the display, [a 3D warning] grabs your attention and forces you to focus on mission-critical information.”
Volvo’s Mixed Reality R&D
As car makers consider new sizes and shapes of screens, mixed-reality applications are becoming critical to their own R&D. At Volvo, designers and engineers are leveraging the Unity game engine and employing XR-1 augmented-reality (AR) headsets from Finland-based start-up Varjo for design iterations and safety trials. Varjo’s headset was suited for the project because of its especially high resolution, according to Timmy Ghiurau, Volvo’s senior lead for XR and virtual experiences. “If you do any type of validation or verification work, you need to be able to tell apart different materials, colors and textures, so for us that was crucial,” he explains. “We were pushing their boundaries when it comes to latency. And we even wanted to drive with it.”
Drive with it? That’s right. Much of Volvo’s design work takes place on ergonomic rigs with VR headsets, where testers can enjoy a simulated ride. But the company wanted the freedom to put testers with AR headgear on the road—the literal road—which posed a new set of challenges. “When I present the project at conferences, I name my talk, ‘Snap Back to Reality, Here Comes Gravity,’” Ghiurau says. “As in, yes, it’s fun to work with VR in the lab, but when you put it on the road, you face gravity. You need to take care of so many other aspects that you never thought would work against you, such as being in motion.”
But it’s worth the effort. Employing closely supervised driving routes on completely blocked roads with no other drivers, with failsafe systems in place to stop the car in case of emergency, drivers wearing Varjo headsets can evaluate different human-machine interfaces (HMIs), heads-up displays (HUDs), and sizes and positions of screens under real, not simulated, driving conditions. Varjo’s gear also incorporates exceptional eye-tracking, which helped Volvo’s engineers analyze driver reactions and identify potential distractions. And, Ghiurau says, using the same hardware and software across the company made it easy for the design and engineering teams to work together and share knowledge.
Extending the Driver’s Senses
Volvo recently announced that its 2022 XC90 will incorporate LIDAR hardware from Volvo’s partner Luminar, a big first step toward autonomous highway driving. (An over-the-air software update will be required to activate the feature.) For now, LIDAR is being used to extend the vision of the car for self-driving purposes. But it’s not too hard to imagine how LIDAR data could be used to extend the driver’s vision, too, via infographic overlays in augmented reality (AR). This might require a head-mounted device, or perhaps displays could be incorporated in the car’s windows, windshield and mirrors.
Autonomous vehicle specialists sometimes refer to SLAM, short for simultaneous localization and mapping. That’s the technology that allows driverless cars to orient themselves in an environment and make decisions on how to maneuver. “When we started the Varjo project, I was thinking that if you have a headset on, together with a car that has LIDAR and cameras, you become the most complex SLAM AR device,” Ghiurau says. “The car’s sensors can map locations and recognize the planes and road around you, and you could augment virtual objects and anchor things in a more stable way that really extends your vision.”
Spotlight on Headlights
At German lighting specialist HELLA, engineers are making adaptive headlights and taillights a part of the automotive information ecosystem. “Styling and safety are two important aspects of our technologies,” explains Dr. Daniela Karthaus, head of the lighting innovation optics department at HELLA. “We have technologies to realize nice welcomes and farewells when you lock or unlock your car, such as an animation of the signal lamps that shows you the car is locked. But there are a lot of aspects related to safety.”
For example, Karthaus cites the company’s glare-free high beams, which detect other drivers and eliminate light shining in their direction. That means you can drive with your high beams on without blinding oncoming traffic. HELLA’s HD84 headlamp, adopted for example by Mercedes-Benz, features 84 LEDs arranged in three rows. When the car senses a car or motorcycle approaching, the corresponding LEDs are shut off. Future iterations of the technology could use different technologies, such as a DLP micromirror system or segmented LEDs with thousands of pixels, each of which has its own advantages and disadvantages, according to Karthaus. “Of course, now we are talking about many more pixels,” she says, referring to the new HELLA SSL HD. “In our current predevelopment, we are looking at systems with more than 30,000 pixels.”
So the next generation of HELLA’s technology could be even more precise in the ways it puts light on the road. Imagine a headlamp that uses light to outline the shape of a bicyclist on the road as you approach, giving you maximum visibility as you safely pass. Or one that projects virtual lane markers showing exactly how wide your car is in relation to the road.
And Karthaus is looking ahead to holographic technologies. Today’s autostereoscopic displays either use eye-tracking technology to generate right-eye and left-eye stereo views or require viewers to be in pre-determined sweet spots (or wear headgear) to see the effect. Truly holographic displays are different — they project rays of light to recreate images with full parallax that are visible from any angle, without glasses, and are visually indistinguishable from real objects. These could contribute to a car’s styling and aesthetics but might also come into play on the road. A car’s rear lamp could display a holographic stop sign if it must make a sudden stop or pop up a holographic warning if the car behind it is tailgating. That’s one of the reasons why Karthaus has a special interest in holography—it was the topic of her doctoral thesis—and why HELLA is an early investor in a Silicon Valley startup called Light Field Lab that is developing advanced holographic display technology.
“We want to use holograms to generate special designs — for example, in the rear lamp,” Karthaus explains. “Classic holography is a little bit different from what Light Field Lab is doing, and we want to evaluate the possibilities for using their technology in exterior lighting, as well as other applications for the interior.” Those are still being researched as HELLA considers whether holographic technology is compatible with the material requirements for automobile lighting or if certain adaptations would need to be made.
Immersive Experiences of the Future
It’s a sure bet that today’s display technology will improve dramatically, no matter how the various solutions develop. Instrument panels that are just starting to incorporate depth effects today may become genuinely holographic. Windshields, side windows and mirrors may all become venues for AR-style HUDs that display more detailed information about the world outside the vehicle. Exterior lights may become more cleverly reactive to road hazards. And new types of gestural control, complete with haptic feedback, may be in the cards. "Light, olfactory, touch and feel, gestures and holographic screens will play an important role, addressing all the senses and making the driving experience extraordinary and unique,” the Bosch team tells us. To that end, Bosch notes that it is also investing in the holographic display developments at Light Field Lab. “Bosch is approaching the next generation of 3D displays and holographic screens, building a bridge between today and tomorrow and between the consumer and automotive worlds.”
And how far could this extraordinary experience extend? Consider the Field Trip to Mars. In 2016, the visual-effects house Framestore helped design a very unique school bus, using Unreal Engine and imaginative VR techniques, that could convince the kids on board, if only for a few minutes, that they were being ferried across a Martian landscape. Framestore replaced the bus’s windows with transparent 4K displays and used Unreal Engine to render a simulated Mars environment, in real time, as it would appear through those windows. A motion-tracking system ensured the rendered environment matched what the students inside felt as the bus rumbled through the streets of Washington, D.C.
Imagine a similar technology that could block out the windows of a passenger sedan, using immersive images, surround-sound audio, and more to transport its occupants for the duration of their ride, and you’ll get a sense of what’s possible in a holographic future. It’s just around the corner.