Exclusive: The Tesla AutoPilot – An In-Depth Look At The Technology Behind the Engineering Marvel


Tesla's Digital Cockpit - Nvidia's VCM

tesla feature 2Before we jump into the details of the Autopilot technology, there is something else that is worth a look into as well - Nvidia's technology in Tesla Vehicles. This is the tech that powers the digital cockpit inside the Model S and Model X. Although the company has been very serious about ADAS (Automated Driver Assist Systems) and DNNs on the road, it actually (currently) only supplies Tesla with the necessary hardware to drive the digital cockpit. This means that the 17.5 inch infotainment screen and the instrument cluster panel behind the steering wheel.

Nvidia's last financials indicated a very strong growth of their automotive department. The reason is ofcourse that their Tegra chips are increasingly in demand as the ultimate choice to power digital cockpit systems for various automobile vendors. Nvidia has also been aspiring to break into the ADAS business with its new Drive PX chip. Infact, Elon Musk was present at CES this year when the CEO of Nvidia demonstrated the capabilities of the Drive PX board. This caused many to speculate that the board was present inside the Tesla models. This is however, not true, and you would be forgiven for thinking that the latest Tesla vehicles contain the module.

tesla dashboard tegra

Tesla vehicles do not (currently) have Nvidia's Drive PX/CX board and instead rely on the Tegra K1 (VCM) to power the digital cockpit

We were also able to find out the exact Tegra model present in the new Tesla EVs.  Previously, the cars possessed a Tegra 3 processor, but all modern Model Xs and Model S cars will have the Nvidia VCM (Visual Computing Module). NVIDIA has a long-standing relationship with many automakers including Audi, Volkswagen and BMW. These partners are now using the Tegra Visual Computing Module (VCM) for their infotainment systems. The VCM is a highly flexible platform, incorporating an automotive-grade NVIDIA Tegra mobile processor with dedicated audio, video and image processors. Nvidia revealed their VCM module a year or so back and it consists of the Tegra K1 with a grand total of 384 GFlops. To put that into perspective, that is more power than last generation consoles (PS3/Xbox 360). 

Now, interestingly, when the VCM was revealed, it was stated to serve two purposes: driving infotainment systems and providing visual driving aid.  The infotainment capabilities include not only advanced material rendering (the speed gauges and instrument cluster behind the steering wheel) but actual gaming as well. This 3D rendering capability is being effectively leveraged in the indicators on the instrument panel that turn on and off during Autopilot. The dynamic lane markings and the car detection that is shown on screen is another example of this.

A slide from the CES presentation. @Nvidia Public Domain

The VCM can be easily inserted into a standard 1-DIN automotive enclosure (the size of a car stereo). This is also the same component that allows the Tesla's car’s dashboard infotainment system to be updated over the air, putting it on a different life cycle than the rest of the vehicle. The chip has 192 Kepler based cores and a TDP of around 5W. The Tegra K1 was  the first mobile chip by Nvidia to have CUDA implementation. It is the first mobile chip to surpass the 300 Gigaflops mark in Single Precision computing, thanks to its general purpose, GPGPU CUDA Cores.

The driving aid portion of the chip was stated to be capable of pedestrian detection, adaptive cruise control, collision detection, land departure warning and blind spot monitoring. However, the chip ended up being used solely for driving graphics and general housekeeping on the Tesla vehicles. All the bulk of ADAS and Semi-Autonomous driving fell onto a chip from a completely different vendor. This vendor is the magician behind the magical act of Tesla's Autopilot and is very well known in the automobile industry. I am referring ofcourse, to Mobileye.