Teslas Karpathy on the technology behind his autopilot project


“We’re in bad shape when it comes to transportation. We have these metallic objects that are traveling very quickly with really high kinetic energy. We put meat in the control system; it is quite undesirable. It basically comes down to the fact that people are not that good at driving. You’re getting into big trouble, ”said Andrej Karpathy, Senior Director of AI at Tesla, at the CVPR 2021 event.

He sued the human driver, calling them “meat computers”. Karpathy said people drive in a tight loop with 1-ton objects at 80 mph, have a reaction latency of 250 ms, use situational awareness mirrors, and dodge distractions. On the other hand, automation in transportation offers many advantages, including a tight control loop, faster response latency (

Complete the survey: Using Behavioral Science to Analyze Customer Behavior

Tesla’s approach

Regarding Tesla’s approach to autonomy, Karpathy said the team is working towards fully self-driving (FSD) capabilities. Tesla founder Elon Musk has spoken in the past about his great ambitions in this direction. In fact, in 2020, Musk claimed the company was on the verge of meeting the basic requirements of level 5 autonomy, which does not require human driver input (however, Tesla later issued a rebuttal that the company is still at level 2).

Autopilot is Tesla’s suite of advanced driver assistance technology that offers features such as lane centering, traffic-aware cruise control, self-parking, semi-autonomous street navigation, automatic lane changes, and more.

hardware: The team builds silicon chips that power the self-driving software from scratch. The aim is to optimize architectural and micro-architectural aspects in order to achieve the maximum silicon output per watt. Other functions include performing floor plans, performance analysis, writing robust tests and scoreboards to verify functionality, implementing compilers and drivers to communicate with the chip, and production.

Neural Networks: Tesla applies deep neural network solutions to problems ranging from perception to control. The full build of the Autopilot Neural Network includes 48 networks that have been trained for over 70,000 GPU hours. Together, these networks output 1,000 different predictions at each time step. Tesla Vision is one of the applications of deep neural networks. Tesla Vision deconstructs the environment of the car on a much higher level than classic image processing techniques.

The on-board computer Hardware 3 operates the neural network developed by Tesla, which can process more than 40 times more data than previous generation systems. This offers a view that the human driver alone cannot access – simultaneous viewing in different directions and on wavelengths beyond the human senses.

Last year, Musk announced that the company was developing a neural network training computer called the Dojo to process huge amounts of video data. At the CVPR 2021 event, Karpathy introduced a predecessor for Dojo, which he touted as one of the fastest file systems in the world. He said the unnamed supercomputer has “720 nodes, each powered by eight of Nvidia’s A100 GPUs (the 80GB model), for a whopping 5,760 A100s across the system.”

See also

navigation: The autopilot’s Navigate function helps optimize the route and makes adjustments so that the vehicle is automatically steered to motorway junctions and exits based on the destination. The Autosteer + function helps navigate complex roads with advanced cameras, sensors and computing power. Smart Summon helps maneuver the vehicle around obstacles to find the right parking space.

Vision system: Karpathy said Tesla’s vision system is one of the best in the business. The cameras do most of the work in terms of perception, and the company is now planning to scrap sensors and move to a vision-only approach.

Evaluation infrastructure: Tesla builds open and closed hardware-in-the-loop assessment tools and infrastructure at scale to drive innovation, track performance improvements, and avoid regression. Tesla’s systems use anonymized extracts of characteristics from the fleet and integrate them into large suites of test cases.

Join our telegram group. Become part of a dedicated online community. Join here.

Subscribe to our newsletter

Get the latest updates and relevant offers by sharing your email.

Shraddha Goled

Shraddha Goled

I am a journalist with a postgraduate degree in Computer Network Engineering. When I’m not reading or writing, I can be scribbled to your heart’s content.


Leave A Reply