OpenAI’s Sora has something to do with Tesla’s FSD?

OpenAI recently announced Sora, a new large language model (LLM) that can generate short video clips and it drew some interest from the online community to make a comparison with Tesla’s Full Self Driving (FSD) technology.

Social media X user @drknowitall posted a video about Sora and how it relates to Tesla’s FSD version 12. The host, known as John, explained the occurrence of this generative artificial intelligence (AI) technology which can create up to a minute of video based on a prompt.

This capability is identical to OpenAI’s ChatGPT text-to-text framework. Instead of text, Sora has a text-to-video framework. The AI company said that it trained text-conditional diffusion models jointly on videos and images of different durations, resolutions, and aspect ratios.

It also utilizes a transformer architecture to operate on spacetime patches of video and image latent codes. Eventually, the artificial general intelligence leads to a video that is created with a three-dimensional perspective.

FSD:

Tesla has been promoting the FSD as the future of autonomy but it’s still not accessible for all users. The company keeps on testing FSD on the road and collecting data to train the FSD system.

FSD is an autosteering feature that operates on its own in city streets and can identify real-time traffic and signals while driving. This is not a fully autonomous technology and it requires the driver’s full attention.

Aside from this Beta stuff, FSD generates the entire traffic and scenarios dynamically and shows it on the screen. CEO, Elon Musk also commented about this capability in the video’s comment saying that Tesla has been able to generate real-world video with “accurate physics for about a year”.

However, Musk mentioned that the video created by FSD was solely based on Tesla cars and trained with driving data. “We have been short on training compute for FSD, so haven’t trained with another video” but he said that the company “certainly could”.

Later in the same post, Musk said that Tesla could try some Sora-type video generation later this year when it has “some spare capacity”.

Tesla’s new FSD BETA V12 is a breakthrough in the lineup, it has unsupervised AI, which can learn and operate on its own. Before V12, the company has been using manual code to teach the car how to perform in certain scenarios.