AWS and Decart team up to unlock the potential of real-time AI video

AI startup Decart has moved into pole position in real-time video generation after striking a deal with Amazon Web Services to leverage its AWS Trainium chips for enhanced model training and inference.

The company said it’s combining Amazon’s custom-designed AI accelerators and the Amazon Bedrock service to create the industry’s first full-stack platform for low-latency, high-throughput AI video generation.

The partnership was announced during the “Infrastructure Innovations” keynote at the AWS re:Invent conference on Thursday, where Decart Chief Executive Dean Leitersdorf joined AWS Senior Vice President of Utility Computing Peter Desantis on stage to showcase how the startup’s models will benefit from running on the recently unveiled Trainium3 chips.

During the presentation, Leitersdorf said his company has reaped immense benefits from deploying its video models on Trainium, the cloud computing giant’s AI accelerators. Going forward, it’ll be one of the first AI companies to deploy its models on Amazon’s newly announced Trainium3 accelerators, which are expected to deliver even greater performance benefits.

It’s a key development that means Decart can now support high-performance video generation workloads in real time without any sacrifices in terms of quality. By running its models on Trainium chips, which are more configurable than traditional GPUs, the company believes it will finally unlock the potential of real-time video creation, enabling new experiences in gaming, entertainment, advertising, social media, e-commerce and more.

The next frontier in AI video

Real-time video generation is an emerging discipline within the AI video segment that can only be supported with extremely low latency and highly-performant cloud-based infrastructure.

Unlike traditional video models such as OpenAI’s Sora 2, Google’s Veo 3.1 and Runway Gen-4.5, Decart’s Lucy is designed to generate video instantaneously, so that the content can be streamed as soon as the user enters their prompt, without any delays. While Sora 2, for example, can take up to a minute to process a user’s prompt, Lucy’s outputs begin in milliseconds.

“Generative video is one of the most compute-intensive challenges in AI,” Leitersdorf said in a press release. “By combining Decart’s real-time video models with AWS Trainium3, we’re making real-time video generation practical and cost-effective at scale.”

Decart isn’t alone in pushing the boundaries of real-time AI video. Vmake, for example, offers models that enable social media creators to edit live video streams on platforms like Instagram and TikTok and add special effects as they’re broadcasting.

However, Decart says its decision to adopt Amazon’s Trainum chips will give it an edge over others, because it means it no longer has to make any compromises in terms of video quality. By running Lucy on Trainium3, it hopes to be able to improve on its current 30 frames-per-second outputs and generate live video at up to 100 FPS, while reducing its time-to-first frame to less than 40 milliseconds. Trainum3, with its centralized SRAM, low-latency interconnects and efficient FLOPs, is key to realizing these ambitions.

“Trainium3 has been a huge enabler for our workload – and creating this new GenAI category of real time visual intelligence,” said Leitersdorf on stage at re:Invent. “At Decart, we have a lot of proprietary IP that allows us to train inference models much faster and more efficiently, and we worked closely with the AWS team to port our model stack, to port our infrastructure to Trainium, to let us work with Trainium for building models and running them.”

New creative possibilities

By generating high-fidelity AI video in real time, Decart says it can power use cases that simply weren’t possible before. One of the most compelling applications will be in live gaming, where video clips can be incorporated into open-ended video games to generate environments based on player interactions.

In social media, influencers can integrate AI video into their live streams to make it appear as if they’re broadcasting from the streets of any city in the world, or even a magical fantasy world. Real-time video also creates new possibilities for interactive entertainment, with directors able to personalize TV shows for each viewer. In ecommerce, brands can create video tools for shoppers to model new clothing and accessories in any setting.

After showcasing some examples of these use cases at re:Invent, Leitersdorf passed the baton to the ecosystem of AI app developers. “Frankly, everything you just saw, all these experiences, we didn’t come up with them – it was builders,” he said.

“It was developers that took our API. You guys know your industries way better than we do, and you can understand how to take these models and extract value out of them for your customers, for your industries.”

Plug-and-play live visual intelligence

The partnership with Amazon is what will make these applications possible. With Lucy now available on Amazon Bedrock, developers can integrate the model into their cloud applications to create advanced, real-time video engines on demand and scale them instantly, without worrying about the underlying infrastructure. It dramatically lowers the barrier to entry for instantaneous video generation, unlocking new avenues for creativity to shine.

“I’m really excited to be here and announce with AWS this new category of GenAI live visual intelligence. We’re taking it to every industry, every market, at any scale, powered by Trainium3, using Decart models. Everything we build, we put it on our API,” said Leitersdorf to conclude his presentation. “We can take stuff that’s in our imagination and connect it to what we see with our eyes, in reality, live. I really can’t wait to see what all of you build with it.”