Deepgram

That made inference efficiency a natural target and Deepgram turned to AWS AI chips to improve it. They worked with AWS to bring its Nova-3 speech-to-text model to AWS Trainium2 for offline batch workloads such as contact-center analytics and medical transcription.

From that working baseline, we kicked off the optimization work by replacing several layers with fused kernels from the public NKI Library (NKILib), a collection of pre-built reference kernels written with the Neuron Kernel Interface (NKI).

At each step, a different part of the Neuron stack accelerates the work: NKILib provides drop-in replacement kernels, Neuron Explorer is the go-to tool for profiling Neuron workloads, and the Neuron Agentic Development (NAD) toolkit speeds up profiling analysis and kernel tuning…

At each step, a different part of the Neuron stack accelerates the work: NKILib provides drop-in replacement kernels, Neuron Explorer is the go-to tool for profiling Neuron workloads, and the Neuron Agentic Development (NAD) toolkit speeds up profiling analysis and kernel tuning…

Deepgram’s production serving stack is written in Rust

porting Nova-3 to Trainium2 required more than compiling the model

we demonstrated how Deepgram and the AWS FDE ported the Nova-3 speech-to-text PyTorch model to Trainium hardware