Back to Portfolio

Local AI Video Pipeline

Pravinraj P
Pravinraj P June 2026 · 5 min read

We generated a full educational video on an Intel i5 with integrated graphics—no GPU, no video generation model, and no cloud.

The output video (embedded above) is a 2-minute and 47-second educational explainer on the Stack data structure, demonstrating the LIFO (Last In First Out) principle through clean, programmatic 2D vector animations of colored blocks being pushed and popped, accompanied by synchronized synthesized audio narration and crisp typography.

We built this system for the AI Video Generation Hackathon at Rajalakshmi Engineering College (REC), Chennai on June 6, 2026. Video production has traditionally been a resource-heavy process. When modern workflows integrate AI, they typically outsource the work to massive cloud computing clusters or mandate high-end local GPUs. This adds cost, complexity, and massive external dependencies to a project.

Our goal was to bypass these requirements entirely. By designing an architecture based on local, CPU-bound execution, we created a deterministic pipeline that runs end-to-end on everyday consumer laptops. Unlike probabilistic generative models that yield different pixel outputs on every run, this pipeline is entirely deterministic because it pairs fixed-seed speech synthesis (Kokoro TTS) with exact, code-defined vector rendering (Manim) to guarantee the identical video output byte-for-byte on every execution.

“The constraint that shaped every decision wasn't compute. It was: what does the next stage actually need from this one?”

By answering that single question, the architecture fell into place. We split the pipeline into two distinct halves: an automated backend that generates the timing foundation, and a manual frontend that compiles code-based visual scenes.


The Automated Pipeline

The first half of the system runs completely end-to-end without any manual intervention. Its job is to generate and align the spoken audio, producing a highly accurate timing file that orchestrates the visual layer.

First, a local language model generates the script as a structured array of sentence strings. This script is fed directly to Kokoro Text-to-Speech (TTS), a highly optimized speech synthesis model that runs exceptionally fast on CPU architectures, producing a single .wav voiceover file.

Finally, WhisperX processes the audio file. It aligns the speech sequence and outputs a scene-level .json timeline packet. In this JSON, each sentence string is paired precisely with its start and end timestamps. This timeline serves as the temporal skeleton of the video.

The Manual Compiling Layer

The output of the automated layer is fed directly into the manual rendering engine. Rather than generating pixels or frames via AI models, we write the animation instructions programmatically.

We use Claude and DeepSeek to read the scene JSON and generate Manim animation code. We chose these models specifically because they do not have video generation capabilities. They can only write code. This is a deliberate architectural feature, not a limitation: instead of producing a video directly, they produce Manim scenes that we can easily control, inspect, render, and audit.

To translate the WhisperX-generated scene JSON into functional Manim animations, we construct a structured prompt that passes the raw timing segments and the corresponding transcript text to the LLM. The prompt instructs the model to map each segment to a distinct visual event, calculate the exact duration of each animation run-time using the timestamp offsets (e.g., run_time = end - start) to ensure audio-video synchronization, and map logical operations to specific geometric mobjects—such as representing pushing and popping as translation animations of stacked rectangles. By defining styling presets, margins, and movement constraints in the prompt, the model outputs clean, syntactic Python code that inherits from Manim's Scene class, ready for immediate compilation.

Manim then renders the vector animations on the local CPU, and the final step merges the synthesized audio track and rendered video frames into a clean, finished output file.

The Trade-Off: Why Top AI Companies Don't Do This

If this CPU-bound, code-driven approach is so efficient and inexpensive, why aren't top AI companies adopting it? The answer lies in the fundamental trade-offs of the rendering layer.

Our pipeline relies on Manim, which is designed to render vector graphics, mathematical equations, geometric shapes, and clean diagrams. While this is perfect for technical explainers and educational walkthroughs, it cannot render the real world. Generating a photorealistic video of a landscape, a person speaking, or complex physical world interactions requires massive diffusion models that synthesize pixels directly from statistical noise. For those use cases, neural pixel generation is mandatory; but for clean, structured data visualization, code-driven vector renderers are superior.


The Pipeline Architecture

The entire pipeline flows through a chain of clear, auditable artifacts. Every stage has a defined input and a defined output. Nothing is hidden, and each component can be inspected or replaced independently.

Local AI Video Pipeline Architecture Diagram

Ultimately, this architecture demonstrates that decoupling generation from rendering yields an extremely robust, customizable, and inexpensive production engine.

What We'd Do Differently

In a future iteration, we would transition from manual visual compilation to an autonomous agentic loop. We would construct an orchestrator that feeds the scene JSON into an LLM generator, runs the output script in a sandboxed Manim execution environment, parses any compiler tracebacks, and automatically feeds errors back to the model for self-correction. Additionally, we would implement dynamic prosody in the TTS engine to match visual pacing, and integrate a pre-indexed vector asset library so the model can insert complex, custom SVG illustrations rather than manually constructing every drawing from raw mathematical primitives.