Models · Research release · 28 October 2026

Drishya Tiny

How far does about $21 of GPU time go for a video model? A research release, failures included.

Drishya Tiny is a 212M-parameter video generator trained from scratch, using the frozen Wan2.2 VAE and Flan-T5-Base text encoder. It turns a text prompt, or a text prompt plus a first frame, into a 2-second 256×256 clip.

We built it as an experiment: how much video model can about $21 of GPU time buy, and what moves prompt-following at this size? So we are releasing almost everything, including what didn’t work. Give it a sentence, or a photo and a sentence, and it makes a 2-second clip. It is good at waves, skies, snow and fire. Ask it for two things in one clip and it mostly can’t.

Read the release post

Try it now · in your browser · free, no login

Type a prompt and make a 2-second clip, on your own device.

Drishya Tiny runs entirely inside this tab: the 665 MB of model files download once and are kept by your browser. Give it a prompt, and optionally a photo to animate. Runs on your device; nothing you type leaves your browser, and photos never leave your device.

Checking this browser…

Needs WebGPU (desktop Chrome or Edge); other browsers fall back to a slower CPU path. Once the files are cached, a clip takes a few seconds on a recent Mac. The browser version uses a tiny decoder (TAEW2.2, MIT, by Ollin Boer Bohan), so clips are softer than the full model’s. Long, descriptive prompts of nature and scenery work best. Every clip carries a visible “AI-generated” mark.

Or try it on Hugging Face (GPU)

The result

Close on video quality. About half on following the prompt.

On VBench, its video quality is 75.6, level with the 256px VideoCrafter-0.9 (74.9) and close to ModelScope (78.1), both built on Stable Diffusion. Its generator was trained from scratch and is about a sixth the size of ModelScope’s 1.35B UNet. Prompt-following (semantic 32.3) is about half theirs (65–67).

Our own sampling and scoring with the official VBench code, not a certified leaderboard entry (21–22 October 2026). The other models’ numbers are published ones, checked on 25 October 2026.

Run it in your browser Try it on Hugging Face (GPU) Weights on Hugging Face Code, logs and cost ledger on GitHub Technical report

Generator: 212M; 211.92M, trained from random weights. With the frozen Flan-T5-Base and Wan2.2 VAE decoder, about 877M at inference
Training: 15 h; One L40S GPU, 180,080 steps at batch 64, 11.53M clips seen
Cost: ≈$21; The release model’s own path: data prep $7.40 plus training $13.76. Everything, failures included: $57.43
Output: 2 s; 17 frames at 8 fps, 256×256, from a prompt or a prompt plus a first frame

What it makes

Text-to-video clips from our own prompts of the kind it does well, at the default settings, each shown at its real size and looped. Two seeds per prompt, nothing else picked, so you see the weaker take too.

“Ocean waves rolling onto a sandy beach on a sunny day, blue sky with white clouds, the camera is static.”
Seeds 0 and 1
“Sunset time lapse at the beach with moving clouds and orange colors in the sky.”
Seeds 0 and 1
“Snow-covered mountain peaks under a blue sky, clouds drifting slowly past the summit.”
Seeds 0 and 1
“Ocean waves crashing against a rocky cliff, white foam and spray, aerial view.”
Seeds 0 and 1
“A sailboat sailing on the open sea on a sunny day, gentle waves.”
Seeds 0 and 1

From a photo

Give it a photo and a prompt and the photo becomes the first frame; the prompt says what should move. Three public-domain photos, seed 0, default settings. Later frames lose detail and the motion is small, as it is from text. The in-browser version at the top of this page does both: add a photo there to animate it.

Photo
Clip
“A large white cloud drifting slowly over a grassy field with trees, summer day, gentle wind.”
Photo
Clip
“An old stone building in a green field, clouds drifting across a blue sky, the grass moving in the wind.”
Photo
Clip
“A view over a park with a lake surrounded by trees, clouds moving slowly across the sky.”

Photos: public domain / CC0 (PD12M).

Where it breaks

The blooper reel: prompts that ask for two things, a layout or written words, with the same settings and the same two seeds. It almost never draws two named objects (VBench multiple objects 2.9%). Here the cat and the dog merge, the cup and the book never appear, and HELLO never gets written.

“A cat and a dog sitting together on a red sofa”
Seeds 0 and 1
“A red cup to the left of a blue book on a wooden table”
Seeds 0 and 1
“A person writing the word HELLO on a whiteboard”
Seeds 0 and 1

How it compares

VBench scores a video model on many separate things. Quality asks whether the clip looks and moves like a video: steady subjects, smooth motion, no flicker, a good-looking frame. Semantic asks whether it shows what the prompt asked for: the right object, two objects, a colour, a place, an action. The fair peers are the models the VBench team scored at the same 256×256 size and about 2 seconds.

VBench at 256×256, about 2 seconds, higher is better. Video quality: Drishya Tiny (212M generator) 75.59, ModelScope T2V (1.7B system) 78.05, VideoCrafter-0.9 74.91, InstructVideo 81.56. Prompt-following (semantic): Drishya Tiny 32.26, ModelScope T2V 66.54, VideoCrafter-0.9 65.46, InstructVideo 56.81. Drishya Tiny is our own sampling and scoring with the official VBench code, not a certified leaderboard entry; the others are published numbers.

VBench text-to-video scores, higher is better
Model	Size and lineage	Total	Quality	Semantic
Drishya Tiny	212M generator (from scratch)	66.92	75.59	32.26
ModelScope T2V	1.7B system (UNet ≈ 1.35B), from SD 2.1	75.75	78.05	66.54
VideoCrafter-0.9	Not stated; spatial layers from SD	73.02	74.91	65.46
InstructVideo	A reward fine-tune of ModelScope	76.61	81.56	56.81

Drishya Tiny: our own sampling and scoring with the official VBench code (commit fd18b3d055cb), not a certified leaderboard entry; 946 standard prompts × 5 videos, CFG 5, shift 5, 50 Euler steps; 21–22 October 2026. The other rows are published numbers: the VBench team’s own sampling and scoring at 256×256 and 16 frames, checked against the official leaderboard on 25 October 2026. Every peer has Stable Diffusion in its lineage.

Every VBench dimension

The split is sharp. It is smooth, steady and colourful, and it really moves (dynamic degree 80.56). It rarely puts the asked-for things in the clip, and almost never two of them.

Quality · %
Subject consistency	87.54
Background consistency	94.52
Temporal flickering	98.78
Motion smoothness	96.87
Dynamic degree	80.56
Aesthetic quality	34.77
Imaging quality	51.49

Semantic · %
Object class	24.97
Multiple objects	2.91
Human action	20.00
Colour	84.39
Spatial relationship	6.03
Scene	6.03
Appearance style	20.40
Temporal style	12.15
Overall consistency	14.53

Image-to-video

Give it a first frame and a prompt and it keeps the frame and animates it. On VBench’s image-to-video suite it is level with VideoCrafter-I2V and behind the two other 256px models.

VBench image-to-video total, higher is better
Model	I2V total
DynamiCrafter-256	85.25
ConsistI2V	84.07
Drishya Tiny	82.62
VideoCrafter-I2V	82.57

VBench image-to-video total on the 1:1 crops (1,118 images × 5 videos). Drishya Tiny: our own sampling and scoring with the official code, not a certified leaderboard entry; quality 75.00, I2V 90.24. Its 355 crops are byte-identical to the official copy. Others: published numbers, checked 25 October 2026.

Where it falls short

Prompt-following is about half the peers’ (VBench semantic 32.3, against about 65–67 for the 1–1.7B 256px peers). It almost never draws two named objects (multiple objects 2.9%), rarely gets spatial relations (6.0%) or named scenes (6.0%), and gets the object class right about a quarter of the time (25.0%).
Faces and hands smear. Text melts.
Short and small: 2 seconds, 256×256, 8 frames a second. Motion is small.
Image-to-video drifts: later frames lose detail and blur over the 2 seconds; faces and text melt.
Visual quality is modest: aesthetic 34.8%, imaging 51.5%.
Data fixes alone won’t close the gap at this size: our controlled test moved prompt-following by 3–4 points.
Not certified: the VBench numbers are our own sampling and scoring.
Non-commercial weights: the training clips come from HD-VILA-100M, whose terms allow research use only.
It works best on nature and scenery (water, waves, sky, snow, boats, fire); people degrade. That is our impression from picked clips, not a measurement.

How it was trained

1
Data
About 300K clips from VidGen-1M (whose clips come from Microsoft’s HD-VILA-100M), cut into 475,157 two-second windows and encoded once with the Wan2.2 VAE. VidGen-1M’s own captions, read up to 77 tokens. No photos, no synthetic data. 2,048 clips held out for testing, never trained on.
2
Model
A 211.92M diffusion transformer (16 layers, width 768, 12 heads) from random weights. Flan-T5-Base reads the prompt and the Wan2.2 VAE turns the latents into frames; both stay frozen.
3
Run
15 hours on one L40S GPU on 20 October 2026: 180,080 steps at batch 64, rectified flow. A quarter of the steps train image-to-video by replacing the first frame. The recipe is the one Stage 1 tested on skies, unchanged by Stage 2.
4
Gate
A test written before the run, on held-out clips: FVD-2048 235.7, motion 0.73 of real, CLIP gap 78% of real. It passed.

The pre-registered release gate on held-out VidGen-1M clips
Check	Result	Reference
Text-to-video FVD-2048 (lower is better)	235.7	Real clips frozen on frame 0: 660.4
Motion, against real clips	0.73	Rule: 0.5 to 2.0
CLIP gap, own caption minus shuffled	20.15	Real clips 25.73, so 78% of real
Image-to-video FVD-512	240.3	Motion 0.89, CLIP gap 23.7

EMA weights, 50 Euler steps, shift 3, CFG 5, on 2,048 held-out clips. Our FVD code matches StyleGAN-V’s official FVD code to a relative 1e-10 on identical frames. CLIP gap uses OpenCLIP ViT-B/32.

What it cost, failures included

The release model’s own path cost about $21: the data step, $7.40 on one RTX 5090 for about 6.2 hours, and the training run, $13.76 on one L40S. Everything we tried to get there cost $57.43 of GPU time, across 55 machine rows, every failed attempt included.

GPU cost per stage
Stage	GPU cost	What it bought
Day-0 checks (Mac, Colab probe, research)	$0.03	Is it feasible?
Stage 1: method check on skies	$7.14	The recipe works
Stage 2: three techniques vs the plain recipe	$7.20	Keep the plain recipe
Stage 3: data and training of the release model	$24.71	The release model
Stage 4: sampling sweep and VBench	$8.48	The evaluation
Stage 5a: three controlled arms and scoring	$9.90	The data test
Total	$57.46	$57.43 of GPU plus $0.03 of research on day 0

Not included: the pretraining of the borrowed Wan2.2 VAE and Flan-T5 (not ours), people’s time, electricity and storage. Stages 1–3 are provider bills or rate × time; Stages 4–5a are rate × minutes. For a smaller bill still, MiniDance trained a scenery-only model for even less (about $2).

The model

Generator: 211.92M parameters: a video diffusion transformer, width 768, 16 layers, 12 heads, patch 1×2×2; trained from random weights
Text encoder: Flan-T5-Base encoder, 109.63M, frozen; prompts are cut at 77 tokens
Video VAE: Wan2.2 VAE, 704.69M, frozen (the decoder, 555.05M, runs at inference)
At inference: About 877M for text-to-video; about 1.03B for image-to-video, which adds the VAE encoder
Output: 17 frames at 8 fps (about 2 s), 256×256
Sampling: 50 Euler steps, CFG 5, shift 5 for the VBench numbers
Training data: VidGen-1M clips (from HD-VILA-100M): 475,157 two-second windows; the clips are not redistributed
In the browser: onnxruntime-web (WebGPU, or the CPU), with the TAEW2.2 tiny decoder (MIT, by Ollin Boer Bohan) in place of the Wan2.2 VAE decoder, so clips are softer
Licence: Weights CC BY-NC 4.0; code Apache-2.0

Licence

The weights are non-commercial because the training clips come from HD-VILA-100M (Microsoft), whose terms allow research use only. We trained on clips from VidGen-1M, which draws on it, and we do not redistribute them: the release lists the clip IDs and the scripts to prepare them. The code is Apache-2.0, so you can train the same recipe on clips you have the rights to. Thank you to the VidGen-1M and HD-VILA-100M authors, to the Wan team for the Wan2.2 VAE, to Google for Flan-T5, to Ollin Boer Bohan for TAEW2.2 (MIT), the tiny decoder the browser version uses, and to the VBench team for an open benchmark.

How to cite

@misc{udgamlabs2026drishyatiny,
  title  = {Drishya Tiny: a 212M video generator trained from scratch with a frozen Wan2.2 VAE and Flan-T5-Base},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Drishya-Tiny}
}

Everything we released

Drishya Tiny on Hugging Face
The release weights (EMA, safetensors) and the card
Drishya Tiny on Hugging Face Spaces
Try it without downloading
GitHub: udgamlabs/drishya-tiny
The code, the recipe, the training logs, the cost ledger and the clip IDs