WHY THIS MATTERSRare conditions cannot always be safely staged or reliably collected.
02 · CONTROLLED COUNTERFACTUALS
Keep the scene. Change the weather.
Conditional world generation can build these counterfactuals. NVIDIA Cosmos Transfer 2.5 combines the original drive, a weather prompt, and an edge guide to preserve structure while restaging the conditions.
controllable generation
COSMOS TRANSFER 2.5ACTIVE PATH: EDGE CONTROL
Inputs
TEXT PROMPTweather: fog
Original drive
Edge video
DepthSegmentation
DIFFUSION BACKBONECOSMOS TRANSFER 2.5
noise
Builds a clean video over time
ADAPTIVE SPATIOTEMPORAL CONTROL
Hold the scene together
tt+1t+2
The guide follows roads and objects through time.
Output
GENERATED VARIANT
Fog
COUNTERFACTUAL VIDEO
Restaged drive
Prompted weather with edge-guided structure.
CONCEPTUAL VIEWInformation flow is shown at a high level. Internal model layers are simplified.
TRY A CONDITION
Highlighted path: edge control. Depth and segmentation are additional structural controls.
03 · QUALITY GATE
Generation still needs a quality gate.
The same brief can yield weak weather, warped geometry, or unstable motion. A judge decides what enters the dataset.
Increasing N expands the search. It also creates more work for the judge.
Sample budget, N
N=1 offers no comparison. The first sample ships.
1 take on screen
YOU ARE THE JUDGE
Inspect the visible candidates. Select a take.
CUT!
03 · THE REVEAL
The lesson's balanced rubric first selects Take 04 at N=4.
DIRECTOR’S CUT · TAKE 04LESSON RUBRIC PICK
WHY TAKE 04?
01
Prompt adherenceFog spreads across the street rather than forming a local smoke cloud.
02
Structural fidelityRoad geometry, lanes, buildings, and vehicles stay coherent.
03
Temporal stabilityThe atmosphere and object identities remain steady through motion.
Take 03 is sharp. It barely follows the fog prompt. Take 07 is dramatic. Its fog looks like smoke. Quality has several dimensions.
THE SAME IDEA, IN NOTATION
x1,…,xN ∼ pθ(x|c)thenx* = arg max s(xi)
Research papers often call this judge a verifier. First apply hard gates. If no candidate is acceptable, abstain. Otherwise, score the eligible candidates with s and keep the highest score.
N=1Forced choice
Take 01 wins because it is the only option.
N=2More variety
We see a stronger atmosphere, but also a pink cast.
N=4Winner appears
Take 04 enters the pool and becomes our cut.
N=8Diminishing return
In this run, 4 more samples cost compute and the winner stays.
04 · THE SELECTOR BOTTLENECK
Build the judge.
The videos stay fixed. Change the priorities and watch the winner change.
LESSON QUALITY RUBRIC
What should count as good?
Try a preset. Then move the sliders. The weights express what your judge values.
These fixed 0-to-100 values are illustrative visual ratings for the ranking exercise. Benchmark evaluation requires validated metrics and reference labels.
JUDGE'S PICK
87.7 / 100
04
Take 04
Balances the requested weather with a coherent street.
⚠
Try Fog only. The judge rewards Take 07's smoke-like plume. More candidates give a weak judge more opportunities to optimize the wrong thing.
FROM LESSON TO DEPLOYMENT
Who gives the judge its scores?
Here, fixed illustrative ratings make the ranking mechanism easy to inspect. A production AV quality gate would combine calibrated human examples with automated evidence.
01Human calibrationDomain reviewers define what usable means and label a reference set.02Visual diagnosticsEdge, depth, detection, tracking, and flicker checks test structure and motion.03VLM reviewA video-language model checks condition match and describes visible failures.WorldModelBench, NeurIPS 202504Safety gateCritical lane, object, or motion failures trigger rejection before ranking.
A VLM can contribute evidence within a broader quality gate. Production signals should be calibrated against reviewed examples before they enter the decision rule. The demo uses fixed ratings; a VLM is presented as a possible extension.
05 · CHANGE THE WEATHER
Roll it again: rain and snow.
Reuse the same 4 seeds. Change only the weather prompt. Variation remains.
“Moderate fog fills the street… distant buildings are softened.”
Compare takes under the same weather condition.
OPTIONAL · BEHIND THE CAMERAHow were the videos made?Open production notes +
Cosmos generated each candidate. Ray ran 4 GPU actors. 8 seeds finished in 2 waves.
NVIDIA COSMOSvideo generation+RAYparallel actors
@ray.remote(num_gpus=1)
class CosmosActor:
def audition(self, seed):
return cosmos.generate(seed=seed)
actors = [CosmosActor.remote() for _ in range(4)]
takes = ray.get([
actors[i % 4].audition.remote(seed)
for i, seed in enumerate(seeds)
])
winner = select(takes, rubric)
Source videoThe scene to preserveEdge controlThe structure to preserve
FARNEAR
Depth controlAlternative structural control
Transfer 2.5 supports several structured controls. This experiment used the distilled edge checkpoint. The depth view above is illustrative.
1Conditionsource + prompt + edge
→
2GenerateCosmos Transfer 2.5
→
3Parallelize4 Ray actors · L40S
→
4Selectrubric + evidence
8 / 8successful fog generations
106 smean generation time
474 sobserved wall clock, including loads
42.1 GBGPU memory per actor
Show the exact fixed settings
Model edge/distilledSteps 4Guidance 4Edge weight 0.65Frames 93Resolution 1280×720 generated
06 · CHECK THE CUT
Got it?
2 questions. 30 seconds.
QUESTION 1 / 2
What changed across the 8 fog takes?
QUESTION 2 / 2
Why might a larger N fail to improve the final output?
0 / 2
Complete the knowledge check.
THE TAKEAWAY
Generation creates options. The judge turns options into a decision.
The generator-selector pattern now appears in reasoning, generation, alignment, and evaluation.
Built on NVIDIA Cosmos. Cosmos Transfer 2.5 source uses the Apache-2.0 license. Model weights use the NVIDIA Open Model License. These videos are educational outputs from the published example input. No endorsement is implied.