The Camera Kept Going Back Down the Alley
A 15-second flight over an old hillside city, rendered at full portrait resolution on a 16 GB card — and the one sentence that kept dragging the camera back to where it started.
TL;DR — The shot was supposed to lift out of a narrow alley and keep flying. It kept diving back in, and no amount of "stay high" fixed it, because every segment began by telling the model it was looking at an alley. Along the way: MiniMax H3 runs at 1088×1920 on 16 GB, out-resolves LTX 2.5, and tolerates a text encoder a third the size.
It begins with a still. A stepped lane in an old hillside district at dusk, rust on the railings, power lines slung across the sky, a lit vending machine halfway down, and above it all a cloud deck torn open with light. I wanted the viewer to leave the ground: rise from the steps, clear the roofs, and go toward the break in the clouds. Fifteen seconds, portrait, full size.
A question of two megapixels
The first attempt with MiniMax H3 came out at 768×1344 and looked flat, so I moved to LTX 2.5 at 1088×1920. The obvious question followed: can H3 not do this size? The honest answer was that I hadn't tried. At 768×1344 H3 already touched 15.1 GB on this RTX 5080; doubling the pixels looked like a guaranteed crash.
It wasn't. The node that feeds H3 a first frame will take any width and height up to 16,384 in steps of 32, so the barrier is purely memory, and kijai's KJNodes has two answers to that. One breaks the feed-forward layers into chunks along the token axis. The other, in its own words, "Reduces peak VRAM of the MiniMax H3 attention without changing the math." With both in the chain and the video decoded in tiles, seven full-size segments peaked between 15.12 and 15.44 GB and none fell over. Each took 868 to 950 seconds — about 3.2 times the smaller size, for 2.02 times the pixels.
What the extra pixels buy
Starting from the same still with the same camera sentence, H3's frame came out with edge sharpness of 14.5 against LTX's 8.3, and nearly two and a quarter times the movement: 18.1 against 8.1. Both held the first frame equally well, at a deviation of 3.3.
The wires and window frames are where you see it — crisp on top, softened below.
There was a twist in that same comparison. The sentence asked the camera to climb between the buildings. LTX rose; H3 took it as an invitation to travel forward down the lane. Only the vocabulary of a crane operator — the camera cranes straight up — lifted it off the steps. And the flatness that sent me to LTX in the first place turned out to be mine: my script had lost the part of a proven prompt that asks for one constant speed "with clear parallax as the foreground slides past the edges of the frame", along with a negative prompt that forbids static, frozen and jump-cut results.
Down again
Fifteen seconds is three H3 segments of 124 frames. The last frame of each becomes the first frame of the next, and the one duplicated frame at each seam is dropped, which leaves 370 frames, 15.49 seconds. The first complete version rose out of the lane as intended. Then, around the five-second seam, it plunged back between the houses and raced along a street, wires whipping past the lens.
Measured in one-and-a-half-second windows, the movement went from around 15 to 41.7. My instinct was to argue with the camera: stay high above the rooftops, hold your altitude, never drop into the streets. Segment 2 dived anyway, at 36.6 against 38.8 the first time.
The culprit was sitting in front of every instruction. To keep three segments looking like one place, each began with the same description of the setting, and that description opened with "A steep stepped alley". By the second segment the camera was already floating over the roofs, yet every time the model was told, before anything else, that it was in an alley. It went back to find it. When the later segments were rewritten to describe what the lens could actually see — high above an old hillside city, roofs and water tanks spreading toward a river — segment 2 stayed aloft at 15.1, and the whole shot settled to 15.8.
It is not finished in every respect. The final five seconds glide level rather than climbing into the cloud, and within the first segment the palette drifts from the still's bruised violet and gold toward something brighter and cleaner than I painted.
A smaller voice for the prompt
While this rendered, word arrived of ClipProj v3.1. H3 reads its prompt through an enormous vision-language model — the project's card describes "a Qwen3-VL-32B truncated to 50 layers — 15.7 GB in NVFP4 — solely to turn a prompt into a [seq, 5120] conditioning tensor." ClipProj replaces it with a 5.2 GB Qwen3-VL-4B and a 26 MB learned mapping from its output into the shape H3 expects. Its author is candid about the trade: "swapping the encoder costs about what re-rolling the seed costs."
I ran the crane-up opening again with only the encoder changed. Motion 14.5 against 15.1; sharpness 17.8 against 17.9; first-frame deviation 5.1 against 4.9. Both climbed out of the lane the same way. The render finished in 863.6 seconds instead of 875.9, and peak graphics memory barely moved, from 15.39 to 15.13 GB, because the encoder has left the GPU before the heavy work starts.
What changed was the rest of the computer. This machine has 32 GB of RAM, and ComfyUI's log shows it staging about 40 GB of models for each H3 run: 14,956 MB of encoder, 19,995 MB of video model, 4,965 MB and 576 MB of decoders. With ten gigabytes gone, the page file at peak fell from 16.1 GB to 4.7 GB and the least free memory rose from 442 MB to 1,980 MB — numbers helped along by a fresh restart before that run. The one worry I had going in — H3's first-frame node also shows the picture to the encoder, per the node source, and the projection was trained on text alone — left no mark I could measure. The encoder swap stays.
Appendix: everything needed to reproduce it
Hardware and software. RTX 5080 16 GB, 32 GB RAM, Windows, ComfyUI 0.37.0, KJNodes at e8e88f7, ComfyUI-ClipProj at c01ba8f. H3 files from the ComfyUI repackage; setup follows the official guide.
Node chain.
UNETLoader minimax_h3_fl2va_pruned_int8_convrot.safetensors
LoraLoaderModelOnly minimax_h3_fl2v_turbo_8step_v1.0_comfyui_bf16.safetensors strength 1.0
MiniMaxChunkFeedForward chunks 4, seq_threshold 4096 # KJNodes
MiniMaxLowVRAMAttention head_chunks 8 # KJNodes
CLIPLoader qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors type minimax
MiniMaxH3ImageToVideo width 1088, height 1920, length 124, first_frame = still
BasicScheduler simple, 8 steps
CFGGuider cfg 1.0
KSamplerSelect res_multistep
VAEDecodeTiled tile 512, overlap 64, temporal 64, temporal_overlap 8
CreateVideo 24 fps (+ VAEDecodeAudio for H3's own soundtrack)
Encoder swap for ClipProj. Files: Qwen3-VL-4B fp8 from Krea-2, projection from the model card, node from GitHub. Use the stock loader, not the node's resident loader — on a single 16 GB card a pinned encoder takes memory from the video model at every step.
CLIPLoader qwen3vl_4b_fp8_scaled.safetensors type krea2
ClipProjApply mmh3-4b-ClipProj-v3.1.safetensors
# wire ClipProjApply's CLIP output into every node the 32B CLIP fed:
# both CLIPTextEncode nodes and MiniMaxH3ImageToVideo
The three segment prompts and the negative, as run.
# segment 1 — first frame = the source still
A steep stepped alley in an old hillside city district at sunset, weathered walls, iron railings and tangled power lines on both sides, packed old rooftops with water tanks spreading out below, under a huge sky of glowing gold and violet cumulus with broad shafts of sunlight pouring through a break in the cloud. The camera cranes straight up vertically from the alley steps like a rising crane shot, balconies, railings and power lines sliding down out of the bottom of the frame as it lifts clear above the rooftops at one constant speed for the entire shot, never slowing, never pausing, never stopping, with clear parallax as the foreground slides past the edges of the frame. The clouds churn and the shafts of light sweep slowly across the scene. One unbroken take, no cut. Audio: wind and distant city hum, no cuts. Cinematic CG footage with the polish of a modern Unreal Engine 5 render: volumetric light, ray traced reflections, razor sharp detail, deep saturated colour, high dynamic range.
# segment 2 — first frame = last frame of segment 1
High above an old hillside city district at sunset, a vast sea of weathered tiled rooftops, water tanks, antennas and tangled power lines spreading far below toward a river and a distant skyline, under a huge sky of glowing gold and violet cumulus with broad shafts of sunlight pouring through a break in the cloud. The camera glides forward high above the rooftops like a bird, banking gently from side to side, the rooftops sweeping past far beneath it and the sunlit clouds opening up ahead at one constant speed for the entire shot, never slowing, never pausing, never stopping, with clear parallax as the foreground slides past the edges of the frame. The clouds churn and the shafts of light sweep slowly across the scene. One unbroken take, no cut. Audio: wind and distant city hum, no cuts. Cinematic CG footage with the polish of a modern Unreal Engine 5 render: volumetric light, ray traced reflections, razor sharp detail, deep saturated colour, high dynamic range.
# segment 3 — first frame = last frame of segment 2
High above an old hillside city district at sunset, a vast sea of weathered tiled rooftops, water tanks, antennas and tangled power lines spreading far below toward a river and a distant skyline, under a huge sky of glowing gold and violet cumulus with broad shafts of sunlight pouring through a break in the cloud. The camera climbs steadily toward the break in the clouds where the shafts of light pour through, the city shrinking away far below, until it glides up into the glow at one constant speed for the entire shot, never slowing, never pausing, never stopping, with clear parallax as the foreground slides past the edges of the frame. The clouds churn and the shafts of light sweep slowly across the scene. One unbroken take, no cut. Audio: wind and distant city hum, no cuts. Cinematic CG footage with the polish of a modern Unreal Engine 5 render: volumetric light, ray traced reflections, razor sharp detail, deep saturated colour, high dynamic range.
# negative (all segments)
blurry, soft focus, low detail, mushy, washed out, flat lighting, static, frozen, jump cut, scene change, deformed, distorted anatomy, extra limbs, watermark, text, interface
The prompts that produced the dive, for anyone who wants to see it happen.
# what segment 2 said the first time (camera dived back into the alley)
A steep stepped alley in an old hillside city district at sunset, weathered walls, iron railings and tangled power lines on both sides, packed old rooftops with water tanks spreading out below, under a huge sky of glowing gold and violet cumulus with broad shafts of sunlight pouring through a break in the cloud. The camera clears the rooftops and flies forward over the sea of tiled roofs and water tanks, banking gently like a bird, the rooftops sweeping past beneath it and the sunlit clouds opening up ahead [+ same TAIL and STYLE]
# the altitude fix that did NOT work — same scene sentence, stronger camera sentence
A steep stepped alley in an old hillside city district at sunset, weathered walls, iron railings and tangled power lines on both sides, packed old rooftops with water tanks spreading out below, under a huge sky of glowing gold and violet cumulus with broad shafts of sunlight pouring through a break in the cloud. The camera stays high above the rooftops and glides forward level over the sea of tiled roofs and water tanks, holding its altitude and never dropping down into the streets, banking gently like a bird, the rooftops sweeping past far beneath it and the sunlit clouds opening up ahead [+ same TAIL and STYLE]
Joining segments.
# hand the last frame of segment N to segment N+1 as its first frame
ffmpeg -sseof -0.05 -i segN.mp4 -update 1 -q:v 1 segN_last.png
# drop the duplicated boundary frame when joining (1/24 s of audio with it)
ffmpeg -i segN1.mp4 -vf "select=gte(n\,1),setpts=N/FRAME_RATE/TB" -af "atrim=start=0.0417,asetpts=N/SR/TB" segN1_trim.mp4
# 3 x 124 frames - 2 duplicates = 370 frames = 15.49 s
How the numbers were measured.
motion mean |frame difference|, greyscale, 160 px wide, frames 0.25 s apart (0-255)
edge sharpness mean of the top 5% luminance-gradient magnitudes x100, mid-clip frame at 1088x1920
first-frame dev. mean |first frame - source still|, greyscale 160x282 (0-255)
ClipProj A/B frames sampled at 6 fps and 272x480, so its sharpness and deviation
figures are on a different scale from the LTX table — compare within a table
FAQ
Why not stay with LTX, which renders five times faster?
For drafts, I would. For the final, H3's detail and movement were clearly ahead at the same size — 14.5 against 8.3 in edge sharpness — and the wait of roughly 45 minutes for 15 seconds was worth it here.
Why didn't telling the camera to stay high work?
Because the scene description repeated before each segment still named an alley. The model treated the place as settled and the camera instruction as negotiable. Change the place, and the camera follows.
What does ClipProj save on a machine with one GPU?
Mostly system memory. Render time fell by 12 seconds out of about 876 and graphics memory by a quarter of a gigabyte, but swap use at peak fell by more than two thirds.
What comes next for this shot?
A final segment that genuinely climbs into the cloud, and a way to keep the still's weathered palette through the first seconds instead of letting it brighten.
Further reading: the model is MiniMax H3, its ComfyUI nodes are documented in the official guide with source here; the memory patches are in KJNodes; the comparison model is LTX 2.5; the encoder work is ClipProj with its ComfyUI node and the Krea-2 encoder files.
Images and video: frames from the author's own renders on an RTX 5080. The first frame is a still generated locally with Qwen-Image 2.1.
← Back to all posts