1. Inside This Guide: Playable Worlds vs. AI Cinema
Modern visual AI splits into two distinct superpowers: real-time playable world models (Google Project Genie) that let you walk inside a living set, and deterministic cinema engines (Runway & Kling) that render locked shots for an editing timeline.
Quick Navigation
Core Mental Model: One Concept, Two Complementary Engines
When to Use Interactive World Models vs. Shot-Based Video Engines
- Live Camera Ownership: Steer with
WASD, jump with Spacebar, and pan with Arrow keys in real time. - Spatial Memory (~60s): Walk around a corner, turn back, and see rooms and props hold geometry.
- Best Use: Level blockouts, architectural walkthroughs, and harvesting exact angle stills for downstream shots.
- Frame-Exact Anchoring: Lock
first_frameandend_framecompositions, character sheets, and 12-bit HDR color. - Acting & Physics Precision: Transfer real human facial nuance via Act-One or full-body motion via Kling.
- Best Use: Scripted commercials, narrative short films, music videos, and client-ready 4K masters.
2. Access Tiers, Credits & Engine Decision Matrix
Every platform prices compute differently. Knowing per-second credit rates before pressing Generate lets you explore freely in cheap draft modes and save premium credits for locked final takes.
2026 Platform Tiers, Technical Ceilings & Credit Rates
| Engine / Studio | Access & Pricing Tiers | Output & Credit Cost | Sweet Spot & Watch-Outs |
|---|---|---|---|
| Google Project Genie Genie 3 World Model |
Google Labs web portal (labs.google/fx/projectgenie); Google AI Ultra tier (18+). |
720p at 24fps live stream; strict 60-second interactive session limit; MP4 walkthrough export. | Best for interactive scouting, Street View world remixing, and angle discovery. Not a full game engine; physics drift past 60s. |
| Runway Gen-4.5 & Turbo Cinema Physics Engine |
Free (125 one-time cr) • Standard $15/mo (625 cr) • Pro $35/mo (2,250 cr) • Max $95/mo (9,500 cr). | 5–10s clips; Gen-4.5: 12 cr/sec (~120 cr/10s); Gen-4 Turbo: 5 cr/sec; 12-bit / HDR mastering. | Best for fluid/fabric physics, Multi-Motion Brush layers, and Gen-4 References. No native voice track; pair with Act-One or NLE audio. |
| Runway Act-One / Act-Two Performance Capture |
Built inside Runway Standard, Pro, Max, and Team plans (no separate add-on fee). | 5 credits/sec (3-second minimum) driving a target character image from webcam/phone video. | Best for nuanced dialogue, eye-lines, and stylized avatar acting. Needs clear front lighting and unobstructed mouth/eyes. |
| Kling AI 3.0 Omni Elements & Keyframes |
Standard ~$7–$10/mo (~660 cr) • Pro ~$25–$37/mo (~3,000 cr) • Premier 8,000+ cr. | Up to 15s multi-shot or 30s Motion Control; 6–8 cr/sec (1080p) vs. ~30 cr/sec (native 4K/60fps). | Best for @Element locking, Start/End keyframe morphs, and athletic stunts. Native 4K burns 4x credits; upscale 1080p when prototyping. |
Three-Step First-Day Studio Setup
Pick one project ratio (16:9 widescreen or 9:16 vertical) and lock frame rate to 24fps across every engine before creating your first reference frame.
Generate neutral 3-angle character sheets and clean location plates first. Reusing one master portrait across Runway and Kling eliminates identity drift.
Scout lighting in Project Genie, iterate motion timing on Gen-4 Turbo (5 cr/s) or Kling 1080p, and promote only winning seeds to Gen-4.5 or 4K.
3. Annotated Workspace UI: World & Shot Director HUD
Although Project Genie, Runway, and Kling organize menus differently, every production workflow centers on the same seven spatial controls. Learn this unified cockpit map to direct across all three engines.
actor_take03.mp4 (6.5s)Seven Core Viewport Controls Explained
@scout_mira), Kling voice profiles, and exported Project Genie still frames for reuse across scenes.WASD + Spacebar + Arrows, or preview Runway/Kling motion trajectories in 16:9 / 9:16.[x, y] directional vectors while pinning buildings stationary.first_frame and end_frame stills in Kling or Runway to control exact opening framing and closing composition.5s, 8s, or 10s) and locked random seed.image vs. video) for stunt choreography.
4. Playable 3D Worlds in Project Genie (Genie 3)
Powered by Google DeepMind's Genie 3 spatiotemporal world model, Project Genie generates navigable 3D spaces that react dynamically to your controls at 720p / 24fps with roughly one minute of spatial object memory.
Describe your terrain, upload a photo/sketch, or pick a Google Street View map pin. Preview the opening frame with Nano Banana Pro and refine elements before compiling.
Pick First-Person or Third-Person perspective, then move with WASD, jump with Spacebar, and pan the camera with Arrow keys as the path renders live.
Download your walkthrough MP4 or click Remix on any seed world to swap the character, weather, or physics mood while preserving spatial layout.
Structured World-Sketch & Plate-Harvesting Blueprint
# PROJECT GENIE WORLD SKETCH TEMPLATE (Copy & Adapt inside labs.google/fx/projectgenie)
ENVIRONMENT : Mist-drenched floating lantern harbor at twilight, wet basalt piers, wooden footbridges
GROUNDING : Optional Street View coordinates OR uploaded concept sketch (16:9 widescreen)
CHARACTER : Agile third-person explorer wearing an amber waxed canvas raincoat and satchel
PHYSICS : Grounded walking pace, buoyant water reflections, gentle cloth sway, soft jump arc
PREVIEW EDIT: Use Nano Banana Pro preview -> [Add] distant lighthouse beam; [Remove] foreground crates
SCOUT LOOP : Walk forward 15s (Shot A wide) -> turn 90 deg left to bridge (Shot B) -> export MP4WASD, export the 720p MP4, and pause on your favorite frames. Those geometrically consistent plates become flawless Start Frames for Runway and Kling.
5. Runway Gen-4.5 Director Controls & Act-One
When your scene requires physical weight—fabric billowing in wind, water splashing against glass, or an actor delivering subtle heartbreak—pair Runway Gen-4.5 with Gen-4 References and Act-One / Act-Two.
Gen-4.5 models cause-and-effect momentum across 5–10 second takes and exports up to 12-bit HDR (hdr10/hlg). Attach clean portrait and prop cards in Gen-4 References so your hero's face and jacket stay identical from wide shots to tight inserts without custom LoRA training.
Record a 5–15 second driving clip on your phone or webcam. Act-One maps your micro-expressions, eye-lines, blinks, and lip-sync onto any stylized or photoreal character image at 5 credits/sec, while Act-Two extends performance capture to hand and upper-body gestures.
Color-Coded Runway Gen-4.5 Shot-List Formula
Programmatic Runway Gen-4.5 Inference Payload (With Seed Lock)
{
"model": "runway-gen-4.5",
"taskType": "videoInference",
"input": {
"prompt": "Slow dolly-in on @scout_mira raising brass flare on rainy pier, heavy coat fabric physics.",
"image_url": "https://cdn.example.com/plates/Genie_Dock_A.png",
"duration": 8,
"ratio": "16:9",
"resolution": "1080p",
"fps": 24,
"seed": 4291
}
}6. Multi-Motion Brush & Precision Camera Paths
Text prompts alone often swirl static walls or freeze hair that should blow in the breeze. Combining Multi-Motion Brush masks with explicit camera path trajectories separates environmental motion from camera movement.
Paint independent brush regions onto your start frame—assigning Brush 1 to rising steam, Brush 2 to flowing river water, and Brush 3 to swaying tree branches.
Keep brushes strictly inside moving silhouettes and avoid feathering onto brick walls or horizons. Unpainted or zero-vector regions stay anchored like real set walls.
In Gen-4/4.5, replace legacy sliders with crisp shot-list verbs (slow dolly in, hold, orbit 90 degrees) or structured camera axis vectors via the API.
Multi-Motion Brush & Compound Camera Trajectory JSON
"camera": {
"movement": "pan_right",
"speed": 3,
"zoom": "in",
"horizontal": 2.5,
"vertical": -0.8,
"roll": 0.0
},
"multi_motion_brush": [
{ "id": 1, "mask_coordinates": [120, 440, 310, 620], "vector": [1.5, -3.2], "trajectory": "linear", "intensity": 7 },
{ "id": 2, "mask_coordinates": [520, 190, 780, 410], "vector": [-2.0, 1.0], "trajectory": "orbit", "intensity": 5 }
]camera and multi_motion_brush arrays via API, keep your text prompt concise and free of conflicting camera directions (do not write fast whip pan left in text if your JSON sets pan_right).
7. Kling Elements, Keyframes & 4K NLE Finishing
Kling AI 3.0 Omni shines when characters sprint, fight, dance, or transform smoothly between two exact still frames. Pair Kling's consistency tools with DaVinci Resolve or Premiere Pro to finish a broadcast-grade cut.
Register multi-angle portraits and hero props inside the Element Library, bind a matching voice signature, and attach up to 3 Elements (@element_1) to lock outfits and faces across head turns.
Supply first_frame and end_frame (image_tail) to morph cleanly between two compositions—or pin up to 10 timeline milestones in Kling 4.0 for multi-beat transitions.
Upload a character portrait plus a reference stunt/dance clip. Set character_orientation to image (max 10s) or video (up to 30s) to transfer exact body mechanics.
Kling API Start-to-End Keyframe + Element Lock Payload
{
"model_name": "kling-v3",
"prompt": "Continuous tracking push-in as @element_1 steps from sunlit pier into neon fog.",
"negative_prompt": "blurry, warped fingers, flickering clothing, watermark",
"duration": 5,
"mode": "pro",
"cfg_scale": 0.5,
"first_frame": "https://cdn.example.com/plates/day_pier_start.jpg",
"end_frame": "https://cdn.example.com/plates/neon_alley_end.jpg",
"elements": [
{ "element_id": "elem_scout_mira", "weight": 0.85, "position": { "region": [0.2, 0.1, 0.8, 0.9] } }
]
}NLE Assembly & 4K Upscaling Workflow
- Drop Genie (720p), Runway (1080p), and Kling clips onto a single 24fps Rec.709 timeline.
- Trim the first and last 8–12 frames of AI clips to cut cleanly on physical motion.
- Unify disparate renders using one subtle Kodak film print LUT and 35mm grain overlay, plus layered Foley stems.
- Instead of paying ~30 cr/sec for native 4K in Kling during exploration, master locked takes at crisp 1080p (6–8 cr/sec).
- Upscale final locked timeline exports with DaVinci Resolve Studio Super Scale (2x/4x) or Topaz Video AI (Proteus / Rhea models) to sharpen fabric weave and eye highlights.
8. 5 Real-World Copy-Paste Cinema & World Recipes
Stop reinventing settings for every new scene. Copy these five battle-tested recipes directly into Project Genie, Runway, Kling, and your finishing terminal.
| Recipe & Engine | Exact Inputs & Prompt / Parameter Formula | Pro Execution Notes |
|---|---|---|
| 1. Playable Game Level Scout Project Genie |
Prompt: Ancient sunken observatory overgrown with moss, third-person explorer with lantern, warm dust motes, realistic stone gravity. | Walk 20s with WASD to test vertical scale; pause and save 3 still frames (wide, corner reveal, hero balcony) for Runway/Kling. |
| 2. Hero Product Orbit Runway Gen-4.5 |
Input: Studio packshot + Brush 1 rising mist ([0.0, -2.5]) + Prompt: Slow 90-deg orbital arc around matte espresso brewer, 85mm macro rim light. |
Keep intensity at 6 so steam swirls naturally while stainless steel edges remain razor-sharp across the orbit. |
| 3. Dialogue Close-Up Runway Act-One |
Input: Locked character portrait + 8s front-lit webcam clip delivering 12 words cleanly with subtle eyebrow raise and glance left. | Costs just 5 cr/s (40 credits). Export ProRes/MP4 and sweeten dialogue in DaVinci Fairlight with room-tone noise bed. |
| 4. Day-to-Night Location Morph Kling Start/End |
Inputs: first_frame (noon cafe) + end_frame (rainy neon midnight) + @element_1 + 5s Pro mode, cfg_scale: 0.5. |
Match horizon height and camera distance in both stills so buildings transform seamlessly rather than melting sideways. |
| 5. Action Stunt Transfer Kling Motion Ctrl |
Inputs: Character portrait + 12s parkour vault video; character_orientation: "video", keep_original_sound: "no", Pro 1080p. |
Always set orientation to "video" for clips over 10s or turns exceeding 90 degrees so limbs do not snap backward. |
Recipe 5B • Conforming Mixed Renders for DaVinci Resolve (FFmpeg)
Before dropping 720p Project Genie walkthrough grabs, Runway H.264/ProRes renders, and Kling MP4s into one timeline, normalize frame rates and scale everything cleanly into an edit-friendly 4K ProRes 422 HQ intermediate:
# Normalize mixed 720p/1080p clips to unified 24fps 4K ProRes 422 HQ plates (zero re-compression stutter)
for f in raw_renders/*.mp4; do
base=$(basename "$f" .mp4)
ffmpeg -i "$f" -vf "fps=24,scale=3840:2160:flags=lanczos" \
-c:v prores_ks -profile:v 3 -pix_fmt yuv422p10le -c:a pcm_s24le "conform/${base}_4k24.mov"
doneseed into your upgraded Gen-4.5 or Kling Pro call so you refine the composition instead of rolling the dice again.
9. Team Pre-Viz, IP Hygiene & Frame Provenance
Shipping AI-assisted films for clients, festivals, or studios requires clean rights hygiene, reproducible pre-viz packets, and verifiable provenance credentials under 2026 disclosure standards.
Share lightweight creative packets before rendering expensive masters: bundle your Project Genie 60s MP4 scout, seed prompt text, 3-angle Element sheets, and a rough 24fps animatic cut so directors approve blocking upfront.
Generate client deliverables strictly on paid commercial tiers (which exclude inputs from model training and remove watermarks). Never upload copyrighted characters, celebrity faces, or client competitor logos into Elements.
Google embeds SynthID watermarks directly inside pixels and audio—resilient to cropping and compression—while C2PA Content Credentials attach signed metadata manifests. Always re-sign final NLE renders after editing.
Split team monthly credits into an 80% R&D / Draft Pool (Genie scouting, Turbo 5 cr/s, Kling 1080p) and a locked 20% Master Finishing Pool (Gen-4.5 12 cr/s & Kling 4K) gated by creative-director sign-off.
Re-Signing Edited AI Masters with C2PA Content Credentials
Exporting from DaVinci Resolve or Premiere Pro strips upstream generation manifests unless you append an updated C2PA assertion documenting both AI source ingredients and human editorial finishing:
// manifest.json — Declare hybrid AI source lineage + human NLE assembly before delivery
{
"claim_generator": "Studio_Post_Pipeline/1.0",
"title": "Neon_Harbor_Director_Cut_4K.mp4",
"assertions": [
{ "label": "c2pa.actions", "data": { "actions": [
{ "action": "c2pa.created", "digitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia" },
{ "action": "c2pa.edited", "parameters": { "software": "DaVinci Resolve 20 + Topaz 4K" } }
] } }
]
}
// Sign final MP4: c2patool input_master.mp4 -m manifest.json -o final_signed_4k.mp410. The Cross-Tool World-to-Cinema Synergy Pipeline
No single engine dominates every film task. Use this five-stage handoff pipeline to pair Project Genie's navigable worlds with Kling's action physics, Runway's acting polish, and studio 4K finishing.
Three Critical Handoff Rules Between Engines
Keep Project Genie still exports and downstream Runway/Kling keyframes at identical aspect ratios (16:9 or 9:16) with matching horizon lines.
Carry identical time-of-day and color temperature descriptors (e.g., overcast cyan fog, warm tungsten lantern) across every tool's prompt.
Mute per-clip AI music during generation or strip audio tracks in Resolve so a single continuous score and Foley stem bridges every cut.
11. Troubleshooting Clinic & Pre-Flight Checklist
Keep this final page next to your keyboard while directing. Every recurring world-drift, mask-bleed, lip-sync, or keyframe-warp snag has an immediate 10-second fix.
Top 7 World & AI Cinema Problems & Instant Fixes
| Symptom / Failure Mode | Root Cause & Exact 10-Second Fix |
|---|---|
| Project Genie forgets geometry after turning around | Exceeded the ~60-second spatial memory window or walked too far afield. Keep scouting loops tight inside the core set, grab angle stills early, or Remix a fresh seed. |
| Runway Motion Brush moves walls or bleeds silhouettes | Feathered brush spilled past object edges. Paint tighter masks strictly inside the moving element and leave surrounding architecture untouched as static anchors. |
| Act-One facial expressions look stiff or miss eye-lines | Dim or side-lit driving video confused landmark tracking. Re-record your phone clip facing even soft light at eye level, remove glasses, and keep hands away from your mouth. |
| Kling Start-to-End morph twists or melts mid-transition | Camera horizon or subject orientation jumped too far between first_frame and end_frame. Align horizon lines across both stills or shorten duration to 5 seconds. |
| Hero face or jacket drifts during fast turns | Single front portrait lacks side-profile cues. Upload a 3-angle sheet to Kling Elements (@element_1) or Runway Gen-4 References and drop repetitive clothing text. |
| Cuts between Genie, Runway, and Kling feel disjointed | Mixed frame rates, resolutions, and contrasting digital sheen. Conform all clips to 24fps in Resolve, apply one unified color LUT + 35mm grain, and upscale to 4K together. |
| Monthly credits disappear before final cut lock | Rendering 10s Gen-4.5 (12 cr/s) or Kling 4K (30 cr/s) on unverified prompts. Draft in Genie, Gen-4 Turbo (5 cr/s), and Kling 1080p, then upscale locked winners. |
16:9 / 9:16) & 24fps set everywhere.camera path do not contradict text.@Element or Gen-4 Ref.c2patool signing.labs.google/fx/projectgenie to scout a 60-second location plate, bring your best frame into Runway Gen-4.5 or Kling 3.0 Omni with a locked character reference, and cut your first 24fps sequence in DaVinci Resolve today.