From 7a3135c0204e11a322b98d337bc81ead4da89757 Mon Sep 17 00:00:00 2001 From: Brandon Thomas Date: Fri, 30 Jan 2026 01:19:36 -0500 Subject: [PATCH] essaaaay --- .../lib/content/news/world-models-for-film.md | 40 ++++++++++--------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/frontend/libs/markdown-content/src/lib/content/news/world-models-for-film.md b/frontend/libs/markdown-content/src/lib/content/news/world-models-for-film.md index fb41209381..cff42b5d46 100644 --- a/frontend/libs/markdown-content/src/lib/content/news/world-models-for-film.md +++ b/frontend/libs/markdown-content/src/lib/content/news/world-models-for-film.md @@ -4,50 +4,51 @@ abstract: How to use World Models and 3D Rendering to Get Consistent AI Film Loc date: 2026-01-30 --- -# "AI Slop" +# The "AI Film Trailer" Era You've seen a lot of these: -@youtube(QMtGiLJJVd0) +@youtube(pGn-1BKo3nY) -For the past year, it been relatively easy to make "clip shows", trailers, -and montages like this. +For the past year, it been relatively easy to make montages, "clip shows", and trailers, +like this. -But characters need space to live, breathe, and interact. They don't just journey from point -A to B and that's the end of the story. They spend time in places. They deal with -problems, they explore interpersonal relationships, they think, they overcome. +Real stories have characters that need space to live, breathe, and interact. You don't just fast track +everyone through a journey from point A to B and have that be the end of things [1]. Characters need to spend +time in places, explore interpersonal relationships. They need to take time to think, talk, fight and argue, +process emotions, and sometimes do absolutely nothing at all. -In *"film language"*, even the camera's relationship to the characters conveys meaning. - -# Location +# Location and Blocking We're all filmmakers at ArtCraft. The old school, photons-on-glass kind. -One of the biggest problems we've had over the past few years with AI video is precisely controlling what's -on screen and maintaining consistency of location. +One of the biggest problems we've had over the past few years with AI video is giving our characters room to occupy. +Setting them up on set, precisely controlling what's on screen, and maintaining consistency with their location. The majority of AI video right now looks like a series of disconnected clips, which results in AI film having a sort of "film trailer" feel to it. Lots of jumping around, very little continuity. -We've figured out how to overcome this problem. Let me show (don't tell) you what we've discovered. -You'll intuit what's up immediately: +We've figured out how to overcome this problem. And in the spirit of "Show, Don't Tell", +let me show you what we've discovered. You'll intuit what's up immediately: @youtube(wJCJYdGdpHg) -# Text is Coarse Grained +# Text is a Coarse Grained Medium -Let me try a single, rather elaborate prompt for Nano Banana Pro: +Here's a simple Nano Banana Pro prompt: ![Example gif](./images/blog/sam4.png) > Sam Altman in a room. He sits at a desk, looking towards a desktop computer on the left. > The room is dimly lit. The glow of a computer terminal, sitting in front of him bathes the room in blue light. -> Behind him, there is a wall with posters. There’s an AC/DC poster on the left, and a window on the right. +> Behind him, there is a wall with a poster. There’s an AC/DC poster on the left, and a window on the right. > Through the window, there’s a view of moonlit trees. What happened to the location here? Why are none of these the same? Why do the locations of the poster, computer, window, and so forth keep changing? +Moreover, how do we adjust this? Iteratively prompting? + You already know the answer: text is a poor serialization of the physical world. Text-to-Image is coarse grained. It lacks the ability to convey precise physical attributes and spatial relationships. @@ -56,6 +57,8 @@ Outside of proper nouns, text has limited ability to convey notions of location. store any physical attributes, it can't track relative positions between objects, and it's obtuse to describe spatial relationships and appearance. +In *"film language"*, even the camera's relationship to the characters conveys meaning. + - Example image Most people have been stuck using this. @@ -83,7 +86,6 @@ so moving the camera results in hallucinations and inconsistencies. --- -@youtube(pGn-1BKo3nY) # 3D to the Rescue @@ -119,3 +121,5 @@ and local GPU support is coming soon (subscribe to stay updated). p.s. did you catch the typo(s)? The pixels may be generated, but the text was artisinally hand-crafted. — Brandon Thomas, that one silly engineer in your drama club making you Meisner to Tolkien. + +[1] Unless you're Game of Thrones...