mirror of
https://github.com/storytold/artcraft.git
synced 2026-11-04 09:28:11 +00:00
essaaaay
This commit is contained in:
1 parent
dfcf827a41
commit
7a3135c020
1 file changed
+22
-18
@@ -4,50 +4,51 @@ abstract: How to use World Models and 3D Rendering to Get Consistent AI Film Loc
|
||||
date: 2026-01-30
|
||||
---
|
||||
|
||||
# "AI Slop"
|
||||
# The "AI Film Trailer" Era
|
||||
|
||||
You've seen a lot of these:
|
||||
|
||||
@youtube(QMtGiLJJVd0)
|
||||
@youtube(pGn-1BKo3nY)
|
||||
|
||||
For the past year, it been relatively easy to make "clip shows", trailers,
|
||||
and montages like this.
|
||||
For the past year, it been relatively easy to make montages, "clip shows", and trailers,
|
||||
like this.
|
||||
|
||||
But characters need space to live, breathe, and interact. They don't just journey from point
|
||||
A to B and that's the end of the story. They spend time in places. They deal with
|
||||
problems, they explore interpersonal relationships, they think, they overcome.
|
||||
Real stories have characters that need space to live, breathe, and interact. You don't just fast track
|
||||
everyone through a journey from point A to B and have that be the end of things [1]. Characters need to spend
|
||||
time in places, explore interpersonal relationships. They need to take time to think, talk, fight and argue,
|
||||
process emotions, and sometimes do absolutely nothing at all.
|
||||
|
||||
In *"film language"*, even the camera's relationship to the characters conveys meaning.
|
||||
|
||||
# Location
|
||||
# Location and Blocking
|
||||
|
||||
We're all filmmakers at ArtCraft. The old school, photons-on-glass kind.
|
||||
|
||||
One of the biggest problems we've had over the past few years with AI video is precisely controlling what's
|
||||
on screen and maintaining consistency of location.
|
||||
One of the biggest problems we've had over the past few years with AI video is giving our characters room to occupy.
|
||||
Setting them up on set, precisely controlling what's on screen, and maintaining consistency with their location.
|
||||
The majority of AI video right now looks like a series of disconnected clips, which results in
|
||||
AI film having a sort of "film trailer" feel to it. Lots of jumping around, very little
|
||||
continuity.
|
||||
|
||||
We've figured out how to overcome this problem. Let me show (don't tell) you what we've discovered.
|
||||
You'll intuit what's up immediately:
|
||||
We've figured out how to overcome this problem. And in the spirit of "Show, Don't Tell",
|
||||
let me show you what we've discovered. You'll intuit what's up immediately:
|
||||
|
||||
@youtube(wJCJYdGdpHg)
|
||||
|
||||
# Text is Coarse Grained
|
||||
# Text is a Coarse Grained Medium
|
||||
|
||||
Let me try a single, rather elaborate prompt for Nano Banana Pro:
|
||||
Here's a simple Nano Banana Pro prompt:
|
||||
|
||||

|
||||
|
||||
> Sam Altman in a room. He sits at a desk, looking towards a desktop computer on the left.
|
||||
> The room is dimly lit. The glow of a computer terminal, sitting in front of him bathes the room in blue light.
|
||||
> Behind him, there is a wall with posters. There’s an AC/DC poster on the left, and a window on the right.
|
||||
> Behind him, there is a wall with a poster. There’s an AC/DC poster on the left, and a window on the right.
|
||||
> Through the window, there’s a view of moonlit trees.
|
||||
|
||||
What happened to the location here? Why are none of these the same? Why do the locations of the poster,
|
||||
computer, window, and so forth keep changing?
|
||||
|
||||
Moreover, how do we adjust this? Iteratively prompting?
|
||||
|
||||
You already know the answer: text is a poor serialization of the physical world. Text-to-Image is coarse
|
||||
grained. It lacks the ability to convey precise physical attributes and spatial relationships.
|
||||
|
||||
@@ -56,6 +57,8 @@ Outside of proper nouns, text has limited ability to convey notions of location.
|
||||
store any physical attributes, it can't track relative positions between objects, and it's
|
||||
obtuse to describe spatial relationships and appearance.
|
||||
|
||||
In *"film language"*, even the camera's relationship to the characters conveys meaning.
|
||||
|
||||
- Example image
|
||||
|
||||
Most people have been stuck using this.
|
||||
@@ -83,7 +86,6 @@ so moving the camera results in hallucinations and inconsistencies.
|
||||
|
||||
---
|
||||
|
||||
@youtube(pGn-1BKo3nY)
|
||||
|
||||
|
||||
# 3D to the Rescue
|
||||
@@ -119,3 +121,5 @@ and local GPU support is coming soon (subscribe to stay updated).
|
||||
p.s. did you catch the typo(s)? The pixels may be generated, but the text was artisinally hand-crafted.
|
||||
|
||||
— Brandon Thomas, that one silly engineer in your drama club making you Meisner to Tolkien.
|
||||
|
||||
[1] Unless you're Game of Thrones...
|
||||
Reference in new issue
Block a user