Corridor Crew explain how they made an anime short by filming themselves and running the footage through custom-trained Stable Diffusion models. The video walks through the technique and its open-source foundations.
Why Hacker News points people here: HN cites it as an example of temporally stable AI video stylization built from ordinary production gear.
“They recently created a full 7-minute anime using Stable Diffusion with their own models and their existing video production gear”
Behind the scenes of making a 7-minute anime with Stable Diffusion
Full comment
I thought Midjourney was better as well, until I saw some recent videos from Corridor Crew on Youtube. For those who don't know, this is a VFX studio in LA that tries to keep at the cutting-edge of video production techniques and posts content to their Youtube channel, and they have a massive number of followers and several viral videos. They recently created a full 7-minute anime using Stable Diffusion with their own models and their existing video production gear, I'll post the links and let the results speak for themselves The actual 7-minute anime piece produced using SD: link Behind the scenes: "Did we change anime forever?" this video "VFX reveal before and after" link
Cites itsschuellerFeb 2023
“what corridor crew[2] did to stabilize the randomness when you want to make videos”
Corridor Crew stabilizing Stable Diffusion output for video
Full comment
The whole thing with SD is extremely interesting and difficult to keep up with as there are so many new things coming in almost daily. Just last week I saw ControlNet[1] which ads a lot more control. Today I saw what corridor crew[2] did to stabilize the randomness when you want to make videos. Very exciting. [1] link [2] this video
Cites itKelamirMar 2023
“where they demonstrated how it came to be”
Explains how the AI-processed anime characters were made
Full comment
The characters here are processed with Stable Diffusion. This is an impressive progress in the field of video generation with help of AI, considering how consistent the generated characters look like across frames, in my opinion. Also here is another video this video "Did We Just Change Animation Forever?" where they demonstrated how it came to be.
Cites ityummypaintMar 2023
“It unsurprisingly received alot of pushback from people in the industry but I think it's really impressive.”
Technique for turning live action into animation
Full comment
This reminds me alot of this techniquefor turning live action into animation: this video It unsurprisingly received alot of pushback from people in the industry but I think it's really impressive.
Cites itdbspinMar 2023
“Corridor digital show off a similar technique here”
Example of temporally stable animation with Stable Diffusion
Full comment
The issues you mention with Stable Diffusion are teething issues. Here's an example of a current technique to get temporally stable animations with Stable Diffusion - link Corridor digital show off a similar technique here - this video GPT4 with its new plugin architecture is rapidly becoming more capable. I think you're absolutely correct in the tendency of all LLMs to hallucinate is a major issue that we don't know how to correct even in principle. But there are techniques to moderate its impact on responses - triangulating using multiple models, citing sources etc. You could build an assistant like say this, today - link to make your appointments and answer the phone today, using technology like Uberduck for voice generation, GPT for conversation analysis and DeepSpeech, Kaldi or Whisper for speech to text. I agree with the criticism that we're not likely to develop AGI this way. We're not going to have a wilful agent with it's own goals trying to escape the bonds of its server to destroy humanity any time soon. But that's not necessary for this technology to be incredibly dangerous. A mindless philosophical zombie with the capacity to convince people it isn't could lead to an extinction event too. Given that we don't understand precisely how these models process information, how to imbue them with goals or motivation (or measure those), how to accurately assess how well they've been trained - or any of a dozen other major alignment issues - there's a strong argument that
Cites itsomepersonFeb 2024
“They used videos of themselves running through their own models to make the characters.”
Anime with consistent characters made from footage of themselves
Full comment
> There was a video of some guys doing this (sorry can’t find it) to make an anime type animation. With consistent characters. They used videos of themselves running through their own models to make the characters. That was Corridor Crew: this video
