Google's Genie 4 World Model Adds Sound and Speech

Google DeepMind has announced Genie 4, the next version of its "world model" that builds an explorable 3D scene from a text prompt or an image. This one has sound. It also lets you change the scene while you're still moving around inside it, which is the part I find most interesting.

What happened

In its Genie 4 announcement, DeepMind says the model now creates native audio, including ambient sound, music and spoken dialogue from characters you can talk to. You can also type new instructions partway through a simulation to change the camera, swap the weather or add characters, without starting over.

Genie 4 is built on Gemini. Google's own post says Gemini models handle the reasoning, so a world behaves more like you'd expect. The Verge reports that sessions now run for about two minutes, double the one minute Genie 3 managed.

For now it's a research preview for trusted testers. Google says it will reach Google AI Ultra subscribers through Project Genie and early-access partners on the Gemini API and AI Studio. TechCrunch says that should happen in the coming weeks. Google didn't share separate pricing.

Why it matters

World models are where a lot of AI labs think the next big jump comes from. They matter less for games you'd actually play and more for training robots and agents in simulated places that are cheap to make. Google itself pitches Genie 4 at filmmaking, journalism, gaming and manufacturing, so think storyboards, rough game prototypes and factory or robot simulations.

The limits are still obvious. The team told TechCrunch that objects can still drift the longer a session runs, and that accurate audio and readable in-world text aren't solved yet. Two minutes is also short. It's a demo of where things are heading, not a finished product.

What to do

• Curious? Watch for Project Genie. You'll need a Google AI Ultra plan when it opens up.
• Developers should keep an eye on AI Studio for the early-access API.
• Don't expect to run anything like this at home yet. If you want to play with AI on your own PC, start with running local models in LM Studio.

Credit to TechCrunch, whose X post was where I first saw the news.