I can’t wait to see how tech like this totally upends image creation & editing as we know them. That’s just my hunch, mind you, but converting images into this kind of interactive world makes it seem almost inevitable.
This is Genie 4, our most realistic and interactive world model yet.
Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.
Today, we are announcing Genie 4, our new world model, which unlocks unprecedented realism and improves real-time interactive simulation. Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
The Genie 4 model incorporates and builds on capabilities from the Gemini family: Gemini Omni anchors the simulations in real-world appearance and behavior, while other Gemini models provide the complex reasoning needed to help maintain a simulation’s coherence and continuity.
With this foundation, you can set an event in motion, and Genie 4 simulates what happens next: how the world responds, how it sounds, and how the people in it move, react, and speak.
Understanding and simulating the world is a crucial aspect of building AGI. Genie 4 advances our ability to create safe, infinitely variable simulations that can help with everything from developing training environments for robots and agents to unlocking the creation of new interactive experiences.
My wife & son are huge fans of LIDAR-heavy shows like Expedition Unknown, and I can only imagine how they and many others will use these new techniques to shed light on archeological finds:
Reconstructing ancient ruins in 3D with Gemini 4 Argon
Using Google @Antigravity and Blender MCP, see how Gemini 4 Argon supports digital heritage preservation by turning raw, fragmented archaeological data into fully explorable 3D worlds. https://t.co/ySvI57WIlq
I’m a fan of these guys. Charles is awesome, and my friend Philip (with whom I got my start in his pre-ILM/Pixar days) is leading product design. I’m excited to try out their “semantic scene architecture,” which promises to focus on outcomes rather than inputs:
My cofounder @cmigos (ex-Apple & Unity) shows how to animate cameras in Intangible’s semantic scene architecture.
Watch him stage the action, track a subject, and move the camera in 3D like he’s directing on set.
Insane that 3D worlds can be conjured straight from language, then rendered natively or used as guidance for AI-based rendering. Check out this demo from my former Adobe teammate Tom:
I had an early access to GPT-6 Astra and I can say, everyone in the world now has a 3D designer at their fingertips. I gave it an image of a house and asked to create it in 3D with all the details including toys, appliances and furniture. This a full 3D model reconstruction in… pic.twitter.com/VuBOPQH0Ht
Astra is very good at 3D modeling, and I can’t wait for all of you to experience it, for now here is a little walkthrough on how I built the demo house for our launch blog post. From a Blender scene to a Unreal Engine 5 walkable experience. Thread. pic.twitter.com/xGqe5iXoRv
I guess we’re allowed to talk about it now… Astra has absolutely blown my mind. I did get early access to test it out and it’s definitely felt like the biggest leap we’ve seen in a long time!
I asked it to make me a humanoid wolf in Blender, it used computer use, took control… pic.twitter.com/NFW9K073EJ
“It started as a slightly ridiculous question,” says Bilawal Sidhu: “How much of the planet can you make sense of from public data and a browser tab?
Being able to navigate & understand the world with a Jarvis-style UI is just magical.
Once aircraft, vessels, satellites, active fires, earthquakes, traffic, public cameras and physical infrastructure live on the same photorealistic 3D globe, it stops feeling like a pile of dashboards. It feels like a command center.
In this video I walk through the whole thing: ask the planet what’s happening over a cluster of military flights; jump into cockpit mode; voice-fly to points of interest, draw real boundaries and routes; move from city traffic into street cameras; inspect the physical internet; tune into radio around the world; and replay a recent Falcon 9 launch.
I have no words for this kind of witchcraft. In case the embedded tweet doesn’t show up in English, here’s a translation:
Wow. Maciej Dobrodziej – a digital creator – has “brought to life” one of Warsaw’s most famous photographs. The photo of a girl running in the rain on Puławska Street was taken by Zbigniew Siemaszko in 1968. Fun fact: After many years, it was possible to find Ms. Grażyna, who is the heroine of the photo. She recognized herself upon seeing… the photograph on FB.
Wow. Maciej Dobrodziej – twórca cyfrowy „ożywił” jedno z najsłynniejszych zdjęć Warszawy. Fotografie biegnącej w deszczu dziewczyny na ulicy Puławskiej zrobił Zbigniew Siemaszko w 1968 roku.
Ciekawostka: po latach udało się odnaleźć panią Grażynę, która jest bohaterką zdjęcia.… pic.twitter.com/tvNJJjlka3
Meanwhile the “Scroll World” Claude skill promises to interview you (!), then whip something up with the help of generative text-to-video tech (gotta get Omni in there!).
I built a skill to let my Claude Code build premium landing pages like this in one shot. The 3 sites in the video are one-shot results, ~$10-15 each.
How it works is intriguing enough to quote at length from the GitHub page:
—–
It leans on Higgsfield for the art: cohesive isometric diorama scenes (GPT Image 2 — via Higgsfield, or the Codex CLI on a ChatGPT subscription) and the camera flights themselves (Seedance or Kling image-to-video — only models that can frame-lock a seam), scrubbed by scroll position — the same technique behind Apple’s scroll-through product pages. The camera genuinely moves; scroll only drives time. It’s framework-agnostic: you get the Higgsfield pipeline, the prompt templates, and a portable vanilla-JS scrub engine that drops into plain HTML, Next.js, Vue, or a Python-served page — nothing assumes a stack.
When invoked, the skill:
Interviews you — the subject/industry + pitch, a brand kit (import from a URL, hand it over, or have it proposed), art direction, the ordered scenes the camera visits, whether you want the mobile version (a second chain rendered natively in 9:16 portrait — composed for phones, not a crop of the landscape film), and the budget — render tiers and stills source shown with estimated credit costs, approved before anything generates.
Generates the assets — one still per scene, one “dive-in” camera clip per scene, and the connector clips that join consecutive scenes, generated from the actual rendered frames of their neighbours so every seam is frame-identical. Mobile opt-in renders a parallel portrait chain the same way, frame-locked against its own 9:16 renders.
Wires it up — a config-driven scroll engine that plays the whole chain as one flight, serving the portrait clips and posters automatically on phones.
At Google I/O 2026, DeepMind shipped Maps Imagery Grounding for Genie 3 — their real-time world model can now generate interactive 3D worlds conditioned to any of the 280 billion Street View images Google has captured over 20 years. Pick a location on Google Maps, choose a style, drop in a character, and walk around.
Check out his accessible & illuminating tour of the new tech:
My 16yo is lowkey impressed that at Adobe I got to work with Utah Teapot creator Martin Newell. At this point, anything that impresses a teen is very welcome. 🙂
I wonder what he’d think of Gemini Omni turning real teapots into geometry just by saying the word:
Google Earth now allows importing ANY 3D model, so I used @tripoai on @fal to get Godzilla into Tokyo, and my Chrome extension to transform it into a movie scene pic.twitter.com/150iiy5UjX
Today we are announcing a new approach to fix scene alignment after a photo was taken. Our method, now available as part of the Auto frame feature in Google Photos, uses machine learning (ML) models to understand the scene and its spatial layout and uses generative AI to imagine the photo from that new perspective. In contrast to classical photo editing, our method interprets a photo as a 3D scene — think of a real moment frozen in time — and change the camera position automatically within that space.
“Being early is the same as being wrong.” — Marc Andreessen, Vol. ~900
We put 3D into Photoshop nearly 20 years ago, and it got used by nearly 20 people total, lol. For many of the past several years, it was on the team’s “gotta throw overboard, as soon as we can find time” list—but happily that time was never found.
I am so glad to see this foundation now finding a meaningful niche, and I have high hopes for its generative future. Posing a person or thing directly is so much more intuitive than trying to precisely describe an outcome via prompt, and simple 3D manipulation + generative rendering could well deliver game-changing best of both worlds.
“3D scenes, websites, games, apps,” promises Spline. “Describe anything and Omma builds it for you in seconds.”
Omma combines code generation (LLMs), 3D AI mesh generation, and Image generation all in one place for you to build and ship. Deploy to production, assign custom domains, and more.
This new tech from Meta promises to create geometry from video frames. You can try feeding it up to 16 frames via this demo site—or just check out this quick vid:
Huge drop by Meta: ActionMesh turns any video into an animated 3D mesh.
The moment I switched on gravity was the moment everything changed.
Lines I had just drawn started to fall, swing, and collide like they were suddenly alive inside my room. A simple sketch became an object with weight. A doodle turned into something that could react back. It is one of those Vision Pro moments where you catch yourself smiling because it feels playful in a way you do not see coming.
Of course, Old Man Nack™ feels like being a little cautious here: Ten years ago (!) my kids were playing in Adobe’s long-deceased Project Dali…
…and five years ago Google bailed on the excellent Tilt Brush 3D painting app it acquired. ¯\_(ツ)_/¯
And yet, and yet, and yet… I Want To Believe. As I wrote back in 2015,
I always dreamed of giving Photoshop this kind of expressive painting power; hence my long & ultimately fruitless endeavor to incorporate Flash or HTML/WebGL as a layer type. Ah well. It all reminds me of this great old-ish commercial:
So, in the world of AI, and with spatial computing staying a dead parrot (just resting & pining for the fjords!), who knows what dreams may yet come?
Apple’s new 2D-to-3D tech looks like another great step in creating editable representations of the world that capture not just what a camera sensor saw, but what we humans would experience in real life:
Excited to release our first public AI model web app, powered by Apple’s open-source ML SHARP.
Turn a single image into a navigable 3D Gaussian Splat with depth understanding in seconds.
On Friday I got to meet Dr. Fei-Fei Li, “the godmother of AI,” at the launch party for her new company, World Labs (see her launch blog post). We got to chat a bit about a paradox of complexity: that as computer models for perceiving & representing the world grow massively more sophisticated, the interfaces for doing common things—e.g. moving a person in a photo—can get radically simpler & more intentional. I’ll have more to say about this soon.
Meanwhile, here’s her fascinating & wide-ranging conversation with Lenny Rachitsky. I’m always a sucker for a good Platonic allegory-of-the-cave reference. 🙂
From the YouTube summary:
(00:00) Introduction to Dr. Fei-Fei Li (05:31) The evolution of AI (09:37) The birth of ImageNet (17:25) The rise of deep learning (23:53) The future of AI and AGI (29:51) Introduction to world models (40:45) The bitter lesson in AI and robotics (48:02) Introducing Marble, a revolutionary product (51:00) Applications and use cases of Marble (01:01:01) The founder’s journey and insights (01:10:05) Human-centered AI at Stanford (01:14:24) The role of AI in various professions (01:18:16) Conclusion and final thoughts
And here’s Gemini’s solid summary of their discussion of world models:
The Motivation: While LLMs are inspiring, they lack the spatial intelligence and world understanding that humans use daily. This ability to reason about the physical world—understanding objects, movement, and situational awareness—is essential for tasks like first response or even just tidying a kitchen 32:23.
The Concept: A world model is described as the lynchpin connecting visual intelligence, robotics, and other forms of intelligence beyond language 33:32. It is a foundational model that allows an agent (human or robot) to:
Create worlds in their mind’s eye through prompting 35:01.
Interact with that world by browsing, walking, picking up objects, or changing things 35:12.
Reason within the world, such as a robot planning its path 35:31.
The Application: World models are considered the key missing piece for building effective embodied AI, especially robots 36:08. Beyond robotics, the technology is expected to unlock major advances in scientific discovery (like deducing 3D structures from 2D data) 37:48, games, and design 37:31.
The Product: Dr. Li co-founded World Labs to pursue this mission 34:25. Their first product, Marble, is a generative model that outputs genuinely 3D worlds which users can navigate and explore 49:11. Current use cases include virtual production/VFX, game development, and creating synthetic data for robotic simulation 53:05.
My friend Bilawal got to sit down with VFX pioneer John Gaeta to discuss “A new language of perception,” Bullet Time, groundbreaking photogrammetry, the coming Big Bang/golden age of storytelling, chasing “a feeling of limitlessness,” and much more.
In this conversation:
— How Matrix VFX techniques became the prototypes for AI filmmaking tools, game engines, and AR/VR systems — How The Matrix team sourced PhD thesis films from university labs to invent new 3D capture techniques — Why “universal capture” from Matrix 2 & 3 was the precursor to modern volumetric video and 3D avatars — The Matrix 4 experiments with Unreal Engine that almost launched a transmedia universe based on The Animatrix — Why dystopian sci-fi becomes infrastructure (and what that means for AI safety) — Where John is building next: Escape.art and the future of interactive storytelling
Improvements to imaging continues its breakneck pace, as engines evolve from “simple” text-to-image (which we considered miraculous just three years ago—and which I still kinda do, TBH) to understanding time & space.
Now Emu (see project page, code) can create entire multi-page/image narratives, turn 2D images into 3D worlds, and more. Check it out:
Turntable is now available in the Adobe #Illustrator Public Beta Build 29.9.14!!!
A feature that lets you “turn” your 2D artwork to view it from different angles. With just a few steps, you can generate multiple views without redrawing from scratch.
I love seeing the Magnific team’s continued rapid march in delivering identity-preserving reskinning
IT’S FINALLY HERE!
Mystic Structure Reference!
Generate any image controlling structural integrity Infinite use cases! Films, 3D, video games, art, interiors, architecture… From cartoon to real, the opposite, or ANYTHING in between!
This example makes me wish my boys were, just for a moment, 10 years younger and still up for this kind of father/son play. 🙂
Storyboarding? No clue! But with some toy blocks, my daughter’s wild imagination, and a little help from Magnific Structure Reference, we built a castle attacked by dragons. Her idea coming to life powered up with AI magic. Just a normal Saturday Morning. Behold, my daughter’s… pic.twitter.com/52tDZokmIT
The capturing work was led by Harry Nelder and Amity Studio. Nelder used his 16-camera rig to capture the recent winners. The reconstruction software was a combination of a cloud-based platform created by Nelder, which is expected to be released later this year, along with Postshot. Nelder further utilized the Radiance Field method known as Gaussian Splatting for the reconstruction. A compilation video of all the captures, recently posted by BAFTA, was edited by Amity Studio
Putting the proverbial chocolate in the peanut butter, those fast-moving kids at Krea have combined custom model training with 3D-guided image generation. Generation is amazingly fast, and the results are some combo of delightful & grotesque (aka “…The JNack Story”). Check it out:
God help you, though, if you import your photo & convert it to 3D for use with the realtime mode. (Who knew I was Cletus the Slack-Jawed Yokel?) pic.twitter.com/nuesUOZ1Db
Part 9,201 of me never getting over the fact we were working on stuff like this 2 years ago at Adobe (modulo the realtime aspect, which is rad) & couldn’t manage to ship it. It’ll be interesting to see whether the Krea guys (and/or others) pair this kind of interactive-quality rendering with a really high-quality pass, as NVIDIA demonstrated last week using Flux.
3D arrived to Krea.
this new feature lets you turn images into 3D objects and use them in our Real-time tool.
Powered by advanced AI, TRELLIS enables users to create high-quality, customizable 3D objects effortlessly using simple text or image prompts. This innovation promises to improve 3D design workflows, making it accessible to professionals and beginner alike. Here are some examples:
The world moves on, and now NVIDIA has teamed up with Black Forest Labs to enable 3D-conditioned image generation. Check out this demo (starting around 1:31:48):
For users interested in integrating the FLUX NIM microservice into their workflows, we have collaborated with NVIDIA to launch the NVIDIA AI Blueprint for 3D-guided generative AI. This packaged workflow allows users to guide image generation by laying out a scene in 3D applications like Blender, and using that composition with the FLUX NIM microservice to generate images that adhere to the scene. This integration simplifies image generation control and showcases what’s possible with FLUX models.
Just a taste of the torrent the blows past daily on The Former Bird App:
Rodin 3D: “Rodin 3D AI can create stunning, high-quality 3D models from just text or image inputs.”
Trellis 3D: “Iterative prompting/mesh editing. You can now prompt ‘remove X, add Y, Move Z, etc.’… Allows decoding to different output formats: Radiance Fields, 3D Gaussians, and meshes.”
Blender GPT: “Generating 3D assets has never been easier. Here’s me putting together an entire 3D scene in just over a minute.”
Adobe’s new generative 3D/vector tech is a real head-turner. I’m impressed that the results look like clean, handmade paths, with colors that match the original—and not like automatic tracing of crummy text-to-3D output. I can’t wait to take it for a… oh man, don’t say it don’t say it… spin.
Amazing, and literally immersive, work by artists at The Weather Channel. Yikes—stay safe out there, everybody.
The 3D artists at the weather channel deserve a raise for this insane visual
Now watch this, and then realize forecasts are now predicting up to 15 ft of storm surge in certain areas on the western coast of Florida pic.twitter.com/HHrCVWNgpg
I’m old enough to remember 2020, when we sincerely (?) thought that everyone would be excited to put 3D-scanned virtual Olympians onto their coffee tables… or something. (Hey, it was fun while it lasted! And it temporarily kept a bunch of graphics nerds from having to slink back to the sweatshop grind of video game development.)
Anyway, here’s a look back to what Google was doing around augmented reality and the 2020 (’21) Olympics:
I swear I spent half of last summer staring at tiny 3D Naomi Osaka volleying shots on my desktop. I remain jealous of my former teammates who got to work with these athletes (and before them, folks like Donald Glover as Childish Gambino), even though doing so meant dealing with a million Covid safety protocols. Here’s a quick look at how they captured folks flexing & flying through space:
Back when we launched Firefly (alllll the way back in March 2023), we hinted at the potential of combining 3D geometry with diffusion-based rendering, and I tweeted out a very early sneak peek:
Did you see this mind blowing Adobe ControlNet + 3D Composer Adobe is going to launch! It will really boost creatives’ workflow. Video through @jnack
A year+ later, I’m no longer working to integrate the Babylon 3D engine into Adobe tools—and instead I’m working directly with the Babylon team at Microsoft (!). Meanwhile I like seeing how my old teammates are continuing to explore integrations between 3D (in this case, project Neo). Here’s one quick flow:
Here’s a quick exploration from the always-interesting Martin Nebelong:
A very quick first test of Adobe Project Neo.. didn’t realize this was out in open beta by now. Very cool!
I had to try to sculpt a burger and take that through Krea. You know, the usual thing!
There’s some very nice UX in NEO and the list-based SDF editing is awesome.. very… pic.twitter.com/e3ldyPfEDw
And here’s a fun little Neo->Firefly->AI video interpolation test from Kris Kashtanova:
Tutorial: Direct your cartoons with Project Neo + Firefly + ToonCrafter
1) Model your characters in Project Neo 2) Generate first and last frame with Firefly + Structure Reference 3) Use ToonCrafter to make a video interpolation between the first and the last frame
Being able to declare what you want, instead of having to painstakingly set up parameters for materials, lighting, etc. may prove to be an incredibly unlock for visual expressivity, particularly around the generally intimidating realm of 3D. Check out what tyFlow is bringing to the table:
You can see a bit more about how it works in this vid…
Pretty cool! I’d love to see Illustrator support model import & rendering of this sort, such that models could be re-posed in one’s .Ai doc, but this still looks like a solid approach:
3D meets 2D!
With the Expressive or Pixel Art styles in Project Neo, you can export your designs as SVGs to edit in Illustrator or use on your websites. pic.twitter.com/vOsjb2S2Un
Man, what I wouldn’t have given years ago, when we were putting 3D support into Photoshop, for the ability to compute meshes from objects (e.g. a photo of a soda can or a shirt) in order to facilitate object placement like this.
I still can’t believe I was allowed in the building with these giant throbbing brains. 🙂
Create a 3D model from a single image, set of images or a text prompt in < 1 minute
This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or… pic.twitter.com/sOsOBsjC8Q
