World Labs Blog

Homepage for World Labs mentioned in parent comment: https://www.worldlabs.ai/

This from Fei-Fei Li's lab looks more promising. https://www.worldlabs.ai

Yes - I had similar thoughts when I saw the word "startup" used alongside something so far-out (same 'critique" should apply to Fei-Fei Li's World Labs - https://www.worldlabs.ai). These are VC-funded research labs (and there is nothing wrong with tat). Calling them "startups" as if they are already working on an MVP on top of an unproven (and frankly non-existent) technology seems a little disingenuous to me.

Once it has been proven to be possible, other companies [1][2][3] can and will reproduce it, and will attempt to push the frontier. As far as we know, there's no bottleneck that's stalling development here. [1]: https://www.worldlabs.ai/ [2]: https://wayfarerlabs.ai/ [3]: https://runwayml.com/research/introducing-general-world-models

Dr Fei-Fei Li talks about this as the LWM (Large World Model) during this interview: https://youtu.be/fQGu016AlVo and with https://www.worldlabs.ai/

OpenAI launches Sora (quite a while ago now), Google needs to fire back with something else groundbreaking. I love the advancement of the tech but this still looks very young and I'd be curious what the underlying output code looks like (how well it's formatted, documented, organized, optimized, etc.) Also, this seems oddly related to the recent post from WorldLabs https://www.worldlabs.ai/blog. Wonder if this was timed to compete directly and overtake the related news cycle.

Y'all did such a good job with this. It captivated HN and was the top post for the entire day, and will probably last for much of tomorrow. If you don't know already, you need to leverage this. HN is one of the biggest channels of engineers and venture capitalists on the internet. It's almost pure signal (minus some grumpy engineer grumblings - we're a grouchy lot sometimes). Post your contract info here. You might get business inquiries. If you've got any special software or process in what you do, there might be "venture scale" business opportunities that come your way. Certainly clients, but potentially much more. (I'd certainly like to get in touch!) -- I believe volumetric is going to be huge for creative work in the coming years. Gaussian splats are a huge improvement over point clouds and NeRFs in terms of accessibility and rendering, but the field has so many potential ways to evolve. I was always in love with Intel's "volume", but it was impractical [1, 2] and got shut down. During the pandemic, to get over my lack of in-person filmmaking, I wrote Unreal Engine shaders to combine the output of several Kinect point clouds [3] to build my own lightweight version. The VGA resolution was a pain and I was faced with fpga solutions for higher real time resolution, or going 100% offline. Today, World Labs and Apple are doing exciting work with image-to-Gaussian models [4, 5], and World Labs created the fantastic Spark library [6] for viewing them. Most recently,…

Yep, they call them World Models. Here’s the cutting edge: https://www.worldlabs.ai/blog/marble-world-model Google deepmind is working on one too.

The thing to understand is how well Marble's Gaussian splat models integrate into virtual production for visual storytelling. Then it's filmmaking! We worked with filmmaker Joshua Kerr on this project that took him back to his 8 year old zombie epic roots: https://www.worldlabs.ai/case-studies/lightcraft It's hard to appreciate how magical this is until you get beat up by traditional photogrammetry-based set acquisition or trying to build it from scratch in 3D. In a couple of years you are going to see high school drama groups everywhere staging film productions with this tech.

Brazen brute bashes bank! Czar’s zealots embezzle zillions! Cops can't quell criminal caper! https://theshadowoftheczar.com is at large for World Labs (https://worldlabs.ai) global launch! Last week I released ‘The Shadow of the Czar’ in partnership with World Labs, the AI research group co-founded by Ben Mildenhall, Dr. Fei-Fei Li and more. ‘Shadow’ is an action-packed, jazz-infused narrative web game in film noir style, produced in just six weeks by a single creator to push both my storytelling and World Labs' incredible technology to the limits. Play via your desktop or mobile browser at the link above, or check out the trailer, or read the case study on World Labs' site! Trailer: https://youtu.be/4lOYJk3D7kQ Case study: https://www.worldlabs.ai/case-studies/2-shadow-of-the-czar Game: https://theshadowoftheczar.com Creator: https://jamesckane.com

Blog post: https://www.worldlabs.ai/blog/marble-world-model (https://news.ycombinator.com/item?id=45907541)

To be fair, multiview-consistent diffusion is extremely hard - it's an accomplishment of it's own right to get right, and still very useful. "World model" is probably a misnomer though (what even is a world model?). Their recent work on frame gen models is probably a bit closer to an actual world model in the traditional sense (https://www.worldlabs.ai/blog/rtfm).

> if they have anything figured out besides "collect spatial data" like imagenet I mean she launched her whole career with imagenet so you can hardly blame her for thinking that way. But on the other hand, there's something bitter lesson-pilled about letting a model "figure out" spatial relationships just by looking at tons of data. And tbh the recent progress [1] of worldlabs.ai (Dr Fei Fei Li's startup) looks quite promising for a model that understands stuff including reflections and stuff. [1] https://www.worldlabs.ai/blog/rtfm

On a very similar theme, here is the work from World Lab (founded by Fei-Fei Li, ImageNet dataset, et al.) about creating 3D worlds: https://www.worldlabs.ai/blog