Just wanted to say I love watching these videos. Keep up the great work!

@TwoMinutePapers

Жыл бұрын

You are very kind, thank you so much! 🙏

@brodoxl

Жыл бұрын

Yes!

@stevefranklin8452

Жыл бұрын

except for your cadence and your penchant for 2 word sentences, which means I can only watch your videos with subtitles turned on....

I can't wait for games in the future to incorporate these kind of AI simulations in real time!

@shukrantpatil

Жыл бұрын

Basically a virtual universe ? You will be the playable character and all the NPC's will be AI controlled characters with their own personalities ? The number of crazy stuff you would be able to do in there would be incredible........wait a minute , this sounds extremely similar to a ...... metaverse

@owenb6499

Жыл бұрын

It’s probably just around the corner! I know unreal can do fluid sims but they do tank performance. I don’t see why they can’t add this to!

@richard_d_bird

Жыл бұрын

video games need to be applying advances in ai in all kinds of ways for sure

@MitsumaYT

Жыл бұрын

They already do, indirectly. DLSS is basically trained to upscale images on 16k resolution images, to then create a 4k image upscale from the 1080p original.

@Sylarleft

Жыл бұрын

Exactly my thought. This is perfect for a pre-made action scene sequence, where you don't interact with the simulation

You have to love this type of storage. It's incredible how much data it can store, even if it isn't lossless. Maybe there could be further applications in other fields in the future.

@TasX

Жыл бұрын

Maybe fluid simulations for engineering applications. Doesn’t need to be perfect, but enough to know where to place sensors and stuff.

@MaakaSakuranbo

Жыл бұрын

Kinda sounds like the human brain

@khiemgom

Жыл бұрын

@@MaakaSakuranbo they are both neural network ig

@pneumonoultramicroscopicsi4065

Жыл бұрын

@@TasX pi = 3

GPT3: NeuralVDB is a new AI tool from NVIDIA that can compress simulations by up to 60x, making them run faster and require less memory. This video discusses the tool and how it will be available to the public soon.

@godofthecripples1237

Жыл бұрын

NIIIIIICE

@pile_of_kyle

Жыл бұрын

Are you a GPT3 bot?

This reminds me of the original "tech breakthrough" that the HBO show Silicon Valley was about... "Middle out compression" - In the show it was promising size savings similar to what NVidia is now promising with this new AI compression method. it seemed so impossible (and incredible enough to launch a "robin hood" startup in the show) just a decade ago!

@IceMetalPunk

Жыл бұрын

I mean, it works completely differently, but now I'd like to think the inspiration for this was somehow similar 😂

I kept looking for the compression artifacts and finally found one near the end. It is remarkable just how well it manages it considering the staggering amount of compression it handles.

@360Fov

Жыл бұрын

I thought it was compressing geometry data?

@riveraluciano

Жыл бұрын

@@360Fov I think it does, you'll see mostly in transparent/translucent parts there is a loss of definition seen as noise.

@LegitChipmmunk1002

Жыл бұрын

Keep in mind KZread also compresses the video

@riveraluciano

Жыл бұрын

@@LegitChipmmunk1002 yes, but those artifacts are different, you don't see the typical blockiness of video compression.

@user-qv2wd2jc6m

15 күн бұрын

EXACTLY!!! This type of improvements in technology get's me SO excited like a kid in a candy store! Seriously, this is the kind of tech that improves experiences in video games!!! Can see developers using compressed micro VDB's for clouds interacting with ray-tracing casting shadows over fields, and massive 3-d volumetric fires/smoke on battlefields all reflecting ray-traced light in real-time! Not yet of course, but in a few years or when we finally get the next PS6 and Xbox for sure!!!

4:25 Weather forecast: Sorry we have a technical problem. Casual NVIDIA home user in 2022: Fine i'll do it myself.

Less memory footprint means that now they can scale up the resolution or the simulation space? Which is more important for science or for industry? What I heard that for a car mirror simulation it is required at least 6 times of the length of the vehicle to simulate it properly.

@Android-ou3qv

Жыл бұрын

NeuralVDB is lossy compression (?) so I'm not sure how useful the technique is for the scientific or industry. It might be okay for rendering/games and such but not for actual physics unless you want to introduce some artifacts. Correct me if I'm wrong

@nebelwaffel8174

Жыл бұрын

I think it could be used like superresolution for rendering. If you render an image at 4k and scale it down to 1080p you will get an better image than the native 1080p render. In the same way you could make a finer simulation and compress it afterwards back to normal filesizes. However often is the computational power the limiting factor, not the storage

You're one of the few youtube creators I will stop everything and watch every new video, the implications of each study are just mindblowing and I'm so excited for the future! What a time to be alive! :D Thank you for making new studies entertaining and accessible!

Conspiracy theory: Two minute papers is written, edited and narrated using AI. The voice, the topics, the way everything is explained, the selection of stock footage, it all checks..

With compression levels that good we could fit VBD caches into freaking MOBILE games. Imagine a world where you don't have to download 100gb just to play the latest AAA games, where it could fit in just 5-10gb.

@rubengildenhuys8962

Жыл бұрын

Nah, still a 100 GB download, but a 100 times larger, more intricate game. Imagine a game like ARMA 3 with full terrain and building destruction. All in real time on map 8x8 Km

@GalactusTheDestroyer

Жыл бұрын

@@rubengildenhuys8962 There's no reason it can't be scaled both ways. Smaller footprint means a potentially wider audience while larger basically equates to more detail and scale. Future looks bright!

@Operational117

Жыл бұрын

@@rubengildenhuys8962 8x8 km?! That's just 64 sqkm. That's already been achieved in the last decade of video games (even if they do take up the full 100GB of space). However, 8x8 km with 16x16 tiles per square meter (equating to roughly 128000x128000 tiles) would be beyond impressive, as would 128x128 km maps with one tile per square meter! All within the same 100GB of space usage! EDIT: Oh, what about 8x8x8 km maps? Now we have TRUE terrain features like overhang cliffs, caves and the like!

@mikakorhonen5715

Жыл бұрын

Nobody wants 10GB game.

@Ben-rz9cf

Жыл бұрын

@@rubengildenhuys8962 building destruction isn't about VDBs. The typical method is voronoi fracture. So a step up from prefractured meshes like the ones they use now is impact based fracture like the kind they have in houdini. VDBs are far more useful for anything that involves smoke, clouds, water. Volumes specifically.

NVIDIA IS Dominating the Future of AI...

I guess this is part of the reason why NVIDIA is dramatically raising prices on their GPUs haha

@mirroredvoid8394

Жыл бұрын

Yeah, Nvidia doesn't a shit about gamers their tech useful in so many other industries.

Nvidia is doing sooooo much work. I wish them well 🥺

Long time subscriber just wanted to say thanks for all your hard work on these videos

Super cool stuff. I like to get a glimpse of the future and your channel is great at it

Your videos are always fascinating, but for some reason I look forward most to your "hold on to your papers" logo appearing (and the "what a time to be alive" catch cry)!

Could be a complete game changer for us in the VFX industry. We are very limited by the memory footprint of the volumes we are using, sparse solvers have recently given us much higher resolutions, but this could increase the size our volumes by almost a factor of 4. I can't wait to do my first nuclear explosion with this.

I'm totally looking forward to read the entire paper, but in case it's beyond my comprehension, does this mean that it could be used to expand the processing capabilities of the graphic cards using less memory (because of the compression) or just reducing the size of the finished simulation?

The spaceship rising out of the water is in the videogame "Between the Stars." Makes me wonder if it is an asset that is just up for sale. I thought they had built it for the game, but maybe not. I'm also interested to see how long it will take before these kinds of physics simulations get so cheap that they can be used in videogames in real time.

I wonder if this can be implemented in houdini. Would be a game changer for houdini because so much of it relies on vdbs

@spyral00

Жыл бұрын

I recently did a lot of fluid simulations in Houdini and yes, this would've been a game-changer. In 2022. it's still very complex, unstable and very difficult+slow, even on the best existing software. So it feels like it's one of these areas of computer graphics where there is tons of room for improvement.

Man I really love how you resume so quickly, and how passionate you are speaking about them, it's lovely and joyfull to me to hear everyword you say.

I've been a PC gamer since I could type Load "*" ,8,1 on the family computer to start up my favorite games. I've been a fan of nVidia's technology and the groundbreaking work they do to push the industry forward. I am also a science enthusiast, so I love the fact that my hobby since childhood is still pushing the industry forward, now my additional hobby of following what various scientific communities are discovering and predicting using technology like AI. I have a smart home powered by the Google ecosystem running on nVidia hardware. I am so glad that the money I "wasted" upgrading my GPU faithfully over the years so I could shoot pixels at pixels is now doing real work to help all of us understand the universe. What a fantastic time to be alive indeed. Came because I saw what looked like a cool nVidia tech demo that I hadn't seen, subbed because the video creator is a kindred spirit. 🙂

One of the only channels I always watch when a new video drops. Such an exciting field of truly world changing technology!

Just Yesterday I was Simulating some flames to be used from a Jet booster in an animation. the simulation is a small background element, but with how much space they take I have to use simulations on a project by project basis. If we can get Neural VDB working at a consumer level that would easily open up the ability for artists to create libraries of their own simulations to drag and drop into multiple projects over time, not to mention having the option for much larger simulations to play a main part in their work! Nvidia has a good history of working with Blender to implement their technologies so I'm hopeful of seeing this soon!

I wonder when are we gonna get AI enhanced weather models? I don't even know if it'd be possible, or how it'd work, but I'd imagine you could have a neural network study current weather models, and then guess which one is most accurate in each situation, then have it model alongside our current weather models, and as the training got better, we'd be able see further into the future, as well as get more accurate predictions. The only issue is, we'd need sensor data for the whole planet. I'm sure it could also be used for more precise weather modeling, and if you could train it on how different landscapes and mountains affect wind patterns on a small scale, you could scale it up for regions and could hypothetically reduce the butterfly effect. Even if you didn't get it down to exact resolution, you could get a topographical map of land surface and cut it down to half a kilometer for training data, then train it on how different atmospheric conditions, wind speeds, and wind directions interact with the landscape, then scale it up to simulate all land on earth. Of course you'd have to deal with the weather effects of the ocean, and to solve that you'd probable wanna decrease the resolution to every 10 kilometers and track the surface ocean temperature, as well as waves and atmospheric conditions.

@kodfkdleepd2876

Жыл бұрын

Unlikely, AI is not good at prediction. AI is simply a highly optimized compressor/interpolator. The only reason we think AI is amazing is because past methods have been very crappy. AI will never solve the chaos problem. Yes, it will be more effective in some ways(given the older methods are pretty bad) but it will always be susceptible to chaotic indeterminism. We pretty much, though, have a decent interpolator for weather. During the summers it gets hot and winters it gets cold. We pretty much have historic temperatures that are relatively accurate. Not sure why we would need to know the temperature 3 years out to 0.1C(assuming we could). In fact, AI fails at this problem pretty catastrophically(maybe with enough data it might be able to model it but that would require far more computation, data, and memory than we have). All this stuff you see with AI now days is only works well because it tricks the human mind well. Before, say, the hard coded hand crafted algorithm were designed very poorly and so a human could easily see the flaws in it. With AI, all the flaws are distributed/encoded in a way that does not ruin the experience for the human mind. This does not mean there isn't error, it's just that the error is distributed in a way that doesn't interfere with the intent. E.g., old video compression would use blocks and these blocks then would be obvious at low rates. Using AI that isn't block based allows one to get low rates without the blocky effects making the experience better. Better to waste time on more important things like solving homelessness or financial terrorism that will have a real impact on humanity.

@martiddy

Жыл бұрын

There are simulation algorithms that can make a lot of weather predictions with the help of supercomputers by taking into account all the climate factors that you already mentioned (and more). However, I don't know if such algorithms uses some kind of neural network or are just pure physics calculations.

@IceMetalPunk

Жыл бұрын

@@kodfkdleepd2876 "AI is not good at prediction. AI is simply a highly optimized compressor/interpolator." Um... what? One of the first families of neural networks, and one of the most common, are classifiers, which are in effect prediction systems.

@kodfkdleepd2876

Жыл бұрын

@@IceMetalPunk Wrong, you completely clearly do not understand the work predictor/prediction/predicting. (or you do not understand what classifying is). "A prediction, or forecast, is a statement about a future event or data" By future they mean *unknown*. Classifying something is not based on the unknown, it is precisely based on the own. A classifier is a in interpolator. If you have two points you can interpolate *between* them. That is, you guess the value between them using linear approximation. Extrapolation is not finding an approximation between data but is guessing the future trend. In some sense it is interpolation but with one point being that of the point at infinity... but this changes the behavior completely. You do not understand as much as you think, I suggests you go learn about what really is going on. AI is just large dimension regression. Classifiers are interpolators that the very training is setting up the sampling of the domain and application of the AI is simply finding what the unknown input best fits(which sample it was trained on most represents(usually through least squares) it). All the fancy AI we see now is this and this is why they have a differential manifold based parameterization that everyone loves. That is, one can adjust the inputs(which is just a vector) smoothly through the space to generate outputs. This is running the NN backwards and, really, generated data from AI is actually just the errored inputs that do not mach the training. Everyone thinks this is amazing and special but it's not. It's not any more special than taking a sampling of a 1D function and generating some curve based on the sampling. The only difference is AI is working on 10-100's of orders of magnitude of dimensions higher than what people generally think about. There really is nothing special going on with AI. It just seems that way. It's only special in that it is a very powerful tool but if it's used improperly, e.g., people like you use it to predict if nukes should be used or if the rich should loot the world economy because they are gods then, well, it's extremely dangerous. AI will never do anything more than interpolate. That is all it does. Anyone that has ever designed a NN and understands the various mathematics of it should understand this. When AI starts to become highly modular(It's now starting to be used in that direction since the foundations have been solved) then we'll see a paradigm shift. Of course, in fact, the human brain is also an interpolator BUT there may be more going on which isn't in AI(but maybe not). In fact, extrapolation may be impossible and we just get lucky when we predict things because of the structure of the past(e.g., if it is usually differentiable). E.g., I will give you one of the most simplest problems. Prove to me you are a genius/god: f(0) = 0, f(1) = 1. Approximate f(1/2) the best that can be done. Approximate f(2) the best that can be done. (if you are smart you will notice that what you are doing for f(2) is the same as for f(1/2) and yet they are fundamentally different types of points. With some constraints on f, such as limiting it's bandwidth or derivative in some way it changes things up but only superficially, still the underlying issue exists. This is the difference between global and local. In fact, predicting f(342342) is really no different than f(2) as one can show with transforming the data using a homeomorphism)

@kodfkdleepd2876

Жыл бұрын

@@martiddy Mandelbrot was working on weather prediction using algorithms in the 50's when he stumbled on chaos and indeterminacy. These are real problems that cannot be solved(although maybe a completely new type of mathematics would not have them). There definitely are patterns in the weather... that isn't' the issue. Every chaotic system has order. The issue is that one can't quantify the error in the prediction to any meaningful degree(at least weeks out). The error compounds on itself so after several iterations of the system the error starts to become dominant and eventually takes over. This is a problem of all recursive systems. Every recursive system will tend towards it's fixed point. f(r) = r, f(r_k) -> r. This is inherent in the math and there is no way to escape it(except for completely new math). So any algorithm/ai that would be designed to predict anything would require it's fixed point to be known and that means to already know the outcome you are looking for which defeats the purpose of designing the system. What this means in practice is that no AI will ever be perfect(they are recursions) for doing anything, eventually they will become more and more off(humanity is also an AI, as is your brain, if you take a general approach to AI). Hence there will always be a tweaking that is required, and updating to fix the "bugs in the software". Over time though they will get more "accurate". The only time this game stops is when people say "Ok, we have a good enough approximation". E.g., it's like predicting the digits of Pi. At first even a few digits were hard... then 7 was enough for anything... now we have billions of digits. Until humans decide enough is enough the game will continue going. We'll never even become close to knowing every digit, it's an impossible task and yet people will continue. This is not necessarily bad but can be. Not knowing when to stop lets you go further but eventually you fall off a cliff.

Using this for gaming and VR would be amazing. Real-life graphics on a tiny budget. Something VR desperately needs is real world weather and better fluid simulation.

holy crap thank you for the content two minute papers each one you release in mind blowing ! I subbd and am here for the long haul now. cant wait to see yearly updates on these techniques and how whacky this gets ! enjoy being human while we can folks

Thank you so much for such a wonderful review! 🙏

Putting that Hyperion cloud in to such a tiny footprint was NUTS!! Very well done

This is huge improvement, must read the paper!

3:11 - This would be awesome to see in game engines. Only 25mb of memory??? When we now have 8-12GB starting to become more mainstream the sort of visual effects we will be able to have in games in 5 years is going to be mind blowing.

I'm a visual effects artist and this gets me SO EXCITED. Thanks for the video!

Would have loved to see a minute of evaluation on how the compression loss affects the final product. What exactly are we giving up?

1:06 - LOL I was like "wow...this looks just like the real thing..exactly like it !" then I realized it was a real footage haha

Wonderful video!!! Very VERY exciting!! I cannot wait to see this being used in video games! We've seen wonderful advancements with ray-tracing and graphics overall(like Nanite with UE5) but we still have always lacked real-time water physics and special particle Fx like smoke, fire, etc. and really sucks you out of the immersion when playing, ugh!! I would really love to see what Nvidia's next GPU the 5090 can bring to the table and what improvements that they might have made with this technology since the 4090 2 years ago!

This is very impressive. I look forward to the day when we get real-time ocean wave simulation

If i had a looking glass holographic screen and powerful GPU, i would use to see insane holographic simulations

@TasX

Жыл бұрын

You could always pre-render them

thank you so much for your work !! can this ne used for storage space of games ??

It's surprising how many things can be improved drastically by throwing artificial intelligence at them. Truly, AI is a bringer of a new era.

@IceMetalPunk

Жыл бұрын

"This would require a ton of math that we haven't even invented yet, so much calculation time, somewhere to store all the data..." "What if we just tell the computer what we want it to do, and then let it teach itself how to do it the best way?" "..GENIUS."

Can't wait to run a one trillion particle simulation on my nokia

This channel scratches that learning itch just right.

What software did you use to do the leaves on water simulation?

Amazing! Now I can do huge projects on my PC in Blender

This stuff is blowing my tiny mind. Love your vids man.

Amazing when I look at this I am imagining that unreal engine would be a great place to start with this provided the fact that they have such a breakthrough with lumen. Thanks for this information.. just wanted to add that mainly I was trying to say Nanite , which handles huge amount of meshes.

@wobbers99

Жыл бұрын

a great idea.

@litjellyfish

Жыл бұрын

I think Nvidia probably want to first make sure people use it in their own “engine” Omniverse? Sorry what has lumen to do with this really?

@supertaurus2008

Жыл бұрын

@@litjellyfish hi i think didn't add in Nanite. Which is actually what I was coming to say.

@litjellyfish

Жыл бұрын

@@supertaurus2008 yeah for sure that is great combo. Although I guess some of those things don’t work with it. But for sure the hierarchical non deforming meshes could work for this physics / simulations and nanite I would say some fields is also to use this to construct lighting and point clouds simulations fast and then use the data for lighting or volumetric mesh generation or to create liquid meshes real-time.

in the near future, we will be able to see movies like Avengers, but not pre-rendered but directly post-produced in real time by our GPUs, with real physics and maximum RayTracing And the file will be so small that it will fit on a 1.44mb floppy disk! What a time to be alive!!

@jackieclan815

Жыл бұрын

@PerúCriticoAU lol wow that's interesting!!!

amazing possibilities in VR, Art, Predictions, and more. What a great time to be alive!

That's impressive. I want this for music.

I didnt understand if it is only a matter of computational power saving, or it is really the simulation itself that is going to be more easily achievable, ie without the very complex pipeline necessary with such a program like houdini for movie special effects

@mohandasjung

Жыл бұрын

It seens like both

@alexmcleod4330

Жыл бұрын

Neither. It's for storage of the simulation data. You usually wouldn't run your simulation and rendering tasks at the same time. You'd simulate as much as you could in one go, storing gigabytes of data per frame, and you'd schedule the rendering to happen later. All that stored simulation data for a typical simulation can be a few terabytes, even when compressed. NanoVDB and NeuralVDB mitigate those costs - but NanoVDB is read-only, and NeuralVDB is lossy, so they aren't really useful during the actual simulation itself.

0:42 This legitimately made me amazed

What program was used for the water simulation

Your narration is one of the few examples where a text-to-speech robo-voice would be an improvement.

Is this an open library? It's incredible. really shows power of neural networks

What a time to be alive!

da phuck, it looks so smooth also so less space it is damn amazing

Reminds me of Leigh Orf's work

This will effect so many things. This compression could be applied to all media allowing far greater detail on all media everywhere on the internet.

Is this compression or reduction ?? ...is the principle of compression to reduce the size but, to retain the original data with no change ??? (rar / zip / tar) ...I'm confused with all the definitions :) . There was no mention of lossy compression ( example: jpg/ mp3) ...Regards ...another super interesting video :)

It's not lossless but for the compression it looks amazing.

power of holding onto papers - just magical

Can this method be used to store pure structure data? Like gene sequence. We are running Seurat for RNA sequencing and each pipeline we ran is humongous, and as you can guess, eats up memory quite rapidly. Since they are all eventually binary, it will be cool to see something like this.

I want to see this kind of thing implemented into an aerodynamics model in games.

Amazing video !!

Insane ! One day we will have this in video games !

I wonder if we could get faster simulations through AI as well. Realtime fluid simulations for video games would be cool.

In GDC they did say it will also be in video games, they should racer X something demo. I think this is the same thing.

What a time to be alive.

Is this scalable? Say you still run the simulation in a huge datacenter but with this technique, would you be able to do even more complex simulations then? (For example that meteorological simulation mentioned in this video)

@sypialnia_studio

Жыл бұрын

Everything is scalable. But this compression is not lossless so the compression noise is also scaling up. Which is not good for meteorology simulation because of the 'butterfly effect'.

we are seeing life-time achivements every day. the speed of these improvements never cease to amaze me. sometimes I think we are going a little fast haha

I am curious to know on what gpus these simulations run on. After all, not everyone has workstation GPUs or a 3090 lying around. Still amazing though.

Lol I recognise the guy in the example 1:56 , I think it’s Coleman Huges(?)

I'm scratching my brain from the walls right now... Blew my mind ! 🤯

I want to do aerodynamics simulations with this

That's a massive leap, I wasn't squeezing the paper enough

2:06 if you pose during this part you can see the video transmited with ai compression has not the same face than the original !

I can't help but think of videogaming applications lol what a time to be alive!

"NVIDIA" and "cheaper", two words I didn't think I was going to hear in the same sentence anytime soon 😂

I wish blender's fluid simulator Mantaflow will integrate this useful feature! Cant wait to create stuff like 0:23 in blender without putting my disk and CPU on life support

I wonder if this comes at the cost of precision and accuracy in the data that is stored

Damn ! So it seems, as I was theorizing, that after ray tracing, the next new big technology coming to games might just be real time physics simulation. Not gonna lie, that sounds awesome.

So is it available right now in any practical form?

NanoVDB is already in Blender... Thanks nVidia 😀 !

@wobbers99

Жыл бұрын

really? is it a plug-in?

The implications for weather and climate modelling could be tremendous!

Would youtube use this to compress the videos?

I would use it to make even more complex simulations.

Again amazing video. This would bee soo much help. Im currently doing a comercial for a product I designed. It is rendering just now. That 64 times more detail to fluid simulation would make a hell of a difference. There is soo much stuff going on right now that soo I will be able to make even better animations for my company.

as a water artist.. I hope one day in the next 10 years.. we would be able to use some of these..

I need to do a realistic flood simulation for VR, my options are so limited

AR with real Particle Physics! Lovelace is pretty wild

What are the weaponized/misuse scenarios for this technology to be avoided

man, from an entire DC down into a single GPU! holy mackerel!

Real time realistic simulations in games incoming?

those "gta 5 in 100mb download" might become possible

Train one of those physics simulation AIs to work with this data representation.

This isn't a simulation. It's an animation. A simulation is what we are composed of and exist in, namely, quantum particles and spacetime. Regarding animations, a computer and display is one way to create them. You can also use a robot, or a puppet perhaps.

It would be awesome if have a breakthrough with AI and then we are be able to play games like Death Stranding on a mini laptop from 2005

What a time to be alive

does this mean that we can soon use all those amazing/unreal simulation Realtime?

Is this interesting for CFD simulation in racing?