FINALLY getting around to some of the topics you guys have asked for! What do you want to see next? In the meantime, you can try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/TheCherno . You’ll also get 20% off an annual premium subscription.

Would love to see a video on serializing the assets! Also really hoping we'll get to see how an Asset Packer is implemented in the Game Engine Series

I'd love to see a video on Cooking assets. It's always been a mysterious process to me

@Jorven

Ай бұрын

Yes, I haven’t got a clue as to how that works.

@neil_m899

Ай бұрын

In my game engine, I have an AssetProcessor that takes in source assets and generates product assets (.asset file extension) for each individual asset. This way, I can keep each product asset as separate file. For final packaging, I can just create a zip file with all the product assets programmatically. This is my preferred method 🙂

+1 for binary files ser/deserialization video/s. Presumably FileStreamReader wraps ifstream and ostream calls. Great content.

I was looking forward to this video. Thank you sooooooooo much Cherno

Good Video Cherno! A video on your approach to serialization/deserialization would be interesting.

Love this kind of content - extremely interesting, especially as a modder who deals with these kinds of "packs" frequently lol, but also as an aspiring software/game developer who has great interest in optimising game files like this

This is a lot like the Unreals stuff. I’m loving where Hazel is going

Great video! Thanks!

Thanks!

I really miss the outros you used to have 😭 It was like your signature for me 😂

I wish you went into detail on *why* you have an asset pack in a single file rather than just having a folder with the optimized assets as individual files. Is it mainly performance? Is it about avoiding lots of small files? About having an "atomic" representation of all the assets, identified by build version?

Always the best!

Great video, Charno!

Awesome thx 👍

I found that using generic packed file format (ZIP/PAK) makes things easier and scalable; and you get all the compression for free. Thinking about security through obscurity does make it harder to manage for very little benefit (if someone wants to, they _will_ figure it out anyway).

@hopperpl

Ай бұрын

zip is not used anymore, because the format can only compress single files, but not compress over many files. modern game engines use a similar format to tar, omitting all irrelevant information like time stamps or names, and then compress the tar file chunk-wise. the format is not tar, but it is similar to tar. And no, you dont need to decompress the whole file from the start to find your asset, you calculate the needed chunk offset and decompress that chunk; usually chunks are 256 KiB in size and span over many files

@dealloc

Ай бұрын

@@hopperpl Indeed. I also meant in general archive formats with some examples. "packed" is maybe not the right use of the word here in context. And yes, random access decompression is definitely more ideal when you deal with larger archives.

Thanks, very interesting. I'd like to hear more about the binary serialisation/deserialisation. I wrote a library for writing PSD files and want to know if I did it in a sane way.

Hey Cherno can bring the C++ gamedev series back. like updated and with even more fun projects.

Yes to 3D model data video please!

Great video

I'm making a photo editing app and I had a very similar idea for presets recently Since I need to pack more that JSON-serializable parameter arrays for what I'm doing I've been on the fence between sth like LZ4 and this approach And since I'm already borrowing a lot of principles from game dev for this project - might as well do this too, now that someone broke down the approach with a nice flowchart 😌

@TurtleKwitty

Ай бұрын

You'd be well served by sqlite so you don't need to make your own format

@MaxRovensky

Ай бұрын

@@TurtleKwitty I do, there's multiple concerns beyond just serializing the data

Duck is back, good content is guaranteed )

Nice t-shirt Cherno

Nice the autogenerated subtitles say that your name is CH, not too far from the truth.

Yes, yes, yes binary serialization would be amazing! :D Really interesting video

You should add an app-specific signature at the end of the asset pack to ensure that the pack is not altered.

@futuremapper_

Ай бұрын

someone really hates modding :)

@TurtleKwitty

Ай бұрын

@@futuremapper_ That wouldn't actually stop modding if the engine allows the runtime to continue anyways a simple popup "The games assets have been modified. Continue?"

Amazing video as always :) it was really informative. one question i have is, how do you handle integer endianness? if you build the asset pack in a little endian machine and then someone wants to download the game on a big endian machine, will it still work? I have seen in the video you had a template function to deserialize the binary data and it takes whole structs. are the templates specialized for the different structs? thanks again

i wonder if using vector and doing binary search would have any significant speed difference from using map, as the indirections of a map might cause more cache misses, at least this is something i hear on discussions of vector vs map, but this might be dependent on the size of the map

I might be asking too much here, but I feel like it would help if you actually make a video of you doing something like this on a small demo project, at least just a way to be able to have exe's working without putting the actualy asset fbx files in the packaging folder.

Will this system support modding? What if you want to allow mods to replace assets or add any?

What happens if you create a patch to the asset file over idk steam and it's like 1GB large. Can you make an efficient delta so the end user doesn't have to wait an hour to patch their game every time an update comes out?

Yes, more on serialization pls Bring raincloud back :P

Is there a version of Hazel available for testing purposes? Also supports XR (OpenXR - not openVR)?

what happens in the case of a corrupted file that still has a valid HZAP header? would you need to check for corruption for every single part of the file before loading them?

@mattiapezzano1713

Ай бұрын

You might as well let it crash at that point 😂 is prob some cheater changing meshes and stuff 😅

This is great and everything, but what I really want to know is who do I throw money at to get the duck shirt you are wearing?

A video about binary serialization would be awesome

how do you handle the byte order?

@BurgerKingHarkinian

19 күн бұрын

My best guess is just using little endian. But Idk

A question comes to mind. If you have 1 big monolithic file for the game, how do you plan to make steam updates not basically force download the entire game with each update. Even if only 1 asset was changed. Ofc I don't know how steam deals with synchronization, they might have a system in place. But it would be interesting to know if that has been considered.

@Rhadamus

Ай бұрын

Steam doesn't redownload the entire file. It computes the difference between the old version and the new version and only downloads and applies that difference to the files.

@Speiger

Ай бұрын

@@Rhadamus ok, but he just tells we dump everything asset wise into 1 file. Isn't that a redownload of the entire game by that logic?

@IgnoreSolutions

Ай бұрын

@@SpeigerA single file can still be “diff’ed”. When you do that you’re applying a patch to the file only changing the bytes that have been modified.

What IDE is that? VS 2022 right? What is that written "not equal" symbol in there? @23:03 How does this work? Is != translated into this?

build your own format is okay but i would like to put stuffs into a encrypted zip file instead of messing around.Yeah loading text based formats can be slow but not that much i doubt,look at how fast blender load files.

Great ✨✨ 🔥☺

How will you get the 8 byte integers working in 32 bit builds?

@anon_y_mousse

Ай бұрын

Probably the same way they already worked before. You could manipulate 64-bit integers in C and its various offshoots back in the 90's. Most of the time the compiler would generate special code inline to work with them, and then as compilers got better they would use SSE registers and then we got proper 64-bit a pretty long time back.

@raffiepro

Ай бұрын

@@anon_y_mousse Okay thanks 🌚

@foobar4114

25 күн бұрын

Hazel doesn't support 32 bit builds, iirc.

Currently the App Binary decompiles perfectly and has all the names just like Unity games without il2cpp. Are you planning some kind of obfuscation?

A video about binary serialization woud be awesome.

7:00 if anyone is more of a foss guy, I've found a liking to ImHex

This is interesting, but why didn't you just use a .zip archive? There are libraries for dealing with them in C and I'm sure there are some C++ specific ones as well, and C# has it in the standard library. Also, you shouldn't store libraries inside the asset packs. It would be far faster to load the code if it was stored external to any asset packs. My suggestion is that you should have a file in the root directory of a .zip archive for all the metadata you wish to store and then just have the assets as if they were in a directory whose root is the location of the game, but keep all of the code separate from the asset pack. Do you have a Linux build working yet? Will you be accepting community code contributions?

daddy

any security you might think is smart will be broken, by the local access security policy. if you even once get the asset as plain data, its unsecure. people will hack it. any such efforts are only making more burdens to yourself. multiple duplicate work for no actual reason. would be better to just have all plain readable files. also for all debugging reasons during editor operations. yep same files. even encrypting would not benefit you at all. you are going decrypt and load the files on the local computer. and online DRM will shuggers. speed-up reasons are not valid either, you can do that smart using the original master files. or generic export files. you just gave two excuse reasons to make things harder for programmers. engineers keep it simple. why would you promote unnecessary work. bad design. same goes for gov and taxes. no use. only extra hand waving. dont pay taxes to your king. independence day. managers not needed or required. at all. three times advertising something not required. 3rd time the charm. hmh. wasted work. no wonder time spent doing stuff that would not be required to be done. ever. fluff. money makes you doing nonsense. cut it straight up. why you need a trainer to keep you in focus. omg. bloat ware, and shovel middle ware. integrated not sparse. I hope you are not talking your own, or copy from people. because you did not heed to God's advice.

@gsestream

Ай бұрын

even if you stream remotely, you know those casette tape copies, or just screen capture. or NVIDIA RTX remix on the fly. etc. also making law again only shows you did not learn your lesson. emphasis on yours. oh yeah, money to serve.

@kirillvorotov1169

Ай бұрын

it is less about security and encryption and more about performance. You definitely don't want to load asset files one by one at runtime, it'll probably kill the performance of the game. Another thing, as being said, is that common general purpoise formats usually contain mode data than you need. For example, typical fbx file will contain scenes, dummies, lights, meshes and more. Instead of converting this data into a format that you actually need at runtime you can preprocess you data. Actually, scratch that. Your last sentences clearly show you a bloody lunatic.

@gsestream

Ай бұрын

at least I will not be implementing any of the nonsensical stuff for you. if you teach it you do it yourself. will not take any part in this. what, you cant code a fast text parser. even that binary program you use can do it. nonsense, anything you judge, comes on your head. no thanks.@@kirillvorotov1169

@gsestream

Ай бұрын

tanking performance? nope its just fine. its all the other issues, including extra non-useful work that tanks all money projects.@@kirillvorotov1169

@gsestream

Ай бұрын

what do you think about xml parsers or xhtml on the web. fast enough.@@kirillvorotov1169

dude, please don't make your vid personal-stories based, make them generic same as C++ series there might be a lot of ppl come to your channel for the first time and they find you talking about some stories of you engine in the middle make more of C++ tricks, vids, .... etc

But what about assets that are instantiated at runtime only? Do they belong to any scene?

No Cherno, I do want my file to be the raw modal file, that makes you game easily moddable. c'mon...

@tony01riggioni

Ай бұрын

I don't see how that's a negative.

@IvanStamenkovicSeemsIndie

Ай бұрын

@@tony01riggioni let's say you want to re-texture the existing game, if a file is a custom binary format good luck editing that in Photoshop or similar. Where if it was jpeg or similar you would be able to edit it quickly

@WhoLeb7

13 күн бұрын

I guess then you would need to create a tool for that, which allows you to modify the said maps without allowing you to retrieve asset data, at least not in transfer states(gltf or obj or whatever)