Thank you all so much for the support this year! Wishing everyone a happy new year and I can't wait to show you what else I've been up to in 2024 🎉

@ralphwarom2514

6 ай бұрын

Why not just have a mini-QR in every picture file that the game can read and start you out from that position. (Sending this before I watch the full video.)

@toby818

5 ай бұрын

@@ralphwarom2514 what abou a qr code using slight variations in the alpha channel to represent black and white

@Zak-AZ

5 ай бұрын

Combine the transparency method with the QR code method. Generate the QR code, but instead of using black and white to represent it on the image, use alpha values 254 and 255. Then to read the save file, just look at the alpha values to re-convert it to a QR code behind the scenes.

@ragingram2

5 ай бұрын

​​@@Zak-AZi came here to say this exactly lmao. Since the QR standard has built in error correction it should be possible to encode the QR code with LSB(i wonder if you could modify LSB to work with the least 2 significant bits 🤔).

@BIGGESTxUSERNAME

5 ай бұрын

@kylebanks What an awesome idea. I just implemented something similar and it works when using compression level 9 and halving the image size with Pillow. However, I tried using compresspng online and it doesn't work. I'm happy to send you the code if you're interested.

This reminds me of Spore made back in 2008. You could make 3D ships, creatures, houses etc, and I recently tried to get back my old creations but was surprised to see only thumbnails in the save folder. Turns out they did exactly what you presented: the pngs WERE the saved creations, containing all the 3D information hidden inside. But I think it's hidden differently, using color information inside the parts of the png that are transparent (and this is the majority of pixels). This enables much more information to be kept, or allows to significantly reduce the image size for the same data. Not 100% sure since I've not looked into the thumbnails data myself.

@kylebanks

6 ай бұрын

Ah that's so cool! So the thumbnails were transparent cutouts of the creations I guess, with the rest of the unused pixels being used for storage? Very clever.

@kuromiLayfe

6 ай бұрын

i remember Koikatsu Party using BMP files for their save files containing all the character data and location data in the sandbox game … it had 1px wide borders containing the location data (least important), and the character data (name , outfit relation status etc) was like a invisible watermark (64 px x 64 px tiled in the image (1 bit colorshift). because it was BMP it didn’t have the compression issues JPG or PNG has and can even be resized

@Nebulaoblivion

6 ай бұрын

Yeah, I can't for the life of me remember how specifically it worked, but one of the Maxis devs explained it pretty clearly in a talk.

@Darkxellmc

5 ай бұрын

@@kuromiLayfe Never ask a women : her age. A man : his salary. KuromiLayfe : how they knows this.

@kuromiLayfe

5 ай бұрын

@@Darkxellmc the HF game modding site had a full explanation on how to add modded content into the save file while preserving game updates … thats how i learned about koikatsu save method 😂

All this for a small feature. I admire your dedication to the game. Hope this game becomes a hit

@kylebanks

6 ай бұрын

Thanks! I just enjoy a good programming challenge :)

@nathannguyen1431

6 ай бұрын

Feels like a key selling point of the game if you ask me 🤔

@lorrdy7640

4 ай бұрын

You just call the game boring with that@@nathannguyen1431

@jakemartinez6894

4 ай бұрын

@@nathannguyen1431 It’s a cute gimmick that would get used once or twice…

@-sleepy-

4 ай бұрын

@@jakemartinez6894 It's used all the time in the games that have it, but it's definitely not the key selling point

10:48 Ok wait wait hear me out. Building on the idea of it being a post card… what if a game used this method, and use the stamp on the postcard to encode additional data. The stamp could be predictable, so you could hide a QR code in it easily, or even just have the stamp (or validation thing on top of it) straight up be a QR code. That would be such a cool feature for like a game the revolves around travel or letters for its theming/aesthetic!

@leonverhaegh4126

4 ай бұрын

If you really want to take it further, you could use the tech used to create QR codes to create your own "standard", which looks like a postage stamp.

If you have static UI elements, such as a map frame, you can store the data in those, and to the user, it'll look like an animated texture.

I don't actually hate the QR code version. With some graphic design work (QR codes are pretty tolerant to deviating from the rigid shapes and colours) it could look pretty nice! You could probably get away with styling it as a rune carved into a tablet, or something of that nature which fits your aesthetic. Using a proper binary format would allow you to further shrink the data too, perhaps allowing the QR code to be a little smaller / less busy. All of that said - my understanding is that this is a story based game, and sharing save files doesn't actually seem like it would make that much sense to me.

@kylebanks

6 ай бұрын

Totally agree :) Ultimately this won't go into Farewell North, but I do think it could be interesting for the right kind of game

@aaronpohle2016

6 ай бұрын

Or you could integrate the data more organically into the stamp or some other feature of the postcard. If you assume that the data is read in image form and not by a camera it should be fairly easy to "make up" your own format. You just have to sample predefined positions in the image and don't have to worry about perspective for example. You could do this as part of the stamp or some kind of letter marking (iam thinking of the old school stripes used for international post). Of course this would yield very little space for data but I think that with the right optimisation (not using Json at all. Just raw data positionally encoded and in the best case not even strings or something like that) this could work wonders for combining beauty of the screenshot with resilience of the data.

@Sollace

5 ай бұрын

@@aaronpohle2016 Since we're going through creative ways to hide data in the game, how about this: Turn the stripes on the acacia trees into barcode stripes. :D

I wonder if you could combine the QR code with stegonography. You could make it much larger, covering most of the image (using multiple pixels for each QR code block) and hide it the same way people hide watermarks. Since watermarks are made to be resistant to compression, this should be pretty consistent.

@eragonawesome

5 ай бұрын

I was thinking the same thing! I hope they see this and give it a go, I'm still convinced this could be an *awesome* feature

@heyjakeay

5 ай бұрын

This is what I was thinking too. If you're already handling the decoding, you can make the encoding any hacky way you want since you know the exact way you need to undo it later. Making a scaled-up QR over the image but hidden in the colour information would make it a perfect disguise and work fine when resized or compressed.

@aSaProgrammer

5 ай бұрын

I just did that myself, I will try to improve it and upload a video of how it works

@lunaponta594

5 ай бұрын

@@aSaProgrammer im looking forward to watching it

@EmilioBPedrollo

5 ай бұрын

​@@lunaponta594 The QR code have a lot of features dedicated to make it easier to be recognized in different skews, rotations, light, etc. and it sounds like a huge waste of image real state to codify those bits on the image. You should ate least try to get rid of those and encode just the data parts of the qr code. Also you can try to separate the save data in blocks add some simple ECC like humming code to it. And finally do like NASA did on Apollo and have the data stored 2n+1 times in the image then make of this redundancy it's own ECC.

Super interesting topic and video, Kyle! Fun fact: I'm not sure if it's still the case, but in World of Warcraft, screenshots used to encode data like user ID, timestamp, and realm information into the image. You can check this by capturing a completely white screenshot. Afterwards, you need to adjust the sharpness/contrast of the image significantly, and you'll be able to see these artifacts.

@kylebanks

6 ай бұрын

Ahh that's neat! Just did a quick search and it looks like it was to detect where leaks were coming from

@wooviee

5 ай бұрын

This technique is exactly what I was thinking would be a good solution. It's like a QR code, but the entire image is the QR code.

@SimoneBigozzi1998

5 ай бұрын

woah didn't know about that!

Postcard saves feel right at home in and they sound really cool in terms of sharing online, maybe with a way for the player to write text on it too? Could get extra fancy and turn the load save screen into a table full of postcards the player can pickup, look at, etc Many postcards have little stripy border edges so 32 edge stripes each slightly varying in individual rgb channels (4 variations per channel) could be subtle enough to not be too noticeable of a difference (which could be helped by alternating the base color every other stripe) but not so minuet it gets destroyed by compression, with this you can store 2kb

IIRC, Exapunks (a Zachtronics game) uses LSB encoding in images to store the player's programs for the fictional Redshift console - arguably a perfect use case for the format, given the small data size. I enjoyed seeing the different steps in progressing the "postcard" idea further!

@tuple5982

5 ай бұрын

zachtronics >>>

This a fun problem. I like the idea of a stamp, like you had at 0:05 . Real postcards have them. You could probably store as much data as a QR code. Maybe, (and this isn't the only way to do this, or the best) make each element of the postcard represent a thing being saved. Say you had 16 levels, have 16 icons. A lighthouse to represent a coast level, a house to represent another. Then have the accent marks be other bits of saved data. A white border for one thing, cream for another, navy for another. That feels like a pretty brute force way to save data, but since you're representing things with blocks of pixels it would be fairly robust to image compression and filters. (Nothing would be perfect there) It would also have the added effect of generating unique stamps, which some people might want to collect.

Parkitect uses LSB steganography to encode ride blueprint data into PNGs. Apparently Spore also uses it for creature data.

@kylebanks

6 ай бұрын

very neat!

Maybe combine the qr code with the screenshot by blending it together. E.g., like a slightly transparent overlay or similar? Photo editing tools with layers often have various options, maybe there is a combination for you, to add a almost unnoticeable qrcode that is recognizable to machines but not at first glance to humans? Might still have an issue with image recompression...

@PopeGoliath

5 ай бұрын

You're describing the steganographic approach with added features: larger metapixels, more significant bits used from the channels, and error correction built into the data. That could definitely increase reliability. Tuning the magnitude of these interventions could allow data to survive the image compression standards that most big uploading services use. If your data survives upload toFacebook, Google, KZread, etc while being minimally noticable, that sounds like a success.

@FracturedFantasia

5 ай бұрын

@@PopeGoliath I totally agree. Especially if the screenshot included some pretty overlay (name of the game, Polaroid vignette, playtime so far, artsy swooshes) that is actually a locating feature so the QR reading function knows where to look for steganographic bit flips. I think this technique could get you all the way to printable, scan-able screenshots.

@JamesRaynor11

5 ай бұрын

Or, if you can’t hide a QR code subtlety using LSB manipulation, lean into it and stylize it. Use the QR code as a masking layer for some kind of blending. E.g. have your screenshot desaturated everywhere except where the QR code is, or vice versa

Woah, all of these methods are really cool, and it's also cool that this rabbit hole led you to shrinking down the file size of your saves!

i think the neural network was matching the color palette of each region, thats why it scores perfectly and has 100% confidence, since i dont think any of your post processing messes with the colors too much either

Another way I thought of doing this was by combining the Stenography with QR Code. Assuming: - QR Codes are always square; - You can force it to generate a fixed row x column count. It's easy to create an algorithm that resizes the QR Code to fit in the maximum possible size and encode the black-white squares in the image's LSB (maybe with 2 bits instead of 1). With this, you have redundancy, because more than one pixel represents the encoded data, and you can take the average of the LSBs to reconstruct the QR Code (with these two possibly dodging compression and resizing). Plus the QR Code's error correction algorithm. --- These random ideas are excellent, though, because they forces you to search through and learn different things!

I passionately love the dedication that you're putting on this game, I wish you the best and I can't wait for playing this masterpiece

@kylebanks

6 ай бұрын

thank you :)

Realistically the "best" solution for this, I think, would be to store the raw values of the save data in a custom, aesthetically pleasing "stamp" that was placed on the image. Like, each checkpoint could have a unique image, you could embed collectible data in varying decorative border patterns, etc.

This reminds me how you can save screenshots of characters from a game called Koikatsu, where you have a very indeph anime character creator, and the game can just load characters from those screenshots including all character creator settings. To this day I have no clue what method they use. You can even share screenshots of modded characters and other people, if they have the mods/files installed, can still load them and use ingame just fine. Since it's so uncommonly used, it always amazes me.

This is really creative. The image loading would fit your game really well - I absolutely love the design you made for it. Keep going!

I feel the need to point out that you can add arbitrary data to png files png files consist of chunks of various data, these could be critical (image data) or ancillary (metadata), you can find a list in the png spec. eXIf that you mentioned is an ancillary chunk. ofc you can't use it since it is removed by many apps for privacy reasons. But you can just use a custom chunk, all you need to do is use an unused (publicly) chunk type ID so it doesn't conflict with anything and that the chunk is marked critical so it isn't removed. Then you can just add whatever data you want in the chunk. Ofc you need to specify the Len of the chunk data and the CRC (corruption detection number) this isn't affected by image modification or apps removing private info (almost certainly)

@SumeaBizarro

4 ай бұрын

This is likely what Illusion games do with their "cards" (this is Japanese smut games from past 8 years or so) Critical thing for their system is being able to share the "cards" and I was interested how they avoid places stripping their arbitrary data

I think seganography could work for most things as long as the screenshot resolution is within the range for twitter to not re-compress, I know koikatsu uses a png-based system to share data, but I'm not the kind of person to try/find out how that works... There are also "watermarking" methods (which are functionally steganography) being implemented for deepfake detection/digital rights that are resistant to being resized supposedly, but IDK how they work.

I didn't even know this game existed. KZread decided to just show me your video because I was essentially doom scrolling down the rabbit hole of recommendations. Now I'm patiently waiting for your game. It looks amazing!

I still think the qr code is a better solution, you can decorate it in a way that blends with your environemnt a little and it will be an awesome feature to have.

using a neural network for save-data will definitely result in some *very* interesting speedruns. I imagine someone will find a way to trick the A.I. to take their 'new' save, and with just a reload, put them right at the end.

Such a cool and creative concept! Very educational devlog. Great stuff! 🔥

@kylebanks

6 ай бұрын

Thanks, glad you enjoyed it!

Interesting idea and a wonderfully made video. Best of luck for your game launch.

What a fun video! IIRC the "Kingdom Hearts" games do something similar with their save files. From what I remember, each individual save file (99 of the suckers!) is a png with *slight* differences in their makeup. How Square Enix transfers those images into your save data may as well be wizardry to me though haha.

this was a really good video! cant wait for what else you have

just found your channel. very clear and useful explanations of all the things in the video. subscribed

This seems like something straight out of viewfinder, what a cool idea!

This is SOOOOO cool man I love the idea Edit: can't wait to glitch myself out of bounds, take a screenshot and wreak havoc on everyone I send it to

@kylebanks

6 ай бұрын

Ahah yea you could definitely do some griefing with this 😅

This guy has INSANE determination. Continue with the good work!!!

Really cool concept! Happy new year! Cannot wait to see what happens with Farewell North in 2024😊

@kylebanks

6 ай бұрын

Happy new year to you as well!

There is a small sandbox game called trailmakers that saves blueprints (basically anything you build in the game) as images with steganography, they have a transparent background, but the rgb values change depending on the data, giving a whopping 24 bits per pixel (i know this wouldn't really work in your case, unless you add a transparent stripe, just some fun fact)

I love this concept. Will steal when I have a suitable scenario for it to make sense

The thing with the QR code is that it makes it obvious to players that there’s information stored in the image, so if you wanted to make a postcard feature that allowed players to drop into a specific location then it’s probably the best choice.

If you're still thinking about this as a cool idea, I've personally done this myself by adding new compressed text chunks to the PNG. You can even give a name to each chunk if there's extra metadata you want to keep. As far as I know, it doesn't work if you upload it somewhere that fully reencodes the image data, or one that converts it to another format. There are some other weird PNG examples across Twitter so it may work there actually? I've only tested this over Discord, but it works a treat. They don't strip the tEXt or zTXt chunks when they do their processing, so all the data stays intact.

Loved the Stenographic encoding section! To address your original issue: why not use a concatenated file (like a tarball)? 1. In a SaveWriter method, construct a file stream for your serialized gamesave data, and another with your image data, each in their native formats 2. Concatenate the two file streams together in a well-known order (ie, either save-first or image-first), along with a final (or leading) integer size of the first stream 3. Save it as a ".sav" or something cool 3. Write a reader lib to deconstruct it by reading the file into memory, splitting the size-number, splitting the two original files at the byte location indicated by the size-number, and then loading each file part as needed (the image for menu display, and the serialized gamedata only when a load is initiated). You now have a proprietary, but useful, savefile format that, if written competently, needs only a single IO operation per save/load and should not add a significant amount of compute as all you're doing is some basic in-memory manipulations of binary streams you were going to create anyway. This approach technically requires you to do unnecessary IO by loading gamedata into memory along with the images, but with 12KB per gamedata block, this is negligible compared to the image previews you'll be displaying in the load menu. Of course this doesn't allow you to pass a viewable screenshot around and use that as the savefile, but I don't see why that's really a requirement versus a curiosity!

Fascinating rabbit hole! The idea of stenanography and least significant bit is really cool.

This would be fun for Minecraft or other Games which have procedural generation. You could store the following data in a screenshot: - Seed - Position - view direction & angle The data could be hidden in the stamp found on the letter.

I always wondered how this worked. Thanks for sharing.

Fun fact but smut games by developer illusion has a system of making png "cards" that can include custom character data because that is what their lates games are about. There even is a small screenshot that can include entire custom scene data for supplemental studio included with the game because that is what their latest games are about. The only thing is that no... "Main save" is stored with PNG's but "sharing your creations with others" and also being compatible with popular image boards have been a boon. No idea why it has to be a Japanese smut game that ships with most robust if not only version of this feature but hey.

As someone who also makes systems that are far too complicated than they need to be simply because they seem fun I relate

I wonder if you made the image grayscale - that way you can use an entire channel of the texture, for example red, as long as it averages with green and blue to form the same shade of gray. That way you could hide the QR code in just the red channel without it actually appearing in the final grayscale image. It wouldn't look as pretty, but it might work.

@gamechannel1271

6 ай бұрын

Grayscale doesn't work that way. All pixels need to be the same value to make gray. This means whatever value you are storing in the red channel is also the darkness of an individual pixel.

@drewviesta4878

6 ай бұрын

@gamechannel1271 yeah I guess it would only work if you do like a grayscale post processing effect in game, but then you couldn't share it. Nvm i guess

@azai.mp4

6 ай бұрын

This does actually remind me that you can mix a red QR code, a green QR code and a blue QR code together to get 3 times as much data in the same amount of pixels. Every pixel does have 8 different possible colors that way instead of 2, which doesn't necessarily look very nice. Though you could map them to any palette you like and as long as you know what palette you used you can get the original colors back and decode the message. So there is artistic freedom there.

@Waitwhat469

5 ай бұрын

@@azai.mp4 That might be a neat combination of techniques. Could have it analyse the screen shot generate some varieties of multi color QR codes so that they blend in with the image as much as possible.

This was a joy to watch.

you could use qr codes but change their design. like you could have them as a frame so they wont be as noticeable or make them almost transparent and make the game do some photo adjustment so it can read it

Haven't seen this yet but I'm really amused by how you came to this idea independently. The Anex emulator for PC-98 games saves its states in bitmap images, so this is in fact a thing!

I saw a video a while ago about people who developed a way to embed meta data in aesthetic designs like QR codes. The designs had flowery/swoopy designs (and I think penguins too, it was weird.) I imagine when you press pause a unique opaque design of a complementary color design of gameplay encoded in this way vignettes the game. You get a prominent continue option in the upper third center. Then options, achievements, collectables, exit game, etc. in the lower third framing up your player at center center. The data can be encoded in 4+ of these designer qr code variants, including velocity, direction, collectables, missions collected, missions completed, etc.

The game MidBoss does something similar with their "Death Cards" and those seem to survive social media sharing: "Share your crushing defeats with your friends through the use of death cards, images which contain embedded data which lets players load the card in their game, inspect your run's stats, and replay it using the same seed and settings."

I'd probably just append the raw JSON data to the end of the PNG (which seems to be similar to how Trailmakers stores its vehicles, they are also stored in their preview PNG). This makes it impossible to share on social media, but it extremely easy.

Fantastic video! The least significant bit demo is super interesting! Though the dataset at 7:33 really needs more variation, like noclipping around the map and turning the camera to cover all angles of the playable space. Using correlated frames from a video will overfit since the goal is classifying random screenshots.

with 2d pixel art games, creating an algorithm to map pixels to areas in the game sounds like a fun challenge and even with additional metadata like an inventory system/items/upgrades/etc, if its displayed directly on the screen in the hud, you can also specifically check and match those pixels

Oh yes, I remember TerraTech did this to save the techs you build, I was really impressed at the time.

This is a great exploration into the feasibility of these different methods. I’m surprised you didn’t end up implementing any of them at the end, but your reasoning made sense! I would have liked a little more in-depth explanation of potential file i/o issues.

*proceed to download a picture of the endscreen of the game and called it a **0:00** speedrun*

Trail makers does this to save its creations! Was very surprised and confused when I first tried to share a design with a file (didn't want to put it on the workshop, just wanted to show a friend). So yeah, saving data in images is a really effective way of saving data for "plug-ins" style loading like Spore creations (as someone else pointed out in the comments), trail makers, and probably a ton more games. Wouldn't be surprised if Besiege and Scrap Mechanic used a similar manner of saving and loading creations.

Kyle Banks: Tries to store the save file in an image using various techniques. Fans: "Release Farewell North already."

Honestly, I thought you were going to say the stamp franking was a transformed QR code 😄 cool work!

@kylebanks

6 ай бұрын

That would have been pretty neat :D

A cool idea, and definitely room for other options too! I would've encoded your QR code inside the LSB stenography technique. That should achieve the best of all worlds, and avoid the wasted compute of a ML network.

I'd highly recommend anyone that found this interesting to check out how the game Monaco used steganography as well. They hid their level data in the actual side view level preview. So the preview of any level actually contained the entire level itself lol

Honestly, I think something similar to a QR code but not quite exactly a QR code would work - perhaps encoded in the stamp specifically. If you imagine a stamp pattern with a certain number of design elements, then by altering each of those elements you can encode data. QR codes are designed the way they are (large blocks, black and white) because the data needs to be readable when photographed, even with difficult lighting or an unusual angle, but since the main destructive process these postcards are likely going through is just compression you can use far more detailed encoding and still have it be readable - for instance, as well as encoding by having elements be present/absent, there's also the potential to alter colour, location, and angle, such as by having a franking mark that is positioned differently relative to the stamp. Very interesting thought!

This is a really good idea!

Good job with your game, I found you around the time of your firsts Dev Logs, i'd liked so much your game back then (even now tbh) that I even shared it with one of my friends (who has done the same with another group). Keep up with the project. A note though (maybe i've lost some devlog), why aren't you using a Binary Serializer for your save data? The problem with json is that it need both field name and parsing, Binary serialization, uses a struct (this a few years ago, i don't know if they have added support for classes) as a reference for type and a termination byte for splitting field reducing your save even more! For the image it's better to save everything in one file (both the data and the image). Then you can apply 2 level of optimization: - First you apply the jpeg (or png) compression algorithm, then, you use an array to remap from 0 to n every single color that are in the images (once) and you save both the remapping array and the image (with the color index) in the save file. - Second train an AI to upscale those images so that you can save them at 240p in the save file and save space. Another option, that i'm really not fond of, is to save both file inside a zip file with a custom extension. Hope this ideas can be helpful!

All things considered, even if you didn't end up using this, you still managed to compress your save files by a wide margin

could overlay multiple of these systems for fallbacks 1: optional visible data stamp 2: say in the file name 3: say in the metadata 4: at like 1/4 resolution, bake least significant bit stuff

I have an additional suggestion. A decorative border or decorative corners on the screenshot that has the data embedded. Also, thank you for the validation about the lack of helpfulness of the multiplication table.

Thank you for not really using it in the game my imposter syndrome will kill me without the last part of your video. Really nice proof of concept too.

There is another possiblity, that would work for very linear story games , that is making every "level" or "key part" from a different range in terms of contrast and collor pallete so when "reading" an screenshot that hasn't been altered by a photo editor the game will calculate the average of contrast, brightness and amount of color of each channel and then place the character in the point of the timeline from the game that matches that combination or the closest aproximation to it. obviously what you wanted is to store data inside an image instead of just make the game "know" where that image came from in terms of simple color information, but that was just another approach.

great video, taught me a lot, especially in the first 3 parts!

Love this experimentation. 2 thoughts that come to mind: 1- The QR code could be stamped on the image without it being just black and white. That wouldn't be as distracting, maybe. 2- you could add the metadata as hud in the image. Similar to BOTW and how it adds the divine beasts, shrines, hearts, etc. to the loading screen. Or Celeste, where at the end of the level you get a nice painting and it contained your completion time.

I guess this would be specially neat on a procedural game, it could encode the world seed (if the game generates multiple worlds) and the coordinates, either as a way of getting a spawn location, or as some kind of post card system "hey look at this place I've found, if you import the image, you can get directions to there" Also, a technique that could be also done: having a grid of squares containing the data data applying very slight shifts to the image below (like overlaying the 'data grid image' on the 'image image' with the 'data image' being at 10% opacity and using dodge or burn blending modes), in a 1080x720p image you can have a bit over 12k squares of 8x8 pixels (quite a few of those could be use for error correction) The game loading then then scans for a pattern of slight colour variations of the image. You can also have some heuristics for optimizing storage and error prevention

I wish I had this video when I was taking steganography in college. I was able to understand the LSB algorithm much more easily with the way you presented the information; this is compared to a professor who only used "Unregistered HyperCam 2" videos he recorded years ago, and he continues to recycle the content suggesting it is "prime learning material." A teacher enshrouded by their own ego and knowledge, unable to teach. The teacher makes all the difference! Thank you for sharing this, your passions, and innovative ideas. I hope others can become inspired and educated through your journey! It's never too late to learn something (-:

I enjoy pushing things to their limits, just like Kaze recoding sm64 or Gamehut talking about anything from the genesis and snes era. Maybe you could do a bit of a cross between qr code-like icons and machine learning. Level specific icons that don't ruin the screenshot that can quickly be picked up by the load system.

The TEC RedShift from EXAPUNKS uses low resolution images for software distribution EDIT: for clarity, the images aren't QR codes or raw data, they look more like a mini disc with SD card pins

Damn, this would be an amazing feature for the game Viewfinder, which is all about bringing pictures into reality.

If I recall correctly in the game "Spore" you could create various creatures and share them with friends by sharing a png screenshot of that creature

You could also essentially encode the qr code as average changes across chunks of the image.

The process(es) you went through were definitely interesting, but I think what you really need for 100% "drop/open" fidelity is like, a built in "glyph" system or something, where you effectively use some kind of more advanced data storing system to imprint on the image the exact coordinates/time/etc. data you need to land right at that point. If you boiled the save files down to lookup tables consisting of all the different variables, then you could likely come up with a unique way of communicating that data cleanly in an image without compromising the total aesthetic. Just a little "stamp" on the bottom, similar to a QR code, but using your own custom language. ;)

It sounds like you need to essentially one-hot-encode the save file data in its most dense representation. You have millions of pixels, each with the ability to encode 4 bits of data, and each unique save file only has hundreds or thousands of unique variables. So you encode properties of the save file like “Player has item X”, “Player has item Y”, “Player is at checkpoint Z”. If you write down enough if statements and encode them as pixels, you can probably restore any player’s state. This method should also be thousands of times more data-dense than the compressed save files that you showed us.

I love the steganography idea, and am sad you aren't using any of these, but oh well, maybe you'll use it in a future game of yours :D

This feels like it would work for a game like Viewfinder, where the whole game revolves around taking photos of stuff

you can actually append a zip archive to an image, basically hiding it after the image data. This works because zip archives are read back to front, while most common image formats are read front to back (PNG, JPG and GIF). Also GIF gives me an idea, just have the QR code as the second image in an animated GIF, where the second image will never be shown.

You could add in your own version of a QR code - something diagetic like a picture frame around the image or a mail stamp. Something that would fit with your game's setting. That way the encoded data could look like a part of the photo even though it has a purpose outside the game world.

i like the qr code solution as anyone could scan it without a load barrier (assuming you encode it as url). as for the look and feel - you can get away with other colors and even distortions etc

Kingdom Hearts does/has done this. It’s a great fun way to store saves and I love that you figured out a method of doing so for other people to use and learn

my first thought on how to do this would be appending a single layer of pixels around the edge of the picture. then either each pixel could represent a collectable or something. or the pixels could contain the data for the save when converted back into text

You've got 4 channels. Blow up your QR code to take up the whole image, but only change the lowest N bits, where N is determined experimentally. You can split your 2kb up into 3 or 4 chunks, depending on if you want to use alpha or not (I can't with the tools I could use, which is why I'm thinking about it, but maybe contrast change is more noticeable than color change?) The idea is, most image compression preserves low frequency changes, so if you can spread your change to the image over a bunch of pixels in a low frequency way, you can sample them and common compression algorithms will preserve your information. The smaller each QR code is, the fewer pixels it needs, the bigger you can spread the change over your image, the lower frequency it is, the more compression it can withstand. And even literal down scaling. But, you'd perhaps need more than the literal least significant bit, so you'd want to test to see how many you can change before it becomes too obvious to the eye, and then see if that survives any meaningful compression. I... realize I could run the first half of this experiment in a limited sense with tools I already have. I could take an arbitrary 3 unique QR codes of a a few different reference sizes, decompose an image of some standard resolution into color channels, alter them in increasingly large increments, put them back together again, see how much editing it takes before I can see it. But I don't have a way of getting the image back out without sitting down and writing software, which I could do, but really don't feel like right now. Well, free rabbit hole to anyone reading youtube comments. I might try it myself later if I work up the motivation.

Mind blowing stuff. Just a thought, “postcards” in reality are actually 2-sided; one side has the image and the reverse contains all the postal info and personal notes. In this case it could mean two images (one for the back which has the metadata, and one for the front which has the image), but it could also lend to the effect of being a postcard as opposed to just an image. Such a good video. Definitely have to rewatch that and take notes. Lol

If you add a save server, you can store the bulk of the save data on the server and just use some form of steganography to hide the file's UUID. Having much smaller data line that would give you much more leeway for the error correction space.

cool as hell idea!

Tbh if you run an ai to figure out where the player is in the world/story, you can just as easily build your game so every ability unlock, every item you acquire, every collectible or secret found is physically on your character in some shape or form, as it will be presumably present on the screenshot. Maybe a tattoo, or a badge, or something similar. Then you train the ai to look at the character too, and from the badges figure out what is unlocked, what is not.

My proposal: hide the data in the grass/trees/clouds/lens flares. These can be modified without drawing attention because they already look random. 1. generate a HUGE qr code with tons of redundency that spans the full image (newer QR standards allow for arbitrary sizes) 2. define some types of "focus points" that can be found by a neural network (grass tips, tree branch splits/ends, cloud vertices, bumps on a stone...) 3. Align all these points to either "even" or "odd" columns in screenspace, depending on the QR code. Colums should be a few pixels wide. This can be achieved by changing the geometry and rendering a new frame just for the screenshot. Train a model for each of the types of focus points to extract them again. As long as the screenshot contains at least a few thousand of these points the data should be recoverable. Bonus: double the data by also aligning points vertically.

It blows my mind that this is even possible.

you might be able to create something like a qr code but have it look nicer by having it look like a stamp, you could have the main image of the stamp signify the checkpoint you are at and have small adjustments on the stamp (i.e. a different cost of the stamp, tears in different corners, additional scenery in the stamp image, angle of the stamp) that can be used to store the collectables and other data

This is amazing.

TBH I wouldn't worry about the re-compressed images losing their save data, and just pop a message "Oh no, it looks like this picture didn't make it through the intertubes. Maybe try again with the original screenshot?"

First thing I thought of was hiding the data in the Fourier transform of the image, which I think could resolve many of these issues. The Fast Fourier Transform algorithm represents the color values of an image as sine waves of a range of frequencies that can be added together to produce those color values. Data hidden in the outermost (highest-frequency) pixels of a Fourier transform would produce changes unnoticeable to the human eye. This still carries the issue of image compression destroying the data, as most compression algorithms involve mangling the outermost pixels in the Fourier transform for that very reason. But by moving the hidden data further inward, you could fine-tune the trade-off between compression resistance and the degree to which it noticeably changes the image. That plus some amount of error correction could probably solve this problem.

Very interesting feature!

Very cool idea! Seeing all of the other suggestions in this comment section, I though I'd put my own hat in the ring. Perhaps you could reduce the number of bits needed to represent a save state by interpreting the bits directly instead of going through ascii and json. For example, you could have the first four bits represent the checkpoint the player is on or have some bit representing whether the player has gotten a certain achievement. A more error-resilient method of encoding the data into the postcard might be to actually change the underlying screenshot while making it believable to the user. For example, you could have tears on the border of the image encode information based on their length or shape. The only hard part would be detecting whether a tear is in one state or another.