Geez... I thought I knew disk formats, having designed the hardware and written low-level device drivers for two (very) proprietary hard and floppy disk controllers in the 1980s, but this stuff just made my brain explode... Just brilliant, and huge respect for the level of research that you've put into understanding this subject and using these tools to their fullest capabilities.

Also for anyone like me who does disk imaging, make sure you create flux images with multiple revolutions - preferably at least 5 revolutions. All the major flux imaging tools and hardware support this and more than 1 revolution is necessary to fully capture many types of copy protection. It makes the flux images bigger but more flux information is always better than not enough. For GreaseWeazle the --revs=5 parameter is all you need to ensure 5 revolutions are captured. Kryoflux captures 5 revolutions by default. I believe SuperCard Pro captures 2 revolutions by default but can be easily increased in the SCP software.

@markusfritze

2 ай бұрын

Yes, absolutely! I sold a copy protection scheme for the Atari ST in the 80s which had a sector which started at the very end of the track and went well over the index mark. It could be read just fine, because the FDC ignored the index mark during a sector read. If you stop reading right at the index mark, aligning the beginning of the track with the end is probably not possible.

@skilletpan5674

2 ай бұрын

@@markusfritze Some PCs back then had great control of the hardware like the APPLE ][ and the C64. The IBM also had fairly good control. Thanks for the info about Flux. I only used to use some software like Anadisk/Teledisk for IBM,locksmith (v6 is the best) for apple and some copy protection/disk copying manuals for Apple ][ that talked about track arching etc. You can still find good manuals online in archives like Asimov that discuss all of the protections of the day right upto the 1990s.

You want me to forget everything I know about how floppies store data... I'm way ahead of you, I forgot that over 20 years ago :)

@erichollar5503

2 ай бұрын

I never knew anything to begin with. 😁

Realistically I'm personally never going to have any use for this information but your presentation just makes it so fascinating I can't help but watch! Thanks for the video and esoteric knowledge!

Our university library is working on creating a digital preservation workstation, focused on older media at the institution first, but there's probably no restrictions on what they'd work on. This video was shared with them as it was the perfect length and detail for what they need as an introduction to working on floppies. A lot of the media that needs to be worked on is pre 2000. Seems the important flash drives and zip disks mostly got copied to network storage, but when storage was more at a premium, floppies archived a lot of stuff. Thanks for sharing this!

Disk formats are something that I watch from afar these days. Just not enough time for my regular systems job to work with them myself. Retirement is only few years away. Maybe I will come back to it.

Having had to work all this out for myself to preserve an extremely rare HH Tiger machine, I found this video is greatly reassuring and wished it had been around more than a year ago! Being able to rescue and even create new floppies for my Victor 9000 with its ZBR arrangement was a godsend!

Another use case for raw backups of floppy drives, is to recover discs that are very hard to get, by using several drives. I did it with some bad amiga discs. Thanks to the "guided raw" option of the kryoflux, you can dump, and check which track is good, and which one is bad. As every track is backup as a file, and different floppy drives, get different recovery success you can overwrite the successful tracks files on top of the bad ones, to get a perfect raw backup of the disc. and then, do a successful binary conversion.

@networkg

2 ай бұрын

Love my Kryoflux. So far I have had no problems using Kryoflux to achieve everything from archival quality flux copies to just converting the disks to use with an emulator. Of course, I use it just for Commodore (and maybe a Nabu if I can get a disk drive.)

And now I understand why PC floppy drives cannot read 800k mac disks! Thank you so much for this! You're very good presenter ❤

Excellent video as always! I would add though there are formats out there where not even sector header information is valid which can be a problem with disk imaging software. The original Quantel Paintbox (DPB-7001) used 8" DSDD disks, the formatting essentially is one sector per track. When writing a track the machine looks for the physical index pulse, then writes a few 0x00 then 0x53 ('S' for sector) and then 19200 bytes of data payload and that is literally it... no sector IDs, no checksums literally nothing else. In later years they did add more checks into it, checksums etc but it's a big problem imaging and converting as you are stuck at working at the flux level.

We can add hard sector floppies to the to the mix. The North Star floppy disk system used ten sector disks with each sector marked with a hole marking each sector. A bit more than that, but that is the gist of it. They started out with 90k floppies, then 180k. but that was ok, as the system, at the time only had 24k of RAM. The controller was an interesting board, 1k of read only memory. I have a note on how that worked if anyone is interested.

I used to fiddle with sectors on disks back in the good old days on my C64 and Amiga. It was great fun and something I miss about old computers and tech gone by. This was a refreshing insight into the things I didn't get when I was learning about raw copies and sector meddling to copy those protected floppies. :) Man, I'm old.

@tonylipford7310

2 ай бұрын

I recall that the hardware on our Amiga 1000 could leverage 82 tracks!

Your explanation of this and the laserdisc archiving in previous videos are really excellent. You have a real talent for making this highly technical stuff understandable, and that's a valuable skill.

I have a bunch of Floppy disks in "T-Format" formatting, which worked perfectly in DOS and Win9x, but dead in the water in NT.

@coffee115

2 ай бұрын

T-Format claimed 1.72 MB max capacity, with DriveSpace this could give you 5+ MB of capacity if you used basic Text documents.

Wow! This is super timely with my own backup project underway. I cannot believe the massive depth this actually goes into.

I lived through this time of computing, admittedly as a dead end user. I had no idea about any of these complexities. Thank you. This was amazing.

I've got a TRS80 M1 giving me fits on the disk system. This video helped to solidify my understanding. I've been waiting for my GreaseWeazel to arrive from the other side of the pond. Thanks for making this video.

What a fantastic video. This will be so useful for generations to come. Make sure you backup the flux map of this video!

This is epic, what a community contribution both in explanation and projects!

Fantastic review of use of disk preservation systems. Thanks!

I first started using floppies in (I think) 1986. And yet, you've taught me some things. Well done!

18:56 YES! jaut-76, this is also exactly what we see on your 8" SWTPC disks that I imaged for you, so take note! Shelby knocks this explanation out of the park...way better than I was able to articulate!

@jaut-76

2 ай бұрын

Indeed he does. Hopefully this means we can have the disks available and working for the masses

Shelby, I LOVE this topic...excellent video, thanks so much for creating this! This video is the BEST EXPLANATION I've seen on KZread YET on how flux encoding works on floppy disks...superbly done!!

Great guide that I've come back to multiple times. Thanks!

Props to YT for recommending your video ! I pretty much only work with TRS-80 disks (I have 2 M1s, 4 M4s and a 4P), and use my greaseweazle to image to .scp files for the HxC software to decode for me. That dummy disk image view has been great for dealing with all the different kinds of disks I see. I've got pretty much the same drive setup, but keep the ds40 drive on a shelf for rare just-in-case use. I might copy your triple-drive setup, though. It looks pretty handy. And now it's time to go scroll through your entire videos list.

It's vanishingly unlikely I will ever be doing any of this myself, but this info and the way you discuss it is absolutely freakin' FASCINATING, and I am extremely grateful to those who are preserving the data on these disks for future generations. Also "Greaseweazle" is a freakin' awesome name for a piece of tech. ;)

I finally bought a Greaseweazel about a month ago after hearing so much about them from you and other channels, and I'm blown away with how incredibly easy to use and affordable it was. Absolute life-saver for somebody like me who collects all types of systems, but can't afford to seek out and buy original software. Now if only I had one when I still had my Northstar Advantage...only reason I sold it was because I had no software! Great video as usual.

Thank you for this! I grew up at the tail end of floppy and have been struggling some trying to get into it.

Wow! Amazing explanation of how this all works! Now,I finally try some myself. Thanks for sharing

This was a great video! I have a really hard time understanding computers because I usually do analog electronics and mechanics. Starting with the low level stuff really helps me understand!

I found a box of old floppies, but didnt know a thing about reading them! Thanks for this perfect video!!

Man, do i feel stupid now. I had no idea it was so complex. Thanks for the video!!

My linux brain lit up at the kde oxygen window decoration + nimbus GTK theme :D

@MarekKnapek

2 ай бұрын

I was like: Is this Java with Nimbus look and feel?

TY for the video it finally made me buy a Greaseweazle i was thinking about buying something alike for years. Also i like the style a lot, it feels like there are no "things you should've just known" left.

great video! As a suggestion, I would love a guide like this for CDs and DVDs, specially with dealing with DRM. I made an effort to make images of all my games, but I'm not sure if I did a good job, some times my imaging would fail and I resorted to manually the files and creating a new ISO.

The amount of knowledge you have is simply astounding. I'm always amazed and I SO like to listen to you, although majority of this stuff will be of no use to me. Never the less, thank you for taking the time to explain!

Love these explainer videos. I don't own any retro computers but I enjoy watching them.

I bought a Catweasel 20 years ago to archive my old Commodore and Amiga disks. Nice to see contemporary tools to work with these images.

thankyou 1000x for making this intro video to this topic.

Great video, thanks for sharing your scripts. I wouldn't have been able to archive the Zobex boot disks without your help.

I didn't understand a single word, and yet I loved every second. Shelby rules!

This video was awesome. Helped me a ton to better understand disk formatting. In the process of trying to write downloaded images to physical floppies for multiple vintage machines, but mostly for Kaypro and TRS-80; Apple is easily taken care of with ADT Pro. Thanks for the education! 😃

You're going to want to make a part 2 for this discussing the Copy 2 PC Option board, and further discussion of various copy protection schemes like weakly recorded bits, intentional bad sectors, and how the exact floppy controller might be required in order to read copy protected discs (preventing some clone and 3rd party controllers from working). Also you kind of omitted the fact that floppy discs are analog, and their controller is basically a software controlled mechanism. Oh, and the hacks we used to use to make 360KB discs hold 800KB. And an expansion on the 'booter' programs. Which correlates with why off-the-shelf USB floppy drives are a bad choice for archival purposes. The floppy rabbit hole goes pretty deep.

@elphive42

2 ай бұрын

For some reason, the “floppy rabbit hole” made me picture those portable holes from Toontown.

@penfold7800

2 ай бұрын

Yeah, there's an endless probability of dark abyss scenarios created by the evolution and de-volution of data storage

@davidwillmore

2 ай бұрын

Tell me you didn't watch the video without telling me you didn't watch the video.

@hicknopunk

2 ай бұрын

​@@davidwillmoreKZread has video now?? 🤔

@blakecasimir

2 ай бұрын

I would very much like to see all this covered in a follow up video. Fascinating stuff!

Thanks for presenting. I remember Copy to PC and tools of such and made a BIN file of a Floppy and image of Zork (way back when). I still have some of the old floppies around.

Great video. Your video on this topic made it to the front page of Hacker News. Good job

Thank you Shelby!!! Great content

Wow great explanation! will have to watch again for it to sink in, lol. Does clear up alot for me though.

Brings back memories looking at all your software. I feel like I'm at Fry's Electronics or CompUSA, those were cool times. 👍 👍

Interesting video. I do agree that this is the best form of preservation. Kudos on this and keep up the good work.

Sector interleaving is quite interesting as it shows how slow the CPUs at that time were...

There was also M2FM format disks mainly used on some 8 inch drive systems. It was a hybrid FM MFM system

6:35 I've used this exact feature if HxC for YEARS...and I still love it! Use it all the time!

It could be great if someone could simplify the command line interface by adding step by step screens that ask questions to the user, providing guidance as to what's being asked. This idea has already simplified many workflows like jailbreaking consoles, upgrading node modules, etc.

@TechTangents

2 ай бұрын

If it helps, I added some rough documentation of reading, converting, and writing to my written side of this to see some basic examples of the commands: wiki.techtangents.net/wiki/Floppy_Disk_Imaging#Format_Structure_and_Converting

Having listened to more than a few Jason Scott talks it's really nice to get a technical intro into this subject. I'm not worried about the magnetic artifacts though. The amount of data/media that has slipped through our fingers in this digital age is way scarier than getting a proper flux track of some 80s floppy. A few days ago the Yuzu repo imploded into a logic cloud. Git has the awesome characteristic of every clone being the full source of truth yet there's so much more to it (and with naive googling I haven't found a mirror so far,, maybe I just to wait till the smoke clears).

I see that copy of Nox in the back. Still one of my favorite games!

I would love to see a detailed video on preserving Apple2 disks using an IBM 1.2MB drive, since finding 360K drives is a nightmare.

@TechTangents

2 ай бұрын

If you're using a greaseweazle just use ` --tracks="step=2" ` to tell the software to double step the drive which will make it read every other track. That's really all there is to getting flux off of the disks with a 1.2MB drive.

Great video dude, well done !

man, it feels like i've been watching Druaga1 and AkBKukU forever now

Thanks for the guidance on HP Series 200 disks. I have a few from A Long Time Ago -- it'll be interesting to see if they're still readable.

I love your videos, can you please do a video on using the domesday duplicator for VHS archival.

I bought a Greaseweazle device a year or two back. I had full intention of backing up all my old PC floppies, BBC Micro floppies, Atari ST ones, and finally my Acorn Archimedes discs before they suffer any bit rot. I can do most of them, except the PC 5.25" ones as I can't find my old 5.25" drive. I really thought it was going to be in storage in my attic, but I emptied most of that recently to do some home renovation and can't find it. For now I'm not prepared to pay the silly prices that the PC 5.25" drives are listed for, so I'll just have to wait. One will cross my path eventually.

When you talk about DRM, I remember when you had to install Lotus-123, you had to set the clock on the computer to 8Mhz from 12MHz, for their copy protection to work.

The analogy I use to explain the difference between informants and formatted capacity (2MB as opposed to 1.44MB) is this. Imagine you are going to use a garage wall for storage. The wall is ten feet high by twenty feet across. 10x20 gives 200 square feet of storage. Except, it doesn’t. By the time you cut the two by fours to size and arrange them to support the shelving and cupboards, you end up with about 144 square feet of storage. Floppy formatting is similar. Some of that magnetic real estate has to be used to set up the areas where data will be stored so that it doesn’t spill over into other data. The file allocation table and directory structure needs somewhere to sit and the physical limitations of the floppy drive head limits cannot reach the edges of the disk. For a while, the 2.88MB floppy disk was available. It relied on a floppy drive that could take shorter steps so that it had twice as many steps as a 1.44MB drive. 1.44MB floppy disks could be read in the 2.88MB drive but only 2.88MB disks would provide reliable storage due to the method to record the data and the hysteresis used to ‘fix’ the data. It was a step up from the same technology that allowed single sided, single density disks to be read in new double sided, single density drives. The 2.88MB format was superseded by the availability of ZIP and JAZ drives which provided 100MB of storage and had more in common with hard drive technology.

AMAZING!

The analogy I like to describe why you would want a kryoflux image as opposed to a floppy image file is like the difference between a high resolution scan of some archival newspaper article as opposed to a text file with the text contents of that article in that newspaper.

18:40 that computer look so cool!

Great video, I have a few old 286 XT era shareware games on disk I would like to archive, this explains a lot that I was having issues with. thanks 👍

My first and favorite archival of game install discs would be for Tribes Vengeance, which used regular DVD's or the like... quite literally was worried due to the few scratches and many years since I played a cracked version, and simply copied the discs to an ISO which I then installed the game from. Not sure how much I went into getting the game to run though, as it required the first disc inserted to play, and almost immediately looked for a patch/workaround which I found. Was nice to get it working, and I still have the original discs in their rather readable state.

This guys goatee and hair go hard I respect it

this is cool... well done...

You are definitely the OG for this stuff on KZread. Adrian Black is the OG for repairing old computers, down to board level components. Also CRTs. And the 8-Bit Guy is the OG for cleaning up and retro-brighting old systems.

@XenotypalTV

2 ай бұрын

adrian is legit, i envy his abilities

@tschuuuls486

2 ай бұрын

The 8bit guy used to restore Clamshell G3 iBooks back in the day and nowadays writes software in assembly for vintage platforms.

@TheErador

2 ай бұрын

Thought you were gonna say that 8bg is the OG for blowing up systems lol

@XenotypalTV

2 ай бұрын

@@TheEradorsame when I first read it.

Fascinating, but… I don't have a PC. I'm looking forward to requiring my Atari Home Computer Systems that were taken after I moved to California thirty years ago. The photo I saw showed at least one 1050 floppy drive and a couple of cases full of floppies. Originally, Atari DOS 2.0S formatted 5 1/4" floppies in single density. Then Atari DOS 3 came out and offered an "enhanced" density format. I don't remember now if it provided more sectors per disk or just stored more data per sector, but the result was about a 50% increase in data storage per disk. I added a bootleg of a third-party ROM to my floppy drive to provide true double density. If these old computers actually still work, I plan to see if I can read these disks and copy the files to an SD card using modern accessories for my old computers. I think I had some disk editor software back in the day and I have a detailed source listing of at least the original DOS 2.0S software.

Also may wish to cover Write-PreComp at track 43 on some 8" Floppy Drives. Ref the Centurion minicomputer from the 1970-1980's.

Gotta love that name: "Greaseweazle" 😀

@thromboid

2 ай бұрын

Maybe alluding to the old Catweasel floppy controller (and the even older TV show Catweazle)?

Should do an episode like this on zip disks at some point, I have a few that decided to stop working because I think the tracking sectors have gone bad. And of course they're the ones with data on them that's not backed up which I was trying to do when I found out.

@nickwallette6201

2 ай бұрын

I don’t think this is possible without hacking a drive to replace the onboard controller. You don’t get flux-level access through IDE or SCSI.

The OSI 610-Floppy controller board was one truly weird beast. It basically just sent the output of 6850 serial uart to the disk heads (FM?) It didn't do sectors as far as I know, You basically just read "whole tracks" ... But given that a whole disk image (40 tracks) was a mere ~60k that was fine Loading a program was basically just reading in a track to memory :)

I wonder if there are any flux readers/writers for mitsumi quickdisc (QD) drives, as found on some early akai and roland samplers, a few sequencers, and word-processing typewriters... also used on the famicom disk unit.

Fun stuff. I was hoping you were going to get into things like the old Apple ][ quarter tracks and things like that. Maybe a future vid.

Cylinders are 3D tracks, basically multiple stacked tracks. That's why we talk about cylinders for hard drives and tracks for floppies.

That Mac software on an MFM disk is interesting! I think it won't run on anything older than a IIx, since the Integrated Woz Machine used up to that point only handled GCR and not MFM.

11:22 So, since I always load my raw flux image file into HxC to view how good the read was, I also always use HxC to export known formats, by simply clicking the Export button and choosing the IMG file (.img) RAW Sector file format, which should accomplish the same thing you've demonstrated here. Aha, I see you also show that at 14:15, thank you! But, could you speak to why going to the extra step of using the GW command line to do this is beneficial? 14:24... Ah, this works in HxC "ONLY under ideal circumstances, though". Excellent that you explain why. Thank you! I also thought that HxC could allow you to specify the same parameters on export, thus fixing those strange circumstances you show, but apparently not? I should experiment with that more and report back. THANK YOU for showing how this is done in GW...I've certainly learned something, and I've also been working with strange format floppies for years...again, excellent content. I love your channel!

Very interesting, especially since I have some Apple2 disks to save. What drives do you recomand to do so? (If you have a tool compatible with original Apple2 drives to do so, I have said drives.)

Hey can you do a video on RLL/MFM ST506 hard disk and formatting? Is Interleave really needed on SSD replacement systems… just curious about any caveats you might know about.

I imagine that a reason you don't see that copy protection on modern software is that not a lot of people would want to reboot their computer just to run one game. Also with that weird track layout I can imagine there may have been a few drives or floppy controllers that could not recognize it. The last reason you don't see it is probably cost. I know other computers had floppy copy protection too. I know there is a popular DOS game for the PC that is protected. I knew someone who tried to copy the files from the disk to one of his own. All that happened was that his copy didn't run. It was unusual because it was something we didn't see a lot. I just know he never had any luck,. That game was Lemmings. It has some catchy music and some levels that seem to be just about impossible to me.

Awesome video. Greaseweazle and HxC can be less than intuitive to someone new to flux reading.

I was going to say there were some weirdly formatted disks for copy protection. I think some early Mac programs and some Apple II programs used it. Also, what about the IBM 2.88 MB disks?

Are any of these able to identify and/or write half/fuzzy/weak bits yet? FYI half/fuzzy/weak bits are when certain bits are purposefully written incorrectly so that the flux is halfway between a 0 and a 1. This makes it so that about half of the time it will be read as a 0 or a 1. The copy protection keeps track of those bits and if it sees that it’s been read as a 0 or 1 multiple times in a row (instead of alternating) then it triggers copy protection. It’s normally been impossible to deal with as most software can’t identify the half bits in the flux stream and if you don’t know where they belong you can’t even attempt to write the halfbits. I have quite a few X68000 games that I really want to back up for preservation but the half bytes make the images unusable without someone cracking the game which I don’t consider to be a true real backup.

“I want you to forget everything” *half-life death noise*

What about the old Apple II copy protection format Spiradisc?

I remember interleaving, and being stoked to finally have a PC that was fast enough to not need it anymore...

Much more information.

22:16 Microsoft Adventure - Copy protection example...cool! I wonder if the flux of this has been uploaded to archive? I should check... I'd kinda like to play with that... 23:19...Unless you want a REALLY big challenge.....YES...I DO, thank you!!!

I just have one question: Where do I get raw dumps of Kings Quest (PCjr)? I have bought anthologies and GoG collections, but none of them go that far back. Apparently the copy protection was a bear, and before flux imaging, there wasn’t a good way to get a perfect 1:1 copy that didn’t require a crack.

Hey I think I saw you driving , does your plate say pentium?

Is there a way to capture the flux data without a laptop or desktop?

great video. although I must admit, I spent half of it trying to figure out how you'd get a 5.25" disk with different formats on each side. My guess: they formatted it first in a system like a PC with a double-sided drive, then needed to reuse it on a single-sider like a C64/Apple II, and only bothered reformatting the side they used for that?

@TechTangents

2 ай бұрын

That's about my nearest guess, it could also easily happen with any single sided system if you put a fresh out of the box pre IBM formatted disk into it. I had to learn it the hard way with some disk images from the HP Museum that are for some reason HP 9121 formatted on one side and IBM 720k on the other and needed to get only the HP data.

@jeff-network

2 ай бұрын

There were specific bulk mastering hardware/software to achieve this.

@FooneTuring

2 ай бұрын

​@@jeff-networkI know, I have some. but still, the idea that someone would do it on purpose is silly! I have to assume it was done on accident

3:38 374x type data entry. 3742 was typically a 'dual station entry", two people sitting face to face. 3741 was a single unit. I don't recall having dual drives on a single user machine. The image is older than my brain implant.

i have a load of 720 which have been changed to 1.72 disks. also used pcbackp and cpbackup for backups. so kinda stuck with them. wouldn't winimage also work?

wonder if any hidden easter eggs in those old weird old floppy formats

5:36 "Multiple reads of each track". I've recently discovered that The Applesauce does excellent at this too... They are way more expensive and NOT open source like the GreaseWeazle, but worth mentioning...