CORRECTION: The resource kit was apparently a lot more available than I realized. I knew it was in the book, but I didn't imagine the book was *that* common. I never saw it at any of the shops I worked, but that's a small sample size.

@butsukete1806

Жыл бұрын

I worked as admin in a small lab, the Resourse Kit seemed to be standard must have at the time. The books filled up half a shelf, and occasionally helped, but that's not why you bought it.

@flapjack9495

Жыл бұрын

I remember the NT Resource Kit well. I was an NT admin back in the 3.51 / 4.0 days and that kit was indispensable. The book was also riddled with errors - documentation that was flat-out wrong. "It works like this!" No, Microsoft, it does NOT work like that.

This isn't just half-baked, it's god darn frozen.

@RemnantCult

Жыл бұрын

This beast's still walking in the forest, that's how unbaked it is.

@mollthecoder

Жыл бұрын

And it wasn't even a dessert.

@tylern6420

Жыл бұрын

Its not even there its literally just assorted piles to create a beast to walk around

@DanTDMJace

Жыл бұрын

@@RemnantCultThe beast's name? Windows NT.

@pingwinit8611

Жыл бұрын

it was just a jump for money, so why bother even unfreezing it?

I guess for people wanting to use the POSIX subsystem back then, enterprise grade customer support from M$ was basically a necessity.

@randomtidbits7695

Жыл бұрын

It's been a long time since I had to fool with it, but iirc "enterprise support" didn't get you far and in our case, lead to migrating to an entire MS shop against the wishes of everybody who didn't have their hands in the piggy bank. Which if you boil it down, that seems to be the intended result anyway.

@yukisaitou5004

Жыл бұрын

@@randomtidbits7695 can confirm that this was exactly how it went at the law firm that I used to do IT work for

@TimSedlmeyer

Жыл бұрын

Enterprises which wanted to use POSIX software on Windows NT were very rare and those that did used one of the variety of 3rd party solutions on the market and not the POSIX subsystem. These 3rd party solutions mostly targeted software companies looking to provide their product to Windows only shops. Most POSIX software running in Windows back then was 3rd party software from these companies and the terms of the POSIX solutions allowed distribution of their software with the POSIX runtimes.

@bloepje

Жыл бұрын

M$ does not support. There were some major bugs reported to Microsoft in their SNA implementation (another embrace and extinguish, but then again, who really wants SNA). Bugs were send with fixes and reproduction instructions to their enterprise support. 6 months later a reply came that they found it was a bug and that they will look for a solution. That's how enterprise grade their support was.

@TimSedlmeyer

Жыл бұрын

@@bloepje SNA Gateway was a particularly bad product. Many of my employer's customers at the time unfortunately didn't discover this until replacing the reliable Novell SAA with it. It was so bad but also so key to my employer's customers accessing our primary product that eventually my employer was able to strike a deal with Microsoft. They loaned some of their developers to Microsoft to help improve SNA Gateway. Besides the poor quality of that particular product my experience with Microsoft support was ok, but at that time my customers were also very large enterprises or large government agencies which had large expensive contracts with Microsoft in order to get a high level of fast support.

I've always kinda knew this subsystem existed, but wow I didn't realize it was this cursed. Crazy to think that presumably many many hours were put into designing the subsystem, only for it to be basically unused. I wonder what it was like for the Microsoft developers back then.

@starfrost6816

Жыл бұрын

NT has a native subsystem capability that was leveraged for this. Microsoft simply added another subsystem, although it certainly took a long time to get it going. Maybe there were plans to improve it, maybe there wasn't Microsoft replaced it with "Windows Services for UNIX", which was SUS compliant (probably not getting away with that half baked POSIX bs anymore), and then WSL1, and then WSL2 (which is literally a linux VM running inside hyper-v with some clever stuff to get it communicating with windows) Btw, NTFS is also case sensitive, fwiw, windows just...kinda kludges around it

@74HC138

Жыл бұрын

The devs probably not only knew but fully supported developing this "malicious compliance" POSIX subsystem.

@jordanl5444

4 ай бұрын

Those "many hours" were spent figuring out how they could do as little as possible

The POSIX subsystem for Windows was superior to WSL in one very important respect: the name wasn't backwards like "Windows subsystem for Linux" is.

@MaxQ10001

8 ай бұрын

Yeah, WSL sounds like a Wine competitor, not the opposite 😅

@jeremyandrews3292

8 ай бұрын

Yeah, I was going to say... a Windows subsystem for Linux sounds like a way of running Win32 applications on Linux.

@shallex5744

7 ай бұрын

technically i think the grammar is correct, but regardless it does intuitively sound backwards indeed Windows Subsystem for Linux the Windows Subsystem (the subsystem that exists within windows) that is used for running Linux

@Thanatos2996

7 ай бұрын

@@shallex5744 yeah, I get that’s what they were going for, but it’s still backwards. Anywhere else, the noun adjunct to “subsystem” describes what the subsystem does, and “for” indicates what subsystem is meant to be a part of/run on. It’s not a Switch emulator for N64, it’s an N64 emulator for Switch, for example. Even if the grammar is technically ambiguous (it probably is, English is not especially precise), WSL is still strictly backwards to the standard convention used in any other similar context.

@shallex5744

7 ай бұрын

@@Thanatos2996 yes, it is a very strange name choice indeed

I worked with the NT beta versions and release versions from 3.1 onward, porting a product which ran on various Unixes. We initially thought we'd be able to use the POSIX subsystem, but it was quickly obvious that this was nothing but a box ticking exercise. Our product which ran on NT, on MIPS, alpha and x86, was completely win32 with no use at all of the POSIX subsystem.

@matsstenfeldt1549

Жыл бұрын

I worked on a project in 1995 where management had the vision that we should have a common codebase for Unix and NT. The team actually tried to use the POSIX subsystem whit the same success as showed in the video... Our solution was to write our own libraries as a version of pthread for NT and not to tell management. 😁

@hypercube33

Жыл бұрын

Embrace Enhance Extinguish

The UNIX API begun in the early 70s, when the system was ported to C. POSIX provides a standard for this API, as many UNIX systems got further and further apart from the AT&T sources over time. Currently POSIX means at the very least you can run console apps on Linux, *BSDs, and Mac

@ShinyQuagsire

Жыл бұрын

tbh I'm not surprised this is such a mess, I've never had POSIX accomplish what it was supposed to. Anything beyond basic apps has always required a pile of ifdefs even for stuff like memset/memalign/malloc. They didn't even add endian swapping functions until *2014*... for a standard which is literally centered on standardized IO.

@chri-k

Жыл бұрын

@@ShinyQuagsire it’s way better than nothing though

@llothar68

Жыл бұрын

POSIX is C source code level. Unix API is some kernel calls wrapped by standard library.

@simonfarre4907

Жыл бұрын

@@ShinyQuagsire I think you are conflating Windows adherence to the POSIX standard with the actual POSIX standard. The reason why WSL exists is because of Windows attempting POSIX compliance.

@andreyv116

Жыл бұрын

Yup, posix compat means compiles on both AIX and Linux even though AIX feels like a barren shithole compared to gnu linux

Back in the 90s I used to work for a company that implemented Windows NT and for a while had the largest NT network in Europe. I was aware of the POSIX subsystem which I understood was present to allow sale of the product to US Government entities that had strict policy on procurement where certain features had to be available regardless of use-case. Another feature was C2 security which Windows NT complied with although security capabiltiy has been enhanced since then. I recall anecodally that some software vendors had made their previously Unix targetted software available on NT using the POSIX subsystem but never saw it myself. I had access to Technet and MSDN so had the resource kit and SDK as well as development tools and there was loads of interesting stuff included. For example the source for the built in screen savers using OpenGL library. At that time Cygwin didn't exist and even doing development on Unix/Linux relied on the runes aligning to setting up Windows for this wasn't vastly more complex!

@benhetland576

Жыл бұрын

Yeah, I remember M$ bragging about that C2 compliance from the very start during the presentations of NT 3.1, and how hard work it was to get that certified. Later I found out that this approval was only valid for a certain subset of hardware/software components, and anything with a network (even just LAN) was _not_ C2 compliant. Even back then running without some form of network was pretty useless.

“It just existed so MS could claim POSIX compliance” yeah, pretty much how most people knew at the time as I recall

4:56 tar is not a single format and I'm guessing you were trying to unpack a GNU format tar file. With GNU tar, you might have better luck creating archives with one of "--format=posix", "--format=ustar", or even "--format=v7".

I always wondered what the POSIX subsystem option was in Visual Studio's linker options for C++ projects, I guess it still has some support even in Windows 11 and VS 2022, how interesting.

@NCommander

Жыл бұрын

The linker flag specifically sets the subsystem file in the PE header; I don't think it affects anything else besides that, but I'm not 100% certain.

The literal definition of malicious compliance.

@JeffreyPiatt

Жыл бұрын

Ironically Windows NT is a fork of OS/2 there joint project with IBM. It's likely it's posix compliance was also this messy

@hye181

Жыл бұрын

@@JeffreyPiatt NT is not a fork of OS/2, its a nearly 100% independent codebase

@user-yg4kj2mf1p

Жыл бұрын

POSIX itself is an example of malicious compliance though, since it was all about taking the UNIX API and whitewashing it as a standard for all OSes despite it being obviously intended only for Unixes. They even made the acronym rhyme with Unix, just to rub it in. Which is why POSIX is so limited, once they removed all the bits that were too Unix-y, there wasn't much left. Microsoft made a good call by implementing just the standard to point out the ridiculousness of the standard.

@hye181

Жыл бұрын

@jankrynicky no, it is literally a fresh reimplementation, the only thing taken directly from OS/2 is the command interpreter

@SenileOtaku

Жыл бұрын

Perhaps *that's* what MSWin's "Malicious Software Removal Tool" is for. Because I install it on a MSWindows system, yet MSWindows is still there afterwards.

I love the NCommander In Realtime series. Your livestreams are a bit long for my attention span, so I do like these re-narrated shorter versions, since it lets me see what curse-ed things you're doing... and I totally love the mindset. "This thing is available, I wonder if it's usable, it's probably not but they said it worked so we gonna try it". I always knew the POSIX compatability was kinda trash but you've shown it's total trash, thank you for sinking hours into this, great to watch!

@AMcAFaves

Жыл бұрын

My short attention span appreciates these live stream summaries, too.

Looked up the WeirdNIX contest in old Usenet posts. One of the examples in Donn Terry's 1986 announcement of the contest pointed out that the existing specification considered a single dot to be a valid return value for getcwd().

@NCommander

Жыл бұрын

Oh awesome, which group? I went looking but couldn't find anything but references to references.

Uh, assuming it did pass the NIST test suite, then the problem was with one or more of: the standard, the test suite, or people's expectations. Undocumented, 'soft' standards are a problem.

Considering that at that time Microsoft has already have enough experiences with developing XENIX, the half-baked POSIX implementation were deliberate moves by Microsoft.

POSIX on Windows is so funky, the default C library (msvcrt) fools you into thinking it might be POSIX compliant until you actually try to use it, add the 10 official implementations of POSIX, each for a different version of Windows, third party libraries like Cygwin and MKS, as well as Microsoft simply integrating a Linux kernel into the OS.

@MaxOakland

Жыл бұрын

How’s POSIX on Mac?

@adora_was_taken

Жыл бұрын

@@MaxOakland _so_ much better. fully functional and works like you would expect it to.

@MaxOakland

Жыл бұрын

@@adora_was_taken That’s awesome!

@kreuner11

Жыл бұрын

@@MaxOakland macOS is weird since it combines Old Macintosh, NeXT, Unix, and Mach operating system heritages. Macintosh gives you the GUI style and file-systems, NeXT gives you the premier programming API, Unix gives you the more standard C API and file-system structure, and Mach gives you some of the kernel and executable file format

Reminds me of the time when Java applets were a thing, Microsoft shipped their very own Java Virtual Machine. It was so bad that Sun sued them and eventually won.

@urlhnd

7 ай бұрын

This lawsuit eventually brought us C#.

Fun fact : Linux isn't actually POSIX-compliant. Stallman thought some of the decisions of the norm were stupid and never actually got the certification. You CAN make it essentially compliant with the variable POSIXLY_CORRECT or POSIX_ME_HARDER, but it's not official, making macOS the only fully POSIX OS outside of "real" Unix like AIX or HP-UX.

@LonelySpaceDetective

Жыл бұрын

"POSIX_ME_HARDER" is the funniest variable name I've ever seen.

@MrCed122

Жыл бұрын

@@LonelySpaceDetective For some reason, I want to add DADDY at the end of it.

@kFY514

Жыл бұрын

A couple of Linux distros (specifically, EulerOS from Huawei and Inspur K-UX, both RHEL derivatives) got UNIX 03 certification at one point, proving that Linux indeed _can_ be made "fully compliant" if needed.

@Sumire973

Жыл бұрын

There have been 2 UNIX-certified Linux distros tho.

@vvgr409

Жыл бұрын

There were few Linux distributions that passed Single UNIX Specification tests and were registered as UNIX systems.

The idea the resource kit was obscure or hard to get a hold of is just incorrect. If you wanted the book along with it every bookstore carried it in their technical book section. It was advertised and well documented in magazines of the era. It was available for free via the Microsoft BBS and the Microsoft FTP site. It was very commonly installed on NT systems.

@NCommander

Жыл бұрын

I worked in IT in this era. I remember the 9x ones. The NT one was rarely, and it didn't note that the POSIX tools were included. The book itself doesn't even list that; I had to check the disc myself. The book just lists traverse checking has to be disabled, and nothing else.

@TimSedlmeyer

Жыл бұрын

@@NCommander I worked in IT during this era. Your recollection maybe that the NT resource kit was rare but it certainly isn't mine. It was everywhere.

@MK-of7qw

Жыл бұрын

My brother and I had the NT 4resource kit though he had to order it. A few years later I was able to get the 2000 resource kit but that was a lot more common..

Here I am, (in Greece)helping with decorating the New Year Cake while taking breaks to watch your contribution to Computer Necromancy. Excellent, fact-packed video. May 2023 bring more of this. You are a gem.

@DOSdaze

Жыл бұрын

"Computer Necromancy", I love this term and will be stealing it 👍

The POSIX subsystem is definitely not "batteries ready" UNIX, but you should consider the amount of heavy lifting Microsoft did to make the NT kernel compatible with UNIX architecture. For instance, process creation is very different in Windows and UNIX because Windows has no notion of forking. NT therefore supports an efficient implementation of forking, using copy-on-write, even though Windows didn't need it. Furthermore, NT had to be designed to permit both a case insensitive (for Windows) and case sensitive (for POSIX) naming conventions. I'm fairly confident that NT would have a significantly less complicated kernel if POSIX was not a requirement. They probably wouldn't even have gone with the server/client architecture of having different subsystems if not for POSIX. At the time NT was released, the only subsystems were Windows, OS/2 and POSIX. If they dropped POSIX support, I'm pretty sure they could have found an easier way to do just Windows and OS/2. Even though CYGWIN is a "batteries ready" UNIX, it doesn't take advantage of any of the NT kernel's built-in support for POSIX. This means in principle if you have a UNIX program that does a lot of forking (like a web server), it should run much more efficiently on NT's POSIX subsystem than on CYGWIN. In essence, Microsoft went to the effort to make the cake (everything difficult about POSIX, they implemented) but they didn't bother to frost it or decorate to make it look appealing. It was of course designed for government contractors who presumably had the ability to code up whatever bells and whistles MS didn't provide. One wonders if some government agencies were making use of this back in the day, and we just never heard about it.

This might be interesting. The subsystem I've heard of existing ever since I learned about NT4 in my teen years, yet never heard about anyone using in the real world.

@klausschmidt982

Жыл бұрын

Because, for all intents and purposes, nobody did.

@fss1704

Жыл бұрын

I saw one extremely edge case once, but that's it.

@fss1704

Жыл бұрын

Also, it was executing a file that was not .exe

The resource kit was readily available. MSDN subscriptions included it. I do agree with you 100% on the posix mess that MS shipped.

@NCommander

Жыл бұрын

I put a pinned comment about this.

@CarlenHoppe

Жыл бұрын

A TechNet subscription (for SysAdmins, not developers) should have also provided the necessary resource kits (but not SDK's).

The windows NT networking stack meets POSIX by including the required network diagnostic apps ported directly from BSD very early versions of Windows ipconfig would display the Berkley copyright with the proper switch in DOS prompt so they were compliant with the BSD license.

@lawrencedoliveiro9104

Жыл бұрын

That Windows NT networking stack was in fact copied directly from the BSD one.

Microsoft: "See, look how terrible POSIX is. You should just give us a no bid contract to deploy Windows APIs."

@JeffreyPiatt

Жыл бұрын

Windows NT is still Posix compliant like it's sibling os/2

@JohnDlugosz

Жыл бұрын

@@JeffreyPiatt OS/2 was not POSIX compliant. "it's" is a contraction for "it is". "its" is the pronoun.

@szt1980

Жыл бұрын

@@JeffreyPiatt NT had a os/2 subsystem, BTW

@ssl3546

10 ай бұрын

@@szt1980 Only for 1.x text mode programs. Which was actually super useful because OS/2 1.x was a better DOS than DOS, much better for software development. Many dev tools were bound to work under both DOS and OS/2 1.x.

They really did the absolute minimum they had to do to be certified. Shockingly honest.

@NCommander

Жыл бұрын

I'm pretty sure cc.exe and make.exe exist entirely to run the NIST test suite.

@Pesthuf

Жыл бұрын

It's a miracle those aren't just one massive if-else block that spits out fixed machine code that passes the test depending on the input file name.

@falsefalse

Жыл бұрын

@@NCommander running NIST tests suite incidentally is a good candidate for “something to do with it”

On a technical level, it would've been impressive if it was completed and documented properly. The NT Kernel has support for subsystems besides Win32. Would've been fun to actually use it. Ah well.

@NCommander

Жыл бұрын

That was Services for UNIX and Windows Subsystem for Linux v1.

@timothygibney159

7 ай бұрын

Wsl is a more 21st century solution with containers and virtualization

No random and srandom, so no proper C Standard Library support or a really broken build environment. With no networking and very limited interop(ooh shared filesystem), They really didn't want anyone using it. Meeting the bare minimum for government contracts, just like any business would. WSL 1 pretty much ended up doing The Posix subsystem the right way, aiming for actual binary compatibility not just poor-quality source compatibility.

@R.B.

Жыл бұрын

WSL1 isn't binary compatible, it is syscall compatible. It also wasn't the first time it had been done either. Circa 2002 out of MS Research a similar project was done which could run Linux ELF binaries on a Subsystem. To demo the ability they were running an Apache webserver and had Firefox accessing it. It was never going to see the light of day with Balmer leading the company. WSL1 is more like WINE in that the syscalls were handled by Windows and translated on the fly to an equivalent set of Windows API calls. WSL2 is more like And Linux or CoLinux where the kernel is actually running in a VM and is isolated in its own Hypervisor. Each have their own strengths and weaknesses, but WSL2 with how it visualized the file system and hardware is more resound and can keep up with Linux kernel changes in a way that WSL1 can't. The magic is how they've bridged WSL and Windows so that they are both working together with only a few seams if you know where to find them. For most of my Linux needs, Windows is the best Linux system I need and it is easy to switch between different Distros. POSIX on the other hand. It was available for very custom work and not expected for more than supporting those very bespoke customer solutions. The subsystems were initially designed to support more than the Win32 API, but with IBM and DEC usage being significantly anemic, OS/2 and VMS compatibility wasn't a priority. NT 3.1 and 3.5 were intended to offer more, especially on the POSIX side of things seeing as Dave Cutler and many early NT engineers had built VAX for Digital before coming to Microsoft. The bones were there and have been from the beginning -- NT kernel was going to run Win16, Win32, DOS, OS/2, and POSIX applications, each in their own environments -- it was soon clear that Win32 was the Subsystem to design around. I think it would be interesting for Microsoft to revisit supporting more subsystems. There'd be a great MSR project to make modular environments to host things like a Node/Deno container, one for .Net CLR, Android, or a modern LISP machine, with a Win32 host machine providing a way to keep those other systems running in isolation. It would look a little like Docker Containers in some ways, or a NT-based Kubernetes OS. I think Windows 10X was sort of heading that way with Win32 apps running in their own container, and I think it is still a concept worth going after. WSA and WSL demonstrate that it is possible, but it still needs refinement.

3:51 It kind of reminds me of early "compatible" computers that could run MS-DOS applications, but no real software because programs didn't use DOS/BIOS calls for display etc. Now, I also remember there was a company who made something that rounded out the POSIX subsystem and made it truly useful. MS _did_ provide the critical foundation -- stuff that must be implemented in the OS itself -- of having a 'fork' function and providing for case-sensitive file names.

@lawrencedoliveiro9104

Жыл бұрын

Windows has the equivalent of fork(), but it is expensive to use. For example, back in the days when Git was becoming really popular, Mozilla were looking to switch to a DVCS (I think they were using SVN or something). But they had a requirement that whatever it was would work well on Windows. And Git, at that time, made too much use of fork/exec to work nicely on Windows. So Mozilla went with Mercurial instead. Git has been modified since then, to tone down its appetite for spawning processes, to cater better for Windows installations. On Linux, of course, there was never a problem.

@ailivac

Жыл бұрын

Cygwin has a spawn() function that basically emulates fork/exec with native Win32 calls in the event that that's what you really want to do. But it does provide an implementation of fork that's kind of hilarious in the lengths they had to go to. Obviously it's incredibly slow, but the best part is that because of address space randomization it can sometimes just fail if Windows by chance decided to stick a DLL in the wrong place first. It will retry a finite number of times but there's always a remote possibility that it will just have to give up eventually.

@markevans2294

Жыл бұрын

@@lawrencedoliveiro9104 On early Unix systems fork() was also an expensive operation. Since every writable memory page in the parent's address space would be copied to the child. Hence there was vfork() which would share the address space and suspend scheduling of the parent until the child called exec() (or exit()). In more modern systems writable pages are shared and marked Copy on Write.

FWIW I dont think there was any malicious intention in "sandboxing" the posix subsystem. In Windows the POSIX subsystem sits above the kernel just like Win32 and OS/2. I dont think this was anything malicious just Dave Cutler drawing a natural boundary around the kernel and implementing what was needed. I worked on this operating system and I can tell you that there was a TON of stuff going on in Windows at the time and it was simply the case that OS/2 and POSIX were very quickly not worth spending developer resources on.

@NCommander

Жыл бұрын

No, that's completely fair. I think it got left out of the actual cut, but in the original notes, I actually did compare it to the OS/2 one, which can access NetBIOS, and IPC structures, and has more integration. Like, you could actually use the OS/2 subsystem as a migration path from OS/2 server to NT, and Microsoft's own pre-NT 3.1 early NT toolchain was OS/2 based and ran through the subsystem.

@williamtell1477

Жыл бұрын

@@NCommander Totally. Around this time we were building Window 95 with direct X, then merging all of that code into the NT codebase to produce Win2k, figuring out how to service the OS, doing the IE thing, I mean the origin of Windows was a rocket ship. It was a very diff env from today, tbh noone particularly cared about POSIX. Linux was nothing even close to what it is today (a trajectory which changed dramatically when Google invested in Android) and it was definitely not a high priority. The situation has changed 1000% and WSL is a much more committed attempt and IMO absolutely critical for Windows' - survival given the importance of Linux today dev machines need to be good to work with Linux and OSX has an advantage there. But IMO Windows has made great strides here e.g. passthrough GPU virtualization for WSL usage is a major deal for machine learning workloads. Anyway im rambling.... Awesome video, awesome channel, thank you - subscribed!!

I remember using 4DOS at that time. I don´t know how much POSIX-compliant it was, but it extended a lot the scripting functionality of DOS and OS/2.

I just found your channel and watched about 3 years of content in a week. Hope you're doing well and have more fun in the works :)

11:27 Remember that Windows didn’t have a decent terminal emulator anyway (and as far as I know, still doesn’t). Microsoft didn’t like the traditional idea of sending inband escape sequences to trigger terminal functionality like cursor addressing, text attributes etc. So they developed their own terminal API that required special system calls to control all these things. And, surprise surprise, nobody could be bothered writing software to use these special system calls.

@RadikAlice

Жыл бұрын

They do not. And haven't had a CLI text editor since 'edit' in the 16 bit days No wonder WSL is a thing, CLI workflow in the most basic sense is basically non-existent

@SenileOtaku

Жыл бұрын

Strange and incorrect characters in the stream? Sounds like logging into a non-Microsoft IRC server using MS' Comic Chat.

@SunnyRayPL

Жыл бұрын

@@RadikAlice Yes they do have a terminal (console host, using pre-11 nomenclature), a bunch of improvements was added to conhost in 2019, and now Windows Terminal is default on W11

Nice! Wasn't this a big deal at the time? This is crazy stuff! As an economic history researcher, I swear this story deserves a paper, if there isn't one already. Subbing right now! ❤

I was but a starting developer in 2001, but this is an eye opener! Glad M$ came a long way since then - through no choice of their own it seems.

@watchm4ker

Жыл бұрын

The real punchline is that Microsoft developed and sold a very popular commercial UNIX versions of the 1980s: Xenix. Even considered pushing it as a replacement for MS-DOS... But then AT&T started selling UNIX directly, and Microsoft recognized pretty quickly it couldn't compete with the company it was already paying licensing fees to, and decided to work on a different successor to MS-DOS - OS/2. As for Xenix? That got sold to a company that was co-developing Xenix with MS: The Santa Cruz Operation, or SCO...

@Rouxenator

Жыл бұрын

@@watchm4ker I would love to get more history on the NT name, some say it actually came from a version of Windows developed for the Intel i860, code-named N10 ("N-Ten")

@watchm4ker

Жыл бұрын

@@Rouxenator That, I don't know. I know it used to stand for New Technology, at launch. And that it's... Basically the original version of OS/2 3.0, after Microsoft decided to abandon THAT partnership.

@JohnDlugosz

Жыл бұрын

@@watchm4ker There's no relationship between OS/2 Warp and Windows NT.

@JohnDlugosz

Жыл бұрын

@@watchm4ker Ah yes, back when getting a Unix box at home was scathingly expensive. Linux really did change everything.

Windows terminal stuff has always been atrocious and it was the first thing I thought of when you mentioned Rogue. Even now it's quite bad, though it has improved significantly. It's still decades behind

@nuglord2084

Жыл бұрын

SSH wasn’t enabled by default in PowerShell until W10 1809. It was a huge leap forward.

Amazing stuff and very informative as usual. Also lol'd @6:50

I bought a copy of Services For Unix back in college where I used Unix and NT workstations in the engineering computer labs and ran Windows on my dorm PC. Writing portable c++ code back then was frustrating, especially not standard Unix filesystem structures for headers/libs.

Very cool video...didn't know a posix subsystem existed for windows NT...I only found posix on c functions that looked like yea that might exist on say unix...and it would say depreciated...and I was like ...yea I am using that...for it was something simple I was familiar with...just needed to refresh on the parameters, etc. I guess not knowing about this was a good thing...kept me from attempting it when I was moving from DOS to Windows NT...and didn't know much about how to get there...I found the win32 library interface and I was like ok...winner going with that...worked out ok...well less painful than anything like this.

I remember popping in for a bit during the live stream. Didn't realize you had to use nmake at one point... I'm so sorry. Happy New Year!

The Windows subsystems for OS/2 and POSIX were supported up to and including Windows 2000. I used the POSIX subsystem on Windows 2000 back in the day, and was fairly satisfied with it, except that it didn't support fork(2) (the call returned -1 with ENOTSUP / ENOTIMPL). Since I needed an implementation of fork(), I created one in native Win32 code (which was pretty ugly but worked). BTW, most of the tools and documentation for Windows POSIX were available on the Internet by the time I used it. The Windows SDK was downloadable from Microsoft as well.

Nowadays, this could be seen as perfect example that a company changes according to their CEO and its decisions.

@MaxOakland

Жыл бұрын

I dunno. I don’t trust them with Linux. I’m sure they’ll ruin it to promote their own stuff if people get too comfy with Linux Subsystem

6:43 in Windows 10 and newer, you can enable or disable case-sensitivity on a per-directory basis now

@chri-k

Жыл бұрын

that is cursed

@lucass8119

Жыл бұрын

Sounds like a super convenient way to pull out your hair at some undetermined time in the future! Thanks Microsoft!

@N....

Жыл бұрын

@@lucass8119 It's actually really useful for debugging when you are doing iteration on Windows but the target device uses a case-sensitive filesystem. You can set the folders as case-sensitive and catch those bugs early before trying to deploy and debug on the target hardware.

Would love to see a comparison to the later Unix subsystems in NT: SUA/Interix and Windows Services for Unix (SFU). And maybe a side look at Microsoft's own standalone Unix implementation, which also influenced these efforts: Xenix.

@NCommander

Жыл бұрын

It's on my TODO list, although I've covered xenix before I did youtube: casadevall.pro

The problem with IE4/Active Desktop is just how resource-hungry it is compared to the original incarnation of Explorer. Plus, on 95 at least, the shipped IE 5.5SP2 runtimes make it leak memory. To each their own, but anywhere the original is an option, I take it.

@Irongrip62

Жыл бұрын

Back in the day I used active desktop to have a live stream of some kittens playing in a cat shelter stream from a webcam as wallppaer. It used flash and you had to restart explorer everyone once in a while to fix the memory leaks.

I really like your channel, it has V I B E S. IDK how else to put it. Something about the music and the calmness of your voice and the highly interesting subject matter. Really great, thank you! :)

You mentioned a book "The Unix Haters Handbook" in a video of yours, one of the solaris ones maybe, anyway, I had to tell you this book is absolutely hilarious. I haven't laughed this hard reading something in years. Thank you.

POSIX subsystem was never mean to be a replacement for Unix but a runtime time environment for applications that had been designed to be posix compatible those applications didn’t make use of all the extended c libraries , Unix tools etc because by design they were meant to be posix compatible. What most commenters and the author should remember is that before nt no other major operating system had ever offered completely different runtimes running on the same kernel out of the box. This allowed specific customers to run these applications side by side with general windows applications. Remember that every other posix compatible os offered it natively but could not run other runtimes requiring customers to have multiple systems to achieve the same results. And the windows resource kit visual studio and related documentation was included with every enterprise level msdn subscription which back then was included in major enterprise agreements with Microsoft. At the first company I you just walked into the it administration office to get these cds. So a lot of the downsides mentioned in this video are really only applicable if you were a hobbyist to smal enterprise in which case this function wasn’t designed for you.

@IngieKerr

Жыл бұрын

100 times this ^ POSIX isn't there to run some nerd's *nix nethack, it's there to run POSIX compliant software. That's all. I think folk confuse _most *nixes are POSIX compliant_ with _all POSIX machines should be same as some *nix_ POSIX is designed to be _Unix-like_ , not _to be_ Unix. It's very basic, and that's the point of it. [ and aye, as a software dev since the 80s, every company I've worked for from aerospace, thru shipping, to pure IT, has had a full MSDN if they were an enterprise business, it's the way it was - I've got folders and folders of the stuff in my spare room :) ]

@hypercube33

Жыл бұрын

Technically for a short while, Windows had: NTVDM - 16 bit, POSIX (lol), Win32 together. Technically also could run DOS apps.

There also seems to be an OS/2 16bit character mode support in NT up to Win2000

Hi. What I remember, but back in win2k world and appropriate MV VS, there was an option to configure a project output to be for Windows POSIX subsystem, as just one setting. I tried it literally once or twice, it worked, but there was absolutely no practical usage.

And I thought the RS/6000 shipping with reference implementations of core features was bad (which it is) but this makes that preferable, I bet that if Linux and open source hadn't taken off in the business world we'd still be dealing with this kind of unsubtle yet subtle enough middle finger towards UNIX as a whole

@lawrencedoliveiro9104

Жыл бұрын

“Unix” is a trademark that only proprietary products can afford to license. And all those products are either dead or dying right now. What matters nowadays is not “Unix”-compatibility, but “Linux”-compatibility.

@RadikAlice

Жыл бұрын

@@lawrencedoliveiro9104 While I understand your point, the wording made me roll my eyes and kinda want to say _something_ out of spite It feels insufferably pedantic, but maybe that's just a gut reaction from a bad first impression

I'm sort of tempted to get a VM setup with the POSIX environment on my end so that I can try and run neofetch in it so my guess is that I would have to get bash working in this environment

*Babe wake up, new NCommander!*

Thanks must go out to the FOSS Community for WINE and Bottles. Kudos to Valve for Proton!

@SenileOtaku

Жыл бұрын

Now we just need ReactOS to proceed past pre-Alpha stage...

This may not be the case, but I read that the reason NTFS supports hard links (that is, more than one directory entry pointing to the same file) is that they had to implement this POSIX subsystem. Hard links are _so_ useful. For example, prior to a large scale batch rename, I can clone a directory tree without duplicating the file contents. (This uses cygwin btw.)

@bootmii98

Жыл бұрын

I think they were using HPFS at the time

@ccoder4953

Жыл бұрын

Don't even need cygwin. Lookup "Link Shell Extension". NTFS supports both hard and sym links. And it has for quite some time (NT4!). That plugin has some cool functionality they built on top of the base functionality, so it's definitely worth a look.

@LonelySpaceDetective

Жыл бұрын

@@ccoder4953 hell you don't even need to install anything, the standard command prompt windows ships with has a mklink command for doing this though having it as a right-click context menu is _very_ handy if you make links a lot

@lawrencedoliveiro9104

Жыл бұрын

Symlinks were not added to Windows until Vista. Even now, they require special privileges to create.

@ccoder4953

Жыл бұрын

@@lawrencedoliveiro9104 Actually, we're both sort of wrong. According to Wikipedia: "Symbolic links to directories or volumes, called junction points and mount points, were introduced with NTFS 3.0 that shipped with Windows 2000. From NTFS 3.1 onwards, symbolic links can be created for any kind of file system object. NTFS 3.1 was introduced together with Windows XP, but the functionality was not made available (through ntfs.sys) to user mode applications. Third-party filter drivers - such as Masatoshi Kimura's opensource senable driver - could however be installed to make the feature available in user mode as well. The ntfs.sys released with Windows Vista made the functionality available to user mode applications by default. " As far as the special permissions thing: "The default security settings in Windows Vista/Windows 7 disallow non-elevated administrators and all non-administrators from creating symbolic links but not junctions. This behavior can be changed running "secpol.msc" the Local Security Policy management console (under: Security Settings\Local Policies\User Rights Assignment\Create symbolic links). It can be worked around by starting cmd.exe with Run as administrator option or the runas command. Starting with Windows 10 Insiders build 14972 the requirement for elevated administrator privileges was removed in Windows "Developer Mode", allowing symlinks to be created without needing to elevate the console as administrator. On the API level, a SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is supplied for this purpose" So, the policy can be changed and it appears Microsoft will soon make the change the default.

Does this have something to do with me not being able to get Python's ctypes library to recognize files that use pthreads?

Now that I come to think about it, shouldn't WSL be instead called the Linux Subsystem for Windows (LSW)?

Well in the those days there was a lot of arcane stuff to get a Linux system up and running as well (i still remember modelines for the display driver or wading through long lists of kernel options "fondly"), but at least there was some documentation (read those READMEs) and it didn't have this feeling of malicious intent.

Are you going to do a follow-up covering Interix? That looks like it was more of a serious attempt at providing POSIX compatibility

@NCommander

Жыл бұрын

I will at some point. It's on the topic list.

Did you try searching old newsgroup archives? That's where I would look for interesting commentary from that time period.

@NCommander

Жыл бұрын

There wasn't a lot, most people complaining they lack parts, or being told it was useless.

Windows has another POSIX subsystem called SFU,any idea to try that?

@NCommander

Жыл бұрын

It's on the topic list.

Imagine trying to admin this in the pre-Internet days

that background music is the exact same as used in the Mountain Beast Mysteries channel (Bigfoor stuff). Weird to hear it on other videos...

@NCommander

Жыл бұрын

It all came from the KZread Music Library

I was interested in that thing but never actually had time and motivation to try this out. Now after your video I know that it's not a bad thing that I didn't try. Good thing that it was replaced by SFU which was much more complete and useful. Also I used Cygwin and it was pretty convenient and useful as well. I was even able to run some Unix desktop environments with it and Cygwin/X.

Thanks for sharing. 😉👌🏻

@NCommander

Жыл бұрын

Thanks for watching!

I saw mention of the posix system in Windows NT back in 1993. It never even occurred to me to go check it out, since I had access to Unix. What would have been the point?

@NCommander

Жыл бұрын

That's what a lot of people have told me. SFU was really when it was taken more seriously.

@scottfranco1962

Жыл бұрын

@@NCommander I've been in software for 40+ plus years, an I have been burnt many times by emulating other peoples environments. Its always a moving target, and its really amazing how common it is for programs to skirt the rules of an interface and take advantage of undocumented quirks of the interface. This is one reason why I am not a fan of either Wine or the Linux on windows subsystem. If I want the Unix/linux environment, I'll just go use Unix/Linux. If I want a program to port to a different environment, then recompiling it for the new environment is the way to go. For programs that cross multiple platforms, that's why god made cross platform porting kits. Put it this way: a program designed for porting via a kit had portability in mind from day one. A program written for a given interface (Windows/Linux/Mac) was designed to be bedded in cement from day one.

would be nice to have a stream on SFU to see if it's more sane.

@greatquux

Жыл бұрын

SFU is absolutely much better than this. Wow.

@vvgr409

Жыл бұрын

SFU is much more complete and better than this. Not only it provides much more Unix utilities and POSIX API but later versions even contains X11 libraries. SFU is much more close to something made properly than this thing ever was.

As a counterpoint, you should try out OpenSTEP YellowBox NT. It's basically all of OpenSTEP (fka NeXTStep) 4.2, both GUI apps and command-line, but running under NT 4. (I once had it installed under XP, as well.) It's pretty complete for 1996-97, but good luck building modern stuff. Ironically OpenSTEP 4.2 (including OpenSTEP Mach, the full OS which was derived from NeXTStep and later evolved into OS X) is worse at this than NeXTStep 3.3 in some cases, as the POSIX compatibility is more broken.

@NCommander

Жыл бұрын

We've been having a long discussion on it on Discord, and how it works (which is abusing cygwin). It's on my to cover list as a stream if nothing else.

Wanting to say compliant on the box but doing as little as possible to make it so, and making it so damned hard to have any real practicality as a move to make such compliance a thing of the past.

What is the background music called, beninning at 0:04?

Good 'ol malicious compliance. I think I'm preaching the choir here when I mention embrace, extend, and extinguish as MS's modus operandi back then, hell, they coined the term.

Boo at Microsoft for doing the bare minimum but what'd you expect from them back then ;) The Resource Kit, however: I don't know anyone that used NT 4 that did not have it installed. If you used NT Workstation at home you were pretty much sysadmin-level to begin with, had a work MSDN subscription, and knew what was out there.

This video gave me flashbacks when Microsoft decide not only to implement ODF in the most half backed way inimaginable but also publish it's own OOXML format as a standard in the most useless way to make difficult the implementation by the competition 😬😵

@JohnDlugosz

Жыл бұрын

Yes, my thoughts exactly. And didn't they send their own people to the standards meeting just to vote for them, and never show up again? I remember that causing a problem because they couldn't get a quorum or something. IAC, what they shipped was the original version without any of the feedback fixes needed by the eventual standard.

@lawrencedoliveiro9104

Жыл бұрын

Microsoft itself does not support ISO 29500 (OOXML) properly. Microsoft -Office- 365 still only implements it in “Transitional” mode, since “Strict” mode apparently causes too many compatibility problems.

@benhetland576

Жыл бұрын

For a while back in the early days of those standards I remember discovering that Word's implementation of the ODF appeared more true to the official standard than OpenOffice Writer's was. This was because Writer had already progressed to a later version of its native ODF format by the time it became standardized, which had more extended/updated features. I cannot recall that it had any way of saving/exporting to the older but standardized version. Word's way of supporting ODF was rather cumbersome though, as one had to install and deal with it as a separate plugin. Conversely OOXML has always been a mess from its very inception, allowing for including BLOBs in proprietary formats that only Microsoft itself ever has a chance of fully supporting.

I have a feeling that the relevant FIPS standard was written with a close guidance from someone with Microsoft's interest at heart (or in his pocket more likely). So reaching the bare minimum for compliance was easy, even though the implementation was a complete joke and an insult.

From what I understand, the posix subststem is same mechanism as WSL1 (not wsl2 though, thats just a linux vm)

@NCommander

Жыл бұрын

Indeed. I understand why they went with WSL2 in the end, but WSL1 seems more ... elegant, for want of a better word ...

Looks a bit like what they did to prevent the netware client from running properly...

"While I could bore you with more dry, technical mumbo-jumbo" I think the title already weeded anyone that would be bored by that out.

@NCommander

Жыл бұрын

You know, this is a fair point.

Pretty cool stuff, the system architecture seems decently apart from lacking pre-compiled binaries and dependancies. Also pretty amazed you where able to find these resources from decades ago!

it's not half-baked, it's fully baked, exactly what they wanted to ship :P

lol what a complete mess. Appreciate this kind of dive into such a topic though, it was always something I was vaguely aware of but never researched the true extent of "POSIX compatibility" in Windows NT, never heard of anyone using any such functionality and never used it myself. I admit though that in modern day Windows it is rather nice to not be constantly thrown off by using Unix commands like I used to be in the old days, I still have no pressing need for things like Linux application compatibility but to not need to make that switch between Unix and old DOS commands is much appreciated, that was quite annoying in the past.

UNIX and companies that sold UNIX were their own worse enemy. One of the reasons it was the POSIX standard and not the UNIX standard. X-Windows is another such standard that was crippled from day one. Many UNIX based companies also were not POSIX compliant. Microsoft was hardly alone in that regard. The world wanted a graphical OS that didn't cost $2000 per year per work station. IBM, HP, Solaris and the others charged ridiculous prices for UNIX and hardware. Linux at the time was a hobbyist toy and wasn't an option. Today people get software almost free but in the 1990's it wasn't cheap and Windows NT cost 1/10th of other OS's. Not a small thing when you have 1 million employees who need computers. POSIX might have been a government specification, but the reality was it was only used by a few people in IT. 95% of computers were used for email, Word, Wordperfect, Lotus and Excel. Managers and IT departments wanted a single type of OS to support, or at least that seemed the vision. The reality was/is that different use-cases required different hardware, different OSs and different IT support. Even at the time people were aware of the limitations of NT and POSIX. They didn't care as long as the legacy box that only mattered to the old guys running mainframes was ticked. Remember, UNIX was really only run on mainframes or very expensive workstations at the time. Google wasn't even a twinkle in the eye of Silicon valley. It was only later in the 2000's when Linux was mature enough to save UNIX from obscurity that UNIX compatibility became important. Today things are much different. So it isn't really fair to judge MS from the 1990's. POSIX wasn't all that important and it looked like UNIX was going to die a slow death. If not for Linux, it probably would have.

Trek on NT on DEC Alpha?

@NCommander

Жыл бұрын

Need DEC Alpha hardware. The compilers are all native.

Dave Cutler was a VMS guy, and it shows.

Thank you for sacrificing your sanity so that we may behold this monstrosity. Compiling Star Trek for MIPS Windows sounds like a great idea!

I wonder how later editions faired.

Is this the reason Windows has system32/drivers/etc/hosts? I've always thought this implementation seemed very at odds with everything else in Windows.

@NCommander

Жыл бұрын

The original Windows NT 3.1 TCP/IP stack was a direct port of the one from BSD 4.3 by Spyder Software. I've always assumed it was a holdover from that, although having a hosts file is de facto standard (and might even be in one of the RFCs as required).

@JohnDlugosz

Жыл бұрын

@@NCommander The hosts file predates DNS.

@lawrencedoliveiro9104

Жыл бұрын

On GNU systems, whether you need a hosts file or not actually depends on settings in your nsswitch.conf.

I hope you don't disappear for 4 months again!

@NCommander

11 ай бұрын

I've been livestreaming, but making video content has been difficult. I've made some headway on a new video, but its been very up and down in real life

@partlyawesome

11 ай бұрын

@@NCommander It makes sense, don't let my comments build any pressure. I really just wanted to say your videos are awesome!

Hey Hey Hey Just came back for Another ReWatch!

This madness appears to be the inspiration for Gentoo.

I feel like this could have all been avoid with a generic ‘no, don’t even think about it’ clause. MS: what if we just…. POSIX: No MS: But how about… POSIX: (slaps MS with a wet fish)

@hye181

Жыл бұрын

its funny you say this because there are actual email exchanges dated 1990 between the posix committee and the NT team in leaked windows source which basically go like this

I would imagine they did this to be able to take government and military contracts since they required POSIX compliance

Between this and subsystem for Linux was the subsystem for unix

2:52 POSIX also has its own standards number: IEEE 1003.

I remember there was something called Windows Services for UNIX

"when Microsoft implemented *teh* POSIX standard."

Interix and Exceed were the tools of choice for POSIX on NT 4.0 en.wikipedia.org/wiki/Interix

@butsukete1806

Жыл бұрын

Remember Exceed for X11, but I mostly used Cygwin and Mingw32 for terminal stuff.

@ElliottFrisch

Жыл бұрын

@@butsukete1806 MKS Toolkit was another early player. I think Cygwin became popular a bit later (and obviously I used that once it did).

@lawrencedoliveiro9104

Жыл бұрын

This was part of the original architecture of Windows NT: there was a core kernel, on top of which you could have multiple “personalities”, each with their own userland API. Win32 was itself one of these “personalities”, and so was the POSIX layer. In the early days, Microsoft did let the documentation get out on how to write your own personality, and the Interix folks got hold of this. Then management changed their minds and decided to restrict this info. So they bought up Interix to prevent the details getting any further, and the Interix product became Microsoft SFU. You’d think present-day WSL would be built on the same architecture, but it is not. WSL 1.0 might have been built that way, but they could never get it working right. Which is why WSL 2.0 falls back on good old virtualization instead. And still doesn’t work completely right. So one could say that the original “personality” architecture has pretty much bit-rotted away.

@ElliottFrisch

Жыл бұрын

@@lawrencedoliveiro9104 You meant I can't run my OS/2 1.1 and 1.0 apps anymore? And yes. That was another personality. And yes, I'm still bitter about OS/2.

Great✌✌

6:04 Being able to traverse a directory (e.g. CD into it or one of its subdirectories) is a separate right from reading (listing directory entries) and writing (creating files) -- I think Unix-likes use the "execute" bit for it, and Windows NT and NTFS do the same in their ACLs. However, looking up this permission for every ancestor directory in addition to the permission of interest for whatever file you're working with takes time, and since the concept didn't even exist on old-technology Windows there was little confidence software would get it right. So all users were by default granted the right to bypass these checks: if they have permission to a file, they have permission to a file, regardless of some execute bit on some parent directory. Needless to say, that's not POSIX-compliant. While the government would have to revoke that right (compliance!), it's irrelevant to what you're doing so feel free to leave it on.

@NCommander

Жыл бұрын

It actually doesn't work correctly with Traverse Checking left enable. sh.exe doesn't load its prompt properly and there is a bunch of weirdness.

"Tell me you hate standards without telling me you hate standards!" Microsoft: