“You dont think of software being able to fail.” As a software engineer, I laughed out loud at that. If only people knew had bad it really was.

@SebAnders

Жыл бұрын

But why does it fail? Worked yesterday but doesn't work today, what has occurred?

@baumholderh8425

Жыл бұрын

@@SebAnders what’s crazy is electronics are so sensitive literally radiation from space can fuck up your data. You could do everything correct, program your software perfectly, use the hardware as intended and then bam: universe said no you will get error

@leilagotspaz

Жыл бұрын

@@SebAnders software is more finicky than a nervous system made entirely out of worms. Sometimes I beg my computer in a sweet voice to obey because I have about the same chance in that working as anything else. Software and computers generally can be really finicky, depending on what they are.

@realAniram

Жыл бұрын

Totally wrecked my immersion lmao, whoever wrote it obviously hasn't even poked html.

@nullFoo

Жыл бұрын

@@SebAnders black magic

If you're shouting "it's not possible" over a mound of dead and dying, it's probably time to acknowledge that you could be wrong

@dexterpoindexter3583

5 ай бұрын

Sure... unless you make cigarettes, thalidomide, or homebrew IEDs. Then it's lie through your teeth baby, and keep those profits rolling in!

@coolguyhino92

4 ай бұрын

@@dexterpoindexter3583 oof. Thalidomide was good pull. That shit was literally/deliberately sold to _pregnant women_ , only for each baby to be born with fucked up limbs. All of them.

@SuperPerry1000

4 ай бұрын

...One of those is VERY much not like the others lol

@derrickferguson9998

4 ай бұрын

@@dexterpoindexter3583 can’t wait to buy my ied’s at the nearby gas station!

@LoneWolf343

4 ай бұрын

Yeah, never asked the question "Then where did that radiation come from?"

I can’t imagine receiving 20,000 rads of radiation and just be GASLIGHTED by everyone around me. I would go crazy

@umi2751

4 ай бұрын

Ikr? Medical gaslight is absurd. If a patient reports a damage, the staff must register and report it! I work at an hospital and, unfortunately, some people think that they'll be punished if the report such a thing, when there are literal peoples' lives at risk!

@Dan-fc1hp

4 ай бұрын

Crazy?

@ahme7424

4 ай бұрын

it’s actually disgusting how much medical gaslighting happens

@jamesmeadows6297

4 ай бұрын

​@@Dan-fc1hpI was crazy once

@ChaseThePinballWizard

4 ай бұрын

"hey my chest is burning with the force of a thousand suns and all my skin is peeling off." "nah you buggin."

The fact that text with MALFUNCTION in it being displayed by the software that controls a radiation machine didn't at least discourage the operators from proceeding is absolutely insane.

@user-sl5zk3vc5p

5 ай бұрын

as soon as it said MALFUNCTION, it should have been reset and restarted, all would have been gone well

@bosstowndynamics5488

4 ай бұрын

You'd be surprised how many completely benign errors that critical equipment throws - they *should* have formally troubleshooted them but I wouldn't want to be too harsh on a group of not-actually-very-well-paid technicians who were likely under substantial time pressure and had been given *no* training on how to troubleshoot these devices, or even a reference as to what the errors mean.

@tncorgi92

4 ай бұрын

​@@bosstowndynamics5488yup, management is more concerned with the bottom line, and having cash-producing equipment sitting idle is the last thing they'd want. Safety they say is paramount... until it's not.

@nikhilshah9060

4 ай бұрын

You have NO idea how the technician-level people work, do you? They are just instructed what buttons to press and what to do when some error comes up - they have NO technical expertise or knowledge to know the consequences. It's not their fault, that's how business works. This is entirely the fault of the manufacturer. Any system must, must, must be idiot proof assuming that a monkey is going to operate the machine and under no situation will such a malfunction happen. God we're talking lives here!

@highquality5140

4 ай бұрын

there was only one dangerous malfunction out of 64, so the likelihood is that the operators experienced many false alarms before

A very clear example of, "yes, a medical company will absolutely put profit above your life."

@BowTie8Bit

Жыл бұрын

8 billion people on the planet, they wills say. It's not a big loss if one person dies, but affect the company and many more people will be drastically affected when we lay them off, they will say.

@stab74

Жыл бұрын

And nothing has changed.

@rileypowell5354

Жыл бұрын

What do you mean will? All of them just do, that is their business model. For most companies if they could just feed their employees and customers into a meat grinder and be rendered the profit expected from that person over their lifetime, that's what the company would be doing

@safe-keeper1042

Жыл бұрын

I don't know, this screams incompetence to me on every level more than malice.

@timewave02012

Жыл бұрын

The company in question is wholly owned by the Canadian government.

The idea that a machine will constantly throw up error messages but no one questions is safety even after multiple deaths.

@nostrum7278

Жыл бұрын

and its not like it was a phone that threw up error messages. it was a machine that shoots you with fucking radiation.

@Nyerguds

Жыл бұрын

Yep... as a hospital, I would at least demand a list of the meanings of all error codes before using such a machine again.

@Dowlphin

Жыл бұрын

That's the common sense part, and we could also call it empathy, integrity, self-respect. I have seen this problem many times. There are psychological studies about it, sociological studies, about authority, fear, all that. We don't seem to learn, only adapt/comply temporarily through coersion, then repeating the folly, but with even more deadly tools developed by now.

@ZuraTheCat

Жыл бұрын

Doctors did ask for meanings behind these errors. Company never gave them. I wonder why. What was the reason they didn't want people knowing what these errors meant and also the fact that there were errors and the company knew about it in the doctor's knew about it and they just simply ignored them is a bit odd if I was going to the doctors I wouldn't want their medical equipment coming up with any errors ever and if they were I would hope that my procedure would be halted until further notice

@edmondhung6097

Жыл бұрын

@@ZuraTheCat just guessing, they got too many sensors and error sources to fit 64 error code. So error 54 maybe just some generic hardware error. So they think the machine will be not useable if it have to reset for such generic error

Honestly, the fact that the software EVEN ALLOWS YOU to procede after displaying a Malfunction 54 error is insane. But hey, I guess times just really were different in the 80's.

@Gelatinocyte2

5 ай бұрын

They really were, and effects of that are still felt to this day.

@Elemblue2

4 ай бұрын

Oh man. Just oh man. They are not different.

@homeoffice3524

4 ай бұрын

That's to. But why there was even a option to blast deadly dosages? There shouldn't be a function for that.

@qar_ty7732

4 ай бұрын

im sure software design was uh both undeveloped at the time as well as assummed that the people operating the machines would know to not

@k.m.186

4 ай бұрын

Reminds me that the worst alarm is one that is correct half the time. If it’s correct more that’s great, if it’s correct less that means if you do the opposite of what it says it’s still better than half right. When you’re working with something that gives you constant errors that don’t seem to do anything or have an explanation it’s easy to skip them because they’ve never done anything before. How often do you read terms of service when it pops up? Everyone’s used to clicking past them. Documentation and Testing is Everything in software ethics, what a massive failure.

There is a missing piece to this story. If it was able to fail with strong-beam, Tungsten disengaged, it would also be able to fail with weak-beam, Tungsten engaged. Many patients likely got near zero radiation when they were supposed to get a couple of hundred rads.

@HuyNguyen-iv3kg

5 ай бұрын

Underrated comment

@donothesitate1198

4 ай бұрын

Do you think it killed more people by not killing their cancer cells than it did by radiation poison?

@YouKnowMeDuh

3 ай бұрын

Bingo. As there are overflow errors that went unchecked, there were likely also underflow errors present that just didn't get noticed at the time, if it ever was before the machine became antiquated.

@vitoc8454

3 ай бұрын

Lol, homeopathic radiotherapy

@AndyGoth111

3 ай бұрын

​@@HuyNguyen-iv3kgMore like: underirradiated comment

The most infuriating part of this story is how AECL repeatedly denied their machine injured someone, even though that individual clearly showed symptoms of radiation sickness. Where else did they think these individuals received a dose such as that? It borders on criminal indifference, which is a condition for a murder charge.

@ETXAlienRobot201

Жыл бұрын

and corporations, by their legal status, are never held responsible for murder or any actually serious offense. funny how that works, huh? also, their legal status prevents the individual, which in cases like this is 100% guaranteed to be top-level executives, from taking the full blame/consequence of their actions.

@0Clewi0

Жыл бұрын

@@ETXAlienRobot201 If corporations are people when is the US going to give the death penalty to one?

@saaah707

Жыл бұрын

@@ETXAlienRobot201 only in America. Look up the 2008 Chinese milk scandal. Executives got death sentences for their negligence

@VestigeFinder

Жыл бұрын

capitalism moment

@Andrew-wv7qp

Жыл бұрын

@@VestigeFinder such a situation could also happen in communism. Imagine if this was China, and AECL was owned by a high ranking member of the CCP. Probably thousands would die and nothing would come of it.

I’m a Software Quality Engineer for a medical company and sent this video to my team. It’s important to see the real life effects that can happen if we don’t do our jobs right.

@Lunar_Blacksmith

Жыл бұрын

I assume this would be very sobering in your position. I hope they take it seriously and to heart.

@emilysmith2965

Жыл бұрын

Thank you for your advocacy!

@danarosenthal9472

Жыл бұрын

You potentially saved lives doing that. Congrats and please don't stop being an awesome and thoughtful human being

@joespartan1305

Жыл бұрын

Thank you.

@duckfilms3662

Жыл бұрын

You are a good boss.

The fact that AECL didn’t investigate it themselves, and it took a hospital to do the diagnostics work for them, is haunting.

@JackTheripper911

6 ай бұрын

No, its infuriating.

@trevorrogers95

4 ай бұрын

Negligent, irresponsible, a dereliction of duty.

@Sockem1223

4 ай бұрын

Victims were not shareholders

@davem.4903

4 ай бұрын

​​​@@Sockem1223Shareholders are the Canadian State; AECL is what we call a Crown corporation.

@Jay-pi5vq

3 ай бұрын

Don’t work in a hospital, but as a software engineer, at my job the number of times we have to tell a vendor what is wrong with their own product is extremely alarming.

The fact that you just had to pres "p" to proceed and it worked regardless of error is craaaazy. My phone won't even let me uninstall an app without confirming that's what I meant to do

@what-hq1gl

4 ай бұрын

frr

Rather than a horror story about technology, this is a horror story about cynicism, selfishness and apathy

@justalittleguywithsomeproz1162

Жыл бұрын

almost as if technology is a blessing until someone who can decides to make it a curse. it's not the technology, it's always the operator who fucks up. because even if the machine isn't running properly, it's up to the operator to fix it

@Tulanir1

Жыл бұрын

@@justalittleguywithsomeproz1162 Ok, explain how the operator could "fix it" then bro. The software did not change the beam type after they had explicitly typed it in. It instead gave an obscure error code that did not exist in any manual, and lied about the beam type being changed (displaying the wrong type). You are not going to turn this around on the operators.

@Palladiumavoid

Жыл бұрын

@vacuum sealed Garfield capitalism amiright

@GuntramEverum

Жыл бұрын

@@Tulanir1 You are on the same page I think, just labeling the operator differently. In this case, no it was not the doctors working with the device, it was the people greenlighting its use over and over again after it was proven unsafe.

@minecraftjack6439

Жыл бұрын

@@GuntramEverum In this case it was mostly the fault of the company that developed it and denied the malfunctions and those who did not force them to fix it properly. The operators could be blamed for continuing to use the machine after it was found to be dangerous but they were told the machine was safe

When you get immediately get a call telling you to “stop making claims” after calling about a concern, it’s a clear indication that something is terribly wrong and you’re dealing with Evil.

@derkevevin

Жыл бұрын

You would get that call if the claims weren't true, too. And it's reasonable for companies to try and avoid PR disasters over small mistakes that happen, but of course when knowingly putting human lives at risk, that's a completely different story.

@HenriFaust

Жыл бұрын

Most of the time corporations are dealing with malicious rumors spread by their competitors or activists, so it makes sense for them to start with cease-and-desist.

@orppranator5230

Жыл бұрын

@@derkevevin Wrong. Non-evil would say they are investigating the matter, instead of just telling you to shut up.

@Pluto137

Жыл бұрын

@@derkevevin fair , things like that can easily be falsified and used for worse evil. Just plain blackmail even

@dreadengineer

Жыл бұрын

​@@derkevevin yeah that is a very valid point -- there's no shortcut to good judgment. For every situation like this where there is a true problem, there are 100 activists crying wolf about whatever particular thing they're biased against. An unrelated example I know from a past job: nuclear power plants and the NRC have a (healthy) culture of extreme safety paranoia and publicly report every minor problem or employee failure, e.g. "this widget was supposed to be inspected every week, but was mistakenly only inspected every 2 weeks. The personnel involved have received corrective training." Anti-nuclear activist groups will read those self-published reports, and then repeat them in inflammatory language as if they've uncovered some sort of conspiracy. "DOZENS of safety issues reported!!!! Donate to us today so we can keep up this good work!"

Small correction. Single byte of memory could only tick-up as far as 255, not 256. When it ATTEMPTS to do 256 it becomes 0.

@RedLuigiE

3 ай бұрын

Was going to point this out, thank you.

Im more impressed to hear that the operators used to discard the errors, even without even knowing his meaning. Is really shocking how lightly they treated a machine capable of emitting radiation

@emmettmcnally740

4 ай бұрын

It feels weird, because when I get an error message on my registers at work, I'm there going through the diagnostics I myself can do immediately, and if those don't work, I'm on the phone with IT to see if they have any remote fixes they can do, and if those don't work, a work orders put out for it to be fixed ASAP. And that's just for a register at a regular old retail job, no lives on the line if it malfunctions or crashes in the middle of a transaction, just an inconvenience while I move the customer to another register

@streamerssaymyname

4 ай бұрын

Ok but if you get them several times a day and when you call the pros who sold you the item tell you to just press P to proceed (which they surely did because instead of taking time to explain each malfunction, all technicians would do the same action, pressing P to proceed) it would not be so concerning after a week of it.

@bosstowndynamics5488

4 ай бұрын

The immediate predecessors to the machine in question were physically incapable of delivering dangerous doses of radiation, in that context, and knowing that many of those errors were benign (the Therac 25 killed 6 people but each unit was throwing on average 4 errors every single day) I can absolutely see a tech not realising that some of those errors could be dangerous.

@pappanalab

4 ай бұрын

It was desensitization. If humans experiences something repeatedly sometimes we can get used to the most horrifying things. They were getting errors multiple times a day and usually they didn’t end up with dead people. Why would they assume it would be any different for this particular error message?

@dylan-nguyen

3 ай бұрын

@@pappanalabI used to work hospital IT in 2018 There are still 100s of error codes in 2018 and people just hit ok or reboot the machine 1/3 of the people I helped didn’t even know there’s a computer attached to the monitor

5 orders of magnitude safer doesn't make me think: "oh that is safe!" Instead it makes me go: "How fking unsafe was it that it could easily be made 5 orders safer!!"

@Kyoobur9000

Жыл бұрын

Good news! Its failure rate has been reduced from 99.99999% to only 99%!

@vanleeuwenhoek

Жыл бұрын

Reminds me of the Errol Morris interview of Training Check Airman, Denny Fitch. Fascinating story.

@HenriFaust

Жыл бұрын

Math is not your strong suit. You had better avoid statistics.

@narrowhead

Жыл бұрын

@@HenriFaust L + ratio

@azareii

Жыл бұрын

@@HenriFaust Maybe try explaining the fallacy, rather than just insulting their intelligence.

My dad is a software engineer and he always tells me that "computers are only as smart as the guy that programmed it" I think this is a prime example

@SagBobet

Жыл бұрын

Exactly. This is why modern medical devices have a standardized development and testing process dictated by the FDA so that you don't have to rely on the one guy.

@renard6012

Жыл бұрын

Best thing about computers is that they do exactly what you tell them to do. Worst thing about computers is that they do exactly what you tell them to do.

@deeznuts23yearsago

Жыл бұрын

The best thing about computers? They do exactly as you tell them The worst thing about computers? They do exactly as you tell them

@rictr7421

Жыл бұрын

When the team finds excuses for not writing tests, because of time constraints. 🤦🏽‍♂️

@Reddragon_12345

Жыл бұрын

They can think the Same,but faster, and memorize more possible situations , so technicly, , they are smarter (like chess machines)

As a software developer, I can't believe that my minor changes to accounting programs and websites are given 1,000,000 times more scrutiny and thorough testing than a machine blasting people with radiation. 😢 Devastating and infuriating and unthinkable the total disregard for human lives displayed by AECL even after multiple accidents were reported!! This could have been prevented or stopped early in so many different ways.

@donothesitate1198

4 ай бұрын

You forgot that you're doing software development 40 years later with much stricter safety standards

@franciscocota6440

3 ай бұрын

@@donothesitate1198 Mostly because of cases like this one. This happened throughout all the existing industries back then. In truth, given how many scientific and industrial development leaps were done at the time, a lot was unknown and several horrendous events happened. There was a lot to learn, but hubris an ignorance often got the better of people.

My mom's best friend recieved too much radiation for her brain cancer a few years ago. She went in completely normal, talking and all (this was supposed to be her last dose. The tumor was gone but the doctor recomended one more to be certain). After the radiation she was pushed out in a wheelchair and was brain dead. She went in talking. Came out brain dead. The doctors eventually took her off of life support with her daughter's consent some months later. This happend in Canda by the way. This happens ALL OF THE TIME and so few are talking about it.

@kingsoonkit9234

6 ай бұрын

That sounds fucking terrible. So sorry to hear that

@michaelparker7676

6 ай бұрын

I know. I know. The unwashed masses of lab rats will make real contributions to tech for the elite.

@2jpu524

6 ай бұрын

Do you know the exact or approximate date and hospital?

@emotionalfriendone43

6 ай бұрын

Talking about it? You guys kill people and call it treatment. 🇨🇦

@miguelm203

6 ай бұрын

More info is requiered!

The fact that nobody stopped using this machine after 5 deaths is unbelievable

@SocialLocust

Жыл бұрын

It doesn't seem that unbelievable to me. Constantly seeing medication commercials showing smiling faces while saying that death is a possible side-effect.

@DouglasSilva-bq4xq

Жыл бұрын

5 seems like a small number if you imagine that information was not wide spread as it is today. but still a big uff

@killerzer0x74

Жыл бұрын

Thats American doctors for ya

@alyssahopson5926

Жыл бұрын

@@killerzer0x74 bro... this didn't just take place in america. it literally mentioned canada, among others

@OBSMProductions

Жыл бұрын

@@DouglasSilva-bq4xq Yeah it was the 1980's, if this happened today it would make headlines probably after the first victim

My father was treated with a Therac-25 in late 1985. He died, 15 years later, from radiation-related complications and did not survive to see me graduate high school. Seeing this, all that time later, I consider it ironic that I ended up going into software development, not knowing the history of the Therac-25 until today.

@Shiturd45

10 ай бұрын

I’m sorry for your loss, my dad died in 2022 while I was in college and it’s been lonely without him

@sarahfay5280

10 ай бұрын

@@Shiturd45 I kind of got used to him being gone. I think the hardest part of it all was that the cancer was in his brain, and damage to his brain structures was what got him, but long before he died, he changed in ways that made him very difficult to be around. I hear stories of how he used to be a great man, but I never got to see that. Everyone else mourned someone I'd never met.

@authenticallyyou7475

9 ай бұрын

This brought me to tears. I hope you have had and will continue to have a very successful and impactful career.

@sarahfay5280

9 ай бұрын

@@authenticallyyou7475 Currently working with games industry veteran friends of mine who are tired of how things are done, these days, and our first game as a studio should be coming out in Q2 or Q3 of 2025, if all goes according to plan.

@jammies1431

9 ай бұрын

@@sarahfay5280I’m so sorry for your loss. That must have been so hard. I experienced something similar with my father’s confusion and irritability at the end of his fight with lung cancer. I can’t imagine not being able to remember what he used to be like.

It infuriates me whenever people, be it nurses or technical assistants or technicians, in the medical field ignore a patient's discomfort or even severe pain as if nothing had happened. Just because you can't think of what went wrong doesn't mean everything went right.

These poor women, already suffering from cancer, cervical cancer being one of the worst, and then to be shot with radiation comparable to some of the worlds worst disasters..... it's just sick.

As they say: "all labor laws and safety standards are written in blood" Thanks for another amazing video!

@australiananarchist480

Жыл бұрын

I hate labour and safety laws

@Jemini4228

Жыл бұрын

Why? Because they save lives, stop injury and stop employers screwing over their workers? Because private companies sure as hell wouldn't have come to treat employees well without being told to.

@Wulfslove

Жыл бұрын

@@australiananarchist480 Then you are either a very selfish person who never has to work in dangerous situations, or you are a complete idiot.

@scaper8

Жыл бұрын

@@australiananarchist480 If you're actually an anarchist, you have a piss-poor understanding of how and why those labor and safety laws came to be.

@zerberus_ms

Жыл бұрын

There are so many blood soaked laws and safety protocols in the army... I remember that pretty much every time we heard a safety something, it was accompanied by someone not doing that and someone getting killed. Remember everyone, a gun is a weapon that kills. Don't fear it, but be careful.

The idea that people saw a malfunction, didn't know what that meant and just went ahead in a radioactive treatment scares me

@windsofmarchjourneyperrytr2823

Жыл бұрын

Wasn't stupid proof

@shan8130

Жыл бұрын

It’s actually unbelievable that you can go through med school and not have any common sense.

@sixthsecond

Жыл бұрын

I honestly hope all who did are haunted by what happened to these people

@rythenx

Жыл бұрын

@@sixthsecond I less blame the operators and more blame the people who trained the technicians and the operating manuals for the device. If you face multiple error messages daily, the company tells you not to worry about them and to proceed, and nothing documents what the error messages mean or what action you should take, it's hardly the technician's fault for proceeding with their job. The company who made the software should not have even allowed technicians to bypass the error codes and proceed. Or they should have provided readable messages saying what the technician should do when the error happens. Or failing that, have a book with the error codes that can be looked up and what action should be taken. Sure it's easy to say that the technicians should have used common sense in hindsight and not ignored error codes but when nobody is telling you the severity of the errors or what you should do about them, that is the fault of the trainers from the hardware company and the documentation. There was also a failure in management. The fact that the devices continued to be used after multiple failures resulting in death is baffling to me but this again would not be up to the technicians.

@rythenx

Жыл бұрын

@@shan8130 Technicians who operate xrays and other types of radiation devices generally don't go through medschool if that's what you mean. It's generally a 2 year college program. Technicians aren't MDs. However even if they were MDs, I'm not sure how much blame I'd assign to the operators.. The fact that so many error messages happened constantly, none of them were documented anywhere, none of them had any steps that the operator should take, and some of them were more severe than others without actually indicating which ones were serious and which weren't puts the majority of the blame for this with the company who developed it and trained technicians to use it.

"You dont think of software as something being able to fail." As a scripter, i'm more suspicious if it doesn't.

@PhoenixRBLX-YT

Ай бұрын

what did he mean by that sentence tho god-

@alexjames7144

12 күн бұрын

Most people are idiots, I approach all things with the understanding that most people involved in the process were idiots, with varying degrees of good intentions.

This makes me so sad. These people were already struggling with cancer and they end up getting killed by a machine that’s meant to help them. It makes me so angry because they were already dealing with so much and had to go through additional problems meanwhile nothing was being done

The most disgusting part of this story is how the company got away with it. It's a footnote in history despite how many lives they destroyed.

@Youvko

Жыл бұрын

Yeah. There should be a law, to sue such companies. On other hand making of medical equipment is a really hard thing. Obviously a chance of being sued will increase a price of already very expensive equipment. And people will blame doctors for overpriced treatment(part of price is of course a salary of doctor).

@Youvko

Жыл бұрын

If you have a possibility to go to jail for a mistake while making something, then the price of this thing will rocket jump in to the sky.

@retronymph

Жыл бұрын

They didn't destroy them. They gave them a torturous end.

@adriatic.vineyards

Жыл бұрын

What do you expect? America doesn't prosecute c o m p a n i e s. Even though they can be as psychopathically heartless, deadly, and remorselessly prone to recidivism, as the country's worst serial killers.

@adriatic.vineyards

Жыл бұрын

​@@Youvko The risk of a lawsuit is already there. The company and hospital *were* sued, just not prosecuted. The argument that greater oversight and accountability, as well as increased safety protocol, upheld with legal and criminal ramifications, will lead to increased costs to the consumer, is beyond me. These are changes that can only benefit people. If instigating them puts them out of the financial reach of people, then the next changes need to be structural and aimed at the health care system itself.

As a software engineer, I'm utterly aghast that the machine was actively responding to settings while they were being entered rather than enter all the settings, machine configures itself, double checks all the sensors, THEN gives the user access to the go button.

@Gfious

Жыл бұрын

I had to do an assignment for a college class in which I had to make a vending machine for drinks, with the possibility to add from 0 to 4 ice cubes, and 3 different drinks. It was on VHDL, and we were programming an FPGA board. When me and my mate were presenting the project, my professor started asking why we implemented a button to confirm our selection, that it was not needed and "too many buttons". I explained my reasoning (it was my idea to implement it), stating that because we need to choose the drink, the ice and since the board had limited selection, that it was better to had a confirmation button so that a "costumer" would not make a mistake and end up with something they didn't want. His solution was to put a single input for beverage selection (which would mean it would have to loop around in case you wanted like the 1st drink and you were on the second). Didn't even suggest to have it starting "pouring" the drink after a set amount of time after drink selection. I genuinely was surprised how little that professor cared about something that could cause a problem that could have a simple solution. Got penalized in my grade for that as well. It is somewhat weirdly and morbidly funny how I've heard this and several other stories about how software errors have caused deaths/economic disasters in several classes for quite a while in college, just to have a professor not care about it at all.

@TRAMP-oline

Жыл бұрын

@@Gfious Was your education at least free, or did you spend money on that experience?

@Gfious

Жыл бұрын

@@TRAMP-oline have to pay for it. Not that much (less than 100$ montlhy), but it still is a strain.

@TheBigQQ69420

Жыл бұрын

@@Gfious I mean, honestly, they're preparing you for the real world. This incident was a company squeezing pennies hiring some hobber instead of someone qualified, and your extra bells and whistles, might please the customer, it wouldn't please who you were working for because it cost more than his solution... which is sad, but that's the reality. This practice didn't end what's in this video either; it's still going strong.

@greggv8

Жыл бұрын

@@Gfious You should have told him that he was insisting you create the mixed drinks equivalent of the Therac 25, where a person impatient for a drink could end up with who knows what in the glass rather than what they wanted.

If you've touched any amount of coding software. Scratch even (intro to coding in HS) You'll know that coding is 30% writing new code, 60% fixing that code, 10% trying not to go insane chasing the new bugs found in your fixes.

@aidenpearce5275

4 ай бұрын

"99 little bugs in the code, 99 little bugs in the code, take one down, patch it around, 127 bugs in the code"

The early days of computing were famous for "computers don't make errors"

@rhymereason3449

6 ай бұрын

The computer didn't make an error... the software developers and the machine operators did by ignoring error conditions...

@naturalvids1200

4 ай бұрын

He’s right ⬆️ the computer is never wrong, it’s the programmer

@rhymereason3449

4 ай бұрын

@@naturalvids1200 Computers can be wrong - it's just extremely rare. That's why critical systems like space missions will have 3 computers where 2 must agree. As a 40 year career programmer I myself have seen one case were a random bit-flip passed CRC checks and caused a wrong answer.

@theshadowherself

4 ай бұрын

@@naturalvids1200 The computer *can* be wrong... if cosmic rays flip a bit. It's a joke amongst my software engineering team that inexplicable errors are caused by "sunspots", ie cosmic rays.

@Sha_Fermo

3 ай бұрын

It is funny how this rings so close to, “Guns don’t kill people, people kill people” though…just an observation.

you ever just hire a HOBBYIST to code a state-of-the-art medical device capable of creating 20,000 rads? Edit: I certainly don't blame this entire scenario on the coder, but someone along the line had to have realized maybe they shouldn't reuse code for such a dangerous machine?

@morgansearle3912

Жыл бұрын

To be clear, he coded a DIFFERENT device, and then they just reused the code for this machine as-is. It's even less safe than you're making it sound 🤣

@futuza

Жыл бұрын

The most atrocious thing here, was not his programming, but the horrible management practices of how they handled the code base. It sounds like they didn't even try to code review it, or test it beyond seeing if it "worked".

@joellandry2406

Жыл бұрын

“Science”

@Dowlphin

Жыл бұрын

That's quite rad.

@Eibarwoman

Жыл бұрын

It was a very different time in the world of computing when the only people who could afford it were wealthy hobbyists and government institutions.

The guy who got up and started banging on the door has to be the most spine chilling radiation story ive ever heard Imagine being locked in a room with a malfunctioning radiation machine and being bombed repeatedly with painful waves over and over again while you couldnt be heard because of a intercom malfunction

@bswihart1

Жыл бұрын

Intercom code was Fd up too!!!

@sexygirlmax2019

Жыл бұрын

thats crazy...whenever i bave MRIs they give me a little button to press in case for some reason they cant hear me.

@AMDeZani

Жыл бұрын

@@sexygirlmax2019 And I reckon incidents like the Therac 25 are precisely why.

@shan8130

Жыл бұрын

Okay can we talk about the intercom malfunction? There’s probably a 100 different things that could happen in that room that would warrant REQUIRING an intercom. I get it, stupid things happen all the time in hospitals, but I feel like a soundproof room with no way to communicate is probably not a great idea even if it’s unlikely something’s gonna go wrong.

Machine: "MALFUNCTION!!" Operators: "Nothing to see here...." presses continue button. ☢🤦‍♂

@kaneriseley3124

3 ай бұрын

"it's only radiation"

@victorm7503

3 ай бұрын

💀

@catalayalafaye5337

25 күн бұрын

Here's the thing: bc it was genuinely nothing to see there. 6 instances were mentioned here (not all caused by the same malfunction) but the therac would actually throw malfunction messages daily... and nothing happened. They were told it was safe, and normally it also was. The therac normalized these error messages as nothing dangerous

@bennowakowski3099

10 күн бұрын

@@catalayalafaye5337 exactly. And that was exactly the problem: there was ZERO distinction between harmless errors and dangerous ones.

this is genuinely saddening. I feel terrible for every patient and their relatives that got their lives ruined by this machine.

i'm a metallurgist. computers are also used in metallurgy. yes, having someone who doesn't understand what is actually psysically going on write software, ALWAYS ends in disaster.

@petergamache5368

Жыл бұрын

Test-Driven Development addresses this issue pretty well. Have someone knowledgeable involved in writing the acceptance tests - and fuzz test all inputs!

@JenkemSuperfan

Жыл бұрын

@MY DOG SAYS BJÖRK actually that was due to a design change in the aircraft. The addition of new engines mounted in different positions without any update to software caused the issue.

@Nono-hk3is

Жыл бұрын

@@JenkemSuperfan actually it was caused by deliberate intent to mislead airline operators into believing that no substantive change had been made to an aircraft variant from its baseline model, when in fact it it should have been considered a different aircraft with the unique pilot qualification and service requirements

@sarahsmith840

Жыл бұрын

@@JenkemSuperfan Um, no. The design change did change the aerodynamic of the craft anf software was written to compensate. That software, combined with only checking one AoA sensor, caused the crashes.

@JenkemSuperfan

Жыл бұрын

@@sarahsmith840 the software was designed to deal with an airframe operating different engines in a different position and either not enough or no adjustments were made to the sofware to compensate

8:02 breaking news: company that would lose millions of dollars if their product caused injury and death have concluded that their product did not cause injury and death.

@Bob-the-1-and-only-blob-fish

2 ай бұрын

And in recent news research supports that the sky may be blue

I love how they immediately took the defensive route instead of taking this seriously as the kneejerk reaction. I have a massive distrust of the medical field, so many people are too full of themselves to think they could ever have a possibility of being imperfect because they have a shiny piece of paper. They let the face their job is to save lives go to their head (and pockets) and end up doing more harm than good

"We don't think of software as something that can fail" As a software engineer, this is so deeply concerning to me lmao. I don't know how to tell you this, but our entire field is bad at what we do and you should not trust us for very important things

@mmyz7

Жыл бұрын

It's not a "bad" field, but computers are just unpredictable in weird ways and coding is hard as shit as stuff you wouldn't imagine could happens, happens. So you need to continuously solve for bugs until your product works enough

@CheeseypiPlays

Жыл бұрын

@@mmyz7 didn't say it was a bad field, I love it, and I was just referencing an xkcd lol

@qps9380

Жыл бұрын

@@mmyz7 That's just wrong. Computers are predictable in the best possible way. They will do exactly what you tell them, to the letter, barring rare events like bit flips from radiation. The issue, like is often the case, is human error.

@richardjafrate5124

Жыл бұрын

Many software developers are completely clueless when it comes to machine control. There is a long history of putting a program on a bunch of punched cards, loading them on to a reader, and waiting until the computer gets around to running the program once through and creating a printout of the results, usually by the next day. This is quite different than running said program 100 times per second, as is common in machine control applications. VT100 terminals and VAX computers were only a few years removed from the height the punched card era. Vax computers were never meant for low level machine control (real time positioning and activation of physical devices). Although used extensively in industrial settings, their use is typically limited to process models and retrieving orders from the business computer, while leaving the machine control part to more specialized computers. There is a current trend to use Linux and Windows PC platforms but with a specialized OS extension designed for direct control of machinery or other physical devices.. The punched card paradigm very much resembles the current fashionable functional/stateless programming which is a disaster waiting to happen should those techniques ever be applied to machine control, IMHO.

@JoshSweetvale

Жыл бұрын

@@richardjafrate5124 So if I can rephrase, the stupidity of the day is formatting instructions in a generalist way, making them useless for specific systems?

Okay but the lady who got a dangerous dose of radiation and lost most of that part of her body, but still survived and lived her life despite it, only to keep driving with one arm for 5 years and die in Atlanta traffic is the most Atlanta thing I’ve ever heard

@mech1x

10 ай бұрын

😭😭😭

@ponponpatapon9670

10 ай бұрын

when the fuck will people see that cars and roads are objectively death traps

It's so ironic that a device that was supposed to help people was actually killing them... The fact that those people were cancer patients makes it even more tragic to me.They've already been through so much pain and met with such cruel fate.

@oldtwinsna8347

5 ай бұрын

But it didn't end there as when the litigation started the hospital had a barrage of lawyers with the specific intent to destroy their (the victims) lives even further, digging up dirt on whatever they could find, and then making it turn into their fault, that they were the evil people.

This was a case study in my Computer Science Ethics class because the company paid off the earliest families to keep them quiet. What struck me was they took an industrial machine and put the hardware safeties in place instead of fundamentally limiting the power. They didn't want two machines. They wanted one machine that had two modes. So many families destroyed because they didn't want to manufacture an appropriate device.

@bennowakowski3099

10 күн бұрын

Putting two modes with different power levels is fine, in isolation. The high does was necessary for the x-ray treatment. The actual problem was more just the fact that there was a shocking lack of safety checks and failsaves and hardware interlocks. ALL of those, and more, should have been present.

@morganseppy5180

10 күн бұрын

@@bennowakowski3099 in isolation is doing a lot of heavy lifting here. The lethal doses should have had a physical restraint when shipping out "xray machines". Edit: For example, maybe they could have limited the power, so even in high power mode, it wouldn't produce so much. The company was too lazy (read:cheap) to either create separate machines or to modify existing ones so no harm could come to the patient accidentally. The 80s were wild.

Half Life Histories is legit some of the most interesting documentary content on KZread, love it.

@kaylemcooper452

Жыл бұрын

We need to get this man an award for this series

@zerberus_ms

Жыл бұрын

Not just on KZread.

@drungus3763

Жыл бұрын

​“Those who cannot remember the past are condemned to repeat it.” - George Santayana, The Life of Reason, 1905.

@lillywho

Жыл бұрын

It takes a lot of effort not to make a VALVe joke at this point.

@zerberus_ms

Жыл бұрын

@@lillywho I wish it was connected somehow. We'd only have 2 victims if that was the case.

Man, imagine if we knew who the person who wrote the code was he'd probably get all the blame when really it was the company who was refusing to dig any deeper to help fix this issue. I think it might be for the best at this hobbyist code engineer is unnamed because I don't think it was their fault. Their code was perfect in theory it just wasn't tested properly by the company and the company made very poor decisions on how to proceed

@youtube_username_

Жыл бұрын

Strongly agree.

@kf4293

Жыл бұрын

That was exactly what I was thinking. They would have totally thrown him under the bus. 😒

@ZuraTheCat

Жыл бұрын

@rohanorton I'm glad that they deemed it not his responsibility because in all truth it was the company's responsibility to take action and fix it. Their product not his

@StruggleButtons

Жыл бұрын

As someone who writes code for machinery; you always, always, always test what you wrote. And you don’t just test to see that it works, you try to break it. Then you get more people to test out the code after you think you got all the bugs out.

@ZuraTheCat

Жыл бұрын

@@StruggleButtons yeah I totally agree. The code is made by a hobbyist. He might not even ever seen the machine. Also, it was made for an older machine and poured it to a newer machine and I doubt that they even thought about testing it, which is a shame. But so long as we remember this, hopefully we'll never repeat it and we'll learn from the past

I’ve gotta say, as someone who’s been breaking video games since I knew how to play them and who’s been doing my own computer maintenance and the maintainence for my family’s computers for almost 10 years now, hearing “people don’t assume software is something that can fail” was honestly really funny.

@vibovitold

4 ай бұрын

People aren't surprised by the fact that video games are buggy but they will believe that the code handling eg. their banking transactions must be rock solid.

Therac-25 was one of the cases we studied back in school. After learning that, I always test my codes; even my codes are not doing something critical.

The real error is the code was written for a different machine. If the previous machine T-20 had hardware fail-safes and the T-25 did not, then the original code would be relying on those fail-safe devices. How many T-20s caused radiation accidents?

@bidlis

Жыл бұрын

i dont know.. sorry :(

@mr.voidroy6869

Жыл бұрын

Idk is my name T-20?

@theoddball3850

Жыл бұрын

No clue, but I'm sure the T-800 has much better performance values.

@sdfkjgh

Жыл бұрын

@@theoddball3850: Well, sure, but there's the whole "They were all designed to hunt out and exterminate all human life" thing; if we overlook our inherent biases, then I'm sure we'd all agree with Skynet's annual performance reviews of the entire line. Well, mebbe not the _entire_ line; I heard there was one single unit that was reprogrammed to go rogue.

@neoqwerty

Жыл бұрын

It's almost exactly what investigations found. i still have some of the evaluation reports on this and the medtech ethics class study on the case, but the THERAC-20 did not have radiation accidents. There was a bug in the T-20's software locks that made the code ignore the failsafes, but the hardware locks were functional. They removed the hardware checks for the THERAC-25.

Hats off to the guys who stayed all weekend to try and get the software to fail again. These guys were doing the job of the company that built this infernal machine.

@karmendimas5274

8 ай бұрын

THUMBS UP TO WHAT???? THOSE PEOPLE ARE DEAD!! and the guys who stayed to figure it out did no good!

@DelGTAGrndrs

7 ай бұрын

@@karmendimas5274remove the m in your first name. That’s what you’re acting like

@Yreev

7 ай бұрын

​@@DelGTAGrndrsthat's a good one.

@phoenix9531

7 ай бұрын

​@@karmendimas5274So... You can't read?

@karmendimas5274

7 ай бұрын

so, you cannot comprehend?@@phoenix9531

Living in an age where even motherboards have blinking lights that are associated with specific errors clearly laid out in a manual, the idea of a piece of radioactive surgical equipment having error messages and no way to know what the error meant is horrifying

@bltzcstrnx

2 ай бұрын

High-end motherboards even have error codes indicators instead of just lights.

good god. i cant imagine how far the damage spread. not just the victims of this negligence, but their families, as well as the drs that thought they were just doing what they were suppose to, had to have been so devastated.

My dad (an engineer) always says: "Common sense is not taught in schools" One of his industrial mentors use to told him, referring to any control panel: "If the light is on, it only means the light is on." Then he and his coworkers were ordered to check by hand whatever the hell was going on. No wonder why this story is mandatory for certain careers

@Barnaclebeard

Жыл бұрын

"Common sense" is what one is forced to rely on when one doesn't know any better. It is what lies beyond rigorous and tested methodology. If engineers are depending on their common sense, they should fucking stop, back away, and tell the client that they are not the engineer for the job or the job is beyond modern engineering. Perhaps that is what an engineer DID do, and why they used some highschool kid or whatever their cover story is.

@Rezu55

Жыл бұрын

@@Barnaclebeard You misunderstood their point entirely. The point is "common sense" in a situation like this where an engineer doesn't understand the technology they're dealing with, it's to get help to solve the problem, especially when human lives are at risk. Common sense isn't a synonym for "just try pressing random buttons", in fact, it's quite the opposite lol.

@TheR4gnos

Жыл бұрын

@@Rezu55 I think you're misunderstanding.. Common sense simply means sensibilities that are "common" to everyone (of an average intelligence). To some extent it might mean "flipping random switches" or, more beneficially it may mean having the awareness to back away from the system. The statement "common sense isn't taught in schools any more" is a prejudicial misnomer. It doesn't mean the same to anyone and it really never has been..

@Barnaclebeard

Жыл бұрын

@@Rezu55 You don't hire an engineer so that they can practice common sense. Anyone can do that. It's an engineer's job exactly never to use their common sense, because people are stupid. If an engineer is using common sense, people die. That's the end of it. It's not "common sense" to refuse to work on a project with human lives at risk without an adequate history of testing; that's engineering.

@Gutbomber

Жыл бұрын

"Common Sense" by Thomas Paine, that's a good book. That's how I have mine.

As a medical software engineer who often curses about all the documentation, validation and verification that has to be done, I am yet again reminded of why this is all necessary today. Great video 👍

@Tantejuju65

Жыл бұрын

As a technical writer of software and hardware, documentation and USING it in QA testing is critical. This concurrent activity finds many opportunities for clarity and correction.

@alphanumeric6582

Жыл бұрын

As a high schooler who used to program in grade 9 & 10, diagnosing errors and oversights to a software and fixing them is IMPORTANT for me to keep my grades. Teaches you to be mindful of your code at a very young age

@keisukesakamori

Жыл бұрын

as a bystander that hasnt graduated primary school. AKSJDIAWOESDHUAKSHJCOUAHDELJAHO

@hankkingsley9300

Жыл бұрын

@@alphanumeric6582 and most of your idiots in charge don't understand that you can write a program and if even if you let it sit for a while step back and look at it you're going to miss some glaring error that someone else could see and fix easily it's just the nature of the Beast

@thatsawesome2060

Жыл бұрын

Documentation and validation? Boeing: never heard about it, all regulator in our pocket, by the way here our new MCAS software.

We were required to learned about this in a Computer Science ethics course during my college years. Knowing things like this is extremely important

Every software has errors in it. It takes years to get most of the bugs out, no large software is ever error free.

As a software engineer, I can confirm that users doing things in the software that the developers never anticipated is one of the biggest sources of software error. This is one reason why good UAT (user acceptance testing) is so important.

@fizzinsoda

Жыл бұрын

also maybe have a big team to help code your nuclear device instead of some dude out of his mom's basement

@llrennanll

Жыл бұрын

Reminds me of the Bartender Robot joke, the developers made sure that the robot could make all the drinks in whatever quantity needed, the first real user asks the robot where is the toilet, the robot explodes.

@nogrammer

Жыл бұрын

@@fizzinsoda clinics and hospitals don't have enough money in their budgets for that, it is sad. We need to subsidize the medical industry, not the coal and fossil fuel ones.

@hakimmohamad6216

Жыл бұрын

@@fizzinsoda I guess that this problem was more common in those days when people had no clue about software programming and didn't take the matter serious. If you think that your computer magically does everything it should, you are not going to address software engineering problems with the caution that they should be addressed with. Luckily today most people in the tech industry have at least some basic idea of what computers do.

@CrimsonDoveKarting

Жыл бұрын

@@hakimmohamad6216 as a software engineer myself, no, they really don't. I left a medical device company last year because writing unit-tests were "not in the budget".

Just googled the Therac-25. As a software engineer, I couldn't believe they didn't have their code independently reviewed and they NEVER tested the unit with combination of software and hardware together! Just WTF.

@Milkmans_Son

Жыл бұрын

they tested it, they just didn't test it with operators who corrected their own input errors in 7 seconds or less.

@thewolfin

Жыл бұрын

mRNA is basically software

@scythelord

Жыл бұрын

Early computing was the wild west. Independent review didn't happen.

@hermenegildakociubinska6665

Жыл бұрын

If they didn't test for such an obvious race condition, it's as if they hadn't tested it at all. People are more careful about website layout than this company was about deadly radiation.

@nickthompson1812

Жыл бұрын

There are reasons we have restrictions and regulations today. Sad to say, but somebody had to figure out the hard way before real change happens for the industry.

This incident has a strange similarity to several theme park accidents, notably the haunted mine drop at glenwood caverns. The software says "error", the human operator says "oh probably just a computer glitch" and clears the error. And then someone dies.

16:56 actually it's 255, the maximum decimal value 1 byte can hold is 255, the range of values 1 byte can hold is in fact 256 values from 0 to 255

"You don't think of software being able to fail" Failure should always be the first outcome you think of when coding.

@morganseppy5180

4 ай бұрын

Test-driven design starts with error handling of five right. Tedious but that's the cost of being thorough.

@kfenrisl

4 ай бұрын

No shit sherlock! that is on every coder/programmers mind but mistakes still happeen...

@vibovitold

4 ай бұрын

I think they mean the perspective of an average person, not a software developer. The general public underestimates how buggy most software is, or at least tends to assume that software in critical areas (like healthcare, finances etc.) is cut from a different cloth somehow. (It isn't)

@kashmirwillwin3124

3 ай бұрын

That's like 80% of coding these days

@sayori3939

3 ай бұрын

@@kashmirwillwin3124 java and android cof cof

The failure of the Therac-25 was taught time and time again during my CS undergrad degree. Every time, it was taught to drill one thing into our heads: software. can. kill. people.

@SagBobet

Жыл бұрын

Medical device software is heavily regulated for this very reason. Very rigorous testing and lots of documentation and approvals. Not to mention comprehensive tech support, user manuals and training.

@Layarion

Жыл бұрын

well they taught you wrong then.

@Layarion

Жыл бұрын

@@robmaelstorm23 warmer, but still not hot. the lesson to be learned here as more to do with all the people involved, not anything to do with the code. The company that shirked it off, the regulators that were lazy. those two were the ones in charge. the bill goes to them, not the software. you could remove all code and programmers from the world and you'd still find that people still make these overconfident or greedy mistake with mechanical devices.

@mushyroom9569

Жыл бұрын

Fun fact: cosmic rays can flip bits arbitrarily

@elenafriese891

Жыл бұрын

@@Layarion eh, factual, but that's still a very valuable lesson to drill into the head of someone who might otherwise treat something dangerous with flippancy

Not really a software error, but human error. You never just click proceed when the screen says 'Malfunction'

These errors are also really obvious and easy to predict from a programmer's perspective. Its unbelievable that they didn't have any second person to review the code.

This is why beta testing should NEVER be skipped. If you're developing software, here's some advice: set a layman down in front of it. Explain what to do with it, and ask them to go nuts with it to the point of mental fatigue. To try every oddball thing with it they can think of. Because if there's an issue with your code, even if you think it won't show up in your initial testing, some burned out kid will.

@aluisious

Жыл бұрын

I wrote code alone for a medical device. It couldn't hurt anyone thankfully. It had bugs if the user did weird stuff I never thought of, and my arrogant boss refused to acknowledge that I would have blind spots about how someone might use it considering I made it, knew how it worked, and intuitively wouldn't use it in ways that didn't make sense while testing.

@fawkesrocks

Жыл бұрын

Remembering your code must be written for the average user, and that users are the stupidest humans to your code is so important. People will put letters in phone number fields or say their birthday is Cactus is you let them

@Jkb2002

Жыл бұрын

As a game dev student, I have quickly come to realize how fast a user WILL break your software by doing something you may never think of. I am good at breaking games and software but when you make your own thing, you can try 1000 different things to break it but as soon as someone else who isn't making it gets their hands on it, they find something to break and abuse very fast

@KyleLyre13

Жыл бұрын

@@Jkb2002 It's so easy to have happen because you know the kind of responses the system expects and you can't see past the machine to actually witness the operation.

@Jkb2002

Жыл бұрын

@@KyleLyre13 exactly

My husband and I have been coding for a combined 90 years. We both laughed out loud at “You don't think of software being able to fail.” When it comes to software, failure is ALWAYS an option. This is a particularly egregious case though, even for the 80s. I worked for AECL as a summer student in 1983. Where I worked, a lot of code was written by non-experts. It was a wild and crazy time.

@Albtraum_TDDC

Жыл бұрын

Also any pc gamer from the last 3 decades can attest to constant crashes and stuff.

@MsAnyaBaby

Жыл бұрын

People will forget to hit X on one window and think their entire computer is frozen. It's not just error it's following steps. If it's of that importance REWRITE the whole screen don't make your own "corrections" THEY USED EDIT IN THE WORST WAY

@cloviscareca

Жыл бұрын

If you worked at AECL, probably you know the therac25 programmer. Right?

@randomnotes

Жыл бұрын

@@cloviscareca AECL has many facilities across Canada. I was in Manitoba working at a small research facility.

@cloviscareca

Жыл бұрын

@@randomnotes thanks for your answer. Have you seen a therac 25 in person?

As a software engineer, not only do I expect software to fail, I expect it to fail immediately and in spectacular fashion. Developing software is like always writing a rough draft to an essay. It’ll never truly be finished, and perfect

If an error shows up on devices this important They should literally stop, like emergency stop

13:30 “You don’t think as software as something being able to fail” As a software engineer I strongly disagree 😅

@brickface0

Жыл бұрын

Not just that, we expect it to fail. Not sure if he ever used a computer

@gerrypaolone6786

Жыл бұрын

Totally agree..When I write code "I don't think the software as something being able to WORK" xD

@bellabear653

Жыл бұрын

In the 1980s did they even have programs to check for errors?

@gerrypaolone6786

Жыл бұрын

@@bellabear653 just some more lines of code

@bellabear653

Жыл бұрын

@@gerrypaolone6786 True but important lines that can catch critical errors. Every Coder needs a Spell check.

I remember hearing this story from a professor in college. It’s a horrible story but it highlights the importance of proper software testing.

@olafzijnbuis

Жыл бұрын

Did you mean: ...importance of software DESIGN? Testing finds defects. Correct design prevents defects.

@koma-k

Жыл бұрын

@@olafzijnbuis correct design can still be implemented imperfectly.

@theonewhouploadsnothing1704

Жыл бұрын

@@olafzijnbuis Humans can still mess something up that’s done “correctly.” Same thing that childproof means, you’re kid hasn’t gotten around it YET.

@m__42

Жыл бұрын

I agree software testing is a good thing to do. However even with software testing you may not reliably discover concurrency issues, problems talking to hardware, etc. For anything that is potentially dangerous, what is more important is to have multiple safety layers, from primitive but reliably hardware interlocks to more sophisticated software interlock system. E.g. at the LHC we have a lot of software that is written by physicists who also do programming. This software can and will sometimes fail. However there are multiple layers of interlock systems behind that will ensure that the beams are immediately dumped in case safe operating conditions are violated, equipment trips, or - as a last resort - particle loss rates reach abnormal levels.

@luisenriquemunoz8793

Жыл бұрын

@@olafzijnbuis As a software tester, I can assure you that even properly designed code can contain problems. It is also very challenging to think of all the crazy scenarios a user might come up with while using the software...

This was the case study for my Software Development Ethics module in my Undergraduate Degree in Computer Science, and it hammered home that you really need to think about those harmless, quirky sounding ‘software bugs’. Ironically, my University produced the students who programmed the software that failed and turned an Adrianne 5 rocket into a gigantic firework, so clearly it didn’t work for everyone…

This video is why medical right to repair must happen. None this would have happened if the machine operators had a detailed diagram and manual that explained in great detail what each error meant.

@danielkaiser8971

5 ай бұрын

Are you kidding? The industry was barely in its newborn stage. There were not yet any standards. No one understood anything about computers back then. There was no training. There was no understanding of what to do if something didn't work. And they did have miniscule documentation, but no one would have read it because they wouldn't have understood it. No one knew anything about computers. Literally nothing.

I remember studying this as an undergrad. The software industry has come up with dozens of approaches to improve safety since these accidents, but it's not exactly a solved problem. We have really good tools for testing syntactic correctness, and some languages even allow for proving your code works in a certain way, but ultimately the problems of design correctness and semantic correctness remain difficult to solve - ie. does the design do what is needed, and is the code congruent with the design. Another way of phrasing the problem is: "Did I mean to do the right thing, and does my code do what I meant?"

@kevindaniel1337

Жыл бұрын

it seems to me the biggest change is that its now much more common to assume that software CAN do seemingly impossible problems, rather than assuming that it CAN'T.

@Donbros

Жыл бұрын

@@kevindaniel1337 and doing those impossible problems make a lot of hard tracable errors. You literally tell your managers that and they are fine

@kevindaniel1337

Жыл бұрын

@@Donbros it's unfortunate that so many lessons have to be learned the hard way.

@darrennew8211

Жыл бұрын

Especially difficult when the software is connected to a hardware device that *can* get stuck, wear out, etc. Very few people write software that accounts for brown-outs, power flickers, or other stuff where the hardware doesn't work to specs.

@kevindaniel1337

Жыл бұрын

@@darrennew8211 Excellent points yeah. Kyle mentioned that this particular device didn't have any mechanical fail safes or ways of verifying the software and the hardware were in agreement. I'm sure modern devices have safety switches and sensors in place to confirm the actual state of the machine rather than the state of the software.

I swear, way too many times throughout history, the last words said before something catastrophically fails is "there's no way this can fail".

@thomasmaughan4798

Жыл бұрын

I think the usual response is "That wasn't supposed to happen."

@davidtitanium22

Жыл бұрын

This might be inappropriate but might i say, "overconfidence is a slow and insidious killer"

@KentReynolds

Жыл бұрын

Ditto the Titanic

@trevorthieme5157

Жыл бұрын

@@davidtitanium22 That or it kills you so fast that you don't even realize that your supposed to keel over!

@emilysmith2965

Жыл бұрын

EXACTLY. “Careful consideration” is not usually all that careful.

Yea yea it only took a dozen machine failures and dozen deaths to discountinue use of Therac. Its crazy how errors were ignored as well.

Mindblowing just how many times this machine killed its patients

i had a teacher back in 1962 who had a hand that looked partially "melted". He said it was due to an x-ray malfunction that occurred when he was living in Mexico.

@JacobP81

Жыл бұрын

Why that is terrible.

@cessposter

Жыл бұрын

@@JacobP81 half melted hand.

@soupcangaming662

Жыл бұрын

Can't tell if that would look badass or sickening.

@juniperburton7693

Жыл бұрын

​@@soupcangaming662 I think cool

@cheddarsunchipsyes8144

Жыл бұрын

@@JacobP81bruh💀

The amount of leniency this company was given while its product was actively killing people is INSANE

@JoshSweetvale

8 ай бұрын

Opiates

@JoeGrant-xz1rs

8 ай бұрын

AECL is still around today and despite it’s website saying “transparency and accountability” they still deny any wrongdoing

@arinc9

7 ай бұрын

I haven't watched the video yet but this sounds like something that would happen in the U.S.

@PascalGienger

7 ай бұрын

That repeated already earlier in the US - may I bring up Radium girls? Or Asbestos?

@KrisKringle2

7 ай бұрын

@@arinc9 Oh right. Like arrogance, ignorance, and lack of foresight isn't a rife quality in humans all over the world and shit has, is, and will go on in many places around the world that just staggers belief. The radiation therapy machines that seem to be abandoned and then broken into or carted of to a junkyard. The world is polluting itself and destroying ecosystems just fine (not) without the US's involvement. Got have that palm oil so lets deforest Indonesia and turn it into one big palm tree plantation.

It's crazy to think that the same bug used to get infinite master balls or rare candy's in Pokemon Red, Blue, and Yellow was killing people less than a decade before its release.

I received my computer science degree in 2012. It was mostly a math degree but we had a technology/engineering ethics course. The therac-25 was one of the case studies

@victorm7503

3 ай бұрын

I wish they could teach this in our campuses

Glad I found this AFTER finishing my radiation treatments, my anxiety was bad enough

@j_eezus_christ_bro_chill

Жыл бұрын

I know im just a rando idiot but is everything ok or getting better?

@gchicklet

Жыл бұрын

@@j_eezus_christ_bro_chill yeah, I got covid right after I finished treatment so I feel like I've been hit by a bus, but I'm doing OK 👍 😁

@33screamingfrogs34

Жыл бұрын

we have bug testing today lol

@DieAlteistwiederda

Жыл бұрын

Understandable. My mom went through it over a decade ago and I'm pretty sure even with her being as calm as she was wouldn't have wanted to see something like this. She got a sunburn from her treatment but otherwise was just fine. Also has been cancer free for over a decade now.

@187th-Bricks

Жыл бұрын

So are you okay now?

I really like how you compared the RAD doses to other well-known disasters. Radiation is hard for most people to wrap our heads around, and big numbers are hard to put into perspective. Saying "remember that crazy disaster that scrambled this dude's insides? Well this was way more than that, focused into a tiny beam" is very effective and helps with perspective. You have a great way of distilling complicated ideas without completely dumbing it down.

@Jkb2002

Жыл бұрын

I really liked this too, helped me get a concept of just how bad these accidents were compared to some of the most insane disasters in human history. It's like the doctor on KZread who goes over cases and breaks complicated medical things into understandable bits for the average viewer. I think bring able to teach people complicated things in a simple way is a sign of real intelligence

@kevink2986

Жыл бұрын

True. They say the best experts can teach the layperson in a way they can understand.

@eleanorrobinsonedwards7090

11 ай бұрын

I thought that too, it was incredibly helpful to understand to have these cases compared to other disasters, it helps people understand the sheer scale of how badly they fucked up

@tenshi6293

11 ай бұрын

ironically, that could actually be better

@1980shello

8 ай бұрын

Reply bots

Somehow these types of videos scare me more than videos about unexplained disappearances, like the Yuba County 5 or DB Cooper.

“You don’t think of software being able to fail.” *Laughs in Fallout 76*

Fun fact: The bugs actually existed in previous models of the THERAC. Both the Therac-20 and the Therac-6 had the exact same problem but it was never an issue because there was hardware safety mechanisms in place which were deliberately removed in the Therac-25. This was noticed by a physicist, Frank Borger, because he was using the THERAC-20 for his students and it would typically blow fuses when they were using it at the beginning of the semester, then later on completely stop. This was cause they were doing that same sequence mistakenly setting it to x-ray then quickly setting it to electron. If you read the Standford report on it it has that plus much more interesting info about this! It's actually nuts what the safely standards were like back then.

@victorkhong7654

10 ай бұрын

So the hardware safeties were masking coding errors. Then when the hardware safeties were removed (cost cutting measure or blind belief that the software was safe due to assumed lack of malfunctions reported?), Therac-25 started killing people.

@cosmefulanito5933

9 ай бұрын

It is unfortunate that it can only happen in underdeveloped countries (like the United States) that always put companies ahead of their citizens.

@blearghu

9 ай бұрын

@@cosmefulanito5933 I like to say that the US is the wealthiest third-world country

@DeathnoteBB

9 ай бұрын

@@blearghuWell it’s 1st world, because that label means we were on one side of the Cold War. Developing country is what we are

@poetryflynn3712

9 ай бұрын

@@cosmefulanito5933 The company that produced the Therac-25 is a Canadian State-Funded Publicly-Traded Laboratory. Stop blaming the US for everything. Politically, Neo-Libertarian Beliefs were popular across the western world at the time. State corporatization and free market policies were wildly popular in the west including Europe, North America, China, Japan, and South Korea. The US is only around 20-50 years behind Europe because they're a literally federation twice the size of the European Confederacy.

What really fascinates and very deeply disturbs me about radiation is how it damages living tissue. Radiation doesn't destroy on a cellular level, like some viruses would. Instead, it disintegrates matter at the molecular level. Think about it. The fundamental building blocks of matter, violently ripped apart at an agonizing snail's pace. My stomach furls at the thought.

@goopah

Жыл бұрын

I had to look up "furl", and it is indeed a word. Vocabulary +1 Meaning: To become rolled up. I had heard of "unfurled" before, so that makes total sense.

@user-pr6ed3ri2k

Жыл бұрын

@@gopsaysgodwantedyoutoberap7782 who is she

@KarldorisLambley

Жыл бұрын

i know. i love it all! I am working on making a fusion reactor in my drawing room. using deuterium and about 40 thousand volts. I will be able to make my favourite type of radiation - neutron radiation. At the minute i can only make x-rays, so I am looking forward to making another type of ionising radiation. I want to 'catch them all', as it were.

@HollieMoodie

Жыл бұрын

Yeah, but if you're looking at death, you'll try anything. Including risk being microwaved by that machine. Hell, you'd stick a hot curling iron up your prison wallet if you think it would help.

@socialgutbrain7774

Жыл бұрын

@@HollieMoodie Prison wallet is a new one lmfao

200 rads to a LYMPH NODE? That’s where a LOT of immune cells are. That sounds like a last resort against metastasizing cancer, not an immediate treatment.

I don't know exactly why, but even just the phrase "Malfunction 54" (and come to think of it, all of those error messages) is intensely ominous to me, even without the knowledge of what it does. Honestly, i think its probably the succinct-ness and how quickly it pops up with nothing else to warn you, combined with the insane vagueness of just a single word and a single number. (In history, the vagueness was the reason for disregarding all of the error messages.) Humans fear what we don't know or understand.

As far as I know, only the OS was written by a hobbyist, who probably never imagined his code would be running on an actual death ray. The interface for selecting the treatment type and such was written by the manufacturer.

@adamw.8579

Жыл бұрын

Hobbyists are most brilliant engineers but require proper education. It's also my way of life. Joyful productivity is one side, but learning good habits and discipline is another side. As hobbyist I was chaotic in my early projects, but studying teached me keeping all work in order.

@callak_9974

Жыл бұрын

The other issue really though it was code written for a different machine in the same product line. With enough changes to the actual device, the code is impractical to be using without a revision.

@adamw.8579

Жыл бұрын

@@callak_9974 Yep. I think about same issue, they remove some sensors in new generation and program goes south with errors.

@electroninja8768

Жыл бұрын

@@adamw.8579 Proper education these days tends to not actually prepare people for their actual work. At the end of the day, at least 90% of what a person learns will be learned on his own, without a dedicated teacher. I have known several computer science majors with degrees from proper universities that don't know how to organize and optimize code. Some of them can't even write code without a helper program leading them along. Every software engineer has several dumpsters full of bad test code that they have produced on their way to writing better and safer code. Idk if this is fundamental to human nature, or if the educational system is just bad in general. But it is a weird trend that I have seen.

@adamw.8579

Жыл бұрын

@@electroninja8768 I have other habit: plan twice, made once. It's more effective but often not understood by employer. I'm lucky to work on contract with my former client who understand some cartefully planning hours can save many days later. Just he knows my work style.

This is why you don't just throw up an error code, but instead a plan-text explanation of what the error is

@tristantheoofer2

9 ай бұрын

yeah this or even the error number AND text saying what it is

@jesterprivilege

9 ай бұрын

If your programmer is a good one, you probably won't get errors, and if you do, they will have an explanation. Poor programers don't add notes to error codes, and you get more of them.

@TheYear-dm9op

8 ай бұрын

@@jesterprivilege I don't hink it's as simple as good and bad programmers. Usually the higher ups decide if you actually have time to implement error handling. I'm not a professional programmer, but I do some things for my company. My boss would call it waisted time to implement error management or even explanations for people who don't know how to use my little helpers.

@deathhog

8 ай бұрын

I will actually defend this a little bit. Back in the eighties, every single byte of stored memory was precious. It was common practice that error codes would be listed in a book. Adding them into the computer memory might have cost hundreds per computer. A companion book costs... maybe ten dollars. Still the fact this wasn't documented anywhere is an egregious oversight. Literally killed people.

@imluctor5997

8 ай бұрын

@@TheYear-dm9op Kinda true but it also depends on what is being programmed and for what use. I do believe it should be implemented later on.

My aunt was the first person to receive treatment via the Cyberknife, which is the modern equivalent to the Therac-25. They actually featured her in a documentary called "a day in the life of the Canadian Healthcare system."

This was very interesing as my dad passed from lung cancer in 2009. He did really well with his chemo and it gave us 9 more months with him. Then they talked my dad into doing the radiation and to this day I believe that's what helped end his life more quickly. He had a red mark where they zapped him and he complained about it hurting/burning. He went very quickly after that, within 2 weeks. He was doing ok, we even celebrated Easter with him and then he was gone. Logically I know it was the cancer, he smoked like a chimney and it was his choices that ultimately lead to the cancer but a part of me wonders if he had just said no to the radiation treatment, would we have had even longer time with him. This just gives me more to think about for both yes and no answers. Thank you for this whole series. It has done wonders for my fascination that I've had for decades.

The constant insistence that “an overdose wasn’t possible”, “there have been no other cases like this”, etc. make my blood boil. They’d rather hide the problem and avoid fines/lawsuits (which, btw, don’t cost them more than a minute fraction of the profits they make), than protect anyone. I can’t und how people could be that heartless.

@IkarosTypeAlpha

Жыл бұрын

"an overdoes wasn't possible" Yeah they should have explained how those patients suffered from radiation poisoning immediately after the treatment if it "wasn't possible" God that part pisses me off to no end

@The_Dragon_Tiamat

11 ай бұрын

Companies don't want more money, they want all of the money all of the time. If they can avoid a fine/lawsuit they will, if they can avoid making a new model with intensive testing to ensure it's safe they will. The answer to the question, "How can people be this heartless." Is and so long as capitalism stays the system we use will always be a higher profit margin. If burning your home down made them more money compared to not burning your house down they would do it.

@Hi_Im_Akward

11 ай бұрын

No kidding. It's not like radiation poisoning and sickness is a common thing. Everything comes back to radiation treatment. Honestly the families should have sued and the hospital should have sued the company. The hospital is also at fault for continuing treatment when deadly accidents were clearly continuing to happen.

@the_phantom_cat7912

10 ай бұрын

That's the nature of capitalism for you

@The_Dragon_Tiamat

10 ай бұрын

@@the_phantom_cat7912 Omega based take.

Just to be clear, a byte maxes out at 255; it can't hold a 256, and trying to advance it to 256 is what gets you the "odometer" rollover back to 0.

@blu0065

Жыл бұрын

came here to see if somebody already said this. Thanks. For some of the math behind this, the maximum value of an unsigned 8-bit integer (colloquially, uint8_t) is 255. The maximum value of any unsigned integer is (2^b - 1) where b is the number of bits. The number of values that an 8-bit integer can hold is 2^8 or 256, but the maximum is 2^8 - 1 or 255 since we count from 0 instead of 1. Some of the "safe software" that we learn in uni is to avoid using magic numbers in code and instead rely on flags, vectors, and enumerated types.

@dons6793

Жыл бұрын

Was going to say the same thing but checked comments first. Exactly right.

@samuellourenco1050

Жыл бұрын

Yup.

@pliat

Жыл бұрын

@Vive le Dominique Fabre no, because a computer cannot actually store a negative value, per se. You would be able to have 256 values aka 0-255 (0 counts) you could have -128 to 127.

@deeznuts23yearsago

Жыл бұрын

It the same with when I see people typing out IP addresses and they put above 255 and act like they are threatening And with most colour scales it’s 255.255.255 as the max number meaning you can only have 16million colours

Any time a radiation exposure is compared to Slotin and Daghlian and IS WORSE you know you're in for a ride.

What absolutely blows my mind is that some hobbyist was writing the software to control this equipment.

@richardletaw4068

8 ай бұрын

A HOBBYIST WHO REMAINS UNIDENTIFIED. *That* is the kicker.

@DavidChipman

8 ай бұрын

@@richardletaw4068IMHO, whether they are known or not is not the biggest issue.

@danielkaiser8971

5 ай бұрын

There were zero regulations for software back then. No university courses to learn how to code. No concept of QA. Just manually typing in each and every assembly language instruction in a plain text edit screen, compiling it at the DOS prompt, burning it to a chip, installing the chip on a circuit board, and plugging the circuit board into the equipment the software was supposed to control. THAT was what it was like back then.

@DavidChipman

5 ай бұрын

@@danielkaiser8971 you think all of that woudl have bene needed? Ho about the fact that what thesoftware they were writing could cause harm. They did know that, didn't they?

@James-il1zn

4 ай бұрын

@@DavidChipmani honestly dont think people at the time had any idea how much damage software errors could do, which is why there was never any documentation or verification of software coding. I wouldn’t blame the software engineer, I would blame the corporation that could clearly see the damage the machine was causing but refused to change it in any way.

The fact that they kept using the machine after the first incident astounds me. How do you ignore warnings like that?

@DonNekorio

Жыл бұрын

market regulations where stupidly deficient back then. not saying they aren't now... no state cant keep up with technological advance so regulations will always be outdated.

@alexlevinson8629

Жыл бұрын

I thought you were my brother (Aaron Levinson), ironically enough, born at the first hospital mentioned in this video

@xshowda

Жыл бұрын

Money

@RKroese

Жыл бұрын

When money is in play, bullshit is on the horizon.

@megagamernick9883

Жыл бұрын

It shouldn’t even be a matter of money or standards. It’s complacency if that were me I would be like hold on what happened? And even then hire someone out side the company to look at it.

And here we see another great example of 'don't ever let companies police themselves'

@thewolfin

Жыл бұрын

Pfizer has investigated their own product (Pfizer-BioNTech Covid-19 bivalent booster) by testing it on 8 mice (and 0 humans). The FDA has approved it.

@walkaway6212

Жыл бұрын

Most of all never let government police them selfs. The DEAMONratparty proves that.

@WarPigstheHun

Жыл бұрын

And republicans wonder why federal regulation exists...

@JJ-ub6lv

Жыл бұрын

Or governments

@walkaway6212

Жыл бұрын

@@JJ-ub6lv duaaa

13:26 "You don't think of software as something being able to fail..." This must be a generational thing, because computing in the late 80s through the 90s was fraught with software failure.

@Shimmermon

Жыл бұрын

Anyone who has played a Bethesda game thoroughly should be accustomed to how software can bug out.

@samdancer101

Жыл бұрын

@@Shimmermon it's not a bug, it's a feature!

@paigeconnelly4244

Жыл бұрын

Me, everytime I play The Sims 4 after EA release a new patch/expansion pack: "That is not true"

@SILVERF0X13

Жыл бұрын

I think he more meant that the average person would realize that gears and stuff can jam, but not necessarily know what an overflow error is. There's still a lot of people out there who think computers are magic devices full of blue smoke and that programmers are wizards

@Holoflux

Жыл бұрын

My guy, you wouldn't believe the tech problems i am still having in 2022 If anything it got worse as code got more complicated

Corporate greed is a blight on society

This video is officially my go to for companies and corporations denying accountability for their actions. Even if it happened almost 40 years ago the complete lack of care or action after fatalities just shows the massive incompetence that we still see today

@aysh444

Жыл бұрын

This and DuPont. Absolutely terrible people

@Albtraum_TDDC

Жыл бұрын

Late Stage Capitalism Fail. Corporate Greed, evil manager lies, the usual stuff. When will people learn to vote for their interests, just like the rich 1% does.

@fullmetal3233

Жыл бұрын

That’s what happens when healthcare is privatized and profited on in a capitalistic system. Has nothing to do with saving peoples lives and everything to do with making money.

@P-nk-m-na

Жыл бұрын

​@@fullmetal3233 if anything, it's about only helping people when theyre at their worst. after all, why use cheap methods to prevent sickness when the real money is in trying to treat it?

@bobbirdsong6825

Жыл бұрын

Well… um… because, uh, if we didn’t, then we’d have slow, overcrowded hospitals, like the UK!!! I know one guy who moved to America from there, he said hospitals were insufferable there!! It’s not like the NHS is the most popular and highly approved government service as voted by UK citizens!