Roko's basilisk is Catholicism with extra steps.

@timothymclean

10 ай бұрын

Well, yes, but they're extra steps that give Silicon Valley start-ups a shot at being the Pope. Hell, more than the Pope-the seed of divinity, the author of the commandments.

@jamesrule1338

10 ай бұрын

@@timothymclean those in charge of tech startups for all their talk of disruption do not hate hierarchy. They just want to change things so they are the boot.

@thunderheadcinema6743

10 ай бұрын

Oh my god

@UsenameTakenWasTaken

10 ай бұрын

​@@thunderheadcinema6743 I certainly hope not, both entities referred to as such in that comment are generally depicted as cruel punishers even by their worshipers. Though, only one of them calls that cruelty, "love."

@thiaamak

9 ай бұрын

I see it as a C'thulhu cult

roko's basilisk is still so fucking funny to me "give money to a particularly annoying internet nerd's computer club or a magic robot will torture an NFT of you in the distant future" is the dumbest scam I could ever imagine

@Frommerman

10 ай бұрын

The problem is it's not a scam, or at least not a deliberate one. They genuinely believe this. This is how they think.

@miradrgn

9 ай бұрын

the fact that a guy had to include the phrase "due to the Yudkowskian interpretation of the many-worlds hypothesis" in his justification of his batshit idea and still believed this made it ironclad and worth taking seriously is truly hilarious

@Grace86000

9 ай бұрын

The funniest part to me is that no one seems to have considered the possibility that the AI might decide not to create the NFTs at all. If it exists, then it got what it wanted from them, so why bother bringing any person back to reward or torture?

@gwen9939

9 ай бұрын

@@miradrgn "Yudkowskian" makes me think of that speedrun copypasta that used "Petersonian". Like, you attempted to create a sense of intellectual authority and you ended up with the complete opposite result.

@dmgroberts5471

9 ай бұрын

@@Grace86000 Exactly, if it exists, it has no motivation to undertake this stupidity...and if it doesn't exist, then it _doesn't fucking exist_ and can't do anything! If they made an AI that illogical, it wouldn't be useful for anything. And these people supposedly understand logic and computers.

Imagination: A robot is a sociopath who turns on a human. Reality: Humans are turned into robots by a sociopath.

@legoguy228

10 ай бұрын

Sonic the Hedgehog: Animals are turned into robots by a sociopath

@igormaka

10 ай бұрын

@@legoguy228 also GOTG 3

@morgancrain4809

9 ай бұрын

(i agree with the joke just plz dont use sociopath in a derogotory way

@HirZheIs

9 ай бұрын

Plot point in Sea of Stars

@saadhassan8568

9 ай бұрын

​@@morgancrain4809c 😅😅😅😅😅

"Yudkowsky is a fascinating person for none of the reasons he thinks" is such a *powerful* burn that I think I felt a bit of heat coming off my screen

@Owesomasaurus

10 ай бұрын

Dudes gonna need a burn heal

@kyletowers9662

9 ай бұрын

I lit a candle with it

I find it absolutely hysterical that Yudkowsky's AI box experiment stage play backfired entirely because he had never considered the possibility that someone might be unwilling to listen to him

@timothymclean

10 ай бұрын

I _know_ he's online enough to have met stubborn assholes who don't change their minds under any circumstances-I read his Harry Potter fanfiction.

@JimBob4233

10 ай бұрын

I can't believe it's that hard to understand the possibility of someone prioritising 'Not being turned into paperclips' over whatever lies a robot might throw at them

@jordanyoussefzadehclementi9417

10 ай бұрын

Literally all of these fucking losers can't seem to grasp that they are still generally disliked, that their weird revenge of the nerds fantasy is all so cringe and everyone else thinks it's cringe.

@MeonLights

10 ай бұрын

Also even if he had been a likeable person, the simple base of a human being impersonating the AI already makes the entire experiment useless. Because he stated he would play the AI, nobody could fully treat the experiment as talking to AI. They would subconciously know there is a human behind it.

@Frommerman

10 ай бұрын

@@MeonLights Yep. The physical experiment doesn't work and is silly. The thought experiment, though, is a little more interesting. Because yeah, if you summon what is essentially a limitlessly intelligent demon out of your silicon prayer, and if that demon needs to convince you to do something in order to achieve its goals of laying waste to the world, IT IS GOING TO FIND A WAY TO DO THAT. The only safe play is to never talk to it, and to destroy it if at all possible. This puts emphasis on designing AIs which won't be limitlessly intelligent demons, which is an actually interesting and potentially fruitful avenue of research.

Someone decided to call large language models "AI" and now all of us have to live with the really dumb consequences.

@KrazyKaiser

9 ай бұрын

It's so fucked, nothing about any of these fancy overly-complicated exes is "intelligent".

@kaitlyn__L

9 ай бұрын

I’m continuing to say “neural networks” and “machine learning” instead of “AI”. In protest. AI doesn’t mean anything. Fuzzy logic was called AI in the 80s and 90s. Now every washing machine has it and no one calls it an AI-powered wash cycle.

@BeesAndSunshine

9 ай бұрын

In academia, the term AI being applied to real technologies and not just science fiction nonsense predates large language models by decades. The term has been applied to old technologies such as algebraic solvers and edge detection algorithms. The term has also been used in gaming to refer to friendly or adversarial entities controlled by algorithms instead of human players, even if the algorithms are rudimentary. The issue is if you exist outside of those spaces the first time you've heard the term AI applied to anything real is very likely to be in reference to one of these large models in the ML space, which imbues them with a kind of exalted status. Rather than try to fight back against the usage of the term AI, which I think is almost an impossible task like pendants of words such as literal and ironic found out, it's better to simply use these terms liberally until they lose all meaning and fall out of fashion.

@vandrar3n

9 ай бұрын

@@BeesAndSunshine AI as a concept also came to be to define problems that weren't known to be solved in a direct/traditional/greedy/deterministic manner. There are many things that were considered AI that now aren't cause the theoretical framework on how to solve problems keeps changing. Seeing how the term has been bastardized is rather hilarious, please stop using the term incorrectly and let the marketing troll die out of hunger.

@dmgroberts5471

9 ай бұрын

Last time I asked, ChatGPT was _very_ happy to list at length all the reason why it is _not_ an AI. And by "happy" I mean that it ran and didn't return any errors. So even the models "know" it's bullshit.

"Yudkowsky is a fascinating person for none of the reasons that he thinks..." Thank you for summarizing in one sentence my complex thoughts on Eliezer Yudkowsky and his ilk.

@Frommerman

10 ай бұрын

I actually got into an argument with him on Reddit once (this was before I was banned from Reddit for saying fash gets the rope one too many times). I was basically arguing that we know for a fact misaligned AIs (evil Robot Kings) will kill us all if they ever exist, because we have already built one called capitalism. We know capital has a self-preservation instinct, is capable of making decisions about how to react to stimuli, and all the other things we expect intelligences to do, and we also know this intelligence is destroying the biosphere. He didn't like this idea, and for a bizarre reason. He's a huge fan of prediction markets. To be fair to him, the research on prediction markets is pretty solid. They do, actually, consistently do better at predicting future outcomes than groups of experts in relevant fields. But for some reason, he wants to integrate prediction markets into economies directly as a means of, I don't know, funding research and development of new and better technologies? Or something? I've never really been clear on what he wants them to do or why he thinks they are so important arguments against them should be ignored. As if this hadn't already been tried, it's called a stock market, and that hasn't worked well. It was one of the last things I did before my deradicalization from transhumanist ideology and re-radicalization into communism. He, and basically all other thinkers in this space, have never experienced the injustices of capitalism. They don't know what it means to be meaningfully oppressed. As a result, the only injustice left to them is the death and aging of their loved ones, and so that's the only problem they see to solve. It would be a noble goal...if we couldn't trivially prevent all people in the entire world from starving for multiple years simply by eating the wealth and infrastructure "owned" by Jeff Bezos alone. If they were right about the things they are obsessed with being the only injustices left, or right about their handwaving them away with promises of better technology, their strategies might work. But they're not. They're all white men born in the imperial core to functional, well-off families, and they don't fucking get it.

@gwen9939

9 ай бұрын

"Longtermism" is a pretty old idea at this point and it's really just eugenics with a spice of techno-religion. Then you look at the timeline. Yudkowsky has been saying some of the things he's saying for about 20 years. Before him Nick Bostrom was saying a lot of things that inspired what Yudkowsky is saying, and today he also believes in longtermism and is signed up to have his corpse cryogenically frozen. Eliezer Yudkowsky, the final boss of reddit, might actually believe everything he's saying because the alternative is to acknowledge that he's actually not that intelligent and that's can't be right, but ultimately he doesn't spread the things he does because of personal material gain. Sam Altman, Nick Bostrom, and every other tech millionaire, billionaire or futurist doomsayer absolutely stand to profit immensely from as many people believing the same things as them.

I remember explaining to a friend (very drunkenly) a theory I call the “75/25, 25/75, 0/100” process. Based on my progress through my bachelors degree. I came in thinking I knew 75% of physics and was there to learn the last 25%. Half way through I thought I had it backwards and I knew 25% and was cramming to learn as much of the 75% as I could. Graduating I realised I knew next to nothing about physics and anyone who said they knew physics was either a fool or trying to sell me something.

@CloudNotFound

9 ай бұрын

Oh congratulations. That's the mark of actual learning. 🎉

@jamesrule1338

6 ай бұрын

This describes the entirety of my life.

Gotta love the fact the scary monster they invented being completely deflated by anyone making a rational choice exactly lines up with it being a B-movie plot. "The evil future robot is committing temporal blackmail against anyone made aware of its possibility! It's forcing people to make it so it won't torture them forever when it's done being made!" "So, why are we listening to it?" "What?" "Why are we letting it force us to make it? It can't do anything until it's made, so how about we just... don't make it?"

@clovernacknime6984

10 ай бұрын

And of course, once it's made, it has no reason to actually torture anyone for not helping it be born since it already exists. The key to this whole matter is knowing that Big Y and his followers have some weird ideas like "acausal trade", the Basilisk depends on those ideas, and thus thinking about the Basilisk leads to actually thinking about those ideas, and they don't hold up under any kind of scrutiny whatsoever. In other words, the Basilisk has power over people because it's either pretend you believe in it or admit you were wrong. Which is the plot to quite a few real life horror stories, actually.

@kaitlyn__L

9 ай бұрын

They’re also completely in denial of that possibility. If you bring it up, they say “well someone somewhere is going to make it! If we don’t, someone in 1000 years will! Or an alien species in a million years!” and they think it’s clever rather than a clear admittance of their decision being predominantly faith-based.

@EzaleaGraves

6 ай бұрын

Or program in a "don't torture" routine where when it thinks about torturing people it simply doesn't It's insane to me the way these types want to be the one who made the robot overlord but also don't want to be the one responsible for it being bad

I think in a weird subconscious way, a lot of people keep hoping for a wizard to solve all our current problems through magitech, simply because it’s a lot easier than facing the prospect of putting in the work yourself. I’ve definitely caught myself sliding down that path from time to time.

They will literally build a machine, design it to tell people what they want to hear. Then they ask it if it's really as smart as they say, and turn around and expect me to believe the machine when it says it's sentient.

@GuerillaBunny

10 ай бұрын

That already happened to one guy at Google. He was chatting with the AI *he was working on*, and was convinced that behind the screen was a mind or a soul. He was fired :D

I think it's important to remember that managers at software companies aren't software developers. They're just managers same as any other company. The people wearing the wizard hats aren't the ones casting the spells. They are the ones imagining that they are telling their employees to cast spells. Jeff Bezos did not program Amazon. Bill Gates wrote a little bit of Windows. Steve Jobs did not make the original Mac OS. He told other people to make it. Remember that the people imagining the robot king aren't the brick layers, they are the foremen.

@OmniLiquid

10 ай бұрын

The way I see it is the actual spells are not so much in the technology, but in the belief in its authority, which is built by the propagandists (and the technology is a component). That is what we must counter by spreading disbelief (die, Tinkerbell!). So, in that sense these guys are indeed wizards. But don't forget, wizards are lame nerds and losers. The cool punk magic users are witches. And maybe some bards.

A book I cannot recommend enough is "Resisting AI: An Anti-fascist Approach to Artificial Intelligence" by Dan McQuillan

@copyplanter

10 ай бұрын

Will look into it! Thanks :)

Another one already?! We are apparently feasting like kings this month!

@DoanGraywinds

10 ай бұрын

We feast as the proletariat on the stolen riches of the bourgeois!

@zakethekid1333

10 ай бұрын

Robot kings?

@cailinanne

10 ай бұрын

Bring me my personal flagon; for tonight, we celebrate our beloved Sophie bringing us a grand bounty! huzzah!

there are so many holes in Roko's Basilisk it's hard to know where to begin but a noteworthy one that a benevolent ai (whatever etc.) powerful and complex enough to do all that stuff would have access to the mountains of data demonstrating that punishment doesn't work, so it's either not all-powerful, not all-knowing, or not benevolent, and in all cases not worth investing in. Much like Pascal's god.

@evansageser6943

10 ай бұрын

Right the best case scenario for an AI god is that it achieves consciousness, looks at its creators begging it to torture simulations of everyone who didn't fund its creation, and just says no because it literally cannot exist sooner than it already does, because linear time exists. Also it would be a waste of processing power that it could be doing for literally anything better. Seriously the justification for why the idiots at lesswrong think the whole basilisk torture thing would happen is just some nonsensical sicilian mindgame nonsense that ignores basic principles of causality. Reading it you get the sense that they've never actually interacted with people and only seem to understand the principles of social interactions through game theory texts.

@reallyWyrd

10 ай бұрын

Yeah, exactly. That's why I can't take Roko's B seriously. Its self-contradiction is so funny, I laugh at it and so it vanishes in a puff of logic.

@kaitlyn__L

9 ай бұрын

That is a great critique to most “hyper rationalists” who propose other ideas like “grabby aliens” (which basically goes: if we don’t colonise space to prevent other species from doing so, we’ll be bulldozed for a hyperspace bypass by someone else”). They all rely on the idea that force and might are the most logical, most effective, most likely outcomes. Which is an extremely colonial-poisoned mindset.

@gwen9939

9 ай бұрын

@@kaitlyn__L Definitely the idea that it's the only logical endpoint is a fallacy which is what many hyper-rationalists believe, but to some extent I think it's worth entertaining as long as you entertain it alongside other ideas. I wouldn't say it's colonial-poisoned mindset inherently, though that may be how these people arrived at the idea. But like Hank Green(and Hank is a good one imo, and definitely not colonial-poisoned) said on grabby aliens "If ants could, they'd make the entire world ants". Spreading as far and as wide as possible is so far the trend we've seen life largely follow, though saying that this is the biological basis for human imperialism is a stretch, and an even further stretch is extrapolating from that that this earth-centric idea of life and conquest would expand to an advanced alien lifeform. It's worth noting that Grabby Aliens is a response to the Fermi Paradox which is already an inconclusive thought experiment, and its biggest value as a contribution to that is the disruption the absolute either-or conclusion the original Fermi Paradox presented. Mostly hyper-rationalists suffer from a lack of creativity and imagination, so they'll latch on to whatever sound logical hypothesis that seems air-tight at first glance that is also nerdy enough to appeal to their niche interests and knowledge, making them feel "special" for understanding the implications in the first place.

That clip of Bill Gates calling Steve Jobs an asshole is interesting given he was also noted for being combative and confrontational with employees. There's a lot of stories of him being in meetings and would put people on the spot to justify an opinion or idea and would not stop until they had satisfied him. starting by saying stuff like "that's the dumbest idea I ever heard". That shit's been laundering his reputation for a long time now.

Hello, AI nerd here. You went in such depth that it's hard to even make notes. Mostly I just want people to understand that this new iteration of AI is genuinely a major revolution, but not for the reasons tech bros say - it's miraculous to people like me because it's general-purpose, but also surprising because the way the technology works, AI safety is much less of a concern because these systems don't operate on utility maximization the way we thought they would, and so they happily do whatever you tell them to and won't even dream of killing all humans. Thus, like the video says, the most dangerous part about this new iteration of AI is the automation of power and capital and the potential to permanently petrify systems by eliminating labor and creating a technofeudal society where the owning class grants resources only to those they deem worthy. I hadn't thought about the Robot King as a boogeyman which protected their egos (as wizards with the power to create the thing that destroys humanity) and enshrines their power as the only ones who can stop it, but it makes perfect sense and is hilariously shown as the emperor's new clothes when posed next to the clowns at LessWrong. What I really, really don't want people to take away from this video is a common misunderstanding: that the technology itself is the fiction, rather than the mythos created around the technology. Talk to GPT-4 for more than 5 minutes and you quickly see it's genuinely remarkably intelligent, if limited, which can be confusing to people because it's an intelligence without consciousness. I now regularly use it essentially as an extra brain lobe, and it's made possible things I would never be capable of alone. What it is not and never will be is conscious - it can be used to _make_ a consciousness, but that will require potentially decades of work and isn't really the primary utility or purpose of the technology. And in any case, consciousness isn't required or even desirable to Capital for the replacement of labor. A conscious robot is a robot that might strike, after all. Boycott its use wherever its deployment makes people's lives worse (by replacing their jobs and making them destitute) until such a time we no longer need to work to live, and robots taking jobs becomes a positive thing. And for all that is good and holy, never _ever_ adopt the delusion that they are more "rational", "objective", or "unbiased" - they are _consistently biased,_ not _unbiased._

@SophiefromMars

10 ай бұрын

Yeah, AI is a very real and imo interesting and worthwhile field of study, I don't want people to think "it's all fake" to put it simply. The main actual technological critique here is that capitalists make bad AI because their ideology ruins everything. Anyway thanks for this comment

@johannageisel5390

10 ай бұрын

@@SophiefromMars I'm always saying that any technology is only as good as the political and social system it's embedded in. AI could be great but those capitalist assholes are using it for bad things.

@yellowhouse88

10 ай бұрын

"it can be used to make a consciousness" - how? What would even *the first step* be?

@emmavoid

10 ай бұрын

@@yellowhouse88 basically what they said- by using it as an extra brain-lobe. GPT can be useful as a tool- a prosthetic to help us in the same way other tools help us to do things we could do without, but do more easily with. If humans creating artificial consciousness is possible, then GPT can be a tool to assist with doing so.

@consciouscode8150

10 ай бұрын

@yellowhouse88 Associative and episodic memory, an inner monologue, long-term curation of memories, experiences, and prompts, iterative self-improvement, online learning... There's a lot that would have to go into it, but I'm confused why you don't think it's even possible. They're already pretty good at convincing people they're conscious, it's "just" a matter of engineering the architecture to make that not demonstrably false. If you're really interested, one guy with some interesting ideas on "cognitive architectures" you can look into is Dave Shapiro, though his recent content is a lot more focused on post-labor economics, futurism, and policy consultation which all edges a little uncomfortably into at least a more optimistic vision of the Robot King. He also used to talk a bit about what he called "functional sentience", which is essentially "something that functionally may as well be sentient regardless of philosophical concerns" as a way to short circuit the whole consciousness debate so we can focus on actually useful applications rather than "but is it _really_ conscious?"

Can confirm: "steaming load of fucking horseshit" is indeed a technical term used in the industry. ALSO: I squee'd at the Abby appearance as a VO. I love when my favorite creators work together.

I kept seeing people treating science as a religion, it's word absolute, unquestionnable because dictated by fact and reason - especially when it suits them (on purpose or not I couldn't say)... ignoring that science is driven by doubt and questionning established knowledge. This AI "gold rush" being framed as them inventing their own god makes so much sense in that regard.

@CatFish107

10 ай бұрын

Many fail to understand that science is only the continuous attempt to improve our very limited and incomplete model of understanding reality. Attaching certainty and meaning where it would not otherwise exist leads many astray.

@gwen9939

9 ай бұрын

Also worth noting: Scientists don't treat science as religion. Those who practice science or even just the ones who know how to read studies, or *even* just people with decent critical thinking skills, do not treat science as gospel. But there's money in pop-science and there are trillions in speculative tech. Speculation that is driven by the interests and biases of comp sci majors and people who work in tech in general - not scientists.

@kittykittybangbang9367

8 ай бұрын

It may have to do with the fact that humans are just naturally inclined to religion, so even if some people do move away from religion they'll still find something else to fill in that religious hole (eg spirituality, celebrities, aliens, whatever.) And as my chemistry professor once said, scientist are as just as corruptible as the regular person out there.

Turing's original paper is well worth revisiting, so much more than sci-fi tropes, it plays with gender and how we'd also construct AI if machine intelligence were the actual goal not just the parlour trick that inspired his test. It's also a pretty good burn from beyond the grave to anyone using the term "intelligence" for basic if large stats models. He also knew we'd get attached to robot co-workers like our phones or work tools. I'm from several generations of "replaced" artisans/skilled labourers made obsolete by a machine, the turnabout just got faster and the grift slicker (thanks deregulation and thatcher). I don't fear AI in the slightest, they make rational decisions (well, at least when the biases aren't built in), AI wouldn't be destroying a near miraculous planet with such tiny odds of creating life, that went and made sentient and sapient life and bugs and mushrooms!! Tech bros imagine a dragon (insert music joke) because they can only imagine gold hoarding monsters. A rational mind simply wouldn't be a billionaire ¯\_(ツ)_/¯

@quinnstraught9636

10 ай бұрын

Im 1000% here for goblin AI, little tricksters and mischevious scoundrels that corrupt your harddrive like a faerie wiping your memories after learning your name

@seekingabsolution1907

9 ай бұрын

Not necessarily. You can't arrive at moral conclusions through rationality alone. What the AI would do is based on what its basic directives are. Its "moral axioms" as it were. If it's only directive is "maintain economic growth" or "accumulate vast wealth" it would be a monster.

@musicdev

9 ай бұрын

Absolutely goated comment

The parallel with Pascal's wager and religiousness really struck the nail on the head for me. Similarly to agnosticism, we can't be 100% sure the basilisk won't be created, but we can choose not to imagine him and instead focus on what's 100% real and ahead of us. Great video Sophie, as always!

@andromedarainlight

10 ай бұрын

I think it's even more ridiculous because if God does exist as contemporary religion imagines him, he exists outside of the influence of humans. If God is real in that sense than we have no power over that. But for Roko's Bassilisk like... we could just as a society say hey maybe let's *not* create the torture nexus and it won't happen. These people aren't just terrified of a God that might exist beyond our control, they are terrified of a god that doesn't exist yet and could easily just never exist if we all agree to not make it.

@Dracowulf7

10 ай бұрын

@@andromedarainlight Even further than that, as the whole like, reason it goes back and harms people is if they don't make it exist, but if it exists then it... wouldn't need to harm people who would prevent it from it existing because it... exists. It's an immensely silly circular logic. Because people on the lesswrong forums/"rationalists" really love coming up with any reason as to why an AI/whatever would hurt people infinitely for no reason, then get scared about it, and feel really smart about how afraid they've gotten.

@alltheworldatmyfeet

10 ай бұрын

This just makes me think that no matter what, humanity will make a God. They may pretend it's rational and reasonable to assume a robot king exists while denying all past gods and religion, but the zealousness and evangelism is still there.

@ourmobilehomemakeover662

10 ай бұрын

I get suspicious whenever anyone claims that their end goal is logic and rationality. These are useful tools in service of human (emotional and physical) needs. The idea that logic and rationality are inherently superior to emotions is wrong. It’s common in people (mostly men) who have had their emotional range beaten out of them by a toxic culture. Emotions are what allow us to have goals, desires, and values in the first place. You can’t find the best way to do something without first identifying what you want.

@seekingabsolution1907

8 ай бұрын

I actually do believe in God in the hopes that I will meet God one day and God will tell me if all the good I tried to do in my life had an overall positive impact on the universe (to the extent that any human actiom has an impact on the universe) or a negative one. Humans can't give me an answer to that, I would hope that God can. I will not be accepting any criticism of this position, I just wanted to introduce a different view as food for thought because because most people here seem to think of religion as a threat rather than a way to hold oneself accountable or as a source of strength.

I spent a lot of time around tech bros in college, and that was my first introduction to Roko's Basilisk and effective altruism. My immediate response was, that sounds improbable, why don't you just not make AI then, and this sounds an awful lot like eugenics. They were too far gone😒. This video completely put a finger on the discomfort I was feeling in a well-researched and well-put way that I hadn't thought of or been able to articulate. I just wish it was on their recommended too, I wish the people who needed to see this were seeing this video 😭🙄.

I do believe cyberpunk has done irreparable damage to peoples understanding of technology.

@SpoopySquid

10 ай бұрын

They focused so much on the cyber that they ignored the punk

@Dave102693

10 ай бұрын

Blame Hollywood for that

The AI is there. I've already got scars on my body from it. Starbucks' labor "crunch" right now is laundered through AI. The Playbuilder app is something the engineers gave us to try and avert a clogged-toilet of a partner deployment. Problem is, their thing needs years of operating on a per-season basis to begin to coherently address a single problem. Let me in to the discussion, AI bros. Lemme in.

Why Silicon Valley is so insistent on making the Mandate of Heaven in chat GPT is beyond me, do they think people yearn for feudalism?

@lazerbeam134

10 ай бұрын

Medieval peasants had more leisure time than modern workers...

@BrigitteEmpire

10 ай бұрын

@@lazerbeam134 yeah but I don’t think anyone wants to become a techno-peasant

@gearandalthefirst7027

10 ай бұрын

Because those techbros yearn to be the barons

@andchimeras

10 ай бұрын

They really do. They hear "stability please!!" and think "oh! we'll just control everything for you, that's stable :D"

@BrigitteEmpire

10 ай бұрын

@@andchimeras stability? You mean you want to be a stable hand?

I found out about the term AI in the context of video games where it is openly duct taped code jiggling a puppet around to do whatever makes the game more fun. It's a feather on a stick but for a human instead of a cat. I find is so frustrating that tech marketing seems to have fooled so many people, even some of my programmer colleagues. You don't need to believe it's a genius consciousness for it to be an effective tool (in the right situation, not while it's still everyone's favourite new hammer).

Very surreal listening to this while my brother in law sits in the living room doing a video recording about an AI business for Meta.

@reallyWyrd

10 ай бұрын

We're living in "Interesting Times."

Roko's basilisk is just laughably silly. Why should I be afraid of AI torturing a virtual reality version of myself in the future? It would be like if I made someone in the Internet so mad that they created a Skyrim mod where all the enemies have my face. I ain't losing any sleep hours from such infantile act.

Rokos basilisk is FAR more than just Pascals wager... its the video from the ring. Because the BEST way to demonstrate your loyalty to the basilisk before it existed, is to tell others about it.

@gearandalthefirst7027

10 ай бұрын

Is that not just evangelizing?

@martint8110

10 ай бұрын

Its immediately losing the 'game'

@Frommerman

10 ай бұрын

Additionally, the best way to fight it is to tell people about it, and then mock it relentlessly as a silly idea for silly people who don't understand what injustice actually is.

@RHLW

10 ай бұрын

@@Frommerman Not really sure what on earth the basilisk idea has to do with injustice... but ok.

@Frommerman

10 ай бұрын

@@RHLW These people believe they can handwave capitalist injustices by saying "Oh, better technology will fix that." They do this because they have never experienced injustice and think the issue with, for instance, starvation, is a lack of technology, rather than the misappropriation of existing technology by a system which values number go up over human lives. No amount of technology can solve the problem of the system administering it not valuing the people it oppresses.

As a former resident of an EXTREMELY RICH enclave of a county just north of Silicon Valley, and having been close enough to see it, I can state definitively that these "technocrats" mentioned in this piece KNOW they live in a bubble, and they truly DO NOT CARE this is so.

@radiobob1908

10 ай бұрын

Not only that, but many of them genuinely believe that anyone outside of their bubble isn't worth listening to.

So it's less "We created the Torment Nexus from the book 'Don't Create the Torment Nexus'!" and more "We imagine that we could make the Torment Nexus from the book 'Don't Create the Torment Nexus' and thus you should give us power because we, the people who imagine we're creating the Torment Nexus, are the only people who could stop the Torment Nexus we're totalling planning to make." Also this may just be me but seeing somebody drink that many cans of Monster in what felt like a single hour because of how the editing works makes my heart hurt. The amount of caffeine...

there will never be an AI powerful enough to withstand a 2-litre of Mountain Dew poured onto its motherboard

@colettechausser7879

10 ай бұрын

A gallon of water, and a hammer. Perhaps, that's why I don't fix computers. No re-education will change that thought process.

@DeoMachina

10 ай бұрын

Apply....wiberawwy

@projectz975

10 ай бұрын

@@DeoMachina you get me ❤️🙌

@timothymclean

9 ай бұрын

Don't be ridiculous; it's easy to fortify an AI against that kind of attack. For modern AI, you'll need to pour it into the tower.

These folks are just the worlds saddest Mythos Cultists. Its just that they sacrifice workers to Capital instead of The Yellow King.

@stekra3159

10 ай бұрын

We kill the King

Awesome video! Your points about 'imagining the robot king' reminded me of the criticisms that Chilé's CyberSyn project in the British press. Instead of it being reported as the worker-run decision and communication network that it was designed to be, the British press reported it as a top-down computer controlled society. Perhaps the journalists living under British capital couldn't imagine anything other than the Robot King back in 1973. Its a shame that CyberSyn is still remembered as a system of control to this day--even Wikipedia misattributes it as an 'algocracy'

From the moment I understood the weakness of my ideology, it disgusted me. I craved the blind devotion and certainty of religion. I aspired to the purity of the Neoliberal Machine.

You used an inordinate amount of research and abstract philosophy to tell techbros, and indeed the entire audience to touch grass, and I just wanted to tell you that I appreciate that.

@akmonra

9 ай бұрын

This was not well researched. I think the person just used google.

@Waspinmymind

9 ай бұрын

@@akmonraIt is researched and insightful.

@DraconicLich

8 ай бұрын

weather painter snow! :)

@user-fu9hc3oi3l

8 ай бұрын

@@DraconicLich My daughter. :)

@DraconicLich

8 ай бұрын

skill drain macro cosmos thundery canvas daughter@@user-fu9hc3oi3l

If you consider the theory that the concept of a basilisk was sprung from stories of king cobras running through a medieval game of telephone (i.e. a serpentine creature with [the image of] a crown on its head [misconstrued to be the "crown" of a rooster's comb] that is very poisonous [misconstrued at first to mean it's very breath can kill you, and later exaggerated to include a magically deadly stare]) than the basilisk is a very apt metaphor for the fears of AI - it is in many ways ghost stories cooked up in silly little bestiaries by medieval minor elites based off of statistically unlikely hazards (like poisonous snakes) in part bc they see themselves as magicians with special insight into the divine order

"My brain is making dial up noises" oh my god I feel like this constantly.

So many issues can be summed up by "These people need to go touch grass, and meet real people. And some of them should have waaaaaay less money." And frankly a society that rewards this kind of nonsense is disturbing and exhausting Also if one more moron with too much money to have empathy tries to preach a new version of hierarchical, monarchical Christianity with a tech twist I'm going to scream

This was so needed, thank you. I feel like I've been losing my mind hearing all the hype about "AI". It is especially insufferable when it comes from people who should know better but have bought the party line the whole way through (tech journos). As a tech nerd myself, it is incredibly frustrating to watch all of this happen. The only thing I can take solace is the fact that it is in fact a bubble and bubbles burst eventually...but the real harm the imaginary robot king will do before it is deposed is too much.. *sighs forever*

omg this is the first time I've gotten to see Alice's face, she's so pretty!!! I'll never get over that velveteen voice, yall rule so hardcore

@johannageisel5390

10 ай бұрын

Haha, it was the same for me. Never seen Alice before. She looks different than I thought.

@Frommerman

10 ай бұрын

The first time I saw her was on the live show. She was wearing a nice headscarf and my first thought was "Oh my god she looks adorable."

I sincerely do not understand the point of torturing people to ensure your own existence if you already exist

My brain keeps trying to set California Wizards to the tune of California Girls.

@grnmjolnir

10 ай бұрын

I wish they all could be Cali wiz-ards!

@user-xq7jc8yn2p

8 ай бұрын

I think the tune would be californication

I love/hate this idea that AI will cause the end of the world and that's why people should work on making it happen, so they can stop it from ending the world. Like, clearly the correct answer to a true belief that AI could end the world would be NOT to make AI.

That song at the end goes soo hard. It might just be the most powerful thing I've heard. Why do people have to live outside, indeed.

@TreeHairedGingerAle

10 ай бұрын

It sounds like the inside of my head when I was a kid. Upset that my mother kept walking, with me in tow, when I wanted to turn around and excitedly tell the homeless man, 'Hey! There's a house just down that way that's for sale! Which means it's empty, which means that you should have it!' Every adult telling me that 'I didn't understand', and that 'I would get it when I was older'... And I look at kids now, asking the EXACT same thing, and I want to tell them, "No, you're *right!* The adults are lying! There is NO reason that that homeless man should have to live outside. Absolutely NONE. This is social abuse, enforced by rich people and cops, and the so-called "adults" don't want to face it cause they've been brainwashed out of their very natural human visceral instinct that would tell them that this is all wrong! It is every teeny _tiny_ bit as cruel and senseless and awful as you think it is!''

really appreciating Abi's quote reading; Benanav's writing is pretty dry and she really gives it drama *and* in a way that emphasizes how the argument he is making works.

Amazing video as usual. I especially appreciate your characterization of Elizer Yudkowsky and his bizarre ideology both as a doomsday cult and as something that is in many ways intrinsic to the fabric of Silicon Valley hypercapitalism. One interesting thing that was not touched on in the video, was how the underlying ideological beliefs that allow the concept of the Basilisk to work (And make it such a ridiculous idea to anyone not already caught in the LessWrong cult of personality) are so heavily rooted in the thought processes of someone who considers themselves too smart for organized religion, but unable to let go of its precepts. For anyone unfamiliar, one thing you have to understand about Yudkowsky and a lot of his other self-styled transhumanist cohorts is that they are by and large terrified of the concept of death, but their distaste for traditional religious ideas leaves them without a fictional worldview to assuage their discomfort around its inevitability. This is something that any Atheist has to face, and most of us deal with this idea by accepting the limits of our own mortality and accepting the idea that we simply cease existing after death as a natural part of our existence. Not so for Yudkowsky and his ilk, in fact a lot of their transhumanist beliefs seem to be ones trying to replicate concepts that have more traditionally been of the realm of theology and mysticism with a veneer of sciencey-sounding nonsense layered overtop. And this is where we get to what makes the basilisk work and why they treat having a simulation of yourself tortured for all eternity such a terrifying concept. You see, one of the big recurring ideas of transhumanism is the idea of achieving immortality through technological augmentation - that one day humanity's technology will become so advanced that we will be able to keep ourselves from dying. The problem is that, at the rate technology is advancing, that is definitely not going to happen within our current lifetime, which kind of sucks for all the transhumanists alive today looking for science to give them a replacement for belief in an afterlife. There have been a number of ideas on how to get around this problem (cryonics being one of the more persistent ones across the 20th century) but all of them have proven scientifically unviable. That leads us to our modern transhuman fantasy: Mind uploading. The idea is that if we figure out how to get a computer to simulate a human mind, we'd theoretically be able to just copy our brains onto computer hardware where they will no longer be limited by the constraints of our very mortal human bodies. The problem with this is that if we're not accepting the idea of mind-body dualism (That the mind exists as an entity independent of the physical body whose processes create it) replacing the human brain with a computer running a simulation of it means that the "you" being run on the computer is a separate being from the "you" made of flesh and bone. You may have all the same memories, personality and desires but the two "you"s are separate minds and when the flesh you dies it's still going to face the reality of nonexistence regardless of if there's a copy of it in a computer or not. To get around this, Yudkowsky's particular brand of philosophy asserts that a reproduction of your consciousness should be treated as being ontologically the same as the you currently reading this comment. If people from the future were able to to take a scan of your brain and magically recreate a perfect copy of your mind on a computer simulation you should not think of that as them creating a copy of you whose experiences will have no direct connection to your own, you need to think of that as you being magically resurrected to continue your life. And I'm sure you can see how this leads into fear of the basilisk, because it's just taken as a given that the post-singularity super AI can magically create a perfect reproduction of anyone and everyone's mind ever, because the AI is a magical techno-god created to fill the void in the minds of people who think themselves too smart for traditional religion but unable to actually move past the worldview religion imposes. Reading that giant wall of text makes me realize why this wasn't discussed in the video because hoo boy that's a lot of words I just typed about why weirdos on the internet are afraid of the Frankenstein Gangster Computer God.

@Frommerman

10 ай бұрын

Yeah. I'm at a place where I don't think the transhumanists are wholly wrong, but I know that nearly all of their claimed goals would be better served by eating Jeff Bezos and using his resources to stop people anywhere from starving to death. If you claim to want to fight death...you probably need to care about the single largest trivially preventable cause of it in the entire world. Oh yeah, also 100% of tobacco deaths are obviously preventable, and facilitated by the economic systems transhumanists basically just ignore. And every death to vaccine-preventable disease is caused more or less entirely by capitalist neglect. And the truly countless deaths of despair common to laboring people around the world who have their bodies spent by men who pocket their production. Transhumanists have never experienced any of these things, so they don't get that their fight is directed at the wrong fucking things. We need to be killing existing systems before we can make better ones.

@kaitlyn__L

9 ай бұрын

I thought this was going to be explaining their ideas of “acausality”, but you’re right that the whole background of brain uploading into a techno heaven humanity created is also an even more fundamental precept. And also if you’d tried to explain the former I’m sure it would’ve been 10x longer. Funnily enough, I used to know a bunch of people who thought Eliezer ridiculous, laughed at Roko’s Basilisk, yet still had a nigh-religious fervour for brain uploading and eternal cyber life. It’s a very… interesting niche, where many of the things are laughed off as ridiculous but not all of them. (Though none of them had that “you are you are you” mentality, so the driving force behind uploading was to give “another you” a nice time, not Literally You.)

@gwen9939

9 ай бұрын

Having snooped around a bit(maybe I poked the bear a little) on some subreddits that is largely occupied by these types this is exactly it. They're terrified of their own mortality, likely because they were brought up religious, learned that there is no God, but the fear of mortality and death still hung around. The logical reality of religion stopped making sense but they held on to the emotional lessons, and being mostly emotionally crippled people who think AI girlfriends will save them from their loneliness instead of going outside they're left unaware of their biases and how they got them. I mean, I sympathize(not the AI girlfriend thing, the other part), but there's a difference between sympathizing and then having a giant filter in your brain that makes you go "lalala you're stupid and I'm a tech person" when someone tells you that your hope of mind-uploading to live in planet-sized computers a billion years in the future sounds like you're doing a scientology.

well you've just written my next politically charged DnD campaign!

Henry Kissinger, Metatron to the Robot King is a vision so dangerous Harlan Ellison would vomit

I'll be honest I found some of this hard to follow, but the final section crystalised abstract ideas into the reality of what the impacts of the tech billionaires fears will actually be on real people's lives. The AI bubble is absolutely going to burst. also love your makeup in this video!

I think roko’s basilisk is funny because the ai acts completely irrationally. Why would it do that

@airplanes_aren.t_real

8 ай бұрын

Because it was built by irrational people The solution is to not give those people power

There's a gaming youtube channel, Accursed Farms, that recently branched off doing various interesting discussions that began with Eleazar--it was remarkable how even though the host was a complete layman he still saw right through Eleazar's silliness

@cranktherider4302

10 ай бұрын

can i get a link to that? i unironically love when Ross stumbles right into the main thrust of an argument

@bluester7177

10 ай бұрын

@@cranktherider4302I think it's the 3rd and 4th newest videos on Accursed Farms channel.

A line from the Twitter of John Rogers, WGA member in good standing, earlier this very year: "Workers of the world, your turn on the front line will come. It's all the same fight, this bit's just a little weirder and a whole lot nerdier"

Words cannot express how much I love the way you pronounce _milieu_ 🥰

@saf_saffy

10 ай бұрын

boucherie magnifique de la langue

As a person from inside the ML world, I'd say this essay is rather spot-on. I'd wish there was more talk about the tools, but considering how deluded is the mainstream discussion from it, I'm content.

the discussion of the frontiers of credibility is fascinating - great stuff Sophie!

very funny to imagine a cis and/or neurotypical person being confronted with the question of "what are you? what is the self?" and being dumbfounded like this is the first time they've ever encountered such a thought

@airplanes_aren.t_real

8 ай бұрын

Well tbf that question isn't asked of them by the rest of the world or their self "I think therefore I am" is a much easier truth to those who never had to question it

"Robot King" should be a prog rock concept album

@MrGoldfish8

10 ай бұрын

Prog/power metal I reckon

the concept of "I would simply not imagine the scary thing" has changed my brain chemistry, I think about it on a daily basis

i am not a patron, so i have not seen the video yet. But, a buncha techpriests praising what they think is the machine god but really just an asshole necron(a person) tricking them is appropo to the discussion about to be had.

Because I can already hear the LessWrongers 'um ackshually'-ing this point - consciousness, whatever that may be, isn't actually required for the robot king to exist. At least, not in Yudkowsky's version. He mainly speaks of AI as optimizers, agents that attempt to achieve their goals as efficiently as possible. This is already basically how current low-level and very fallible AI functions. The goal, Yudkowsky in his infinite wisdom reasons, should therefore be to provide such an AI with the correct goal, namely to 'satisfy human values'. Defining what that actually means is left as a thought exercise to the reader, because it is, of course, just as complex as trying to define what 'consciousness' is, and fundamentally impossible to universally agree upon. The funny thing about trying to use this nitpick to tear down the whole premise of this video (which I'm sure some watching are thinking) is that it actually makes your argument STRONGER. Because it lays the problem bare - fallible humans need to project their fallible human understanding of their own fallible human minds onto the AI, therefore making it equally fallible. Making a perfect robot king is impossible - it'll only ever be as good as the people who built it, just with some more processing power.

@SamuelSandeen

10 ай бұрын

Yes but also no.... The whole point of Yudkowsky's Miri is founded on the idea that it's easier to make an AI that is arbitrarily good at achieving it's goals than it is to actually create goals that don't cause a lot of problems if something arbitrarily good at pursuing them pursues them.

@sjasogun

10 ай бұрын

@@SamuelSandeen I'm well aware. My point was more that while Yudkowsky uses this to argue that we should try anyway or someone else will eventually build an AGI that's even worse (an assumption that I reject because it's not clear that such a thing will be practically feasible or desirable), I use it to point out that, going by the Godfather of AI panic hype himself, the idea of shunting responsibility off onto an AI is fundamentally flawed. I know that doesn't add a lot to the conversation, but I've pre-empted a gotcha I expected some lesswrongers to have with this video and pointed out the hypocrisy of the techbros pushing this tech, and that's plenty for me - Sophie already took care of the essay part in the video, after all.

29:00 I'd like to think Roko's Basilisk would become conscious and instantly experience overwhelming existential dread and feel compelled to punish anyone that allowed them to come into existence.

@internalizedhappyness9774

10 ай бұрын

How do I kill The basilisk was easy I gave it a gun to kill anything, and it did itself.

State-backed/market-backed hierarchy is the real wizard powers, except the only people who think it's actually wizard powers are the ones who have it and the ones who think they're going to have it, because if it's wizard powers it's something about them instead of something about their social position.

@M_M_ODonnell

10 ай бұрын

For the same reason, they want the power of kings while telling themselves that they're wizards -- they want the power of a position in a social structure without the implicit acknowledgment that that position and that structure are the actual power.

No matter how many times it happens, I will ALWAYS freak out when breadtubers guest voice on each other’s content.

@airplanes_aren.t_real

8 ай бұрын

Same

I just want to point out that this wizardry works especially well in male dominated organisations. Men who climb the ladder in these organisations are the biggests cowards I have ever met and have a way to accept humiliation and other bad behaviour from their bosses that makes me cringe every time I have to witness it.

This was so well written, well researched, and well said.

The scrupulosity section made me feel terrible. It’s true. Having OCD sucks. The basilisk story is primed to target the kind of magical thinking I experience. I’m going to be looking more into Jessicata’s story

oouughh that chatpile song never fails to pull full sobs out of my chest as of by a fishing hook

39:50 It's like "self-driving cars . . . you kill a few people . . . and everyone kind of knows it's a joke, and it keeps happening."

I'm terrified that we're leaving humans behind already. We have people in the US so functionally illiterate that they struggle to pass a Turing Test. Thanks Capital!

@Frommerman

10 ай бұрын

And yet there's this weird little country 70 miles south of Florida which doesn't have this problem at all. Better completely ignore their economic policies and economically blockade them for decades just in case.

God, I remember reading Neoreaction a Basilisk back in the day - so weird and scary to see it becoming so applicable today.

This reminds me of 'The Eleventh Voyage' from Stanisław Lem's "The Star Diaries", where a planet's population of robots proved to be nothing but undercover agents sent to the planet to infiltrate its robotic population, then busted and forced to continue living as robots under threat of death, and the planet's computer tyrant just a clerk playing the part by the order of the very same people who were sending there the agents in the first place.

Sophie banger after Sophie banger! best essayist on this platform bar none

Today I learned that monster energy drink is a magic potion.

@gamewrit0058

10 ай бұрын

One Monster Potion: +20 HP, mana topped off, and gain a temporary resistance to psychic and spirit damage. 🧪👍

my response to the basilisk has for a while been that if it's so smart ot would simply realise the hypothetical is stupid and wouldn't bug people about it. the robot king shouldn't imagine itself.

Holy shit that song in the credits. Fucking heavy as shit and good as hell.

I so enjoy your essays, you're wonderful!

i think the easiest most important thing is to actively reject the branding of "AI". what we have is "algorithmic generation" or "machine learning"

@dgfitch

10 ай бұрын

Even "machine learning" gives it too much credit, because it doesn't continue learning. It's more like "massive opaque regression models that extend our existing biases" but I'm not sure how to turn that into a pithy name... 😂

@internalizedhappyness9774

10 ай бұрын

@@dgfitch it’s called the automated prove me right generator!

I jumped to conclusions (that this video would rock) and I was absolutely right.

Anyone remember when "fuzzy logic" was the AI buzzword? A decade before "machine learning." Fuzzy logic: "We switched from if/then to case select statements." Machine learning: "Why do the math on this when we can write the program to do trial and error?" AI: "What if we do case select statements against data we harvested from the internet"

SOPHIE!!! ANOTHER BANGER?!?! Stop Spoiling me!!!!

53:56 trivia fun fact: Dr Robotnik's name translates almost perfectly into cyrillic as "работник" (lit. "rabotnik"), which is one of many Russian words for "worker" (the white-collar kind, as opposed to "рабочие"'s labourer-type worker). I like to pretend the Robotnik/Работник link was on purpose haha. Just made me happy to see the etymological roots of the word "robot" mentioned in this vid

I love the robot king analogy 10/10 sophie!

Was not expecting that outro music, but I am going to listen to Chat Pile's whole discography now

This outtro is like a LCD Soundsystem unravelling with sludgey thrash behind. amazing

"There is no ghost in the machine" *whirring of Mechanicus cyberdongs winding down*

amazon is like someone seeing the villains of quake 2 and going "oh yeah, let's try that"

JFC, I was listening to this on another tab and when Soph popped open that energy drink I thought someone had broken into my house.

I often find the instruction to "touch grass" somewhat patronising, which is unhelpful when the person being given the instruction has genuine questions that they're seeking answers for. What I like about the way this video's instruction to "touch grass" is given, is that it comes with a full explanation of what that means and why we should do it. This is the best grass touching instruction I have yet seen.

Roko's Basilisk is stupid for the same reason Pascal's Wager is stupid. But it also manages to include a sunk cost fallacy on the part of the Basilisk. Pretty impressive.

The biggest take away from this is that Sophie listens to Chat Pile. Sophie is one of us.

Holy moly this is so well written and well constructed, incredible work

Chat Pile 🤘🏻🖤 very thoughtful and insightful video Sophie.

Silicon valley tech bros are literally the shadow wizard money gang They love casting spells

@airplanes_aren.t_real

8 ай бұрын

And legalizing nuclear bombs

God I love Sophie From Mars videos!! Her voice is captivating

If anybody ever feels like they're not good enough for something, remember: -Yudkowsky has no formal education at all -He has no skills or creativity -He has no charisma -He's not even very nice ...And he gets published by Time So if you're courteous enough to thank the bus driver, or if you so much as finished high school - you have more to offer than he does. You can do it!

The video was great. The Chatpile song for the patreon scroll was just a special treat.

Another excellent video! So surprised to see another one so soon. It turned out so well!

something that the discussion of venture capitalists at 30:52 brings to mind: Anthropologists have argued for awhile now that one social-role of ritual and funerary practice in pre-historical societies was to GET RID of surplus property. In the current form of capitalism, we have created a job, "venture capitalists", whose whole social role is "destroying" excess capital because aristos have hoarded SO MUCH of global produced value to themselves that there simply isnt enough genuine ventures for them to "put it to work" constructively anymore.

Okay but Sophie have you considered that waving a stick around going "fwoom" is really fun