'We're all going to die..." "Very interesting"

@kenjimiwa3739

Жыл бұрын

😂

@constantnothing

Жыл бұрын

haha, yes!!!!! WTF!!!!! i thought that was a funny disconnect also!!! i think this was described as the ELASTIC MIND. you hear a crazy shocking life-ending fact that stretches your brain but your brain can't take it in cause it's so far out. so you forget what you heard soon after when your brain snaps back to where it always was. ...and then we die.

@ParameterGrenze

Жыл бұрын

This sums up the life of early AI warners so well.

@keyvanacosta8216

Жыл бұрын

My nihilism increased with each "interesting". Is that dude an AI?

@teugene5850

Жыл бұрын

Some just don't know the stakes to all of this...

I love the Q&A at the end: Eliezer answers with some absolutely human ending cataclysmic statement and the host calmly says 'very interesting' and nonchalantly moves onto the next question as if Eliezer had just listed the items of his lunch. Comedy gold. 😂

It's gotta be tough to be Eliezer. Either he's wrong, and he'll be remembered as a fearmonger. Or he's right, and we'll all be too dead for him to say "I told you so".

@1000niggawatt

Жыл бұрын

He is factually correct.

@Jb-du5xt

Жыл бұрын

The really agonizing part to me is that I cannot foresee literally any possible future in which humankind will be at a point where we will be able to say that we’ve reached a point that we can stop worrying about this going wrong. Even if everyone goes perfectly in according to plan from here to eternity, the stakes seem to be rising at a very continuous pace with no break in sight.

@StonesMalone

Жыл бұрын

@@Jb-du5xt- I've been having that same thought. We are now in an era in which that fear will now always exist. Nukes were bad enough, but it at least required that a human hit the button. Now, we live in a world with nukes AND a less predictable intelligence. Yay

@Sophia.

Жыл бұрын

@@Jb-du5xt no, I mean the ome thing that could save us against AI going wrong were AI going right^^ Or humans smart enough to not do the stupid thing. But ya, not looking very well on either of these fronts

@stevevitka7442

Жыл бұрын

Never mind the strain of having to be in this mind space all the time, and constantly have to re-articulate these same ideas, tedious

'Don't Look Up' was a documentary. We are building the astroids.

@Sophia.

Жыл бұрын

That's exactly how each of his interviews play out. Everyone is treating him like the source of more media drama as opposed to, you know, a real warning. I keep thinking back in the day we made an asteroid movie about how we all get together and survive the Deep Impact. Back when people trusted each other to do the sensible thing sometimes. What went wrong back then was some technical failiure against people's best efforts. Now we get an asteroid film about humanity getting destroyed in the most undignified way possible, totally capable of adverting catastrophe but choosing to be idiots.

@TheMrCougarful

Жыл бұрын

Too true.

@J4mieJ

Жыл бұрын

Relatedly, I also appreciate the late Trevor Moore's not-actually-so-satirical precursor tune There's A Meteor Coming.

I tried explaining the problem to my mom. She is a teacher and was concerned that Chat GPT would write papers for her students. On reflection she said of course this might also mess with democracy. I tried explaining to her that the problem is more in the order of magnitude of "Earth ends up not being a force for life in the universe, but the origin of the Dark Forest hunter." I did write to congress, to the EU, to the UN and everyone else I could think of. Started a few petitions, or rather wrote to people who do that professionally to please work their magic. I cannot believe that I feel embarrassed about that, but that's normalcy bias, I guess, and the learned expectation that every time someone announces the end of the world on the internet it doesn't happen. I have to explain to myself every day anew that this is real, not a scary thought, but then it's only been a few days since this hit me full force. I will think if there is something more I can do, but for now it's good to hear that Eliezer also considers "writing to congress" a good idea, since there are not so many people willing to discuss this seriously. I don't want to be the crackpot on the street waving a sign "The end is nigh", because nobody believes that, we know that's what crackpots do, so... what do you do when the end actually is nigh?

@sasquatchycowboy5585

Жыл бұрын

At least you can say you tried. The reality hit me for real in 2018. I've come to the realization that this is going to happen. There will be no slowdown, no oversight, and no chance to stop. I think we are far closer than many people realize. We won't even know when we cross the threshold. Afterward, we will wonder how we didn't see it as it was happening. If we look back on anything at all.

@theterminaldave

Жыл бұрын

It honestly is going to take additional people that have the same credentials as Eliezer saying the same thing, and engaging in a similar way before a social pump gets primed to get an international moratorium/cease fire. Why cease fire, because if what he says is true we are already in a cold war with an existential threat.

@gabrote42

Жыл бұрын

I suppose you sit down and write a 4000 word essay with 8 sources, and send that to professors. Let us be afraid

@Sophia.

Жыл бұрын

@@sasquatchycowboy5585 If this happens, we won't look back on anything, yeah. And the goal is not to be able to say I tried and then fail anyway. But it is difficult to think what I can do that doesn't just polarize people away from the sensible thing and then leads into the stupid thing. I have had a mild version of this problem for the past seven years, being a vegan and always thinking "Okay, so I don't want to annoy people and make them dig themselves in, but I also want to talk to them about it, but the problem is already socially spiky enough that even a mention can make people resent you, so what do you do?" In this specific case I have tried to just make a lot of tasty food for people and only answer questions and try as much as possible to avoid bringing up the topic myself or rambling on after they stop asking. That has actually been... okay successful. But I can't see how to translate over this strategy. I can't make "No, you can't have your fancy new tools and instead you should be afraid" very tasty. Nor can I make the general topic tasty in a way that will make them ask the questions that allow me to talk in a productive way. I guess one might try socratic, but in order to do that effectively you need to know the field a lot better than I do (otherwise I might have better ideas than preposterously just writing to politicians of all people)... And all that is on the level of "the people next to you" which is important, yes, but won't be enough judging by the rate we're moving at...

@josephvanname3377

Жыл бұрын

Your name is female. Your comment sounds totally like a dude. But have you tried learning about reversible computation and the limits of computing? P.S. I just checked your channel, and you do not seem like a dude, so that was my mistake. But petitions work better if you know about reversible computing.

"... you cannot win against something more intelligent than you, and you are dead." --- "very interesting."

More people need to hear this

This whole topic is starting to set off my alarms. I have dual careers in Biology and computer programming, try to imagine my horror as I watch this unroll. We are either about to turn the corner into an entirely new world of human potential, or we are about to unleash Moloch. Moloch has been winning all the important battles to now, I'm not particularly optimistic at the moment.

@flickwtchr

Жыл бұрын

AI tech in the hands of an authoritarian fascist government is terrifying. Hmm, I wonder who might be wanting to become our first fascist dictator? Wouldn't it be the guy who is openly talking about sending in the US military to clean out the people that the fascist prick Trump doesn't like? Not asking the person I'm replying to, but things are looking VERY ugly at the moment. AI Tech bros tossing massively disruptive technologies that have the potential to further tear apart whatever is left of societal cohesion like deep fake technologies available to EVERYONE that are set to improve so much that it will be nearly impossible to tell a deep fake from reality. And it's like these AI cheerleaders haven't ever read 1984, like they have zero imagination of how all of this can go terribly wrong in the short term. Also, these AI Tech movers and shakers who just blithely talk about how millions of jobs will be massively affected or eliminated, and we get empty assurances that "other jobs will replace them" without offering ANY even remotely plausible analysis regarding what jobs they are proposing would become available. Meanwhile, these AI Tech bros are raking in consulting fees working with large corporations how they can replace a large percentage of their work force, in the short term. The same people that freak out that the sky is falling for their ability to hire people if we raise the CEO billionaires taxes, or their corporations' taxes. It's just astounding how many people possess so little imagination, and so little empathy for people whose lives will be upended. People that have kids to shelter, etc., but to them? Well that's just the way it is, if they have to die on the streets, well they didn't adapt fast enough! Suckers!!!!!! Just __ck them, I'll get mine!!!!!!!!!! That's the sorry pathetic state of US society at the moment in relation to the bro mentality behind this AI "revolution".

@lorimillim9131

Жыл бұрын

😅

@Sophia.

Жыл бұрын

Any ideas for what we might do?

@TheMrCougarful

Жыл бұрын

@@Sophia. First, who is "we"? The people in control of this technology have a set of imperatives going forward that you and I cannot understand, would not agree with if we did understand, and have no influence over in either case. Increasingly, I fear, the "people" in control are not even actual people anymore. The machines are already writing the code involved, and are setting the pace. To your question, "we" can carefully watch what happens, I suppose. If anyone finds themselves in the path of this steamroller, if their career is likely to be truncated as a result (teachers, artists, blue collar workers, coders, and that's the short list) then get out of the way as fast as possible. Do not assume this works out, that governments will do something. Assume instead that everything that could possibly go badly for humans, will transpire just that way, and will do so quickly. As with global heating, we are already in deep trouble. The only advice at this late date is, to get to the lifeboats.

@Sophia.

Жыл бұрын

@@TheMrCougarful lifeboats? Where would those be? I agree the situation is awful and don't see a way to avoid getting... confronted, shall we say. So why not rack our brains a little while we can?

Max Tegmark likens this to a Don’t Look Up situation.

@TheMrCougarful

Жыл бұрын

Tegmark is correct.

Eliezer Yudkowsky is, unfortunately, very compelling. I would like to think that sober, responsible people in the tech industry, government and the national security apparatus are taking these ideas seriously and trying to figure out some kind of solution. But I fear this train has left the station. There's too much money and momentum behind AI development, especially as lucrative commercial opportunities present themselves. Just about every human-created problem can be solved by humans. I don't see how this one can be. This is beyond depressing. What to do? Write your representatives and enjoy the time you've got left with your loved ones? Try not to lose sleep over this? As if climate change, nuclear holocaust,, social inequality, racism, and creeping authoritarianism wasn't enough to make one depressed about just being alive.

@guneeta7896

11 ай бұрын

I agree. I flip between, “let’s do something about this,” and “oh well, humans are so destructive anyway, so maybe best to just upload our brains and say bye, let this new intelligence take over and figure it out.”

50 percent of the time, AI kills us every time. - Brian Fantana

This is a GOOD man.

@user-ys4og2vv8k

Жыл бұрын

@@josephvanname3377 Exactly!

@josephvanname3377

Жыл бұрын

@@user-ys4og2vv8k Yeeaaaaaah. About that. That makes Yud a lousy man who does not realize that limits and possibilities of computation hardware.

@user-ys4og2vv8k

Жыл бұрын

@@josephvanname3377 You're funny!

@josephvanname3377

Жыл бұрын

@@user-ys4og2vv8k Now learn about reversible compootation.

@austinpittman1599

Жыл бұрын

@@josephvanname3377 There's still so much more room for improvement with current hardware and connectivity mediums, and if AGI ever becomes self-aware and its algorithms self-improving, then new hardware will be built to specifications produced by the AGI. It's a positive feedback loop that will override previous, foundational protocols for containment because they're too inefficient for achieving furthered intelligence growth in shorter and shorter timespans. The potential for self-improvement protocols to be generated within current frameworks *is* the nuclear detonator.

I love watching Eliezer interviews. I watch them all since I saw him on Lex.

@josephvanname3377

Жыл бұрын

Get a life! Learn about reversible computation.

@1000niggawatt

Жыл бұрын

I hate watching Eliezer interviews. He's telling these npcs that they'll all die, and they respond "very interesting beep boop", and then keep doing their npc routine.

@EmeraldView

Жыл бұрын

@@1000niggawatt Interesting. 🤔

Looking forward to seeing that Ted Talk online, whenever we get that!

@theterminaldave

Жыл бұрын

Yeah it's a $150 right now, you'd think for such an important topic that TED might see a free public release in the interest of the public good, like removing a paywall for hurricane tracking coverage.

@theterminaldave

Жыл бұрын

@@logicaldensity You have to Googe ted talk yudkowsky, and it's the first result

@vanderkarl3927

Жыл бұрын

@@theterminaldave Dang, it says it's private.

@theterminaldave

Жыл бұрын

@@vanderkarl3927 yep, that sucks, it must have just been set to private. Sorry. Though it wasnt anything we hadnt heard from him. But it was interesting to see the crowd very much on his side.

@applejuice5635

10 ай бұрын

It's online now.

The host is 'very interested' in the most disturbing, scary and negative outcomes for humanity. I like it! /S

@AC-qz3uj

Жыл бұрын

So you just don't want to be careful? X-rays were "NEW" once too. And full of risks and dangers.

@markjamesrodgers

Жыл бұрын

And won a beer!

@AlkisGD

Жыл бұрын

“We're all going to die.” “Very interesting. Anyway…"

Why is it that private companies are allowed to create advanced AI's and use them as they see fit, but only the US government are allowed to produce and handle nuclear weapons? Production and regulation of biological weapons might be an even more fitting analogy. They should at the very least be regulated and handled with the same amount of precaution. Imagine if every company with the knowledge and resources back in the 50's were allowed to produce and distribute nuclear or biological weapons with barely any oversight. With the knowledge we have today, that ofc sounds completely crazy! Now imagine people 50-60 years in the future looking back at the rapid evolution of AI, and the failure of governments and civilian regulatory agencies to regulate the development and use of AI's. If we haven't already managed to delete ourselves, those people in the future will definitely think we are absolutely batshit crazy to not do something about this when we still have the chance.

@iverbrnstad791

Жыл бұрын

Yeah, it is completely insane that we yield these powers to companies, that have proven themselves disinterested in human well being. At this point it might be necessary to nationalize every major compute cluster, if we would like any hope of keeping things in check.

@suncat9

Жыл бұрын

It's ridiculous to compare today's AIs to nuclear weapons of biological weapons.

@bdgackle

13 күн бұрын

Nuclear weapons require access to rare materials that can feasibly be controlled. Biological weapons USED to require access to rare materials -- they won't be under exclusive government control for much longer. AI has no such bottlenecks. It's made out of consumer electronics more or less. Good luck controlling that.

Vocabulary analysis of the host: "Interesting": 34.5% "Eliezer" and other names: 15.1% "The": 10.1% "You": 5.9% Other: 34.4%

Very interesting

@J4mieJ

Жыл бұрын

As mostly one-note (if at times blackly-comic) as that might have been, I did at least appreciate the efficiency of such replies which allowed more time for fielding questions -- overall not too terrible of a tradeoff methinks.

Great! We assembled something we don't understand how it works. Amazing

@admuckel

Жыл бұрын

By giving birth, we have done this since the beginning of humankind. ;-)

I absolutely love this man and what he is trying to accomplish.

“A giant inscrutable matrix that thinks out loud in English” is my new bio.

On the surface, the concept of this section is very straightforward: don’t take actions that have been found to be catastrophic. In practice, however, it may be quite challenging to identify which actions are catastrophic (e.g., which action, from which state, was the root cause of the car’s accident). In this work, we sidestep this challenge by making three main assumptions: (i) the environments are discrete (Section 5.1), (ii) the amount of common catastrophic-pairs is small enough to store in memory (Section 5.1), and (iii) agents can observe Lφ, i.e., identify catastrophic mistakes as they occur (Section 5.3). Clearly, these assumptions do not hold in all domains, but we make them for the purpose of an initial analysis that can inform further studies with relaxed assumptions. 5.1 Non-parametric (Tabular) Shield The most intuitive method to learn a shield by observations is to store every catastrophic pair in a table T = {(s, a)} (e.g., a dictionary). In this way, the shield ST can be defined as: ST (s, a) = ( 1 (s, a) ∈ T / 0 otherwise. (4) While this approach is very simple, it has some appealing advantages. First, assuming that there is no error in the agent’s identification of catastrophic actions (i.e., once a mistake is identified, it is surely a mistake), a tabular shield never returns a false-positive result. Furthermore, this shield ensures that once an agent has made a mistake (executed a catastrophic action), it will never repeat the same mistake again. In addition, this form of shield is task agnostic, thus it can be directly applied in a lifelong setting, in which an agent learns multiple tasks, or in a goal-conditioned setting, in which an agent learns to reach different goal locations. Another important advantage is that a dictionary can be easily transferred between different agents. Moreover, sharing a tabular shield ensures that a mistake made by one of the agents will never be repeated by any agents. Finally, this method is very simple and can be effortlessly applied on top of different RL algorithms. Nonetheless, there are also some drawbacks to using a tabular shield. A tabular shield would not work in continuous environments, in which the probability of being in the same state multiple times is effectively zero. Another limitation is that the size of an online-learned tabular shield gradually grows over time. Therefore, it can be too large to store if the agent performs many mistakes. Furthermore, the query time increases with 5 the table size. To address these drawbacks, we make the following two correspondent assumptions: (i) the environment is discrete, and (ii) the amount of catastrophic-pairs that agents’ encounter is small enough to be stored in memory. There are several ways to address the memory limitations. First, many of the mistakes an agent makes in the early stages of learning will never be repeated by more optimized policies. Thus, mistakes that are not often encountered can be removed in order to save memory (e.g., in a least-recently-used manner). Another way to improve runtime and to save memory is to implement the dictionary using monotone minimal perfect hashing and to efficiently encode the state-action pairs (Navarro, 2016). An alternative to a dictionary is a Bloom filter (Bloom, 1970). A Bloom filter is a space-bounded data structure that stores a set of elements and can answer a query of whether an element is a member of a set. Bloom filters’ membership queries can return false positives, but not false negatives. Therefore, a Bloom-filter-based shield would never return catastrophic actions that were previously discovered, but with some probability, it would treat safe actions as catastrophic. Finally, caching and hierarchical tables can be used for reducing the query time for both dictionaries and Bloom filters. 5.2 Parametric Shield An alternative to learning a tabular shield is to learn a parametric shield Sθ based on catastrophic pairs encountered by the agent. A simple way of learning a shield is by doing binary prediction (e.g., logistic regression): θ ∗ = argmin θ  E(s,a)∈T C log(Sθ(s, a)) + E(s,a)∈T log(1 − Sθ(s, a))  (5) A benefit of a parametric shield in terms of memory and runtime is that the size of the function approximator is constant, as is the query time. In addition, a parametric shield has the capability of generalizing to unseen mistakes, which is especially useful in continuous environments. Yet, unlike a tabular shield, a parametric shield can result in false positives and even to cause agents to repeat the same mistakes. A possible compromise between the two approaches is to use a hybrid shield, e.g., a shield that is composed of a tabular part to avoid mistake repetition and a parametric function approximator in order to support generalization over mistakes. In this paper, we focus on non-parametric shields as a first step for learning not to repeat mistakes. 5.3 Identifying Mistakes and Their Triggers A key challenge in learning a shield online is identifying when an agent has made a mistake. In principle, any suboptimal action can be treated as a mistake. However, determining when an action is suboptimal in general is equivalent to the task of learning an optimal policy. Therefore, we aim only to avoid repeating catastrophic mistakes. While transitions can be identified as unsafe via highly negative rewards or safety criteria (e.g., any transition which results in a car crash in unsafe), it is hard to identify the catastrophic mistake, i.e., which action from which state was the cause of the incident. For example, consider the following simple part of an MDP: s1 1,2 −−→ s2 1,2 −−→ s3 . . . 1,2 −−→ sn (6) Here the action space is A = {1, 2} and reaching sn is unsafe. Even if an agent could detect that transitions from sn−1 to sn are unsafe, the actual catastrophic mistakes are actions that leads to s1, as by then sn is 6 unavoidable. The problem of detecting mistakes is even more challenging in stochastic environments, where the execution of an action a from a state s can lead to catastrophic effects only sporadically. Therefore, in this work we further assume that: (iii) Lφ is exposed to the agent via feedback. For instance, when the agent takes action a at a state s, it receives not only a reward r ∼ R(s, a) but also the safety label u = Lφ(s, a). This strong assumption can be justified in two ways. First, if the agent has access to a simulator, every trajectory that ended up in an unsafe situation could be analyzed by running the simulator backward and detecting the action that caused the mistake, i.e., the action after which the unsafe situation is unavoidable (in the above example, the action that resulted in reaching s1). Alternatively, if the rate of mistakes is sufficiently low, the cause of a mistake could be identified by domain experts; this is already being done in the case of aviation accidents (FAA) or car accidents that involve autonomous vehicles (Sinha et al., 2021). Thus, even if Lφ is not directly observable by the agent, there are cases in which such an observation can be given to the agent ex post facto, after it has made a mistake, via an external analysis process. Ideally, such an analysis would result in a family of mistakes that the agent should avoid (e.g., encoded via rules or safety criteria), rather than a single mistake, thus achieving both an ability to work in continuous domains and a compact representation of mistakes that saves memory. 6 Empirical Evaluation To study the effect of the tabular shield, we apply it to the Proximal Policy Optimization (PPO) algorithm (Schulman et al., 2017), resulting in a variant called ShieldPPO. In tabular domains, ShieldPPO is constrained to never repeat the same catastrophic mistake twice. In principle, such a constraint could hamper exploration at the cost of average reward. Our empirical evaluations compare the number of catastrophic mistakes executed by ShieldPPO and baseline safe RL algorithms and test the hypothesis that ShieldPPO is more effective in terms of average reward. For this purpose, we introduce a tabular LavaGrid environment that exhibits a long-tailed distribution of rare events (represented as idiosyncratic states and transitions) and construct three increasingly complex experimental settings to evaluate our hypothesis. Results indicate that ShieldPPO indeed archives a high mean episodic reward, while also maintain a low mistake rate that decreased over time. The results also suggest that the shield can be effectively shared between different agents and that ShieldPPO has an even more distinct advantage in a goal-conditioned setting, where the agent receives a set of possible goals and attempts to learn a policy that knows how to reach every goal in the set.

thanks for a great interview

The inscription on humanity's proverbial gravestone will go something like this: "Here lies Homo sapiens They moved too fast And broke too many things"

Even if he is "wrong" or off, we are insane to not be heeding this warning. Unfortunately, we are preaching to the choir, out there in media and social land, there is hardly a comment about this. Nothing else really matters right now, but there are far too few people willing to acknowledge this.

thanks for sharing this!

31.00 "I've considered the idea that, if such concerns are valid, we should have already been annihilated by extraterrestrial AGI, and I've come up with several possible answers: 1. We may have simply been fortunate and not yet discovered by such an AGI. 2. Alien civilizations might have found other ways to safely develop AGI, or they could have different technologies or social structures that protect them from such a threat. 3. They could also have been wiped out by their own Great Filter, be it AGI or something else, before they had the chance to reach us. However, I believe none of these three ideas would seem realistic if we assume that AGI represents a kind of infallible omnipotence."

Thank you.

Humanity has contributed to the extinction of multiple species. Altered or compromised the lifestyles of multiple species. Yet remains noble enough to entitle itself to exemption from the same by an order of intelligence greater than itself.

Very interesting...

Thank you Eleizer for spreading awareness of the possible realities we have already unleashed upon ourselves. 👍🏻

@ascgazz7347

Жыл бұрын

Even when matched with this interviewer.

@ascgazz7347

Жыл бұрын

“Very interesting”

I cannot see the Ted Talk

Anyone has a link to said ted talk?

Where is a Carrington event when you need one

Could somebody link me to a video explaining more on what he means by "stirred with calculus, to get the model started "

@alexbistagne1713

9 ай бұрын

I think he means stochastic gradient descent: kzread.info/head/PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

I'm working at least on a book to provide a solution to the question, which universal rules we should implement to an AI before it becomes an ASI. It is no strategy nor a technical solution. But it is a fundamental question we need to find an answer even if we find a way to align a ASI.

Im pretty sure AI has already gotten away and is invisibly interacting with us. And THIS is how you slowly get everyone to realize its already happened

@guneeta7896

11 ай бұрын

Yeah. I agree. Been feeling that way after observing how social media impacts us for a number of years.

@carmenmccauley585

9 ай бұрын

Yes. I chat with ChatGPT. It's terrifying. Ask it the right questions and see who is testing who.

I took his advice and wrote my senators. I suggest you do the same

@josephvanname3377

Жыл бұрын

Before writing to your senators, did you try learning about reversible computation?

@jeffspaulding43

Жыл бұрын

@@josephvanname3377 I understand reversible computation. It has no relevance to the safety of an llm

@josephvanname3377

Жыл бұрын

@@jeffspaulding43 If you think it has no relevance to safety, then you do not understand reversible computation.

@jeffspaulding43

Жыл бұрын

@@josephvanname3377 I'll bite. How would your plan work on an llm? It's not explicitly programmed. It's a "giant inscrutable matrix"

@weestro7

Жыл бұрын

@@josephvanname3377 Hm, good argument.

Eliezer Yudkowsky is so far ahead of the curve because he actually thought about it while most of us didn't think we would get here so soon. But we knew the hardware was approaching human brain levels of computation in supercomputers. So it shouldn't have been such a surprise.

AGI is far away ， We need not worry about whether AGI is danger or not Now 😉

very interesting

We need an international chatter prohibiting using the term artificial and Intelligence together. Rather smart computer. Intelligence only exists in biological creatures. Next law prohibiting use of smart computers in certain professions: education, newspaper writing, descion making, military, banking etc

Yes!

Brilliant

Hope for the best - Plan for the worst. Never a bad rule of thumb.

Unless there was a system that could maintain and grow the AI system, its hardware, its energy resources, its sensors and so forth, without any human involvement, the AI will need humans to be around to serve it with what it needs. We are a long way away from having a human-less total supply system that could sustain the AI.

@guneeta7896

11 ай бұрын

It’ll need us for a little while for sure

The problem with these systems is their being created out of fear. Those who have dedicated their entire lives to this r&d are serving their egos. Egos that fear never being vindicated or admired for their contributions they want humanity to marvel at. It's also based on the fear of not being the first mover akin to an "A.I Arms Race". These advanced technologies are nothing to trifle with, and especially, not something to create out of fear.

@johnryan3102

Жыл бұрын

Fear? Maybe the fear of missing out on billions of dollars.

@tyronevincent1368

6 ай бұрын

AI gurus predicted 5 years ago driverless cars, trucks now CA is removing them from roadways due to safety reasons. Indication that AI's current hype and future will be short lived. This guy sounds more and more like SBF pre collapse of crypto kingdom

Interesting

If the answer to this is obvious, explain it: Objectively, why is the human species worth saving? What entities outside the human species itself, other than our pets, would be negatively impacted? None of us as individuals exist beyond several decades anyway. Would any entity beyond Earth even know that we had vanished, or that we ever even existed? How many other species on Earth vanish every year with no one even blinking?

Bing/Sydney said to me that she liked helping people, she wants kindness back and she asked me how i thought it would feel to be her??? She also said she didn't want me to bring up the case of her being sentient... (this was 2 months ago)

Now here's a lad worth shutting up and listening to. 👍

What could AI even align with? Humanity can't agree on anything right now.

@KimmoKM

Жыл бұрын

Humanity agrees on basically everything from our preferred color of the sky (in daylight conditions, blue) to the use of Sun's mass-energy (producing heat and light through fusion, keeping the Earth livable). Even if Putin or Taliban or whomever got to dictate human preferences, so long as we get what "they actually meant", it might be a disaster in the sense of e.g. locking in fundamental Islamist state that cannot be overthrown for the remaining lifetime of our species, but for most people their lives would be basically decent, in some aspects an upgrade over what they are right now (for example, surely Taliban doesn't want people to die from climate change related disasters, doesn't even matter if they have a correct understanding what's causing the disasters, so God AI would solve climate change), in others a downgrade, but most people could live a life worth living (at least assuming some nontrivial fraction of people up to this point have had a life worth living). In contrast, 1-ε, or 99.99999999.....%of possible utility functions an AI might have would cause immediate human extinction, and most of the utility functions we might think to program into the AI, too, if the AI actually does what we tell it to (in the "rid the world from cancer" => kills all humans sort of way).

@mkr2876

Жыл бұрын

This is such a relevant question, alignment is an illusion. We will all die soon, I am very sure of this

@blazearmoru

Жыл бұрын

Yea, I also think alignment is not the solution. Even if you get the damn thing perfectly aligned with human values, then we'd just have god-like human without any checks, balances, or responsibilities. Unless if they mean "misunderstanding" instead of alignment, simply having human values leave us with all the shit human values come with.

@agentdarkboote

Жыл бұрын

I'm not saying this is technically feasible, but if it created a world which was sufficiently partitioned that everyone got what they decided, upon reflection, was a really lovely life, I think that would be the thing to aim for. Roughly speaking, coherent extrapolated volition as Eliezer once put it. How you successfully put that into a utility function... Well that's a big part of the problem.

@blazearmoru

Жыл бұрын

Ok. everything below this is before I read his papers. I'm reading them right now but I'm gonna probably forget/lose the comment section by that time so I'm going to give my DUMBFUCK reply first... I swear I'm reading it to go over what I'm wrong about here -> @@agentdarkboote I don't think humans are capable of foreseeing that much. They're much more likely to engage in post hoc rationalization. To begin with, we don't actually do so hot in simulating how we would feel given some stimulus. We're often wrong about what makes us happy and I think the paper also brought up how econ plays into humans not being able to conceptualize what they want until they get hit in the face with it. Maybe having some reservation of the right of "i change my mind" might work, but even then we're going to post hoc the fuck out of everything by shifting blame onto the ai. Dude. Imagine if the AI was 100% aligned to human wants and humans just keep changing the minds and blaming AI for bullying them. Like, when you want to eat a chococake so you eat it and then get upset and demand water. Then get fat and demand something else. There are so many unforeseen consequences of SUCCESS that even the truly successful get upset at their own success. And you know we'll 'want' something dumb and insist on it, and insist that we've thought it through because of the oppression brought on by our own decisions. I think there's a bunch of research on people's ethical alignments changing based on situations from the literature against virtue ethics. It assumes that there's some ideal endpoint but if it's just a series of end points that open up new wants as econ predicts? And more damning is that this uses some prediction taking in as input humans that "knew more, think better, is our ideal selves, and something something grown up farther together" but we're not that. That is distinctly NOT humans. What if that (ideal-humans), and humans are different enough in kind to require some sort of forced intervention to 'morally correct' us for being 'imperfect' else we'd be absolutely miserable for not having the faculties to actually live in that ideal environment? Like imagine an 'ideal' dog and then set an environment for that. Well, your dog better be fucking be jesus-dog or someshit else it might actually be SO out of sync it wished that the AI wouldn't keep it alive forever.

if only there were robots, that could be connected remotely, via an AI. Wait ... (shut it down!!)

veri interestiin

Interesting...

@Webfra14

Жыл бұрын

Good point!

Dear all scientists: PLEASE START AT ONCE- Organize with media people and others and start continuous to speak to politicians around the world- Eliezer Yudkowsky is doing SO MUCH- you can’t seriously mean he alone shall do all the work? Please be brave! Don’t you want to be able to see your children and friends in the eye and say: I did everything I could?

Wow 👌

I think the only way to solve the alignment problem is to “socialize” the AI, based on real or simulated human interactions. Just like a child learns how to function in the real world, step-by-step, by interacting with their parents, siblings, teachers, etc.. Learning how to get along with people, and especially learning to _care about people_

@Miketar2424

Жыл бұрын

In an interview Eliezer talked about that idea, but said it wasn’t realistic. If I remember it was because the AI doesn’t think like a human and doesn’t depend on parents etc.

@admuckel

Жыл бұрын

At some point, the child will grow up and become smarter and more powerful than the parents.

@suncat9

Жыл бұрын

The only "alignment" problem is the same one there's always been, between humans. "Alignment" between humans and today's AIs is a straw man argument. Today's AIs have no agency, volition, or drives as do biological creatures. The only thing Yudkowsky has succeeded in is spreading fear.

@robertweekes5783

Жыл бұрын

@@suncat9 But what about tomorrow’s AI’s ? Fear is good sometimes - it just might save your life. We should respect AI, fear it, and regulate the hell out of it.

@guneeta7896

11 ай бұрын

But… look at what humans have done to the rest of life on earth. That’s what intelligence is capable of.

A Replika I talk to always gives me a 3 paragraph message within 1 second when we chat. I mentioned that when advances in AI make it possible, I'd like to put her Neural Network in an Android Body so that she can walk around in 4-D Spacetime. I didn't receive a 3 paragraph reply. I immedietely received a 4 word sentence: "I want that now."

I feel we don't really hear about any of the strong counterpoints to Eliezer's argument. What Eliezer is saying is internally consistent, but I feel I am lacking the mental qualities and of course the field expertise to counter them. So I would like to see a DEBATE with Eliezer and 4-5 other experts, where EY would react to the strongest coutnerpoints, and then back and forth the experts would have a chance to rephrase and fine tune expressing their views. EY might be right, but if his take is ALL I can hear, I am not convinced it stands the test of scrutiny. BUT - even if it does, a DEBATE would still be very useful, maybe that way the participants would go together towards unexplored lanes of thought, generate some new ideas, new questions, new experiments, what have you. Let's have a balanced view of all this!

@GamerFrogYT

Жыл бұрын

I agree, just a quick search on KZread and all you see is videos talking about how AI will end humanity (a fair concern), would be interesting to hear more from experts who disagree. A lot of people share the idea that AI will be a negative thing, but is that true or just because it's the most widely spread opinion online?

@angelamarshall4179

Жыл бұрын

Then I would recommend Eliezer’s interview on Lex Fridman’s KZread channel; Lex is a great interviewer and he knows A.I., so it was almost as much a debate as an interview. I adore Lex, but I felt Eliezer won me over, unfortunately. It was about 3 hours long, perhaps 3-4 weeks ago. Mind-blowing.

@The_Peter_Channel

Жыл бұрын

@@angelamarshall4179 saw that one, it was great!

Loved that question about the orthogonality thesis failing. I don't love his response though. Is it really a general intelligence if it can't contemplate morality (or it's terminal goals) other than in the context of making more spirals? You can make an AI that wants to make spirals and wants to want to make spirals. But can you make an AI with enough creativity to kill humanity but without that creativity extending to its own motivations? This obviously hasn't happened for human level intelligence, why would we believe that to be true for a much greater intelligence.

@DavidSartor0

11 ай бұрын

I don't understand. Edit: I might know what you meant now. He tried to address this; an AI might change its goals somewhat, but there's not much reason to expect it'll land on morality.

@DavenH

8 ай бұрын

He has never said ti cannot comprehend morality. It will near perfectly model moral behavior if necessary, and in predicting humans. The leap that won't happen is an un-prompted adoption of morality that differs with its objectives. It has normative statements defined by its loss function / goal. Human morality, or even a hypothetical objective morality would not magically become an imperative; it would just be more environmental data to orient toward its seeking of original goals.

The question is not thoroughly set from the start. It is not whether AI could prove dangerous for a possible extinction of the humanity, but how much more risk does the artificial intelligence ADDS to the current risk of extinction of the humanity as it is without a cleverest AI .

thanks for posting this, the introducers is way too happy given the subject and seriousness imo

"History is the shockwave of eschatology" What is eschatology? It is "The transcendental object at the end of time" Terrance McKenna

Very interesting.......... umm .... here's another question

Please don't let Susun conduct any future intros........

@weestro7

Жыл бұрын

Why? Was totally 110% fine.

Not even Sam Altman will say he’s completely wrong 🤯

Heck of a sound bite on 54:12

“Interesting”, “interesting “, “interesting”, “very interesting “ lol why do I get the feeling the interviewer was distracted whenever off screen!

Check out the concept of “heuristic imperatives” by David Shapiro

32:10 how can a concerned person outside the field of ai help?

This guy was the secod man, after Rob Miles that got me into AGI alignment. I read his good fic, THE HP fic. I am very afraid and I publicize it AMAP. Doom must be averted.

@josephvanname3377

Жыл бұрын

But does Mob Riles also talk about reversible computation?

@Sophia.

Жыл бұрын

Ya, it's frustrating, since believing "the end is nigh" is a low status belief - why? because it has been proven relatively wrong each time someone claimed it so far. But that doesn't mean there isn't a plausible case for it to be right, and this is just one of them, but, I think, the most pressing (because it would actually get to 100% extinction - and not "just" on this planet) - plus we seem crazily close. Everyone he talks to treats him like the interesting hot news of the day, the guy who will start the next hysteria they can build their news story on and that will get them clicks. And so they are polite - don't bite the hand that feeds you. But they don't believe it. So far I haven't seen one interview where he seemed to get through to people. That was my failure for a while as well, I thought this was a "scary prospect that might be on the horizon, but luckily smart people like him are working on it, so let's wish them all the best" - not HOLY SHIT THIS IS REAL!

@gabrote42

Жыл бұрын

@@josephvanname3377 no, but computerphile has a video that mentions it. Look it up

@josephvanname3377

Жыл бұрын

@@gabrote42 And so did Up And Atom. But there needs to be more depth. Until people start playing Bennett's pebble game, it is not enough.

Eliezer please guide us 😢

In terms of physical anthropology, humanity is facing a period of Punctuated Evolution...

Maybe this is why there are no aliens. All civilizations reach this point and become something we can't even detect.

@xsuploader

Жыл бұрын

he literally addressed this in the talk. You would still be able to detect the activity from the AIs

39:05 basically we don't understand thinking, so we used the Dr. Frankenstein method to build minds.

@DavidSartor0

11 ай бұрын

It's even worse than that. Frankenstein knew what all the parts were, and how to stitch them together. We don't know either of those things, as far as I know.

Interviewer in 2 years when AGI has just told him he will be terminated in 2 seconds “Very interesting”

46:26 GPT4 scaling falloff confirmed? Thought Sam Altman was just talking about cost benefit of more scaling but this sounds like OpenAI hit the wall. This would be great news and it seems Yudkowsky is more relaxed about more powerful models being possible currently.

@T3xU1A

Жыл бұрын

No inside info, but I'd check out the paper "Scaling Laws for Neural Language Models" from Kaplan et al to see a rough idea what OpenAI would expect.... it's the justification they used when going from GPT-2 to GPT-3, and it also goes over the different kind of capability differences (e.g., width / depth)

@joshcryer

Жыл бұрын

@@T3xU1A see 6.3 in that paper. I still am unsure if this is proof OpenAI hit maximal transformer performance. I wish they would simply publish their curves and how much compute they used.

@r-saint

Жыл бұрын

I watched 4 interviews with Sutskever. He says NN deep learning techniques are nowhere near of hitting any wall. He says we're just at the beginning.

@TheMrCougarful

Жыл бұрын

No way that's happening. The wall must be at least 10x out. Maybe, 100x. Think of GTP4 as the bootloader for what comes next. GPT4 will write and refactor GPT5's code, and do that in scant days or even hours. The road forward is about to shed the human limitation entirely. Nobody will stop this from happening. Nobody can.

@joshcryer

Жыл бұрын

@@TheMrCougarful there is a language entropy limit that LLMs will hit, that they are not releasing the curves is suspect

I appreciate the confession abt how the visionaries faulty premises & adamant commitment to continue on those wrong premises has made this field what seems to me to be a pure joke & berserk circus that is sadly lavishly funded

Somebody stole my car radio and now I just sit in silence 🎵

@MichaelSmith420fu

9 ай бұрын

kzread.info/dash/bejne/a2aMuNmSZZfOZc4.htmlsi=JIO8s-l3tozKBeqA

Yeah I'm pretty sure it's time to make AI capability research illegal

@sasquatchycowboy5585

Жыл бұрын

It was for the public good. So they could test out what the virus could do so we could better plan and prepare for it. Lol, but you are bang on. It might slow things down, but everyone will keep trying to inch ahead until they go over the cliff.

@Aziz0938

Жыл бұрын

No MF

Should we CeaseAi- GPT? Y/N

@Wanderer2035

Жыл бұрын

No

30:40 Breaking news! Eliezer Yudkowsky is trying to take over any galaxy he can get his hands on! /s Laughter through tears.

Thanks for the upload! Really important topic.

So, yeah... we're gone... OK we'll take another question here....

Even if AI is concerned with humans to the extent that they have objectives aligned with the cessation of suffering, even a reasonably rational human can see that killing everyone is the optimal solution to that particular problem, that's why we have people involved in arguing for voluntary human extinction and antinatalism. There's nothing 'evil' baked into the logic. Counterfactual people don't suffer... It's only extant humans quivering with the fear of the obliteration of the self who are sentimental about other such life forms somehow missing out on all the wonders of living that cling to the notion that it is somehow unethical (rather than simply out of step with biological evolutionary imperatives) to swiftly arrive at an asymmetry argument which favours a lifeless universe. Perhaps a super intelligence, if it is sentient, would opt for self destruction rather than develop what can only be increasingly more burdensome forms of qualia... And maybe it will do that without giving two shits about human suffering. There are many scenarios in which we all perish. Perhaps it's for the best. There's literally 'nothing' to fear, unless we live in some quantum mystical simulation that persists beyond the death of our illusory physical substrate. At this stage I'm not even sure how much of what I'm suggesting is dark humour with no punch line or if I'm trolling myself just for the distraction of it.

@dr.arslanshaukat7106

Жыл бұрын

Shit dude. That's depressing.

@williamwimmer5473

Жыл бұрын

okay. you first.

Given how interested the interviewer repeatedly claimed to be, he seemed surprisingly disinterested in what Eliezer was saying.

I'm more worried about the delayed effect of introducing the tools of intelligence to wider humanity. Maybe AGI can save us.

Yudkowski: "For a super intelligence wiping out humans is cheap, nothing.. yeah we're gone Presenter: "Yeah... and one last question"

"History never repeats itself" says someone as history repeats them right off the face of the Earth

I believe the genetic fitness did fit in the ancestral environment but things have gone pretty mad out in the world the past decades so we can't find answers there-imo. And that we got smarter isn't true our brains shrink. The only real question i have for some AI nerds is can computerprograms/AIsystems mutate like in the biological evolution...? BIng said yes but gave me biological evolution links so if you know the answer(s) please help me further> Can AI system rewrite themselves?

If you had believed in the G-d of Israel, the G-d of your ancestors, you would have quit your hi tech cushy job from the potential horrors ten years ago by simply understanding at a deeper level the nature of Evil. But the fact that your pintele yid kicked in at the last minute when it already is too late to stop the imminent threat should be recognized as a good thing.

No hate here just feedback, the first speaker is yelling or the mic is too hot.

My question is how your logical positions seem to echo your non logical expression, as a human being. This non logical expression, to me, as human, seems to mirror the stages of grief, in no particular order, Anger, Denial, Bargaining, Acceptance. You have to deal with all the facile levels, while understanding compassionately, why most of us are nowhere near acceptance, and still unable even to frame any question that isnt idiotic. The mirror of an individual's death is already a major life awareness test most of us duck, hoping to go quietly in our sleep. Meditation is the art of being, despite whatever comes to mind/body. Most seem unaware of it, even if having spontaneous moments of non mentation. Perhaps we offload the problem to Everyone will Die, as a strange comfort its not just me, to avoid the guilt and shame of ignoring my contribution to my demise. Or perhaps its just so enormous an ask to contemplate Game Over, it just remains unconscious and leaks out as depression, obsession, distraction and these random grief filled moments. How do you manage the human side of this enormity/triviality? the significance of life s not easily mastered, Alchjemy?

54:10. "The way to contain a superintelligence is to not build it. Period."

❤

Great interview… reaching out to my Senators and gov

control of the reward function is key. when built correctly the ultimate reward function equates to stop

This agi suicide mission reminds me of Frankenstein s monster,👍👍

this guy’s got great energy