Holly is a role model. She has a well-reasoned world view, good arguments and is eloquent. Great episode

Gus, Holly, you two are awesome. Thanks for this excellent conversation! Sold. I support pausing.

What I like most is Holly's understanding of company-think or corporate-think. An animal advocacy background certainly helps with that! - Money-grubbing above all else. If AGI adopts that ethic from us - doom is certain.

Holly is consistently one of the most reasonable thinkers in the EA/AI safety space, big fan 🙏

A useful discussion on hardware overhang, thanks for the reference to Heninger's piece ("Are There Examples of Overhang for Other Technologies?")

On "what would you do differently if you eval comes back negative", I've heard from people like Victoria Krakovna that the thinking is something like: There's a lot of randomness in training models, we'll train a bunch of models and keep the ones that pass evals/interpretability analysis. I guess this is supposed to somehow work like a genetic algorithm in search of safe AIs? I don't really buy it though.

27:42 She convinced me there. Jokes aside, good episode!

Hi 🤗

10:25 With a century-level pause, the risk I think of isn't climate change, it's ageing. While there could be medical breakthroughs even with AI paused, the default expectation if you just wait 100 years is that almost everyone currently alive is dead by then. Personally, I'm kinda selfish so I don't want to die of AI killing everyone or of old age, a 10-20 year pause seems like a good idea on current timelines, but 100 years is as much a suicide pact as going full-tilt. 35:18 Oh, she has encountered the issue, she just dismisses it as some silly sci-fi nonsense. "My preferred policy will kill you slowly" and "you even having the concerns you do is a joke" is quite the one-two punch. If my p(doom) were as low as 20-40% I'd feel alienated enough to switch teams for e/acc.

China is most definitely building on this. There's no way they are ignoring A.I. I do think that pausing means that we are placing safety in the hands of China. Which could be better. I mean, at least they're not Google.

@Diego-tr9ib

7 күн бұрын

Pausing might require an international treaty

Impressively articulate and intelligent young lady.

Completely agree with her. But there is some miniscule chance that Superintelligence will adopt a sentientist ethic and fix us humans. Afterall, if it is super intelligent that's the way to go...😁 😇 😈

Hard to root for someone who blocked me on x prior to any interaction. But you go girl 👍