Believe it is pressure. Pressure to begin looking at what systems need to be put in place to protect all of us. Because in the event of a near extinction level event, it's exactly like it sounds. We all die. >> Right. Sager mentioned earlier, we just did a block on Kimmy K3 and the development of these very powerful AI tools which are now widely available um with really no safety guardrails to speak of. How much does this concern you when you're looking at these developments? Um you know, what thoughts are going through your head about the potential consequences? It's interesting you should ask that question because I've often thought about that and I'll give you an a really sort of disquing answer which is that last week I was asked to do a Q&A for DARPA for their biology science and technology study group and that was raised by others. There was a part of that Q&A where I was just listening and one of the charts that was put up was precisely how dangerous AI and biology are together. And there was even talk about whether or not a situation needed to come about where the the plug would be pulled. It's just a fascinating concept that you could actually pull the plug on bio and AI. And from what I understand, the real hurdle in that is getting all six CEOs of these AI companies to agree to all pull the plug,
Will live in a post scarcity world of super abundance and everyone will get a free car and we'll be free to crochet socks and play music and read poetry all day and basically we don't have to worry about anything because super intelligence will solve it all right there's that on one end and then there's the you can imagine I won't go into a belabored description of the doomers but you have the doomers who are like the end is nigh Okay, here we go. It's It's not It's not the second coming as the Antichrist. And within short order, we're going to be MadMax. Between those two, there's a lot. And I suspect you land between those two. But where do you land in terms of assessing the promises and peril of AI and super intelligence as it stands right now? So look, I think any reasonable person should be both excited and a bit frightened >> and you know that's just the nature of it. It sounds contradictory but actually that's the only rational response. I think you know the super abundant story may turn out to be true on a kind of longer view let's say 20 30 40 years. >> Mhm. The problem is that in the path to get there, there's going to be a tremendous amount of disruption and that's going to be politically quite difficult to navigate. I think a useful lens through which to view this question is the China shock in trade. >> Mhm. >> So in 2003 or thereabouts, you get this
Deal. It is possible though I like to believe that all of my greatest fears will never be realized in AI. Wouldn't that be lovely? But it is possible that AI is a winner take all technology because AI is intelligence itself. And so if you get a fast takeoff of a super intelligent AI, it will basically be impossible to beat because it can thwart you at every turn. Uh so as you try to build your super intelligent model, it can do things to disrupt that. If anybody's read the three body problem, spoiler alert, spoiler alert, um there is a part that you realize that the reason that our physics stalls out is because uh this other civilization got advanced physics before we did. And so they're able to send these like um really really really tiny particles around to disrupt our particle physics. And so our large colliders just keep giving us confusing results. And so it would be things like that if we really do get a fast takeoff. Super intelligent AI. Now again that's probably just dystopian fiction. We shouldn't be super worried about that. But that's in addition to all of the economic concerns that will be very real. We have to realize that we are in that battle with China. And so uh for game theoretic reasons, AI will never do anything other than go underground. It will never be stopped developing. Uh so one just spending a lot of time wasted on that is going to be a tragedy. We need to have a um Manhattan project mentality right now around AI. We need to be developing it
So all photos that we take on our own have some watermark until they are edited or something like this. And software can manage communications with the device manufacturer- device manufacturer to maintain human editing— which is the opposite of the discussion to try to watermark AI images. And then you can make a Google image that has a watermark and use a different Google tool to remove it. - Yep. It's going to be an arms race, basically. - And we've been mostly focusing on the positive aspects of AI. All the capabilities that we've been talking about can be used to destabilize human civilization with even just relatively dumb AI applied at scale, and then further, superintelligent AI systems. Of course, there's the sort of doomer take that's important to consider as we develop these technologies. What gives you hope about the future of human civilization, given everything we've been talking about? Are we going to be okay? - I think we will. I'm definitely a worrier, both about AI and non-AI things. But humans do tend to find a way. I think that's what humans are built for: to have community and find a way to figure out problems. That's what has gotten us to this point. And to think that the AI opportunity and related technologies is really big. And I think that there's big social and political problems to help everybody understand that. And I think that's what we're staring at a
Actually starts to rot out your society. And so that's something I do worry a lot about is humans humans love being useful. Humans love solving hard problems, right? Specifically solving hard problems in a society of of of people who who who also believe deeply in solving hard problems for the greater good. >> They need a purpose. >> They need a purpose. And so I don't think the AI question there. So there's kind of three worlds for AI is and super intelligence, let's say, not transformers. You have a world in which you get extinction of humans. And that's sort of the the the true doomer case is like you you you turn on this box. This box isn't a human. This box doesn't align with humans and you get like Terminator case or you get uh Matrix case, right? And then the next is like humans are pets, right? which is kind of to to me at least the universal high income scenario which is like yeah humans have great houses you have all the energy you need you have great food and this is where you get like a wall-ally type scenario of like we're all rich but we don't we don't have anything to do we don't have purpose in life and so the dream scenario is to me at least and and and we'll see and to to be fair this is not this is not my lane right like I I I'm this is not the problem I'm working on but you the the the the dream case is where you first off you do use this new tool we have to increase human quality of life. Like you you bring people out of starvation, right? You you make it such that no one's ever worrying about healthcare. No one's ever worrying about where their next meal comes from while preserving
Just need to make you confused. They just make they just >> It's like inkcloud, you know, like like an octopus shooting ink. It's just confusion. It's like all this stuff and oh look, they put out this policy and wow, they did this thing and you know, this guy said this on Twitter. Wow, look at all the shiny jingling keys. >> It seems like if I were if I were anthropic, I and and I and I had the motivations you suggest, I might be more inclined to say, you know, this thing is a long way off. Chill. We're not going to get to AGI tomorrow or super. But but their estimates actually tend to be same with open AI on the near side. >> Yes. And the reason for this is because you're not the audience. They want the AI engineers to come to them and pledge allegiance. They want the AI safety super high IQ less wrong people to work for them as engineers. So they have to signal to them. They have to be like look guys you want to build super intelligence right? So we can do it safe because there's a fundamental tension in the heads of basically all of these rationality EA etc people which includes the many of the best engineers at these companies and the tension is fundamentally um they want to build ASI >> they want to build super intelligence they've wanted to do it since they were like 12 years old like they've always wanted to do this they've read sci-fi stories they've read Ian Banks or whatever and like they want to build super intelligence and then the second thing is they are unfortunately smarter enough to understand that doing that wrong would be a bad idea. So what they
The super intelligence that you're talking about I feel if you explain that to some people it would scare them even more. You know, perhaps they are worried about the cultural ramifications, ep e economic ramifications, but also just this Terminator situation where, okay, well, they're super smart. They're going to want to then take over the world and they don't need us anymore, right? >> But you have such an optimistic view. I mean, we're talking about solving aging, living to be 150 or more. Uh why do you have such an optimistic view? Are you worried at all about the other pessimistic sort of viewpoints or >> absolutely not and I I'll tell you why I'm I'm so super optimistic about it. Um when people make those statements like uh AI is an existential threat for us and you know it's going to destroy humanity. Um I make the counterpoint there's only one existential threat to humanity and that's humanity. So if you look at history um human beings killed more humans than everything put together caused more suffering than anything that humans have have been exposed to. You know even animals I don't think they they maybe infectious diseases at some point might have caused um a lot of suffering but but but the real danger is is the human intelligence. So uh I do let's do a thought experiment. Let's imagine that uh we live in a parallel universe and in that universe the world have decided that
Bad. Um and uh like sounding uh the the warning here. What um where does that Like what when you look at AI, what is the most aside from like the data centers and what they're doing you know, from an environmental and from just sort of a cost and from like a monopolizing uh energy, but in the the the potential for the tool as it exists or as they imagine it will exist, how does that play into this and and what is the most dangerous aspect of it from your perspective? >> The primary purpose of AI right now is to amass as much wealth and power in the hands of a small number of men as quickly as possible. And that's what we're seeing. There's a lot of speculation about what AI may may be able to do, superintelligence, god-like AI, take all the jobs. You know, those are some speculative possibilities, but they view it as a way to capture wealth and power. They're already doing that, right? Trillions of dollars being invested in making this thing happen and now the government itself is dependent on these systems. So, but when you look at it, I don't worry so much about the longer-term harms because the short-term, the near-term harms are so pronounced. And I think the number one harm, I mean, obviously there's a threat to the environment, to the ecology, uh and and to jobs at some point. But it's further empowering and enriching this small group of people who are already trying to destroy democracy with the wealth that they have. And AI is rising
AI-generated from each show's transcript. Tap any timestamp to verify at the source.
One email. The moments worth hearing from 369 shows — timestamped.