Energy, not compute, will be the #1 bottleneck to AI progress - Mark Zuckerberg
Ғылым және технология
Full Episode: • Mark Zuckerberg - Llam...
Apple Podcasts: podcasts.apple.com/us/podcast...
Spotify: open.spotify.com/episode/6Lbs...
Transcript: www.dwarkeshpatel.com/p/mark-...
Me on Twitter: / dwarkesh_sp
Пікірлер: 699
Zuck is so much better at talking about nerdy shit than he is as a public-facing marketing guy lol
@PeterMilko
22 күн бұрын
yeah noticed that too
@brittney3156
22 күн бұрын
Proof we should let people be real.
@Wobbothe3rd
22 күн бұрын
That's true, but the real question is WHY? The real answer isn't found in Zuckerberg's individual personality, but the wokeness and censoring nature of his employees and the surrounding Bay Area.
@zzappligator
22 күн бұрын
A sign he’s the real deal and not a show man.
@hik_okimori
22 күн бұрын
Literally me lol
Zucc got his latest patch 1.8342 - improved facial animations - improved vocal features - improved body language - custom hair - integrated eye moisturizer (no more eye licking) Still under development: - show emotions on face, voice, body language -
@universalsorrow
20 күн бұрын
kids. kids is probably the biggest patch you can have in your life. after kids, people change. whether they want to or not. your priorities change, your emotions change, your maturity changes. the biggest cares and worries in your life also change. it's not that zucc is more human or anything now. it's that he's a dad to 2 kids now. that demands a different persona
@Mr_Sh1tcoin
19 күн бұрын
-fully working schlong
@Mr_Sh1tcoin
19 күн бұрын
-fully working scchhhlong
@darioc23c
19 күн бұрын
😂 same thought when I saw video
@unrealshrimp
19 күн бұрын
Bro i was inhaling smoke when I read no more eye licking and I died
Zuck got his human-look update
@CPTSwoopty
22 күн бұрын
Stress and pressure will do that to a person.
@Gramooneer
22 күн бұрын
i also noticed that, wtf is happened? 😅
@wheniztheend
21 күн бұрын
@@Gramooneer I think it's mostly that he started growing his hair out. His hair looks much better at this length (makes his head look less like an alien). Also, his skin has more natural color to it. (via being in the sun or taking vitamins or something, idk)
@MatthewMS.
21 күн бұрын
It’s the hair. He got a normal haircut and is in really good shape. We are the same age and I’m very proud of him.
@FanDutch
20 күн бұрын
Yeah it’s the longer hair
The efforts we put into replacing ourselves are remarkable
@CdawgAMVsFilmEditing
14 күн бұрын
Such is the greed and hate of the human spirit.
Dwarkesh cut it out of this clip but he mentioned to Mark that Amazon just purchased a 960 MW (=close to 1 GW) data center from Talen energy next to a Talen nuclear power plant in Penssylvania... around the same time they invested 1 billion dollars in Anthropic, the company that made llm Claude. So I have a feeling we WILL actually see that major leap in capabilities next year, unlike what Mark believes, or seems to believe.
@MrSchweppes
23 күн бұрын
I completely agree with you!💯🎯
@verigumetin4291
23 күн бұрын
Just looked it up online and it's true
@G73Server
23 күн бұрын
Are there pther compnies doigg NB the same?
@fafillionaire
22 күн бұрын
I work for the company that built the switch yard for this data center. When construction started almost 3 years ago it was a for a bitcoin miner, half way through we heard the buyer had backed out. Funny how I find a random youtube video that tells me the outcome, as we have not heard anything.
@ashleigh3021
22 күн бұрын
@@G73ServerNot many companies have the ability to buy a GW plant that produces electricity 24/7.
I never realized how much power was going into making these foundation models.
@tuhaggis
22 күн бұрын
You should see how much energy is used for bitcoin - more than some countries.
@erniea5843
22 күн бұрын
So much for these big tech companies caring about climate, it’s all about AI dominance now.
@rejectionistmanifesto8836
22 күн бұрын
@@erniea5843they only like to virtue signal when they can put the costs on average people
@Matlockization
22 күн бұрын
@@erniea5843 Trust me, if their AI can turn out a portrait of George Washington as an African American, then they care about climate change !
@mitahenare8901
21 күн бұрын
@@tuhaggis you should see how much energy is used for Christmas lights
energy always has been the determining factor of the advancement of society
@KCJbomberFTW
19 күн бұрын
Nuclear energy was always the answer You can have desalination you can fight wildfire you can have infinite agriculture tech EVs etc
@KuddleKittens
19 күн бұрын
@KCJbomberFTW but noooo nuclear always goes kaboom
@sean_haz
19 күн бұрын
@@KCJbomberFTWwhat's the energy constraint for fighting wildfires?
@KCJbomberFTW
19 күн бұрын
@@sean_haz excess water in reservoirs to scoop with planes
@honestabe_9207
19 күн бұрын
Facts
Zuck looks more human than usual. That's sus. XD
@eyeofthetiger7
21 күн бұрын
I'm glad that you like his recent update
@juanramonsilva1067
21 күн бұрын
He was living a fantasy life, where in his mind he was this multimillionaire giga chad pioneering the future of humanity. It seems he has gotten humbled by reality and realized that he isn’t what he thought he was. It’s good character development for him actually.
@Mojbojwoj
20 күн бұрын
@@juanramonsilva1067💯
@harmhoeks5996
20 күн бұрын
They got enough GPU compute now
@rodrigobarraza
19 күн бұрын
@@juanramonsilva1067 Well put. VR and AR isn't really his thing either. You can tell how in-tune he is when it comes to neural networks and AI.
You need to unlock the fusion energy tech tree, before you can build your super AI. Everyone knows that
@WoppyHousen
20 күн бұрын
Fission>>Super AI>>Fusion through Super AI is a much faster track. Just have to keep your population in check during the occasional crises meltdowns during the fission fase.
@dayhillbilly
20 күн бұрын
We don't need super AI. We already have fusion energy☀
@HobbyWoppy
20 күн бұрын
@@dayhillbilly From my GPT friend: "To generate 1 gigawatt (GW) of power using solar panels, you would need approximately 4 square kilometers of solar panels, assuming an average solar panel efficiency of around 20%. This can vary based on factors like solar panel efficiency and sunlight availability. (Confidence: Medium)" And then some batteries of cores.
@KCJbomberFTW
19 күн бұрын
Fission is literally good enough It’s the cleanest cheapest safest energy ever made
@AORD72
19 күн бұрын
What? Just build far more energy efficient transistors. The human brain uses bugger all power. Build biological computers if you have to.
Amazing editing!
glitch in the simulation at 0:40
@dragonsandwarts5644
20 күн бұрын
Also at 0:30 the girl walks behind zucks head then reappears
@dragonsandwarts5644
20 күн бұрын
2:37 another glitch - the man just flat out disappears
@miloshp7399
20 күн бұрын
AI talking about why it's not possible to be developed 😂
@benjaminmathew289
20 күн бұрын
im not sure if ur joking but its just a speed up probably
@brokstine
20 күн бұрын
the zuck can bend the space time
language model startups are building LM models that use a fraction of the computing power
Dude you grew like 20k subs in a span of a few days. awesome!
1.21 gigawatts?!?!
@carkawalakhatulistiwa
23 күн бұрын
Is just 1 nuclear reactor
@GeneralKenobi69420
23 күн бұрын
When this baby hits 88 exaflops an hour.... you're gonna see some serious sh
@jklappenbach
23 күн бұрын
GREAT SCOTT!!!111!! What the hell is a jigawat?
@bigbadallybaby
23 күн бұрын
They just need to make sure their AI gets struck by lightning every time they need to ask it something. The trouble is, we don’t know when or where it’s going to strike.
@somename5632
23 күн бұрын
@@GeneralKenobi69420 "88 exaflops an hour"😂
When the most powerful companies in the US (Microsoft, Google, etc.) need to solve a bottleneck in order to make progress, regulatory and other constraints will be quickly resolved. There is too much at stake when we talk about AI progress. So, expect a 1 gigawatt data center next year.
@ahamuffin4747
23 күн бұрын
Think so too
@Blashmack
22 күн бұрын
AI progress is also tied to national defense interest. So I think they will definitely clear regulat process out of the way
@percywhitehead9228
21 күн бұрын
or just buy one like amazon done. "Amazon buys nuclear-powered data center from Talen" "The data center, Cumulus Data Assets, sits on a 1,200-acre campus in Pennsylvania and is directly powered by the adjacent Susquehanna Steam Electric Station, which generates 2.5 gigawatts of power." probably have it up and running within a year with the hardware installed.
@MrSchweppes
21 күн бұрын
@@percywhitehead9228 🎯
Remember when a computer filled up a huge room 🤔
@BruKfu
23 күн бұрын
i like ur thinking
@Ivcota
23 күн бұрын
Moores law is no longer applicable right?
@the_original_dude
22 күн бұрын
remember when women were the computers, and lots of them were sitting in a room doing calculations
@Instant_Nerf
22 күн бұрын
@@the_original_dudeyeah they still are. Bankers, accountants
@the_original_dude
22 күн бұрын
@@Instant_Nerf no, they don't calculate manually anything
In other words, SMRs will be ideal for powering data centres/servers and super computers.
@chookbuffy
14 күн бұрын
I can just imagine a bunch of private SMRs behind walled compounds that remain powered and cool while the rest of society sweats it out during blackouts
@Rawdiswar
11 күн бұрын
If they ever get built.
AWS has 1GW in the Columbia river basin in dozens of sites and buildings.
@Neojhun
19 күн бұрын
He specifically stated single site. Sure a bunch of service providers have a cluster of data center which equate to 1GW.
@RestraHealth
17 күн бұрын
@@Neojhun riot blockchain will finish a 1GW site in Texas this year.
It's not just energy production, its infrastructure as well. Even if we have the technology, it would take at least five years to build the necessary infrastructure to deliver the energy and thats being optimistic. So essentially they would have to place it next to a plant dedicated for it.
@pin65371
22 күн бұрын
Bruce Power up in Ontario Canada is planning on adding more reactors to their site. They are in the planning phase for an added 4800 MW. Maybe bump that up to 6800 MW and then sign long term energy contracts with Microsoft, Facebook, Google and Amazon. Its right next to a massive lake so cooling water is not an issue and there is lots of land there.
@percywhitehead9228
21 күн бұрын
check out "Amazon buys nuclear-powered data center from Talen", so now they just need to kit it out with AI hardware. so it's max a year away. this probably drive other tech giants to buy something similar or get locked out for 5 years. and it wasn't that expensive at $560m
Energy is the #1 bottleneck of everything my guy
I work for a transmission system operator at the highest voltage level in the grid, and the number of connection requests for new data centers we're getting is just insane!
Yes but only if we keep making current microchip technology. If the transition to carbide silicon chips sooner then a massive 1GW center could ve powered by only a few MW with 5 times the computing power.
“How long can the exponential curve going for?” There’s going to be a Nyquist-Shannon equivalent limit to the max data and compute you can throw at a model before you can’t improve it anymore. For example, if you trained an ai model on flipping a coin, it doesn’t matter if you use a 1-gigawatt cluster and 100 trillion coin flips as your dataset, the model is still going to be 50% chance. The question is how do you theoretically find that limit without throwing 10s of billions of dollars at training clusters.
Energy over time is what all works run on, and how much energy that must be spent ultimately determine the cost.. Perhaps one day, we can all have dividends from Open sourced machine production commons, where the capacity is measured in energy.
AI explaining that AI will reach a bottlneck to come people down
hedge against losing energy suppliers in "the age of wars" (which is over-hyped, but kind of has to be) , do this by buying into renewable energy & refining existing tech, when that's built you've hopefully got it structured to sell it for minimal loss or better yet break-even on it.. above is the unlikely scenario tho, you will most likely, in my eyes get a sustainable growth vehicle for this project and in effect you have AI also relying on the sun like we do, which is kind of nice.
It doesn’t have to be either or, I could see both energy and compute bottlenecks being the case, particularly if these exponentials continue for as long as the “optimists”, for lack of a better word, say that they will.
Just for some counter point the human brain runs on about 20 watts of power. Maybe needing Gigawatts and billions should be an indicator that this could be a wrong approach to reaching human level AI?
@percywhitehead9228
21 күн бұрын
you got a better approach? lets stop everything because the human brain is so much efficient?? it's all progress. you build tech so it improves other tech. the first transistors where astronomically bigger compared to leading edge now. give it 25 years of advancements
@Arowx
21 күн бұрын
@@percywhitehead9228 I just suspect this might be a bit of an AI tech bubble that bursts but gives us great chat bot technology but not real AI. It's a bit like the crypto bubble where we have all this technology and money to push into something that looks good but may not deliver on the hype people are building for it. Mind you if it does go full AI it will probably happen so fast our world would change completely within days. And I do not think we are ready for that.
@Arowx
21 күн бұрын
@qwerty-np2iw There are potential lower energy computing solutions that could just be around the corner e.g. graphine, memristor and photonic technologies could totally change the energy needs of modern computing if they can go mainstream.
@Blazenwave
20 күн бұрын
@qwerty-np2iw you wrote a lot but honestly you're really rather stupid. I feel dumber for having read what you wrote.
@christophermcdermott2443
20 күн бұрын
@@percywhitehead9228 I mean we could get serious about education and up skill the whole world population. I bet even a small uplift in collective cognitive power would make a massive difference in the world
Zuck should look at what amzn did with Talen Energy. Amzn got almost a whole nuke unit.
I would switch the choices of the MODX and Montage (i.e. MODX for the Preset Warrior and Montage for the composer). Also note that the Montage is end of life, the successor is called Montage M (and has an additional virtual analog section)
They should build compute/data centers with power plants, no power lines going elsewhere. Both need cooling and security.
@Matlockization
22 күн бұрын
It's a lot cheaper if you hook it up to the grid.
@percywhitehead9228
21 күн бұрын
check out "Amazon buys nuclear-powered data center from Talen"
@burgundyknight6826
20 күн бұрын
I have seen ideas of a modular mini nuclear plant that you could build into a building, supposedly they are safer and easier to maintain. I haven't even heard of a prototype being built, the government regulates this stuff heavily
Current AI architectures are ridiculously inefficient compared to the human brain.
@CamAlert2
23 күн бұрын
The efficiencies will only improve. Performance per watt is going up.
@rapidreaders7741
23 күн бұрын
To be fair, we still don't know how the human brain works exactly. The neuron-activation model used in NN's is an extremely simplified way of understanding the brain, that doesn't even capture 1% of its true complexity. That being said, I'm surprised just how well our current (general) models surpass most humans in the tasks they do.
@RJ-zm3qk
23 күн бұрын
@@rapidreaders7741 Yes and that is the "problem". We don't even know how human brain works. These models are not self-aware. They just imitate self awareness, and that is 2 different things.
@raul36
23 күн бұрын
@@CamAlert2 Isn't that technological innovation? Thanks for pointing out the obvious.
@RajarshiBose
23 күн бұрын
I don't think this is the right way to put it. Many people like hinton believe nature took the way of training/ learning which is very efficient in terms of power consumption , it rejected other ways which does take a lot of power but could learn from a lot of data/ resource very fast without over fitting etc etc like SGD with back propagation. Today's ai uses learning algorithm which is very efficient in terms of time to train on large dataset and which fits very well, compromising power/computation.
I would love to see a large data center connected to a huge solar plant system that uses full power during the day and is throttled down to a minimum at night for the training over weeks.
A gigawatt is 3 million A100s or 1-2 million H100s. About 10x more power consumption than the biggest data centers today.
@percywhitehead9228
21 күн бұрын
what about the new Blackwell? by the time they build a nuclear power plant you'll have a new gpu
"1.21 Gigawatts!!! Great Scott!!" - Mark EmmetBerg
Will be interesting to see what kind of hardware advancements can be made to reduce the power requirements.
Or transfer learning improves and you just build multiple smal and cheap clusters.
Much more relaxed than talking about corporate responsibility. The tech stuff is what he thrives on.
Who ever is doing his human training is killing it.
The better way for energy not being a bottleneck will be to reduce compute elements to be not based on traditional electronics, but go atomic scale and use extremely low power. Current approach in AI computing uses brute force. Soon progress will depend on finding more intuitive algorithms that converge quicker.
@raul36
23 күн бұрын
It is literally what we have been working on for more than 15 years in laboratories around the world. Thanks for pointing out the obvious!
@burgundyknight6826
20 күн бұрын
Two promising prospects I have seen to make computing more efficient are graphene chips the band gap problem they experience has been improved on recently. Also AI designed chips are designed 1000× faster then human designed ones, they aren't close to the computing power of human made designed chips but at the rate they are improving will soon close that gap
@HakaiKaien
2 күн бұрын
That’s what quantum computing is trying to do. But most power is consumed in transformers. They take the current and split it into a lower current and heat. And then needing cooling systems to dispense of the heat. Which is quite primitive to be honest. The electronic technology needs to mature. We are still at the infant stage of the electronic revolution
Computing and energy go hand in hand. The better the computational efficiency, the less energy will be needed. So, yes, both are a big problem, both current and future.
Build a gas fueled power plant in East Texas and you’re gucci Zuck. Plug it into your zillion gigawatt datacenter and let Jerry Jones take you to the promised land.
nyc typically consumes and anywhere from 5-10 gigawatts
Zack looks great with that haircut
Thank you Mark. That was a very human statement to make.
Facebook et al are putting datacenters e.g. in the north of Sweden close to our hydro resources (and usually cheap cooling). It gives quite few direct employment opportunities, suck up a lot of power and raises energy prices in Sweden. It kindof outcompetes outdoor spas, incentivizes improved isolation in homes, some other energy hungry industries and so forth. We do look at new nuclear but that will take 10+ years to do, unfortunately, because it's so heavily regulation, just as he says.
Hence you see a glance if the future when nuclear fusion is on... no more limit on energy since it doesn't pollute...
The Zuck measuring things only in Gigglewatts
I thought about this for a while: instead of building massive data centers where AI runs, why don't we distribute the compute across the globe in a decentralized fashion? This would also de-risk the power requirements, as it is much easier to power something that requires just a bit of energy.
@unmanned_mission
19 күн бұрын
They probably never thought of that. You are a visionary, they should pay you trillions for your unsolicited advice.
@bitbraindev
19 күн бұрын
@@unmanned_mission ah the internet. Who doesn't love it ♥️
@dhillaz
18 күн бұрын
The volumes of data being moved in AI training necessitates everything being in a small space, even moving data between machines in the same rack becomes a bottleneck. Sometimes even the speed between individual chips can become a challenge (e.g. Nvidia developing NVLink for 900 GB/s)
@fablearchitect7645
18 күн бұрын
Because its inefficient. Parallel compute needs to be in close proximity to provide low latency for AI clusters. The AI clusters also require high bandwidth networking like InfiniBand which is prohibitively expensive over long distances.
After Zuck found MMA he’s been a completely different dude. COMPLETELY different dude.
Yes we might be saved. My day just got brighter. Now im a but more optimistic about the future.
Yep. I do interstate nat gas BD for the southwest US. We often get requests for data center or crypto interconnects so they can power gen sets or even small power plants and they don't know what to say when they learn that we and the competitors are sold out in perpetuity. They spend years planning development assuming that gas will just come out of the ground for them when they show up. There's no capacity left to connect the western US to supply because san juan basin is depleting and getting massive expansions or new builds to permian basin approved is a huge risk today that requires politicians to put their necks on the chopping block. But at the end of the day they aren't going to build the generation to address the massive projections for AI demand growth with unicorn farts. AI is particularly going to be screwed if LNG gets opened up because nobody will bother risking these huge multi billion interstate expansions for domestic use when they can connect to far better markets at the coast.
Seems odd to me that at the mid to high mega scale or even giga scale why they wouldn't be generating power on site with no grid tie. But having done work at high megawatt plants its not the kind of thing that you could build overnight. As far as I know Mitsubishi Power doesn't keep a lot of M501J Gas turbines sitting in stock in a warehouse. If you had a completely off the shelf design and could write a cheque you might be able to build one in 3 years.
The hair def makes him look more human . Also the energy fields sound just like the matrix
This likely means that the path forward is in simplifying AI models so they can do the same work with less energy. We learned about this in my first (and only) coding class - how do you simplify the code so the computer is doing less operations to get the same outcome? We did the same thing when we started building computers. What was once the size of a room is now on a nanoscale. Mark raises an excellent point, though I don’t think we’ll be building reactors and solar fields just to power a learning model - we have enough energy problems as it is. No, we can expect the technology to become much more efficient in the next 5-10 years, doing much more for less work. In the same way we were told to learn code in school, kids are going to be taught to use AI - it’s an incredibly powerful tool.
Very true. The real priority should be A.I macro-engineers (structural, civil, mechanical, electrical, etc.) building energy infrastructure and energy generation plants.
@burgundyknight6826
20 күн бұрын
Priority should be decentralized blockchain infrastructure that tracks all government spending. Once we can track all the tax funds and how the government spends it we can be able to allocate it properly to achieve our goals as a nation
@leslietetteh7292
19 күн бұрын
@@burgundyknight6826 I think using blockchain technology to track government spending is an interesting idea with some potential benefits, but also significant challenges and limitations. Here are some thoughts: Potential benefits: Could increase transparency and accountability around how tax dollars are spent May help reduce corruption and misuse of funds Allows citizens to more easily audit and analyze government spending Challenges and limitations: Technically complex to implement comprehensively across all levels/agencies of government Concerns around privacy and security of sensitive financial data on a public blockchain Blockchain doesn't inherently prevent misallocation of funds, just makes it more traceable after the fact Many government expenditures require some confidentiality (e.g. national security) and couldn't be on a public ledger Blockchain is still an emerging technology with scalability and efficiency constraints for massive datasets In terms of importance and priority, I would rank addressing our pressing energy challenges as a higher near-term priority than implementing blockchain for government finance. Developing clean, sustainable energy sources and upgrading energy infrastructure is crucial for combating climate change and transitioning to a carbon-neutral future. These are huge, capital-intensive undertakings that require major government investments and policy support. That said, I don't think it has to be a strict either/or scenario. Initiatives to improve government spending transparency can be worked on in parallel. And there may even be some synergies, e.g. using blockchain to track and verify how funds earmarked for sustainable energy projects are being allocated and spent. But if I had to prioritize one area for governments to focus on with urgency, I believe the energy transition and building out robust clean energy infrastructure should be at the top of the list given the timelines we face with climate change. Government financial transparency and accountability is important, but feels relatively lower stakes in comparison. Of course, this is just my view based on the information available to me. Reasonable people may weigh the relative importance of these issues differently.
this is why Nvidia is so important, few people are talking about the energy savings from their more efficient chips
Algorithms always had and will have more potential to grow at exponential rates than physical infrastructure. Eventually energy won't matter.
So, we would need modular thorium power units, customized for datacenters, build right on-site..
@dot1298
19 күн бұрын
..about time, that the US legislators wake up and make room in the laws for such new (hybrid) installations.
Sure.. Eventually.. But for now it's not, and it won't be until we're well past the point of major changes.
Zuck has never looked better, more human or more likeable. He's aging nicely. He's letting his hair grow. He's not the centre of attention. He's in observer mode. very enjoyable to see
So symbolic to see people disapearing in the background as the talk about AI progresses.
@futureworldhealing
23 күн бұрын
lmfao
@futureworldhealing
23 күн бұрын
they also appear too
@brozbro
23 күн бұрын
wait til the overlord disappears you too.
@osefaa
23 күн бұрын
@@brozbro I'm planning to change my haircut, no chance...
@yaacheese8643
20 күн бұрын
Looking way too into it little bro
I believe this was a few years ago. Before LLM’s was as prominent as they are now. So, the zuck may not be speaking about AI as we know it right now. 🤔
what if you build the datacenter on Iceland? they kinda have unlimited geothermal energy which is cheap and green
@TheWillvoss
10 күн бұрын
And how do you transfer it? That’s infrastructure cited as a hurdle. Solution isn’t more power it’s engineering gpus and machines to work with less power consumption.
Each time Zuckerberg says "Next Year" a person in this world vanishes. 2:34
Solar + battery can be built anywhere
This new updated version of Zuckerberg is much more human-like
This latest Zuckerberg model really looks realistic.
The solution is distributed compute, networks such as Arbius
@hyliion can elevate some energy restraints . Obviously not gigafactories though Haha.
Gotta get that Dyson Sphere going.
Memes aside and jokes aside that he got an update, he’s got a fucking amazing point, lot of businesses opportunities for the people in the energy sector for the future but as always, regulations and bribery will get in the way that big corps will win these contracts to make the infrastructure and supply the energy.
That’s why free sustainable solar is the only way forward
Pay people to generate electricity via peddling.
@KevinTheNoobie
22 күн бұрын
Holy shit you're a genius.
@llejk
22 күн бұрын
If you don’t have the brain, you must have the muscles.
@wally7856
19 күн бұрын
Human efficiency ~ 25%, diesel engine, fossil fuel power plant ~50%. We can't compete.
Now he looks a lot less like a robot.
If the goal is to develop models that mirror human cognition, then we really need to start building models in the way the human brain builds them. For example, our current strategy basically starts each model at square one. Training starts from scratch. But that's not how the human brain operates. There's a huge amount of latent / default state that the brain builds from: structures defined by genetics, that then guide all subsequent learning. If we took the same approach with ML, I think we'd see a huge reduction in training overhead, and much more focused and sophisticated backprop / weight adjustment algs than gradient descent. Brute force is seemingly miraculous, because it precludes the need for domain knowledge in learning assessment. But this is brute force expensive and wasteful, and our models are growing too large. If my insight is novel, let's call this new approach to learning / teaching "Inherent Learning".
@ishananaguru
22 күн бұрын
You seem to forget the fact that nobody knows how the human brain operates.
You wanna fix that? Cause there is a way, I mention it on the APEC propulsion conference open mic night from the end of April.
now it make total sense why Tesla bought solar city a long time ago
Not if you have localized fusion production. No transmission, local generation, local consumption. Use AI to solve fusion and you’re done.
Human Intelligence is powered by tacos.
What about localized wind and solar? The amount of compute would be proportional to the amount of energy coming in. There would be some amount of batteries, but it might well be cheaper at a certain scale to simply build more solar and go with the fluctuating rate of available energy. If such systems are not connected to the grid, the permitting should be far simpler. Also, if they are not connected to the grid, but are located out in the desert somewhere, they would also be much easier to bomb out of existence.
@tuhaggis
22 күн бұрын
You would need millions of square meters of solar cells to reach a gigawatt. You would be building a solar array the size of a small city.
@stcredzero
22 күн бұрын
@@tuhaggis Yes. But still I could totally see this happening.
@fred6907
16 күн бұрын
@@stcredzero A million square meters is the size of Egypt. And that's just ONE gigawatt datacenter. Still think it's a good idea? Do you Gen Z learn ANYTHING in school these days?
You can imagine a future in Zuck's head where the physical constraint of the AI energy throughput gets lifted such that there is an actual location with a giant super computer like the one in the 2005 film The Hitchhiker's Guide to the Galaxy. LMAO that would be epic
PERFECT SENSE, AS I SAID
And then we will be turned into 🔋 like Morpheus said.😬
i love this guy, so cool & poised
so everything in the world is energy constrained. AI is not the exception. and everything that makes AI so great is now added to the list of reasons we need fusion power ASAP. or maybe we're just blessed for not having it. maybe this is the natural limiter on AI we've been hoping for.
@EnderViBrittania
23 күн бұрын
Meanwhile China building new energy capacity via coal and nuclear rapidly… so China won’t be slowing down.
Character and empathy, not technology, will be the bottleneck to a better world.
META has built and owns multiple datacenter clusters across the country. META also owns the fastest cross-country data transmission network.
Or you just improve efficiency, whether by using synthetic data or more efficient GPUs
We should send datacenters out into space to train on solar energy.
Zuck is apparently aware that a lightning bolt can generate the 1.21 Gigawatts required to produce whatever he requires.
The robot/alien has hair now
So they either need to make energy cheaper or the chips more efficient.
I just read the first few comments, so fantastic to see what kind of focus people have, a bunch of Dudes worried about how a Dude looks. There is a 3 prong issue 1. As Zuckerberg said, adequate power supplies are needed 2. GPU Processing Power and 3. Backhaul of all devices that can share the data sufficiently, traditional TCP-IP structure is insufficient as is copper over UTP. I'm here for my money, he's already got his.
All that Brazilian jiu-jitsu has made him more human!
Wow a nuclear power plant to train the model ? 😯😯
@shrinidhikulkarni479
22 күн бұрын
It's cz they want clean energy. 😅
Can't belive nobody trying to manufacture GPU production. Must be really tough
@jerryspringer6096
23 күн бұрын
Intel
@burgundyknight6826
20 күн бұрын
Tesla is manufacturing DoJo, which is a modular chip designed for machine learning. They are continually adding to it, and it will be one of the most powerful super computers soon. They are planning to use it to crack full self driving.
In matrix machines used people for energy ;) Our word takes an interesting turn 😅
Demand for energy, particularly renewable energy, is going to absolutely explode. Exactly what Zuck is referring to here. No one can build a GW datacenter today because no location has a GW of spare capacity. These companies deploying large cloud datacenters understand their impact on GHG emissions and want to minimize their environmental footprint. So renewable power generation, coupled with grid-level energy storage, is going to skyrocket. Time to invest in ETFs like FAN, GRID, TAN, etc.
Mark looks better with longer hair. This is a good look for him