Building a GPU cluster for AI

Ғылым және технология

Whitepaper: lambdalabs.com/gpu-cluster/ec...
Learn, from start to finish, how to build a GPU cluster for deep learning. We'll cover the entire process, including cluster level design, rack level design, node level design, CPU and GPU selection, power distribution, storage, and networking.
This talk is based on the Lambda Echelon GPU Cluster whitepaper. The whitepaper can be found above.
Slides for the talk can be found here:
files.lambdalabs.com/How%20to%...
Errata:
- Slide 46 contains an erroneous diagram with a connection from the storage server to the compute fabric network, the storage server does not connect ot the compute fabric network. The correct diagram is available in the whitepaper.

Пікірлер: 48

  • @randahan215
    @randahan21510 ай бұрын

    Extraordinary presentation. Covered all the important topics in depth and with real teaching talent. Many thanks!!

  • @dr.mikeybee
    @dr.mikeybee8 ай бұрын

    Thank you. You got me started years ago with your lambda stack -- the only way I could get TensorFlow installed on Linux.

  • @randahan215
    @randahan2155 ай бұрын

    Most professional and holistic explanation I heard about this topic. Thank you so much!!

  • @yassinebouchoucha
    @yassinebouchoucha Жыл бұрын

    Thank you for highlighting an underrated topic/options that company should re-consider within their compute infrastructure.

  • @NSPK-
    @NSPK-9 ай бұрын

    Very expert suggestions for hpc and compute sizing.

  • @ilyboc
    @ilyboc8 ай бұрын

    Really good analysis and presentation!

  • @brianwesley28
    @brianwesley283 жыл бұрын

    Thanks for the video.

  • @carlschumacher5510
    @carlschumacher55102 жыл бұрын

    Its nice to see a holistic explanation of designing / building / installing a complex multi-rack system...As someone that has spent years working on both sides of the "analog/digital divide" (physical data center world / digital world's various segments), the un-sexy physical aspects of available rack space / power / cooling / floor loading / network uplink bandwidth are often overlooked (often assumed)...A semi arrives with a pallet: "Hey Carl, you can have this online in a couple days, right?"

  • @lambdacloud

    @lambdacloud

    2 жыл бұрын

    Hey Carl, thanks for the kind comment. Glad you like the video. It's always funny how difficult it can be to 'bridge the divide' between the physical world and virtual world. Many SWEs expect to be able to "spin up" 1000 servers with an API call and forget that there are actual physical objects and tons of people that actually make that happen when you're on-prem.

  • @peterxyz3541
    @peterxyz3541 Жыл бұрын

    Thanks. I’m planning on building a “massive” 2 GPU system for home use.

  • @fundoo203

    @fundoo203

    8 ай бұрын

    How did it go man? I also want to build something like that and then stumbled on this video, which is excellent

  • @sanaullah-qureshi
    @sanaullah-qureshi Жыл бұрын

    very informative , thank you.

  • @user-us7oi6jw5i
    @user-us7oi6jw5i5 күн бұрын

    Really complete, thank you!

  • @ProjectPhysX
    @ProjectPhysX2 жыл бұрын

    Lots and lots of A100 GPUs. Every single one of them is a monster, almost 2x faster memory than the next best GPU. An entire room full of A100 racks... holy cow.

  • @anatolystrashkevich7621
    @anatolystrashkevich7621 Жыл бұрын

    very informative, thanks!

  • @austynr
    @austynr Жыл бұрын

    Genius bait and switch. Props!

  • @metal_mo

    @metal_mo

    Жыл бұрын

    Lambda needs an explanation on the difference between "building" and "designing".

  • @vtrandal
    @vtrandal2 жыл бұрын

    Excellent.

  • @thePyiott
    @thePyiott2 жыл бұрын

    Great insight!

  • @uzairqarni7782
    @uzairqarni77825 ай бұрын

    This was amazing. Thank you.

  • @natexetan5732
    @natexetan57323 жыл бұрын

    thanks for the inspiration

  • @cyberspider78910
    @cyberspider78910Ай бұрын

    Highly appreciated...KZread should have a separate category called Founder's video.

  • @HarishN.J
    @HarishN.JАй бұрын

    Hey Stephen, this is highly informative. I work on this clustering. Now am able to connect the dots and get the bigger picture. where can i read about the relationship between numa topology and GPU peering capability.

  • @glennisholcomb592
    @glennisholcomb59210 ай бұрын

    I have three computers, and a nas, and a external hub. I think that I don’t need a another server because of the NAS. As far as my architecture goes, is there anything else that you can advise?

  • @petevenuti7355
    @petevenuti735510 ай бұрын

    What if I have a model that I just want to run as provided, it hasn't really been optimized to run around the cluster and has memory requirements greater than any individual system I have. I feel safe to assume that for that specific case a shared distributed memory model would be the solution to run that specific app, yes? Is there any distribution of Linux that has support for such a memory model? It doesn't have to be a full-blown single system image. Perhaps a patch to the memory management driver so storage can be treated as an extension of system memory and not swap memory? Does any such software exist?

  • @eyadmufti
    @eyadmufti Жыл бұрын

    it is a lecture more than a tutorial, Thx.

  • @Bloodycub666
    @Bloodycub666 Жыл бұрын

    I just love this kind things. How do i can start this kind bussnes how i can find customer for like small node and start building up

  • @loadmastergod1961
    @loadmastergod1961Ай бұрын

    I want to build a multi dual epyc 7742 based system for goofing around learning this stuff.

  • @programmingwiththotho4641
    @programmingwiththotho46415 ай бұрын

    Your are insane, thank you

  • @ikbo
    @ikbo2 жыл бұрын

    Do you guys have a gpu cluster optimized for 3d rendering.

  • @chaoticblankness
    @chaoticblankness5 ай бұрын

    Very Based

  • @rosenangelow6082
    @rosenangelow60828 ай бұрын

    Tell me how difficult it is so i can buy your solution kind of talk

  • @nathanthomas9395
    @nathanthomas93952 жыл бұрын

    Does lambda products (gpu cluster) ship with a manual to help you set up the servers for use

  • @mengxu2026
    @mengxu20263 жыл бұрын

    Our group ordered around 10 lambda PCs 1 year ago. Right now more than 5 have problems. Some of them do not start up. Mine gets stuck randomly....

  • @yugr

    @yugr

    3 жыл бұрын

    Have you tried looking into the reasons?

  • @lambdacloud

    @lambdacloud

    3 жыл бұрын

    Meng Xu, you can email support@lambdalabs.com 24/7 or call +1 (866) 711-2025 during business hours. Sorry to hear you're having issues, I'm sure we'll be able to resolve them quickly.

  • @danielleza908

    @danielleza908

    Жыл бұрын

    Our team has 5 lambda laptops, they work perfectly for over a year now.. We also have a workstation with 3 GPUs, works great too.

  • @jleonardoperez5402
    @jleonardoperez54023 ай бұрын

    Looking for work would love to help

  • @ravnodinson
    @ravnodinson9 ай бұрын

    Hell yes Lambda Lambda Lambda.

  • @meng-hub
    @meng-hub11 ай бұрын

    Does it work in man????

  • @julianfiacconi709
    @julianfiacconi709 Жыл бұрын

    Still most relevant today, 2 years later. Thanks.

  • @JustPlainRob
    @JustPlainRob5 ай бұрын

    Now if only I was a billionaire so I could make use of this great information...

  • @thinkinginsomething1859
    @thinkinginsomething185910 ай бұрын

    Half Life man!

  • @huaveihuavei1045
    @huaveihuavei10453 жыл бұрын

    headeggs

  • @harshikamahesh9459
    @harshikamahesh94592 ай бұрын

    Talk about what ur expert.. don’t talk useless stuff without knowing all facts

  • @mikepict9011
    @mikepict9011 Жыл бұрын

    This dudes in full submission mode . Sad

  • @orthodoxNPC
    @orthodoxNPC2 жыл бұрын

    speak UP

Келесі