Your NVIDIA GPU can power a free ChatGPT (Chat with RTX)

Ғылым және технология

Chat with RTX is a ChatGPT-style application that uses Retrieval Augmented Generation (RAG) to allow you to chat with your own data on your own local NVIDIA GPU/PC.
Links mentioned:
Download Chat with RTX - www.nvidia.com/en-au/ai-on-rt...
NVIDIA RAG blog post - developer.nvidia.com/blog/rag...
My RAG tutorial (code from scratch) - • Local Retrieval Augmen...
Sign up to NVIDIA GTC 2024 - nvda.ws/3GUZygQ
PDF links:
Attention is all you need paper - arxiv.org/abs/1706.03762
Embeddings paper - vickiboykis.com/what_are_embe...
Math for ML book - mml-book.github.io/book/mml-b...
Charlie Walks (my novel) - www.charliewalks.com
Nutrition Textbook - pressbooks.oer.hawaii.edu/hum...
Other links:
Download Nutrify (my startup) - nutrify.app
Learn AI/ML (beginner-friendly course) - dbourke.link/ZTMMLcourse
Learn TensorFlow - dbourke.link/ZTMTFcourse
Learn PyTorch - dbourke.link/ZTMPyTorch
AI/ML courses/books I recommend - www.mrdbourke.com/ml-resources/
Read my novel Charlie Walks - www.charliewalks.com
Connect elsewhere:
Web - dbourke.link/web
X/Twitter - / mrdbourke
Twitch - / mrdbourke
ArXiv channel (past streams) - dbourke.link/archive-channel
Get email updates on my work - dbourke.link/newsletter
Timestamps:
0:00 - Intro
0:17 - Chat with RTX requirements
054 - What is Chat with RTX?
1:51 - Downloading Chat with RTX
2:06 - Installing Chat with RTX
2:09 - Make sure to upgrade your NVIDIA Graphics Driver
2:41 - Chat with RTX setup steps
3:04 - What are INT4 models?
3:25 - Opening Chat with RTX
3:48 - Chat with RTX running in the browser
4:33 - Allowing Chat with RTX to work with your own custom data
5:11 - Turning your own custom data into embeddings
6:43 - Question 1: What are embeddings?
7:18 - Question 2: What is the attention mechanism?
7:51 - Question 3: What is matrix multiplication?
8:31 - Question 4: What is Charlie's secret project? (asking a novel)
9:06 - Question 5: What are the macronutrients? And their functions?
10:07 - Outro/NVIDIA GTC
#AI #NVIDIA #GTC24

Пікірлер: 31

  • @rahmatullahbarkat1002
    @rahmatullahbarkat1002Ай бұрын

    very detailed use case. thankyou

  • @astrocoastalprocessor
    @astrocoastalprocessor2 ай бұрын

    🎉 Thank you! This is just the kind of video I've been hoping to find

  • @mrdbourke

    @mrdbourke

    2 ай бұрын

    Good timing!

  • @astrocoastalprocessor

    @astrocoastalprocessor

    2 ай бұрын

    @@mrdbourke It worked flawlessly

  • Ай бұрын

    Do you know whether is it possible to have something similar to DALL-E with this tool?

  • @ibriarmiroi
    @ibriarmiroi17 күн бұрын

    Thank you very much as always. Though out of the context of tis video, i was wondering what would be the replacement for Tensorflow examination Certificate since it would be discontinues on 31st of may this year ? and also , would be possible to make video about the available Certificates online ? since your expertise opinion on each Certificate would be Gold ! thank you

  • @amitkumarchejara6042
    @amitkumarchejara60422 ай бұрын

    This is awesome, but I don't own a GPU yet, still learning!!

  • @gorkemocmenn
    @gorkemocmenn24 күн бұрын

    dude.great job

  • @MK-ce7im
    @MK-ce7imАй бұрын

    this used to work fine but after changing my folder path for the docs, I can no longer open it. I wish I can change it somehow without opening it

  • @justusbatekerd
    @justusbatekerd2 ай бұрын

    Did you have any errors when you installed the software? I have had numerous errors with the sentence-transformers not being installed. Can anyone help with this?

  • @brendendas

    @brendendas

    2 ай бұрын

    Did you find a fix yet? I had the same problem which was addressed on a forum and the solution solved the issue for me.

  • @FernandoLXIX
    @FernandoLXIX2 ай бұрын

    Don't load a huge dataset. I uploaded every college digital file I had and the thing went crazy. After a few hours of it trying to "digest" the data I killed the program and had to reinstall because there doesn't seem to be a way of telling it to stop reading your data. I'll try again with a way samaller dataset as you did in the video.

  • @gordonthomson7533

    @gordonthomson7533

    2 ай бұрын

    What GPU are you running? I’m awaiting 4080 super and hoping to squeeze a load of value via this…

  • @FernandoLXIX

    @FernandoLXIX

    2 ай бұрын

    Don't. After some testing I can tell you it's quite useless. The answers it provides are very bad and short. It doesn't care that much about the data you provide it with. Bare in mind it's a "demo". Maybe in a few months or years it becomes decent. For now, just get chatGPT 4 instead@@gordonthomson7533

  • @MK-ce7im
    @MK-ce7imАй бұрын

    best youtuber out there

  • @kumarvuppala5286
    @kumarvuppala5286Ай бұрын

    what is best hardware required ?

  • @fabiosoave6622
    @fabiosoave662228 күн бұрын

    Thank you for the very comprehensive tutorial! I'm interested in creating an interactive chat BOT to respond to Facebook messages related to medical topics. What I'm asking is, given the issues some have reported regarding the management of very large datasets of scientific articles, how can we obtain a system that is both comprehensive and effective? How can we organize the material questions correctly to make the most of the program's capabilities? Thank you.

  • @dexterroy
    @dexterroy2 ай бұрын

    My GPU is an older RTX 2070. After downloading this nvidia package, I realised it's not going to run on my GPU. I created a similar chat bot that can do Q&A using a pdf file of your choice. I'm using llama-2-7B there. I had to use Google Colab to get 16GB GPU. But man, it's so slowww.. @Daniel Can you please do a video on pdf chat using langchain and one of those free LLM models?

  • @doctorfunhome7842
    @doctorfunhome784222 күн бұрын

    Quadro T600 can run it ?

  • @erentas7391
    @erentas73912 ай бұрын

    you are back at last.

  • @mrdbourke

    @mrdbourke

    2 ай бұрын

    yo yo yo!

  • @bigdhav
    @bigdhav2 ай бұрын

    I have a 4070, but i cant get it to work due to some strange dependency.

  • @mrdbourke

    @mrdbourke

    2 ай бұрын

    Which one is that? Have you installed NVIDIA drivers?

  • @AnshumanKumar007
    @AnshumanKumar0072 ай бұрын

    Will this on my RTX 3060 ? I have a laptop. Wish I bought the RTX 3090 back then, I could have afforded it. Still can but now I am hearing of layoffs in my org, so I am gonna have to save up in case I get axed. EDIT: It doesn't seem it can. I wish my 3060 had like 16 gbs of VRAM :(((((

  • @gordonthomson7533

    @gordonthomson7533

    2 ай бұрын

    That’s odd, minimum is 8gb

  • @sidrajhans
    @sidrajhansКүн бұрын

    This beautiful RTX 4090 is love. I wish I had one. If you ever feel like donating, sign me up.

  • @animetransition1030
    @animetransition10302 ай бұрын

    Yeah make an app like this

  • @mira8950
    @mira89502 ай бұрын

    can it run on rtx2050 ?

  • @pabloarvilla8012

    @pabloarvilla8012

    2 ай бұрын

    no

  • @mrdbourke

    @mrdbourke

    2 ай бұрын

    You could try, however, the system requirements do say to use a 30XX or 40XX series GPU

  • @ybaran
    @ybaran2 ай бұрын

    😮

Келесі