Im a medical researcher and, surprisingly, my life is all about pdfs i dont have any time to read; let alone learn the basics of code. And i think there's a lot of people on the same boat as mine. Unfortunately, its very fucking hard to actually find an ai tool thats barely reliable. Most of youtube is damped with sponsors for ai magnates trying to sell their rebranded and redudant worthless ai-thingy for a montlhy subscription or an unjustifiably costly api that follows the same premise. The fact that you, the only one that came closer to what i actually need - and a very legitimate need - is a channel with

@tonykipkemboi

18 күн бұрын

Thank you so much for sharing about the pain points you're experiencing and the solution you're seeking. I'd like to be more helpful to you and many more like you as well. I have an idea of creating a UI using Streamlit for the code in this tutorial with a step-by-step explanation of how to get it running on your system. You will essentially clone the repository, install Ollama and pull any models you like, install the dependencies, then run Streamlit. You'll then be able to upload PDFs on the Streamlit app and chat with it on a chatbot like interface. Let me know if this will be helpful. Thanks again for your feedback.

@ilyassemssaad9012

16 күн бұрын

hey, hmu and ill give you my rag that supports multiple pdfs and you can choose the llm you desire to use.

@Aberger789

11 күн бұрын

I'm in the space as well, and am trying to find the best way to parse PDFs. I've setup grobid on docker and tried that out. My work laptop is a bit garbage, and being in the world's largest bureaucracy, procuring hardware is a pain in the ass. Anyways, great video.

@kumarmanchoju1129

10 күн бұрын

USe nvidia RTX chat for pdf summarizing and querying. Purchase a cheap RTX card of minimum 8GB vRAM.

@InnocentiusLacrimosa

9 күн бұрын

​@@tonykipkemboiI think most people are in pain now with just this part "upload pdfs to service X". This is what they want/have to avoid. Anyhow, nice video you made here.

Thank you for this excellent intro. You are a natural teacher of complex knowledge and this has certainly fast-tracked my understanding. I'm sure you will go far and now you have a new subscriber in Australia. Cheers and thank you - David

@tonykipkemboi

54 минут бұрын

Glad to hear you found the content useful and thank you 🙏 😊

Welcome on my special list of channels I subscribe to. Looking forward to you making me smarter😊

@tonykipkemboi

20 күн бұрын

Thank you for that honor! I'm glad to be on your list and will do my best to deliver more awesome content! 🙏

Clear instruction, excellent tutorial. Thank you Tony!

@tonykipkemboi

10 күн бұрын

Thank you for the feedback and glad you liked it! 😊

Simple and well illustrated, Arap Kemboi 👍🏾👍🏾👍🏾

@tonykipkemboi

14 күн бұрын

Asante sana bro! 🙏

Top-tier information here. Thank you!

@tonykipkemboi

9 күн бұрын

🙏

this was super clear, extremely informative, and was spot on with the exact answers I was looking for. Thank you so much.

@tonykipkemboi

11 күн бұрын

Glad you found it useful and thank you for the feedback!

Thats a pretty clean explanation. looking for more videos.

@tonykipkemboi

19 күн бұрын

Thank you! Glad you like the delivery. I got some more cooking 🧑‍🍳

Congrats man. Really useful content. Well explained and effective.

@tonykipkemboi

17 күн бұрын

Thank you, @ISK_VAGR! 🙌

This is a fun and potent project. This provides access to a powerful space. Peace be on you.

@tonykipkemboi

5 күн бұрын

Thank you and glad you like it!

Very good! Easy to understand, easy to try, expandable ....

@tonykipkemboi

5 күн бұрын

Awesome! Great to hear.

@DaveJ6515

5 күн бұрын

@@tonykipkemboi you deserve it. Too many LLM KZreadrs are more concerned to show a lot of things than to make them easy to understand and to reproduce. Keep up the great work!

Dope video man! Keep them coming

@tonykipkemboi

12 күн бұрын

Appreciate it!!

Very helpful! Great video! 👍

@tonykipkemboi

7 күн бұрын

🙏❤️

Great job

@tonykipkemboi

21 күн бұрын

Thank you! 🙏

thanks man this is extremely helpful!

@tonykipkemboi

6 күн бұрын

🙏🫡

Good to see fellow Kenyans on AI. Perhaps the Ollama WebUI approach would be easier for beginners as one can attach a document, even several documents to the prompt and chat.

@tonykipkemboi

19 күн бұрын

🙏 Yes, actually working on a Streamlit UI for this

Can you make one video of RAG using Agents? Great video btw. Thanks

@tonykipkemboi

16 күн бұрын

Sure thing. I actually have this in my list of upcoming videos. Agentic RAG is pretty cool right now and will play with it and share a video tutorial. Thanks again for your feedback.

@metaphyzxx

15 күн бұрын

I was planning on doing this as a project. If you beat me to it, I can compare notes

so cool!

@tonykipkemboi

6 күн бұрын

Thank you 🙏

Good video 👍👍👍

@tonykipkemboi

9 күн бұрын

Really userful content and well explained. t would be interesting to see a video but with different types of files, like only PDFs, for example Markdown, PDF, and CSV all at once. It would be very interesting.

@tonykipkemboi

11 күн бұрын

Thank you! I have this in my content pipeline.

🤩🤩

Nice

@tonykipkemboi

14 күн бұрын

Thank you!

Thanks for the share. Quite enlightening. I will def build upon that. Here is the problem I have. Let's say Ihave two documents and I wanna chat with both at the same time (for instance to extract conflicting points between the two). What would you advise here?

@tonykipkemboi

22 күн бұрын

Thank you! That's an interesting use case for sure. My instinct before looking up some solutions is to maybe create 2 separate collections for each of the files then retrieve them separaetly and chat with them for comparison. I'm sure my suggestion above might not be efficient at all. I will do some digging and share any info I find.

Oh I thought you were saying you've embedded an LLM into a PDF document, like those draggable 3d diagrams.

Thanks, Can you please explain one by one and slowly. Especially the RAG part

@tonykipkemboi

16 күн бұрын

Thanks for asking. Which part of the RAG pipeline?

Would love it if can make the streamlit app! I am still struggeling to make a streamlit app based on open source llms

@tonykipkemboi

11 күн бұрын

Thank you! Yes, I'm working on a Streamlit RAG app. I have released a video on Ollama + Streamlit UI that you can start with in the meantime.

@thealwayssmileguy9060

11 күн бұрын

@@tonykipkemboi thanks bro! I will defo watch👌

Great job. Does the file you chat with have to be a PDF or can it be a CSV or other structured file type?

@tonykipkemboi

19 күн бұрын

🙏 thank you. I'm actually working on a video for RAG over CSV. The demo in this tutorial will not work for CSV or structured data; we need a better loader for structured data.

Thanks. Btw, how did you make your your KZread profile photo? It looks very nice.

@tonykipkemboi

20 күн бұрын

Thank you! 😊 I used some AI avatar generator website that I forgot but I will find it and let you know.

@guanjwcn

19 күн бұрын

Thank you

Can this model query with tabular data or image data, can't it?

@tonykipkemboi

22 күн бұрын

I assume you're talking about Llama2? Or are you referring to the Nomic text embedding model? If it's Llama2, it's possible to use it to interact with tabular data by passing the data to it (RAG or just pasting data to the prompt) but cannot vouch for its accuracy though. Most LLMs are not great at advanced math but they're getting better for sure.

Good one.. ok you touched on security- you have here something that doesn’t let things flow out to the internet. I saw a bunch of vids about tapping data from dbs using sql agents. But none said specifically anything about security. So qn- does using sql agents violate data security?

@tonykipkemboi

11 күн бұрын

You bring up a critical point and question. Yes, I believe most agentic workflows currently, especially tutorials, lack proper security and access moderation. This is a growing and evolving portion of agentic frameworks + observability, IMO. I like to think of it as people needing special access to databases at work and someone managing roles and the scope of access. So agents will need some form of that management as well.

Thanks for sharing this. Very helpful. Also, what are you using for screen recording and editing this video ? I see that it records the section where your mouse cursor is ! Nice video work as well. Only suggestion is to increase gain in your audio

@tonykipkemboi

22 күн бұрын

I'm glad you find it very helpful. I'm using Screen Studio (screen.studio) for recording; it's awesome! Thank you so much for the feedback as well. I actually reduced it during editing thinking it was too loud haha. I will make sure to readjust next time.

@xrlearn

21 күн бұрын

@@tonykipkemboi Btw, can you see those 5 questions that it generated before summarizing the document?

@tonykipkemboi

21 күн бұрын

@@xrlearn, I'm sure I can. I will try printing them out and share them here with you tomorrow.

@tonykipkemboi

19 күн бұрын

Hi @xrlearn - Found a way to print the 5 questions using `logging`. Here's the code you can use to print out the 5 questions: ``` import logging logging.basicConfig() logging.getLogger("langchain.retrievers.multi_query").setLevel(logging.INFO) unique_docs = retriever.get_relevant_documents(query=question) len(unique_docs) ``` Here are more detailed docs from LangChain that will help. python.langchain.com/docs/modules/data_connection/retrievers/MultiQueryRetriever/

could you drop a tutorial on building rag chatbots with ollama and langchain with custom data and guard-railing?

@tonykipkemboi

9 күн бұрын

That sounds interesting and something I'm looking into as well. For guard-railing, what are your thoughts on the frameworks for this portion? Have you tried any?

@yashvarshney8651

9 күн бұрын

@@tonykipkemboi realpython.com/build-llm-rag-chatbot-with-langchain/ I've reas this article, and the only guard-railing mech they seem to apply is an additional prompt with every inference.

What GPU do you use ? I have Ollama running on an i5 intel with integrated CPU and so unable to use any of 3B + models. TinyLama and TinyDolphin works but the accuracy is way off

@tonykipkemboi

14 күн бұрын

I have an Apple M2 with 16GB of memory. I noticed that larger models slow down my system and sometimes force a shutdown of everything. One way around it is deleting other models you're not using.

Is it possible to upload multiple PDF documents using the langchain doc loaders and then converse across them? Excellent tut and thanks - David

@tonykipkemboi

53 минут бұрын

That can definitely be possible. Are you thinking of probably two pdfs that each carry different content?

firstly thank you for sharing this entire tutorial, really great, i tried to implement it and got all the issues resolved but looks like i am not getting any output after i ask any question. i am OllaEmbeddings: 100% 5 times and then nothing happened after that, program just quit without giving any answer. will you be able to help me in this regards to see how to get it worked?

@tonykipkemboi

8 күн бұрын

Thank you for your question. Did you use the same models as in the tutorial, or did you use another one? Are you able to share your code?

@spotnuru83

8 күн бұрын

@@tonykipkemboi i copied ur code exactly

@spotnuru83

7 күн бұрын

the reason was i did not use jupiter notebook, i was running in VSCode and i had to save the value that is returned by chain's invoke method and when i printed , it started working, this is amazing.. thank you so much. really appreciate it.

I've been given a story, the trojan war which is a 6 page pdf or I can even use the story as a text , also 5 pre decided question is given to ask based on the story, I want to evaluate different models answers but I am failing to evaluate even one, kindly help, please guide thoroughly.

@ayushmishra5861

21 күн бұрын

Can you please reply, would really appreciate that.

@tonykipkemboi

21 күн бұрын

This sounds interesting! I believe if you're doing this locally, you can follow the tutorial to create embeddings of the PDF and store it in a vector db then use the 5 questions to generate output from the models. You can switch the model type in between each response and probbly have to save each response separately so you can compare them afterwards.

@ayushmishra5861

21 күн бұрын

@@tonykipkemboi What amount of storage will the model take. I don't have greatest of the hardware.

@tonykipkemboi

21 күн бұрын

Yes, there are smaller quantized models on Ollama you can use, but most of them require a sizeable amount of RAM. Check out these instructions from Ollama on the size you need for each model. You can also do one at a time, then delete the model after use to create space for the next one you pull. I hope that helps. github.com/ollama/ollama?tab=readme-ov-file#model-library

I encountered several errors when trying to execute the following line in the code: data = loader.load() Despite installing multiple modules, such as pdfminer, I'm unable to resolve an error stating "No module named 'unstructured_inference'." Has anyone else experienced similar issues with this code? Any assistance would be greatly appreciated. Thank you!

@tonykipkemboi

8 күн бұрын

Interesting that's asking for that since that's for layout parsing and we didn't use it. Try installing it like so; "!pip install unstructured-inference"

Thank you very much for your videos. Please, what's if we have severals PDFs?

@tonykipkemboi

51 минут бұрын

Yes, so you can iteratively load the pdfs, chunk them by page or something else, then index them in a vector database. You would then ask your query like always and it would find the context throughout all the documents to give you an answer.

PDFInfoNotInstalledError: Unable to get page count. Is poppler installed and in PATH? please help with this........

@tonykipkemboi

11 күн бұрын

Are you doing a different modification of the code in the tutorial or using OCR? I would checkout the install steps on their repo here (github.com/Belval/pdf2image) and probably use ChatGPT for debugging as well.

@levsigal6151

3 күн бұрын

@@tonykipkemboi I've got the same error and I am using PDF file. Please advise.

Pls provide notebook if possible. great video.

@tonykipkemboi

11 күн бұрын

Thank you! Checkout the repo link in the description for all the code. Here's the link github.com/tonykipkemboi/olla...

@hectorelmagotv8427

7 күн бұрын

@@tonykipkemboi hey, the link is not working, can provide it again pls?

@hectorelmagotv8427

7 күн бұрын

no problem, didnt see the description, thanks!

@tonykipkemboi

7 күн бұрын

@@hectorelmagotv8427 , thanks. Just to confirm, did it work?

Retrieving answers from vector database takes good one minute on my macbook air, how do I scale this model, can you add pinecone layer to it?

@tonykipkemboi

23 сағат бұрын

So this was a demonstration of running with everything local and nothing online other than when downloading the packages. You can hook up any vector store you like for example Pinecone as you've mentioned. Just beware that since the local models will still be in use, it will still be slow if your system is slow already. Might consider using paid services if you're looking for a lower latency solution.

@ayushmishra5861

23 сағат бұрын

@@tonykipkemboi So tony, what I am trying to build is something like a website, where people come and drop there pdf's and can do Q and A. In my learning and implementation I found out. My 10 page pdf embedding generation is not taking a lot of time, it used to before using the embedding model you used. Now embedding part is sorted. I tried implementing the code with chroma and faiss, results are almost equal. Even for a small sized pdf, it takes a minute to answer. I understand it takes computational resource from my local machine, which happens to be a Macbook Air M1. Do you the a machine with better GPU, lets assume yours produce the retrieved results under 10 seconds? Nobody would like to wait a minute or more than a minute on website for an answer, also I am scared about the part if there are 100's of 1000's of user, do I need to purchase a GPU farm for this to work, lol. Note- I have never made a scalable project before. Please guide. Also share how much time it takes on your Pc/laptop for the answer to come back from the vector db, so I can understand if it's my system which is weak or libraries like chroma and faiss are not meant for scalability.

@ayushmishra5861

22 сағат бұрын

@@tonykipkemboi .

Hi, and if there were 6 or 10 PDFs, how would you load them into the RAG? Thanks

@tonykipkemboi

Күн бұрын

Good question! I would iterate through them while loading them and also index the metadata so it's easy to reference which pdf provided the context for the answer. There's actually several ways of doing this but that would be my simple first try.

I tried the first command %pip install -q unstructured langchain and its taking a super long time to install. Is this normal?

@tonykipkemboi

4 күн бұрын

It shouldn't take more than a couple of seconds but also depending on your system and package manager, it might take a while. Did it resolve?

Nice video. When i try to execute the following commands: !ollama pull nomic-embed-text and !ollama list. I receive the following error: /bin/bash: line 1: ollama: command not found

@tonykipkemboi

7 күн бұрын

This error is means that Ollama is not installed on your system or not found in your system's PATH. Do you have Ollama already installed?

@erickcedeno7823

2 күн бұрын

@@tonykipkemboi Hello, I've installed ollama in my local system but i don't know why i'm getting an error in google colab

hey,if we are using google colab,instead of jupyter,how will we able to corporate ollama with google colab?

@tonykipkemboi

9 күн бұрын

I haven't tried this myself but here are some resources for you that might be helpful; 1. medium.com/@neohob/run-ollama-locally-using-google-colabs-free-gpu-49543e0def31 2. stackoverflow.com/questions/77697302/how-to-run-ollama-in-google-colab

whenever I try pip install -q unstructured["all-docs"] (using win11) I keep getting a subprocess error: Getting requirements to build wheel did not run successfully. │ exit code: 1 any help appreciated!!

@N0rt

15 күн бұрын

AssertionError: Could not find cmake executable!

@tonykipkemboi

15 күн бұрын

Which Python version are you using? It seems there's an issue with latest Python versions and seems to work best with Python

@N0rt

14 күн бұрын

@@tonykipkemboi I'm on Python 3.12

@tonykipkemboi

14 күн бұрын

@@N0rt, I see. The unstructured package has had a lot of errors recently. You could try installing CMake on your Windows machine using the instructions below. Let me know if it works. 1. Go to cmake.org/download 2. Select Windows (Win32 Installer) 3. Run the installer 4. When prompted, select Add CMake to the system PATH for all users 5. Run the software installation 6. Double-click the downloaded executable file 7. Follow the instructions on the screen 8. Reboot your machine

@N0rt

11 күн бұрын

@@tonykipkemboi thank you so much ill give it a try!

can I do this on google colab?

@tonykipkemboi

21 күн бұрын

This is local using Ollama so not possible following this specific tutorial. you can however use other public models that have API endpoints that you can call from Colab. I also want to mention that I have not explored trying to access the local models through Ollama using Colab.

ERROR:unstructured:Following dependencies are missing: pikepdf, pypdf. Please install them using `pip install pikepdf pypdf`. WARNING:unstructured:PDF text extraction failed, skip text extraction... please help

@tonykipkemboi

11 күн бұрын

Have you tried installing what it's asking for `pip install pikepdf pypdf`?

@suryapraveenadivi851

10 күн бұрын

@@tonykipkemboi Thank you so much!! for your reply this got resolved..

@tonykipkemboi

10 күн бұрын

@@suryapraveenadivi851 glad it worked! Happy coding.

E do not need anAPI key for this ?

@tonykipkemboi

6 күн бұрын

Nope, don't need one.

Where is the RAG part?

@tonykipkemboi

4 күн бұрын

The whole thing actually is RAG.

@Hoxle-87

4 күн бұрын

@@tonykipkemboi thanks. So adding the pdfs augments the LLM, got it.

How can download unstructured[all-doc] I can not install this,

@tonykipkemboi

4 күн бұрын

Did you install it like this '!pip install --q "unstructured[all-docs]"'

I want page source after the answer. How to add in the code

@tonykipkemboi

11 күн бұрын

You would need to save the responses from the vector store that matches the query then be able to add that as part of the response. There's probably a more efficient way to do it with the metadata but I haven't played around with it much. Will make sure to add that portion in my next RAG video. Thank you.

@malleswararaomaguluri6344

11 күн бұрын

@@tonykipkemboi thank you, because it will play a major role if we are handling with huge document or multiple documents.

@tonykipkemboi

11 күн бұрын

@@malleswararaomaguluri6344 absolutely 💯. Yes, being able to cite at page level and even maybe showing a snippet of the sources would be great.

Are you Kenyan?

@tonykipkemboi

Күн бұрын

Yes, by birth. I'm located in the U.S. atm.

The pdf refuses to answer.

@tonykipkemboi

12 күн бұрын

Can you explain more about your process?

@maxmuster7003

12 күн бұрын

@@tonykipkemboi I tried to use chatgpt online, but it gave me wrong answers.

@tonykipkemboi

12 күн бұрын

Have you tried the way I did it in this tutorial?

@maxmuster7003

12 күн бұрын

@@tonykipkemboi My PC is broken, i am on an Android tablet.

@tonykipkemboi

12 күн бұрын

@@maxmuster7003 ah I see. Yeah, you have to be on a computer due to memory.

I have a 10 year old PC which has 8GB RAM. Can I use local AI on this machine or do I have to buy a new machine to get it working?

@InnocentiusLacrimosa

9 күн бұрын

That sounds really bad. Why do that to yourself? Anyhow, at that point I would just use Google Colab or something similar: you do not have cuda capable GPU, your cpu is bad and your RAM is barely enough to run the operating system.

@aashishmittal206

9 күн бұрын

@@InnocentiusLacrimosa Is Google colab free?

@tonykipkemboi

9 күн бұрын

Yes, Colab is free and can get you going but you would still need to download Ollama and am not sure you have enough RAM to get any models going.

@aashishmittal206

9 күн бұрын

@@tonykipkemboi If I were to buy a new PC, what should be the specs which is future proof for the next 5 years if all I do is code and do Adobe applications as well? And by coding, I meant using LLMs locally among other things.

@InnocentiusLacrimosa

9 күн бұрын

@@aashishmittal206 You are talking about a desktop, not a laptop? If I were seriously into LLMs, I would try to get an used 3090 as those sell for 600-700 USD/EUR and are the cheapest entry to 24GB VRAM. If that is not an option (too expensive) then the next tier down in cost is 12GB cards and RTX 3060 is the cheapest of those. That is enough to run perhaps the 7 Billion parameter models (?). At least 32GB RAM and a modern CPU (13600K or R5 7600) is the practical entry point to modern CPUs. Without a budget/needs it is really hard to give more advice. You talk about future proofing and doing coding, LLMs, but your current PC kind of states that you are not making money from that work so it is either a study or a hobby. You could be from anywhere and have access to different hardware sets, etc. If budget is not an issue, one could easily just get a 4x3090 or 4x4090 + 14900K/7950X + 128GB RAM or top of the line Macbook with M3 Max and 128GB unified RAM. To invest something into those, one needs to know that it is an investment that pays itself back (rather fast). But then again if you are on that space, you would probably be building solutions based on cloud computing again :-D

dealing with pdf on RAG is the worst thing that you can consider! unreliable

!ollama run nomic-embed-text ------------ /bin/bash: line 1: ollama: command not found please help with this too.

@tonykipkemboi

11 күн бұрын

Should be this instead: `ollama pull nomic-embed-text`

@suryapraveenadivi851

10 күн бұрын

@@tonykipkemboi Tried this got File "", line 1ollama pull nomic-embed-text SyntaxError: invalid syntax

@tonykipkemboi

9 күн бұрын

I think you used "1" instead of "!" in `!ollama pull nomic-embed-text`

why can't i use !ollama pull nomic-embed-text ? someone helps me

@tonykipkemboi

6 күн бұрын

What error are you getting when you run the cell?

@tuanthanh-fg4xr

5 күн бұрын

@@tonykipkemboi i've done it , i must run this line in terminal not jupyter cell