What a great tutorial, clear, concise and informative!!!

Thank you for such a helpful explanation. I wish more tech explainers and presenters were this clear and structured.

Thank you, Matt! Please create more videos like this; they're really clear and simple.

Thank you, Matt! Your videos are awesome and already helped me a lot :)

Im so happy there's someone in the mix who actually has a career in AI/Dev. Seriously, really enjoying your content. Dont listen to any of these jerks.

If you can make a video about crew AI explaining it as simply as you did here, you'd be my hero.

@edwardrhodes4403

2 ай бұрын

And also Autogen and other agents like Devika and how to integrate them

thank you for you job Matt.

Great Overview Matt, your a star!

Thank you for the video

Excellent introduction. Thanks for the help

Hello from Brazil. I really liked your content ! thanks

Verry cool, thank you 🙏

AMAZING!

You're a very smart person 👍👊

Excellent summary, thanks alot.

Nice video

Great content

You explained it perfectly, thank you

@technovangelist

2 ай бұрын

Glad it was helpful!

Thanks! Just downloaded Ollama and was feeling a bit lost. Would really appreciate some guidance on integrating speech recognition and text to speech into the chatbot. But just about anything you say will probably be useful. Please keep 'em coming!

Thank you so much for your tutorials! :D I would like to suggest an idea for a future video that I would be really interested in watching: a more detailed exploration of the various models (such as the instruct/base/etc. ones you've mentioned). Again, thank you very much! You rock! ^^/

Great video, thank you for Ollama introduction! is there a way to add my own data to the model or shrink model to a particular topic? for example TailwindCSS, there is a sorce code, docs, library of the project, is there any way to train model to be able generate layouts and components based on a provided data?

Thanks for this! I just setup Ollama with wizard-vicuna-uncensored:30b-q8_0. Do you know what's different in the fp16 models?

thank you

I successful run Ollama with llama 3 on Mac OS M1 8G, it runs better when you close other apps .

thanks for sharing, prune is something what I missed but very useful due to the fact of downloading large files and loosing them after each restart, was very annoying. I see ollama didn't documented it well or maybe I overlooked it

Hi Matt, do you know what determines the length of a model's answer? How does the model 'know' when to stop? Is it hard coded into the model or is it controlled by Ollama? Thanks

thanks

Do the models return Markdown ...like lists? extended Markdown ...like tables and LaTeX? I have written my own desktop client that I put on all our machines to use OpenAI and their API (cheaper than $20 * 5/month). So I would like to have a network server for my home to run a local model. Can I set it up there and have everyone use it, or would there be performance issues? ...what about simultaneous usage?

Thanks. I really learned a lot by watching your videos. I recently ran into an issue when writing a new template model for few shot learning. Most of the times it responds like expected but sometimes it responds to my prompt and then also inserts it's own command by adding [INST] some other prompt … and also answers it. I probably made some mistake but I cannot figure it out. That's why I would love to see you make a video on templates (if this isn't too much to ask).

@jahbini

2 ай бұрын

I second that request!

Subed ❤️ If only you taught maths too

I like your videos, always spot on and pedagogical! Why did you leave ollama?

@technovangelist

2 ай бұрын

If we find ourselves in the same room I’ll talk about it there.

Do I need to know match, statistics, etc... apart from programming?

Does it phone "home" at all, or is the model I use locally, assuming the machine is connected to the Internet, shared publicly at all, and is it secure?

@technovangelist

Ай бұрын

It doesn’t reach out anywhere unless you write a program to have it do something like that.

Here’s a video request: can you do one on LMSys’s SGLang? Particularly using constrained decoding?

Sometimes your level is so high some others too simple. Cheers.

Great video. The costs are a bit unclear to me. Would a local ollama/llm be free to use?

@technovangelist

2 ай бұрын

What costs? You have to own a computer with a gpu. That’s it

Please create videos on how to fine tune these models 🙏

FYI- My llama2 install is working reasonably fast without a GPU, just a ryzan 5600G CPU, which has some rudimentary graphics capacity built into it.

Thanks Matt, why does everyone use Mistral rather than Mixtral?

@technovangelist

2 ай бұрын

Too slow

I just dont get any sort of CLI when i install Ollama , the app just shows "view logs" and "Quit Ollama"

@technovangelist

22 күн бұрын

so when you run ollama at the command line you don't see anything?

@blackwinegum

22 күн бұрын

@@technovangelist I think i've figured it out, i think my firewall was blocking something, thanks for replying.

great content, a telegram group would be great!

@technovangelist

Ай бұрын

Telegram??? I think I used it once at an Idan Raichel concert but never since. What’s special about a telegram group?

@stebansb

Ай бұрын

@@technovangelist the other option being Discord, I feel is simpler, cleaner user interface, yet very powerful; popular with business and a slightly more mature cohort. The other option is Discord, slower more complex, popular among gamers. Either way, will be cool to have something to build a community beyond KZread.

I have a feeling Duncan Trussell is a subscriber : )

Hello! My video card is an RX580. Is there a way to make it work?

@alexsnow2993

Ай бұрын

Using the rx580, it will be slow? Or not work at all?

@technovangelist

Ай бұрын

I don’t see it on the compatibility list. github.com/ollama/ollama/blob/main/docs/gpu.md

@technovangelist

Ай бұрын

Just won’t work at all. I think ollama requires the newer amd drivers and amd didn’t make it backwards compatible with older cards.

@alexsnow2993

Ай бұрын

Thanks for the info! I can't get another V-card at the moment, and using the CPU it is a no go. Is there any version or any other AI out there, that can be configured locally?

@technovangelist

Ай бұрын

Everything I know of is going to need a decent recent gpu.

it would have been great to know how to install other models not mentioned in Ollama library with specific type of Low Vram and GGUF

@technovangelist

2 ай бұрын

check out kzread.info/dash/bejne/YqOpl5OlXbCyc9I.html

@mrrohitjadhav470

2 ай бұрын

@@technovangelist Thanks a lot❤

I wish there was an easier way to fill in template text, and parsing pdfs. I've seen the 'function calling' video's, but somehow it's still eluding me how do get this done as easily as possible (e.g. sending a pdf over api in a curl request from another machine, and rename it sensibly/according to content)

@technovangelist

2 ай бұрын

The biggest problem there is the pdf. You can’t easily get to the contents of the pdf. The text. It’s often jumbled up. PDF is the worst format you can use if you want the text and to do something with it. That’s also one of the benefits of pdf. It obfuscates the source text so folks can’t do anything with the text.

@userou-ig1ze

2 ай бұрын

​@@technovangelistthanks for the reply! I used pdf2text but it was not exactly perfectly successful. I wonder how ollama frontends (e.g. webgui or webui) solve this for their RAG? Gave me hope that there is a good way of doing it 🎉

Ollama runs very well on a M3 Max

Wait do I need to download anything or not? You say "Now the model should be downloaded, so you can run it with ollama run mistral" Why would it be downloaded? I just installed the Ollama software. Does it download all the models automatically? This seems very unclear

@technovangelist

Ай бұрын

I think you must have skipped around a bit. I very clearly said to install and then run ollama pull to download the model. Then while downloading talked about what’s going on. Then the model is downloaded and you can run it. When you downloaded the model you only downloaded that model. Why download anything? Because you want to run it.

@axeljohannes3464

Ай бұрын

Thanks! I got it to work@@technovangelist

@axeljohannes3464

Ай бұрын

I think what confused me is the term pull, and what that actually meant. So when you got to the point of speaking about downloaded, I was like "Hey, no one said anything about downloading anything"

9:40 To have the same result you just need the same input, seed (and other parameters), no? Reason it's different every time is because the seed is random for every request, right? The seed used (and other parameters) make the result different because it takes a different path in the weight model.

@technovangelist

2 ай бұрын

Using the same seed and temp doesn’t always guarantee the same result

@makesnosense6304

2 ай бұрын

@@technovangelist Ah, because temp is a percentage randomness scale of sort.

@makesnosense6304

2 ай бұрын

@@technovangelist What if temp is 0? Or 1?

@technovangelist

2 ай бұрын

It’s not guaranteed

arch and nix are system-d distros

@technovangelist

2 ай бұрын

Doh!

Please, do more compatibility with GPUs. I have rx580. My processor is crying hehe

@technovangelist

5 күн бұрын

That’s a request for AMD to add support to those older lower end cards I think.

@viniciussilvano4177

5 күн бұрын

@@technovangelist Is there any way I can use a library that allows me to do this. Or is it actually something that depends on AMD? I'm really impressed with what using Ollama as an API has added to my projects. I would like to find some way to speed up processing without having to spend money, at least for now.

@technovangelist

5 күн бұрын

But amd support requires a certain level of the drivers which amd only has working for newer cards. I think the only option is to buy a more recent card. The 580 is 5 years old.

Can Ollama be used in prod on Linux server?

@technovangelist

2 ай бұрын

absolutely. lots of folks are doing just that.

@briannezhad1804

2 ай бұрын

@@technovangelistWow, that is amazing. I would appreciate it if you could provide documents that would guide me in deploying a model for production use and "Function Calling." Ollama is an excellent tool for a startup to keep costs down and avoid OpenAI usage costs.

@briannezhad1804

2 ай бұрын

It also gives us flexibility to keep our data in-house.

@briannezhad1804

2 ай бұрын

​@@technovangelist This is awesome!. Do you have any reference deploy successfully into prod? We are trying to avoid OpenAI and are looking for open-source AI models with "Function calling."

@technovangelist

2 ай бұрын

all the docs in the github repo. but it’s a pretty simple app without many dependencies. I don't know of any guidance though.

thanks for the informative video. some examples should be displayed in a video, rather than spoken, for example, to show how to use environment variables

@technovangelist

2 ай бұрын

There are a number of videos pointed out throughout the video that provide all the examples

Booted up Ollama with the llama2 model and my M1 MBP just froze 💀

@technovangelist

Ай бұрын

That is bizarre…as in you would be the first person that has happened to. Running macOS I assume. Installed using the installer? What else was running? So you installed and then opened a terminal and run ollama run llama2 and then nothing? Probably easiest to solve on the discord.

@robert_kotula

Ай бұрын

@@technovangelist I’ll join the discord channel and try to troubleshoot. I’ve had a couple of tabs open in Safari and one tab in Firefox Developer addition, nothing else. Will need to dig into the performance stats on the laptop.

you should have more views and subscribers

@technovangelist

2 ай бұрын

Thanks. Both are accelerating quickly. But feel free to share. I like to say I am working on my first million subscribers. Only 985,000 short.

First

@technovangelist

2 ай бұрын

First what.

@technovangelist

2 ай бұрын

About 6 hours late to be first comment

My model file refuses to create an awkward silence at the end of its output 😡