Thank you, Matt! Please create more videos like this; they're really clear and simple.

If you can make a video about crew AI explaining it as simply as you did here, you'd be my hero.

@edwardrhodes4403

3 ай бұрын

And also Autogen and other agents like Devika and how to integrate them

I just did all of this yesterday and still watched your video from start to finish. Very clear and concise. I look forward to your other videos.

Thank you for such a helpful explanation. I wish more tech explainers and presenters were this clear and structured.

Im so happy there's someone in the mix who actually has a career in AI/Dev. Seriously, really enjoying your content. Dont listen to any of these jerks.

This is gold. Thank you, Mr Williams!

I have watched a few of your videos. I love the way you explain things simply and clearly. Keep it up. Thank you for your work to all of us.

@technovangelist

16 күн бұрын

Thanks so much

Thank you, Matt! Your videos are awesome and already helped me a lot :)

I really like this channel! The presentation is great and understandable. It is easy to follow along. Thanks.

thank you for you job Matt.

Thank you for the video

Great Overview Matt, your a star!

fascinating. thank you for these videos!

What a great tutorial, clear, concise and informative!!!

@technovangelist

16 күн бұрын

Glad you liked it.

AMAZING!

Excellent introduction. Thanks for the help

Verry cool, thank you 🙏

Hello from Brazil. I really liked your content ! thanks

You're a very smart person 👍👊

Thank you so much for your tutorials! :D I would like to suggest an idea for a future video that I would be really interested in watching: a more detailed exploration of the various models (such as the instruct/base/etc. ones you've mentioned). Again, thank you very much! You rock! ^^/

Excellent summary, thanks alot.

Would have been good to cover what ollama is and what problems does it solve, for about 30sec-1min, before going into hardwarre requirements etc.

Thanks! Just downloaded Ollama and was feeling a bit lost. Would really appreciate some guidance on integrating speech recognition and text to speech into the chatbot. But just about anything you say will probably be useful. Please keep 'em coming!

You explained it perfectly, thank you

@technovangelist

3 ай бұрын

Glad it was helpful!

Nice video

10 out of 10!

Thanks for this! I just setup Ollama with wizard-vicuna-uncensored:30b-q8_0. Do you know what's different in the fp16 models?

Great content

Great video, thank you for Ollama introduction! is there a way to add my own data to the model or shrink model to a particular topic? for example TailwindCSS, there is a sorce code, docs, library of the project, is there any way to train model to be able generate layouts and components based on a provided data?

I successful run Ollama with llama 3 on Mac OS M1 8G, it runs better when you close other apps .

Hi Matt, do you know what determines the length of a model's answer? How does the model 'know' when to stop? Is it hard coded into the model or is it controlled by Ollama? Thanks

thanks for sharing, prune is something what I missed but very useful due to the fact of downloading large files and loosing them after each restart, was very annoying. I see ollama didn't documented it well or maybe I overlooked it

Thanks. I really learned a lot by watching your videos. I recently ran into an issue when writing a new template model for few shot learning. Most of the times it responds like expected but sometimes it responds to my prompt and then also inserts it's own command by adding [INST] some other prompt … and also answers it. I probably made some mistake but I cannot figure it out. That's why I would love to see you make a video on templates (if this isn't too much to ask).

@jahbini

3 ай бұрын

I second that request!

Does it phone "home" at all, or is the model I use locally, assuming the machine is connected to the Internet, shared publicly at all, and is it secure?

@technovangelist

Ай бұрын

It doesn’t reach out anywhere unless you write a program to have it do something like that.

Do the models return Markdown ...like lists? extended Markdown ...like tables and LaTeX? I have written my own desktop client that I put on all our machines to use OpenAI and their API (cheaper than $20 * 5/month). So I would like to have a network server for my home to run a local model. Can I set it up there and have everyone use it, or would there be performance issues? ...what about simultaneous usage?

Subed ❤️ If only you taught maths too

I like your videos, always spot on and pedagogical! Why did you leave ollama?

@technovangelist

3 ай бұрын

If we find ourselves in the same room I’ll talk about it there.

thank you

Great video. The costs are a bit unclear to me. Would a local ollama/llm be free to use?

@technovangelist

2 ай бұрын

What costs? You have to own a computer with a gpu. That’s it

Sometimes your level is so high some others too simple. Cheers.

Here’s a video request: can you do one on LMSys’s SGLang? Particularly using constrained decoding?

Thanks Matt, why does everyone use Mistral rather than Mixtral?

@technovangelist

3 ай бұрын

Too slow

Do I need to know match, statistics, etc... apart from programming?

Please create videos on how to fine tune these models 🙏

I have a feeling Duncan Trussell is a subscriber : )

thanks

great content, a telegram group would be great!

@technovangelist

Ай бұрын

Telegram??? I think I used it once at an Idan Raichel concert but never since. What’s special about a telegram group?

@stebansb

Ай бұрын

@@technovangelist the other option being Discord, I feel is simpler, cleaner user interface, yet very powerful; popular with business and a slightly more mature cohort. The other option is Discord, slower more complex, popular among gamers. Either way, will be cool to have something to build a community beyond KZread.

I just dont get any sort of CLI when i install Ollama , the app just shows "view logs" and "Quit Ollama"

@technovangelist

Ай бұрын

so when you run ollama at the command line you don't see anything?

@blackwinegum

Ай бұрын

@@technovangelist I think i've figured it out, i think my firewall was blocking something, thanks for replying.

Ollama runs very well on a M3 Max

FYI- My llama2 install is working reasonably fast without a GPU, just a ryzan 5600G CPU, which has some rudimentary graphics capacity built into it.

@Thymed

10 күн бұрын

More than rudimentary but still I get your point. much less than modern GPUs or recent APUs

Hello! My video card is an RX580. Is there a way to make it work?

@alexsnow2993

2 ай бұрын

Using the rx580, it will be slow? Or not work at all?

@technovangelist

2 ай бұрын

I don’t see it on the compatibility list. github.com/ollama/ollama/blob/main/docs/gpu.md

@technovangelist

2 ай бұрын

Just won’t work at all. I think ollama requires the newer amd drivers and amd didn’t make it backwards compatible with older cards.

@alexsnow2993

2 ай бұрын

Thanks for the info! I can't get another V-card at the moment, and using the CPU it is a no go. Is there any version or any other AI out there, that can be configured locally?

@technovangelist

2 ай бұрын

Everything I know of is going to need a decent recent gpu.

arch and nix are system-d distros

@technovangelist

3 ай бұрын

Doh!

it would have been great to know how to install other models not mentioned in Ollama library with specific type of Low Vram and GGUF

@technovangelist

3 ай бұрын

check out kzread.info/dash/bejne/YqOpl5OlXbCyc9I.html

@mrrohitjadhav470

3 ай бұрын

@@technovangelist Thanks a lot❤

thanks for the informative video. some examples should be displayed in a video, rather than spoken, for example, to show how to use environment variables

@technovangelist

3 ай бұрын

There are a number of videos pointed out throughout the video that provide all the examples

I wish there was an easier way to fill in template text, and parsing pdfs. I've seen the 'function calling' video's, but somehow it's still eluding me how do get this done as easily as possible (e.g. sending a pdf over api in a curl request from another machine, and rename it sensibly/according to content)

@technovangelist

3 ай бұрын

The biggest problem there is the pdf. You can’t easily get to the contents of the pdf. The text. It’s often jumbled up. PDF is the worst format you can use if you want the text and to do something with it. That’s also one of the benefits of pdf. It obfuscates the source text so folks can’t do anything with the text.

@userou-ig1ze

3 ай бұрын

​@@technovangelistthanks for the reply! I used pdf2text but it was not exactly perfectly successful. I wonder how ollama frontends (e.g. webgui or webui) solve this for their RAG? Gave me hope that there is a good way of doing it 🎉

Wait do I need to download anything or not? You say "Now the model should be downloaded, so you can run it with ollama run mistral" Why would it be downloaded? I just installed the Ollama software. Does it download all the models automatically? This seems very unclear

@technovangelist

2 ай бұрын

I think you must have skipped around a bit. I very clearly said to install and then run ollama pull to download the model. Then while downloading talked about what’s going on. Then the model is downloaded and you can run it. When you downloaded the model you only downloaded that model. Why download anything? Because you want to run it.

@axeljohannes3464

Ай бұрын

Thanks! I got it to work@@technovangelist

@axeljohannes3464

Ай бұрын

I think what confused me is the term pull, and what that actually meant. So when you got to the point of speaking about downloaded, I was like "Hey, no one said anything about downloading anything"

9:40 To have the same result you just need the same input, seed (and other parameters), no? Reason it's different every time is because the seed is random for every request, right? The seed used (and other parameters) make the result different because it takes a different path in the weight model.

@technovangelist

3 ай бұрын

Using the same seed and temp doesn’t always guarantee the same result

@makesnosense6304

3 ай бұрын

@@technovangelist Ah, because temp is a percentage randomness scale of sort.

@makesnosense6304

3 ай бұрын

@@technovangelist What if temp is 0? Or 1?

@technovangelist

3 ай бұрын

It’s not guaranteed

Please, do more compatibility with GPUs. I have rx580. My processor is crying hehe

@technovangelist

23 күн бұрын

That’s a request for AMD to add support to those older lower end cards I think.

@viniciussilvano4177

22 күн бұрын

@@technovangelist Is there any way I can use a library that allows me to do this. Or is it actually something that depends on AMD? I'm really impressed with what using Ollama as an API has added to my projects. I would like to find some way to speed up processing without having to spend money, at least for now.

@technovangelist

22 күн бұрын

But amd support requires a certain level of the drivers which amd only has working for newer cards. I think the only option is to buy a more recent card. The 580 is 5 years old.

Can Ollama be used in prod on Linux server?

@technovangelist

2 ай бұрын

absolutely. lots of folks are doing just that.

@briannezhad1804

2 ай бұрын

@@technovangelistWow, that is amazing. I would appreciate it if you could provide documents that would guide me in deploying a model for production use and "Function Calling." Ollama is an excellent tool for a startup to keep costs down and avoid OpenAI usage costs.

@briannezhad1804

2 ай бұрын

It also gives us flexibility to keep our data in-house.

@briannezhad1804

2 ай бұрын

​@@technovangelist This is awesome!. Do you have any reference deploy successfully into prod? We are trying to avoid OpenAI and are looking for open-source AI models with "Function calling."

@technovangelist

2 ай бұрын

all the docs in the github repo. but it’s a pretty simple app without many dependencies. I don't know of any guidance though.

Booted up Ollama with the llama2 model and my M1 MBP just froze 💀

@technovangelist

2 ай бұрын

That is bizarre…as in you would be the first person that has happened to. Running macOS I assume. Installed using the installer? What else was running? So you installed and then opened a terminal and run ollama run llama2 and then nothing? Probably easiest to solve on the discord.

@robert_kotula

2 ай бұрын

@@technovangelist I’ll join the discord channel and try to troubleshoot. I’ve had a couple of tabs open in Safari and one tab in Firefox Developer addition, nothing else. Will need to dig into the performance stats on the laptop.

you should have more views and subscribers

@technovangelist

3 ай бұрын

Thanks. Both are accelerating quickly. But feel free to share. I like to say I am working on my first million subscribers. Only 985,000 short.

First

@technovangelist

3 ай бұрын

First what.

@technovangelist

3 ай бұрын

About 6 hours late to be first comment

My model file refuses to create an awkward silence at the end of its output 😡