Yes, please. Autogen + Ollama with agents each using a different model. Please mention deployment/resource considerations. Love your videos.

@technovangelist

4 ай бұрын

Great. I will do that. You got 8 thumbs up as of this comment and there have been a few other comments on this so it’s definitely going to get coverage.

@apester2

4 ай бұрын

It will be great to talk about what kinds of things are skills. I have played with autogen but I wasn’t able to get a good grip on what makes a good skill. Too complex or too simple. A few good examples with agents and workflows will be great.

@technovangelist

4 ай бұрын

That’s great. I should definitely go into that. Thanks

@emmanuelgoldstein3682

4 ай бұрын

If you don't need an interface, CrewAI is turning out to be pretty capable as well.

@RickySupriyadi

4 ай бұрын

there was this paper recently... the accuracy increased as multiple models involved working on same thing...

came for the knowledge, stayed for the humor

excellent topic and great delivery. Keep up the good work!

Thank You Would love to see more LangGraph and CrewAI content... but I think you just explained perfectly what I need todo

This video is just 10 minutes but it covered hours of content if you want to reproduce same results from scratch and you are new to these tools. I like this way. It not only helps to have your needs fulfilled but also understand what are some relevant tools that might help you.

Love your persona!! Phenomenal content!!

Ha, I love your humor! I came for the great AI info, but stayed for your personality. :)

Love your videos Matt.

@technovangelist

4 ай бұрын

Thanks so much. Glad you like them.

Flowise plus Ollama with multilingual embedings. Thanks for your videos!

@technovangelist

4 ай бұрын

Great. Flowise looks interesting. Been meaning to play around with it.

@mayorc

4 ай бұрын

I was not able to use LM studio (no error or whatsoever and no display in the LM Studio log at all) or Ollama (error was there but not clear what the problem was) and running a local model with Flowise (with ollama running in a container and Flowise running in another), could you give it a check and see how those could work together? (i think correct export of the ports between containers could be the issue, but I have zero to little knowledge on how to fix that, but that still didn't explain why LM Studio was not working at all) @@technovangelist

@user-cf5jr9bq7i

4 ай бұрын

@@mayorc Yes, check the port forwarding. And the last thing that worked for me was the ollama address: ollama:11434 . Without /api/embeddings/. I watched the logs in docker-deskstop until "200" appeared. And the text is still small in size, there may also be incomprehensible errors. Good luck!

Great video! Noob question, for autogen and ollama, do I need to pass the ollama run command first or does it serve automatically? I can see ollama is running in a web browser, just not sure if I need to run the model in a terminal first. Thanks

@technovangelist

4 ай бұрын

Once ollama is installed it has a service running. There is no web ui unless you also install a community integration for that. Ollama run is just for an interactive cli session. That cli is using the service just like autogen will. So no need to run other things. If you can use ollama run, that is an indicator that the service is ok.

@Royaltea_Citizen

4 ай бұрын

@@technovangelist nice one thanks! It’s a great time to start working more with autogen studio, be cool to see some use cases for it.

Your videos are so great. Informative and clear. Definitely my favorite on AI topics. Thanks!!

@technovangelist

4 ай бұрын

Wow. Thanks so much. If you have any topics let me know.

using ollama and litellm worked quite good for me for generating the oai api endpoint for autogen

@technovangelist

4 ай бұрын

It adds an additional dependency so simplifying is great. And for a lot of folks it may add two dependencies if your project isn’t using python.

@Unicron187

4 ай бұрын

@@technovangelist true, it adds additional dependencies, but i usually need to use python for projects. if you don't mind me mentioning, i am actually here looking for 0:57 and hoped for some link to another video 😜 but i think there will not be much gpu-offloading if i do not define the number of layers in the modelfile 🤪

Very informative with lots of subtile humor. You earned a sub from me! Thanks fir sharing. I have been waiting on this update.

@technovangelist

4 ай бұрын

Great. Thanks. I had fun adding the pew pew pew sounds to the terminator sequence. And now I need to watch the full bluey pregnant dad. It’s on KZread.

Keep it up. Your videos are informative and entertaining. Would like to see the best way to use Ollama on a Windows machine.

@technovangelist

4 ай бұрын

Yes. This is one of the common questions. Folks think it doesn’t run on windows. I need to revive my old windows persona. I worked at Microsoft for 5 years. That persona is still in me somewhere.

Awesome Video! Very Infomative! Can you tell me how you added api-key to ollama server. Would be helpful. Thanks

@technovangelist

2 ай бұрын

Ollama doesn’t require an api key ever. The open ai compatible api uses it so just plug anything you want in there. It’s gets ignored.

Thank you for yor videos. clearly enlightening. If u could share a Linux CLI interaction where bash can parse results or commands o a local LLM CLI would be fantastic.

@technovangelist

4 ай бұрын

Thanks. I think there is a bash focused integration in the community list at the bottom of the readme. And someone pointed out another tool to me that is like that but OpenAI only. So maybe that’s a good next one. I have a few ‘next’ videos

Is it possible to serve the same model loaded on memory both as an Ollama endpoint and an OpenAI endpoint without consuming more memory?

@technovangelist

4 ай бұрын

Yes. Nothing special you need to do. It just works

Are function calling supported in the OAI compatibility?

@chuchanche

4 ай бұрын

This is THE question I had as well.

@technovangelist

4 ай бұрын

So function calling is supported in ollama but the more complicated version OpenAI uses is not yet supported

@JimLloyd1

4 ай бұрын

@@technovangelistIs there documentation or a sample for how to do function calling in ollama now?

@technovangelist

4 ай бұрын

Yes. Look for format json in the docs

Came for Ollama, stayed for Bluey. Nice Video 😊

@technovangelist

4 ай бұрын

Maybe I should do a side channel for a commentary on bluey. We love the Waldorf school toys in all the shots

What API Matt providing in the Autogen's new model creation (5:45 time frame)? and how to get that API?

@technovangelist

2 ай бұрын

It’s in ollama. Nothing you need to do to turn it on

Hi! How web-ui do you recommend for ollama? Thank you 🙏🏼

@technovangelist

4 ай бұрын

I haven't really looked at them in detail. The UI built into ollama has been my main go to.

Thank you. Great video. More Autogen and Ollama. Please.

@technovangelist

4 ай бұрын

More to come!

How do we get function calling will ollama? Any tool or method out there?

@technovangelist

4 ай бұрын

Look for format json in the docs

Wow Wonderful nice and crisp. Yes, please AUTOGEN + Ollama with Multiple agents with deployment -> Waiting for it !!! Thanks in Advance.

@technovangelist

4 ай бұрын

More autogen with multiple agents and deployment. Got it. I am new to autogen. What do you mean by deployment here, to make sure I cover what you are thinking.

so close AutoGen recognized the model but i click on the sinewave button and i get Error occurred while processing message: api_key is not present in lim_config or OPENAILAPILKEY env variable for agent ** primary_assistant**. Update your workflow to provide an api_key to use the LLM. i put in 1234 as the api key and /v1 on the url

@technovangelist

4 ай бұрын

That’s what the docs and the video says to do. You need to set an api key to something and some tools require the url up to v1 and some don’t.

How can I make my own AI that could let me use this as an example like example if I want an AI build just because I invite you not book and that AI could see if that's something was copyright how can I guess something like that made and how can I get them to rewrite it without it being copyright

@technovangelist

4 ай бұрын

I don't think I understand. You want to create your own model? I hope that’s not what you mean, because that takes a long time, and costs millions. Perhaps you can clarify.

Huzzah!

+1 ollama+autogen use case example, like quering a db

yo mom im on tv!! Literally zent on the discord to ask for gpu usage uh

wow its hard to fill 10 minutes, is it?

@technovangelist

2 ай бұрын

Why do you say that. I always target a topic and never think of the time until done. Some have been 3 minutes and some 21

Wish i wasn't on vacation and had access to a real computer....

"people don't even know what it means and they want it..." hahaha🤣

Plus, we'll have nuclear fusion and Ollama will be coming to Windows very soon! 😀

Phew, phew.. Phew

yup

The start made me lol 😂 please fix the Discord 😂

@technovangelist

4 ай бұрын

I think it has to grow a lot more before they add any other channels.

Huh? LiteLLM proxy has an Ollama provider. Just use that.

@technovangelist

4 ай бұрын

But this simplifies things. No need to use a second product that according to lots of discord posts, is full of issues. I haven’t tried it myself. And it removes that additional dependency

Autogen Studio in very much details please please!

@technovangelist

4 ай бұрын

Great. I’ll do that. Thanks

Are you telling me that 'When is there going to be native Windows support?' is not even in the top three!?

@technovangelist

4 ай бұрын

there are a lot of questions like that for which there are easy answers... for windows its wsl2

@technovangelist

4 ай бұрын

and my list was based on absolutely no science

@juanjesusligero391

4 ай бұрын

@@technovangelist Well, wsl2 is fine, yeah, but I meant *native* Windows support ^_^

Finally I can ditch litellm in conda environment

@technovangelist

4 ай бұрын

I know. I didn’t see it coming but if you have to use the OpenAI api this is fantastic

Hey Matt, I've been following your channel and really appreciate your insightful scripting tutorials. 🙌 I'm trying already for a long time to build a LLaVA web agent for tasks like crawling and scraping, and I think Ollama's recent OpenAI API compatibility and LLaVA 1.6 update could be a game-changer for this. I found this video on integrating Ollama in Python script extremely helpful, and I'm wondering if you could guide me on integrating Ollama for a similar use case. Here's a link to a video I believe is similar to what I'm trying to achieve: kzread.info/dash/bejne/e4yGzc-qdbytiqQ.htmlfeature=shared (This is for GPT4 Vision) Any advice or scripting tips you could share would be immensely valuable. Thanks in advance! ✊

@technovangelist

4 ай бұрын

That’s a neat idea. I used pupetteer when I built the training platform at Datadog (I started the training team there...and docs...and evangelism) to auto accept new accounts and generate passwords. But this is using it quite differently.

dude sipping his mug while... might changing somebody life into better...

Maybe my first viewing but I didn’t get the content of this video. Your like touching a lot of topics and not really explaining anything properly

@technovangelist

4 ай бұрын

It’s showing OpenAI api compatibility. How to use the actual thing is irrelevant to the video just that they work.

@technovangelist

4 ай бұрын

But I should make sure that’s more obvious thanks. I will fix it for next time.

@aksingh11

4 ай бұрын

i think i should say sorry because . i saw the video again and realized the video ...nay the channel isn't for noobs like me .. for others api feature announced can be appreciated but i dont the first principles so its no point in begin critical ...have a good day! @@technovangelist