Would be interested in follow up videos that explore interacting with data (eg via SQL) to obtain data relevant to questions asked by the user.

@MaQy

Ай бұрын

Semantic Kernel makes it easy to use RAGs with Azure Search and other DBs. You can also build chat completion tools that the model can choose to call. The example you are going to find everything is a tool that provides the weather.

Ideally I would want to have a local LLM with our training material/documentation, so the users can access that

@HenningKilset76

Ай бұрын

You can have a remote LLM and use the rag pattern to make the LLM aware of relevant documents based on user queries.

@Azagatoth

Ай бұрын

​​@HenningKilset76 Remote LLM is not an option. We have customers that are completely off the grid due to security.

@HenningKilset76

Ай бұрын

Ok - get that. A combination of Semantic Kernel and Kernel Memory can connect to and utilize for instance a local Llama or Phi model as well.

@georgepagotelis

Ай бұрын

The answer you're looking for is Microsoft Copilot 365 (only seen demo online) and there's an copilot developer kit you can "buy" ie. another 100 subscription and do exactly that. Was thinking this while watching too btw.

@francoislepron2301

Ай бұрын

Is it possible with Mistral and train it with my data ?

Directly exposing AI features to my customers would only make sense if it generated higher revenues from existing clients or accelerated the sales cycle (allowing me to grow a bigger customer base), but integrating AI features into the tools I write for myself to generate synthetic data, categorize and analyze server traffic and cloud costs... Yes! That I will start doing immediately!

Interesting, but costly.

@LoganDunning

Ай бұрын

Agreed, calling Open AI Apis ditectly is HUGE cost savings compared to Azure Open AI.

@cheronecom

Ай бұрын

I should note, ollama doesn't require a GPU, but you will want to use a small model if you use CPU - like tinyllama or phi3:mini

@geraldmaale

Ай бұрын

@@LoganDunning Check again, both are the same. (input = $0.005/1k tokens, output = $0.15/1k tokens)

@rogeriobarretto

Ай бұрын

It all depends on the underlying AI Service you want to use, SK works as an abstraction regardless of the service you are using. So if you decide to use it for Local Models you have virtually has no cost.

@HenningKilset76

Ай бұрын

@@LoganDunning It's the same price. GPT-4o 1.5 cents / 1000 input tokens and 3 cents / 1000 output tokens.

The most fascinating feature of the semantic kernel for me was the kernel plugins that allow you to give the AI access to local function to reach the goal you specify in chat. You decorate your plugin functions with a description in plain text and the semantic kernel uses it to orchestrate function calls. This way you can access local data that is highly dynamic (local time, user's todo list, whatever) but it will also call functions to create content or control the app. That really blew my mind.

It would be better if you told about running local LLM with C#

@nickchapsas

Ай бұрын

It’s coming

@KibbleWhite

Ай бұрын

@@nickchapsas Yes please buddy, loving this insight so far! 💖

@irql2

Ай бұрын

Its very easy, just get LM Studio. It exposes an API endpoint you can call in C#. This is what Nick will probably show in the next video because its the easiest to do by far imo.

@alfany6252

Ай бұрын

@@irql2 I want it to be only in my app. I dont want to run other program, run server...

@maskettaman1488

Ай бұрын

It wouldn't be "better", running locally is a totally different scenario for a totally different use case.

I was expecting to see Nick discovering that "Nick: " is not in the loop, but he Ctrl+C'ed early every time.

Nice. I've got some dotnet code that's interacting with OpenAI via their REST API - but I might convert that over to Semantic Kernel after seeing this.

@danclarkeuk

Ай бұрын

Well that was easy! I've just converted it over and it took about 5 minutes and just worked! Thanks for the video! 😊🙏

@vincenthamelin6115

Ай бұрын

@@danclarkeuk Although your post is relatively new, have you seen a cost difference? I've read that using Azure OpenAI Service is MORE expensive than using OpenAI APIs directly. What are your thoughts?

@danclarkeuk

Ай бұрын

@@vincenthamelin6115 Ah sorry - I probably wasn't clear about what I'd changed. I haven't switched to the Azure OpenAI service. I just changed my implementation from talking to OpenAI manually via their REST API, to instead using the Semantic Kernel nuget package.

Would be cool to see a demo on feeding it some SQL table creation statements or some c# objects (perhaps your MovieDB example) and have it generate the code for you. Then ask it to do thinks with the code like "Add the necessary error handling".

Thank you , waiting for other AI videos

didn’t know about that lib. Thank you

Also an example of embedding documents with SK into a local model would be very interesting.

Very nice!

@Nick can we use same code when targeting OpenAI instead of AzureOpenAI ? if not, what would be the diffrence ?

Thank you for the demo. Maybe you could use another LLM instead of OpenAI stuff.

hi, Nick, what is the ide you used in this video? tkx

Thanks for the video. It would be interesting to see how to use RAG as well

Is there any reason why should we use OpenAI thru Azure ? The Open AI api is also directly available. Can someone with some experience with both pls help me compare these two approaches: Pro/Con /Pricing

@DaminGamerMC

Ай бұрын

I have some experience with Azure Open AI. By itself it is also very very cheap. However, as soon as you want to add you own data (like for making a support service for a company) it gets very expensive because you need AI search. A big issue is that you need to be allowed to use the service by Microsoft meaning you might have to wait between a week and a month to really be able to use it.

@MetehanG

Ай бұрын

For my scenario, my company already has access to Azure and we have some free monthly Azure credits. These credits also available on Azure Open AI so it was basically free for us. Why would we pay for OpenAI API while we have that? Other than that, there is practically no difference as far as I know.

@ThaGuus

Ай бұрын

Pro of Azure OpenAI: Runs on Azure so more "secure", Already known SLA, VNET support & private endpoints, if you want to "train it" then the files are on Azure which is a pre. Con of Azure OpenAI: You need to have a business Azure Account with domain and request access, Streaming doesn't work fully like it will stream in parts but not letter for letter like the normal OpenAI API, It's slower

@PticostaricaGS

Ай бұрын

Azure OpenAI is designed having data security in mind. And has easy integration with other Azure AI services. So, for companies internal tools, Azure OpenAI is more suitable.

@nick066hu

Ай бұрын

Thank you to all who replied and also those who are going to. For a proof of concept evaluation I might try the direct OpenAI api first. (I am not very experienced in programming Azure, learning two things at a time might be unnecessarily steep, although the free monthly credits are also tempting) My end goal is to program a technical support service assistant, with knowledge from uploaded manuals and other documents. First in a chat-window text based, in some later phase in voice mode. @DaminGamerMC What do you mean by 'allowed ....and wait a week/month' ? Am I right with assuming, that in this scenario I pay for the uploaded (they call it 'embedded', if I got it right) documents, - I need these the AI assistant find the replies in -, only once, per tokens in the document, and for all subsequent (customer) queries I pay only for the tokens actually in the query and the replies?

Would have been cool it is was with Llama but I dont see any good way with Dotnet?

I also posted a tweet about the behavior of await foreach while building an API to do some AI work. Did you try using await Task.Yield() instead of the Delay(100)? Something fishy is happening over there

How would I use this to stream the response to a frontend application and get the same sort of feel where the chat bot is writing in the response like in the CLI?

@ferd1775

Ай бұрын

Yield return

@ferd1775

Ай бұрын

You could also use llama3 locally and ask it 😉

@neociber24

Ай бұрын

AsyncIterator?

@ferd1775

Ай бұрын

@@neociber24 oh yea. Forgot all about that, I have no use for it generally. Way better solution

What's a benefit to use chatgpt through MS Azure instead of using chatgpt's API?

could you make a video that shows the “chat with your files” feature?

WoW FTW!

That's great. But how much does it cost?

Why not use existing ai agent frameworks here? I think the most critical is how do we get grounding of ai. For example I want it to be able to run my functions. So that it will actually integrate

Does this work with Ollama, or is it just Azure?

@Luke-me9qe

Ай бұрын

Yes. You would need another connector. And it is available

We live in a wild and crazy time.

I've been playing around with Azure Open AI but was just using the Azure.AI.OpenAI packages. Didn't know about this Symantic kernel thing.

This make me notice I have never streamed content to a client side app in C#.

we need big tutorial about blazor and what your opinion in it ?

Cool video as always! But now we want a stream from World of Warcraft 😂

@nickchapsas

Ай бұрын

This has happened already when Dragonflight launched 😂

@JustArion

Ай бұрын

@@nickchapsas Nick the Gamer. Would love to see more of those

WoW streams wen ser?

How to add your domain logic/knowledge?

@nickchapsas

Ай бұрын

I’ll show that in a future video but it gets pricy

@irql2

Ай бұрын

@@nickchapsas Will be interesting to see what you mean by pricy. The technology to do it is openly available.

Recommend Autogen which is also available for .NET

Maybe I have this VERY wrong, but what has this to do with Semantic Kernel? I thought SK was about providing automation to an AI, so that it could invoke functionality within your app. example, my app is a ride booking service, so I provide a "request ride" "feature" and now the general AI knows about that specific "ability" and how to invoke it within my app during a general chat style session?

@etiennebaudoux135

Ай бұрын

You can do exactly this with Semantic Kernel, but Nick only scratches the surface and likely focused on the basic ChatGPT conversation because that's probably what people are the most familiar with when it comes to AI nowadays.

@irql2

Ай бұрын

Honestly all of that can be done without the Semantic kernel. You can define functions in your app and tell the LLM they're available to call. They call it "Function calling" because they're really good at creatively naming stuff lol.

Where's the kill switch 😂

I worked with SK like 4 months ago. I couldnt use because of so many bugs. Maybe better now but I have already gone to python.

Oh I wish you would have said for AI generated title at the start "this title is terrible" but still use it on this video as a joke

We have a cloud platform , clients all scream we want AI. But none can tell me exactly what they want AI to do. Got. POC working with aws bedrock, but still, can’t think of a single use case for it. Our clients want to say they have something AI, seems to pretty much be it.

It's great. Sugestion made a Maui, Onxx, PHI-3 small or Gemini nano , multiple OS's example.

Someone has been playing Cataclysm Classic recently and misses Wrath haven’t they…

@nickchapsas

Ай бұрын

🥲

Not that open implies free... But I find the trend of hosting AI models and charging fees for consumption problematic. Most of the examples I've seen really couldn't be incorporated into real world apps without losing money. I hate that Microsoft kind of pushes people to cloud vendor lockin. I see azure specific stuff and automatically rule it out as a solution

@mekowgli

Ай бұрын

They require serious compute, so it's only fair to charge money for it. Although Azure pricing is, as usual, pretty wild. But the good thing is that, from what I understand, the API schemas are pretty much shared between different providers (including local ones) so you can use the same libraries to connect to different services.

No

i dont wanna and you cant make me.

Lol you cant just easily create this resource on Azure. You have to create a form to request access. And you need company details, you cant just do this as a normal person with personal email. !!!!!!!!!!!!!