NEW AI Framework - Steerable Chatbots with Semantic Router

Ғылым және технология

Semantic Router is a superfast decision layer for your LLMs and agents that integrates with LangChain, improves RAG, and supports OpenAI and Cohere.
⭐ Repo: www.github.com/aurelio-labs/s...
Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions - routing our requests using semantic meaning. This approach unlocks incredibly fast agentic decision-making, the ability to use literally millions of tools, and provides much greater steerability and AI safety using semantics.
📌 Code:
Intro: github.com/aurelio-labs/seman...
LangChain Agents: github.com/aurelio-labs/seman...
🔥 Semantic Router Course:
www.aurelio.ai/course/semanti...
👋🏼 AI Dev:
aurelio.ai
👾 Discord:
/ discord
Twitter: / jamescalam
LinkedIn: / jamescalam
00:00 New Python Library for Better AI
01:57 Using Semantic Router
02:26 Semantic Router in Python
03:08 Defining Guardrails and Routes for LLMs
04:34 Initializing a RouteLayer
07:39 Using the Router with LangChain Agents
11:47 What else can Semantic Router do
12:40 Final Notes on the Library

Пікірлер: 158

  • @broomva
    @broomva5 ай бұрын

    This is great for creating 'fuzzy' like if/else statements, where the statement refers to a cloud of options, really cool. It's like filtering based on the embedding space

  • @itsjustmeemman
    @itsjustmeemman5 ай бұрын

    I have a production LLM endpoint and I realized that 70% of the time it takes to process everything happens on the different guard rails, intent identification and other classifiers in the sequence for a simple RAG 🥲. Thank you for this video I'll implement this and hopefully give some feedback or share what I've learned.

  • @plashless3406
    @plashless34065 ай бұрын

    This is really interesting. Will definitely try it out. Great job james and team.

  • @Darthus
    @Darthus5 ай бұрын

    This may have been within your suggestions, but I can also see this for routing queries to different RAG models, for example, if someone is asking about rules of a game, versus asking to play the game, you could use separate models that are more specifically tuned to information retrieval vs creativity.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    should be doable with current version of library, putting together some examples over coming weeks and will include something like this, thanks!

  • @kavian4249

    @kavian4249

    9 күн бұрын

    @@jamesbriggs Any updates for this?

  • @michaeldimattia9015
    @michaeldimattia90155 ай бұрын

    Awesome work, and great contribution! I have a clunky solution using langchain LLMRouter (which gets me there, but as you pointed out, slow, and a bit awkward). I look forward to implementing this (I can already tell it will be a much cleaner solution than the convoluted logic I've been fumbling with)! Very cool project 🙂

  • @tajwarakmal
    @tajwarakmal5 ай бұрын

    This is fantastic! already have a few use cases in mind.

  • @danielvalentine132
    @danielvalentine1325 ай бұрын

    Brilliant. Simple and elegant solution.

  • @scharlesworth93
    @scharlesworth935 ай бұрын

    Cool I gotta wait two days to watch

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    I need some time to get everything ready 😅 will not make you wait for following videos :)

  • @alivecoding4995
    @alivecoding49952 ай бұрын

    I love your work and content! Thanks so much, James. :)

  • @RichardGetzPhotography
    @RichardGetzPhotography5 ай бұрын

    Building colabs to make James laugh during filming..... priceless!!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yes need more of these haha

  • @avg_ape
    @avg_ape3 ай бұрын

    Fantastic contribution. Thank you.

  • @avatarelemental
    @avatarelemental5 ай бұрын

    This sounds great ! I will give it a try

  • @truehighs7845
    @truehighs78455 ай бұрын

    This is pretty much exactly what Langchain was missing, especially combined with actions and API calls! Well done!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    glad you think so :)

  • @nicolaithomsen7005
    @nicolaithomsen70054 ай бұрын

    As always, mind-blowing. Great video, James. My favorite GenAI educator!

  • @jamesbriggs

    @jamesbriggs

    4 ай бұрын

    glad you enjoyed it!

  • @narutocole
    @narutocole5 ай бұрын

    Super excited to try this!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    hey Jordan! Let me know how it goes!

  • @concretec0w
    @concretec0w5 ай бұрын

    Sooooo soooo coool :) This is going straight into my voice assistant so that i don't need multiple keyboard shortcuts to handle different tasks :D

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    haha sounds awesome, hope it goes well :)

  • @mrchongnoi
    @mrchongnoi5 ай бұрын

    Thank you for the video. Very useful

  • @micbab-vg2mu
    @micbab-vg2mu5 ай бұрын

    Graet video - thank you. I will try it.

  • @jdray
    @jdray5 ай бұрын

    Watching this with interest. A few months ago I identified a hole in the general field of AI stacks that I conceptually filled with a tool called BRAD (which I've forgotten by now what it stood for). In essence the idea was the same as here, except with a tiny, fast LLM deciding on the route rather than routing logic like you describe here. So thank you for (sort of) implementing my vision. Now I can take this item off my list of things to work on. 😁

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    haha I'm glad we were able to accidentally help out with your vision of BRAD!

  • @mooksha
    @mooksha5 ай бұрын

    Very useful, looking forward to more videos on this! Will explore the repo as well. How much work has gone into optimising testing for large number of semantic routes? Is it still fast if there are 200 routes with 5 utterances in each? Also how does it fare if routes have many utterances, like 100 each?

  • @Sarah_ai_student
    @Sarah_ai_student11 күн бұрын

    Your videos are fantastic, offering excellent content. They are probably the best on KZread for beginners in AI and generative AI. I am a student and my school hasn't quite caught up to speed on this subject yet. Could you create a video on how to develop a full-stack chatbot application using Python with Django or another Python framework, incorporating a vector database like Milvus/other, a retrieval-augmented generation (RAG) approach, and a locally hosted language model such as Mistral/other ? A Q&A style format would be greatly appreciated.

  • @carterjames199
    @carterjames1995 ай бұрын

    I love how similar it is to guardrails the semantic search based on preset utterances was amazing when I first saw it in your video last year

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yeah I do love guardrails, it's a great library and ofcourse it inspired what we're building here

  • @storytimewithme2
    @storytimewithme23 ай бұрын

    Getting Thailand vibez off this guy

  • @KarlJuhl
    @KarlJuhl5 ай бұрын

    Great development, thanks for sharing. Personally I have been using LLMs for an intent detection step, to route a user query to a given prompt that has the relevant context to answer a user question. I'm interested to see the latency of this in the same setup. Makes total sense to use vector space to cluster a user query and route to the most similar intent in the space.

  • @ahmadzaimhilmi
    @ahmadzaimhilmi5 ай бұрын

    This has a good use case in research papers if it can be restructured to categorize sentences into type such as result, challenges, methods etc.

  • @user-fs5lb3ce3b
    @user-fs5lb3ce3b5 ай бұрын

    oh, I've been dreaming about this possibility; a kind of dynamic workflow engine that allows the LLM to choose amongst plausible decision paths... will watch

  • @benjaminrigby877
    @benjaminrigby8775 ай бұрын

    this is, of course, intent detection under a different name. the complexity it brings is that now you've got to monitor and update two different systems - so somebody starts asking for a "route" in a slightly different way - or with a different accent that gets transcribed in a different way, and you've got to be able to catch it and fix it across the generative and intent systems.

  • @thebozbloxbla2020

    @thebozbloxbla2020

    5 ай бұрын

    i guess one way to solve that would be to to have all routes sort of defined in a tuple dict. and then have another LLM check to see if the correct semantic route was chosen in async fashion. after that if no useful results were from, say a vector search in a specific category which was defined as a route and chosen by the route selector, then we can fall upon choosing the route suggested by LLM (which could be done in async) in the end we can compare results and send which ever is better as system response. obviously haven't looked too deep into time optimisation. but i have a feeling it could still be faster

  • @larsbell1569

    @larsbell1569

    3 ай бұрын

    Ya but won’t this be an order of magnitude cheaper and faster than an intent system?

  • @mustafadut8430
    @mustafadut84305 ай бұрын

    Man in nice shirt. I can't wait to see an example of it collaborating with a Langchain chatbot.

  • @BradleyKieser
    @BradleyKieser5 ай бұрын

    Very interesting idea, I have been using a similar idea but with a light weight fast locally hosted LLM for the routing decision (your "RAG" technology adds a very good layer of speed and precise control). Weird hearing "route" ("root") pronounced as "rowt". Very Game of Thrones. Guess it's for American viewers. Excellent presentation, clear explanation and very well thought out overview.

  • @nathancanbereached
    @nathancanbereachedАй бұрын

    Have you tested precision in situations where there are multiple routing options that have multiple criteria and are mostly similiar? I wonder if there would be benefit to layering 2 or more routing layers in a branching decision tree format. Would need to test- but theoretically you could have the quality of tree of thought decisioning but at lower cost, higher speed, and each decision point could be human-reviewed before pushing to production. I'm glad I found your channel again- definitely worth looking into.

  • 5 ай бұрын

    Thanks James, this is a very good solution for us moving forwards to using agents instead of chains. How do you see this fits with more advanced rags, that have Query preprocessing, reranking etc.

  • @tonyrungeetech
    @tonyrungeetech5 ай бұрын

    Really looking forward to seeing it applied to RAG! I've been thinking about some sort of process where we search the top K results, then doing a simple semantic evaluation of 'yes I found it' or 'not enough info' to trigger a more in depth search - is it something along those lines?

  • @megamehdi89

    @megamehdi89

    5 ай бұрын

    Great idea

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yeah it feels like it would be a similar process when using the default RouteLayer + static Routes

  • @SimonMariusGalyan
    @SimonMariusGalyan4 ай бұрын

    Great work which can be integrated into apps speeding up data processing and reducing hallucinations… 🎉

  • @caiyu538
    @caiyu5385 ай бұрын

    Great great. Great

  • @dusanbosnjakovic6588
    @dusanbosnjakovic65885 ай бұрын

    I love your work and this video. However I have implemented something like this in heavy production and faced a lot of issues. Chat history, complex multi intent queries and just overall accuracy prevented us from moving forward. Surprised you think this is such a slam dunk.

  • @awakenwithoutcoffee
    @awakenwithoutcoffee7 күн бұрын

    this is incredible James. I have implemented a similar (but less powerful) system inside Botpressand was looking for Langchain intent classification/routes which this seems to cover very well. Are you still using this in production or has Langchain come out with an alternative that you prefer ? keep going my man.

  • @dgroechel
    @dgroechel5 ай бұрын

    Great video. With function calling, how do you generate the arguments and keep the speed?

  • @naromsky
    @naromsky5 ай бұрын

    The name is a banger.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    I wish I could take credit, but it's all the aurelioAI team, my ideas were terrible and fortunately not chosen 😅

  • @DemetrioFilocamo
    @DemetrioFilocamo5 ай бұрын

    Great project and thanks for open sourcing it! What’s the difference with Nemo Guardrails?

  • @dusanbosnjakovic6588
    @dusanbosnjakovic65885 ай бұрын

    Do you have any stats on accuracy differences between this and LLM? Especially in longer queries.

  • @johnny017
    @johnny0175 ай бұрын

    Very cool! I built something similar for a project, but much simpler. I also agree that, for now, it doesn't make much sense to use agent reasoning in production. It takes too much time to output, and it consumes too many tokens. I will keep an eye on the repo and try to contribute!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    I think there are times for agent reasoning, but it is certainly overused - and I believe we will improve semantic-router to replace more of those more expensive+slow reasoning steps with more efficient methods We'd love to have you contribute!

  • @wolpumba4099
    @wolpumba40995 ай бұрын

    *Summary* *Introduction to Semantic Router* - 0:00 - Introduction to the concept of a semantic router as a key component in building AI assistants. - 0:24 - Definition and purpose of a semantic router in AI dialogue systems. *Working Mechanism of Semantic Router* - 0:42 - Semantic router acts as a fast decision-making layer for language models. - 1:00 - The deterministic setup of the semantic router through query-response mapping. - 1:33 - Personal experience of using semantic routers in chatbots and agents. *Setting Up the Semantic Router* - 2:04 - Guide to accessing and installing the semantic router library. - 2:32 - Explanation of the library installation process and version details. - 3:00 - Steps to restart session post-installation in Google Colab. - 3:11 - Creating and testing sample routes for the semantic router. *Practical Examples and Usage* - 4:07 - Demonstration of initializing embedding models for the router. - 4:52 - Introduction to different types of route layers in the library. - 5:46 - Testing and interpreting the output of the semantic router with various queries. - 7:00 - Example of using the semantic router to control dialogue topics (e.g., politics). *Integration with AI Agents* - 7:42 - Demonstrating the integration of the semantic router with an AI agent. - 8:03 - Enhancing agent responses using semantic router augmented queries. - 10:07 - Various applications of the semantic router in customizing agent interactions. *Conclusion and Future Developments* - 12:43 - Reflections on the implementation and effectiveness of the semantic router in projects. - 13:01 - Acknowledging the early stage of the semantic router but emphasizing its effectiveness. - 13:30 - Invitation for community involvement and future instructional content on advanced features. *Closing Remarks* - 14:12 - Concluding thoughts and anticipation for future developments and community engagement.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    thanks :)

  • @socalledtwin
    @socalledtwin5 ай бұрын

    For now, it could be useful for selecting the best RAG pipeline or collection to search through, though it could be the case that future models like GPT-5 are so good at determining the correct function to call or collection to search, it won't be needed. Either way, nice work.

  • @thebozbloxbla2020
    @thebozbloxbla20205 ай бұрын

    hey man, really weird question but i hope you respond. i was wondering if the routes we created got saved in some sort of database, aka all the utterances and what not. or are you just vectorising the utterances each time you open a new instance? can i define a route for the long term?

  • @luisliz
    @luisliz5 ай бұрын

    this is awesome is this similar to guidance-ai? sorry if im confusing concepts

  • @yoshkebab
    @yoshkebab5 ай бұрын

    Very interesting. How do you handle history and context? A lot of times a single prompt can't be categorizes on it's own, and routing will change according to context. I'm curious about your approach to this.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    for now when using the suggestive method I demo in the video the LLM makes the final decision on what to do, so we get around the issue of implementing support for chat history and just use a single query In any case, I 100% agree and adding support for chat history is our next priority, we have already built the methodology for how we will handle it, which you can see here github.com/aurelio-labs/cookbook/blob/main/semantic-analysis/semantic-topic-change.ipynb

  • @nikosterizakis
    @nikosterizakis3 ай бұрын

    That is a great piece of development James. Certainly a very useful 'brick in the wall' of the LLM ecosystem and something we will be using in future projects. Out of interest, are you guys going to branch out to the other two gen AI areas, namely video and audio?

  • @jamesbriggs

    @jamesbriggs

    3 ай бұрын

    For sure, we're already taking some steps into multi-modal, for example using semantic router we're already doing: - Image detection kzread.info/dash/bejne/d6V_zMOFotPLeZc.html - Video processing github.com/aurelio-labs/semantic-router/blob/main/docs/examples/video-splitter.ipynb

  • @nikosterizakis

    @nikosterizakis

    3 ай бұрын

    Great stuff, with permission I am going to test drive that python code!

  • @franciscocaruso4458
    @franciscocaruso44585 ай бұрын

    Hey great video, and very interesting tool!! This looks very simmilar to NeMo guardrails. What is the difference in this case?

  • @gfertt13
    @gfertt135 ай бұрын

    I have been using nemo guardrails for work, but exclusively for the "input rails", which seem to be very similar to this library, with the important difference being that it includes an LLM call to make the final decision of classifying into a route/"canonical form". I'm curious, have you run any tests against a benchmark similar to the way the nemo guardrails paper shows tests against NLU++ benchmark?

  • @gfertt13

    @gfertt13

    5 ай бұрын

    From reading through some of the code it seems like you handle the KNN search with plain numpy and linear search. Any reason for not using a package like FAISS here?

  • @gabrieleguo
    @gabrieleguo5 ай бұрын

    Very interesting framework. What if the utterances are in another language? Does it still work well? I guess it should be matching what the encoder supports, is it correct?

  • @carterjames199
    @carterjames1995 ай бұрын

    This is really cool, can the semantic router or do you have plans to allow the use of open source embedding models or maybe like local mini lm models?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yes you can already, we added it this weekend - see here github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb it works incredibly well, using mistral 7b we get better performance on the few tests I did than gpt-3.5

  • @thebozbloxbla2020
    @thebozbloxbla20205 ай бұрын

    was wondering if we could use the semantic search titles and use it for index searching vector databases to return similarity results more related to the topic...

  • @marcomorales9417
    @marcomorales94174 ай бұрын

    Hey! I've found using rules within the system initial message works great for filtering questions about unrelated topics to the domain the rag chtabot should answer, maybe the win with this approach is reducing cost and time? Also I've been playing around with having 1 chatbot which can have different phases within a conversation which are set by different system messages, for example stage 1 is to extract customer infromation fo obtain a lead and then help them with the question, this information can be outputed by the LLM through a JSON response. Quite interesting

  • @lorenzospataro26
    @lorenzospataro262 ай бұрын

    Have you thought about using Setfit as an alternative to pure semantic matching? If you have a small dataset to train the intent classifier, it would probably perform better for some use cases (at the cost of being a bit slower but still faster than LLM)

  • @GiovanneAfonso
    @GiovanneAfonso5 ай бұрын

    Incredible work! Thank you for sharing, I'm really excited right now, I'll give a try. Could you answer some questions? - Is it possible to have a 1000 different routes? (I'm just curious) - Is it possible to cache / reuse embeddings in multiple services? - Is it using an AI model under the hood for choosing the right router?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    Great to hear! - yes it could have 1000 different routes in theory, I have not tested yet - we don’t cache the embedding yet but we are adding it to the RouteLayer.to_file method - the default RouteLayer uses vector space and embeddings to create what is essentially an inherent classification model

  • @GiovanneAfonso

    @GiovanneAfonso

    5 ай бұрын

    @@jamesbriggs you guys are making the future more shiny. Thanks

  • @georgegowers4037
    @georgegowers40375 ай бұрын

    Is this the same as the planner in Semantic Kernel SDK?

  • @megamehdi89
    @megamehdi895 ай бұрын

    Key Insights by TubeOnAI: 1. Semantic Router Introduction: The Semantic Router is presented as a crucial layer for achieving control and determinacy in AI dialogue. It is described as a fast decision-making layer for natural language processing, enabling instant triggering of specific responses based on predefined queries. The speaker emphasizes its significance in refining the behavior of AI assistants and chatbots, stressing its necessity for deploying such systems. 2. Library Setup and Integration: The video provides a step-by-step guide on setting up and using the Semantic Router library. It introduces the installation process and demonstrates how to define routes and test their interaction. The integration with an AI assistant is showcased, illustrating how the Semantic Router augments user queries and influences the agent's responses based on predefined routes. 3. Enhancing AI Dialogue and Control: The speaker highlights the capabilities of the Semantic Router in influencing AI behavior, protecting against unwanted queries, and suggesting specific actions or information to the AI assistant. The framework is portrayed as a tool for not only steering dialogue but also enhancing the AI's decision-making process and overall functionality. 4. Future Developments and Open Source Collaboration: The video concludes with an outlook on future developments, expressing the speaker's excitement about the potential of the Semantic Router framework. The open-source nature of the project is emphasized, encouraging contributions and promising further insights into advanced features such as dynamic routing and the hybrid layer.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    thanks!

  • @parchamgupta8417
    @parchamgupta84175 ай бұрын

    Hi just had a thought in mind, dont you think dialogflow does this kind of task already although at a much basic level.

  • @TommyJefferson1801
    @TommyJefferson18015 ай бұрын

    So basically, this avoids additional time during inference if I'm not wrong. Also why not finetune LLM and use functional calling? I mean yes it can take some time but how well does this Approach compare to that in production scenarios? Do we have like benchmarks on this?

  • @plashless3406
    @plashless34065 ай бұрын

    I have one question though: in order to make the most out of a route, do we need to have many utterances to cover the whole use case of a route? I mean what if a usee query belngs to a route and it return no matching route?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    You need to cover the routes well, if I see a query miss a route, I usually just add it directly to the route and iteratively do that, adding queries that were missed

  • @plashless3406

    @plashless3406

    5 ай бұрын

    @@jamesbriggs this really is interesting and promising.

  • @dawid_dahl
    @dawid_dahl5 ай бұрын

    Just so I can understand... do you mean that just by 1) providing some example of those various sentences "isn't politics the best thing ever", etc, and 2) user's embedded query - it will give back a route, without ever going to an actual LLM to make the routing decision?

  • @scharlesworth93
    @scharlesworth935 ай бұрын

    This reminds me of that Nemo rails thing you were talking about a while back, is this the preferred tool?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    for me yes, we built this due to limitations we were seeing with nemo (although, nemo is still a great tool)

  • @berdeter
    @berdeter5 ай бұрын

    Very interesting. Few questions: how does it compare to Nemo Guardrails you've covered earlier? Have you considered the case where user's question would trigger several semantic routes? Let's say you make a bot for candidates going to a job search event in a big company. User says "I love AI. Can you propose me something?" Route 1 : go to the IT desk Route 2 : attend to the conference about digital transformation at 11am Route 3: check this job offer that is made fo you So the bot should be able to answer on the 3 suggestions.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    we want to support multiple routes, but it isn't implemented yet - there's nothing in the current methodology that would prevent this though

  • @berdeter

    @berdeter

    5 ай бұрын

    @@jamesbriggs I have implemented multiple semantic routes on a project. I couldn't find a good framework so it's all hand made. I work on a list of intentions such as I want to attend to a conference about digital transformation I perform a semantic search and I have 3 thresholds: Never more than 3 intentions selected Never select intention under a cosine similarity of 82% If I find an intention with cousine similarity above 91% I keep only that one and disregard the rest. Percentages come from experimentation. I think that would be a good starting point to implement something generic in your framework.

  • @JulianHarris
    @JulianHarris5 ай бұрын

    It’d be quite fun to integrate this with the ollama web ui project.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    do it! :)

  • @onufriienko
    @onufriienko5 ай бұрын

    Thanks for sharing James, Are there any examples with LlamaIndex? Thanks 😊

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    not yet, will be working on creating examples w/ different libs over the coming weeks

  • @Bubbalubagus
    @Bubbalubagus5 ай бұрын

    How does this differ from NLU inference architecture?

  • @cuburtrivera1167
    @cuburtrivera11675 ай бұрын

    i think this can be replicated by just using output parser, but this abstraction layer makes it easier

  • @MidtownAI-qi1yq
    @MidtownAI-qi1yq5 ай бұрын

    With react prompting, you can trigger a sequence of actions. How would semantic router handler the same processing ? It appears to me that everything needs to be programmed in and anticipated instead of expecting a "reasoning engine" which can build in theory any workflow based on the available tools. Just thinking about the limitations of this approach. Very creative though!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yes it would require a lot of logic build around it to support multiple steps, I think it would be doable, especially if you have differing sets of routes that could enter the "decision space" but it would be nontrivial with current lib - I think we could build some interesting tooling to attempt to solve for this use-case

  • @roberth8737
    @roberth87375 ай бұрын

    Passing in just the query could often lack context - requiring the usual "create a standalone question.. etc etc" so that statements are correctly interpreted. Although this could be done with a quick 3.5 query, could we somehow combine the past X queries semantically without resorting to LLMs to solve for those cases?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    We’re working on it, expecting to have a v1 feature for identifying most relevant messages required for a query by next week

  • @eyemazed
    @eyemazed5 ай бұрын

    interesting. how do you determine a threshold for whether a user query belongs to a certain route?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    A default value is set based on the encoder being used at the moment, I want to add route specific thresholds and auto optimization of those soon

  • @codingcrashcourses8533
    @codingcrashcourses85335 ай бұрын

    I don´t really like the LangChain integration, accessing attributes like this and overwriting prompts like this: agent.agent.llm_chain.prompt = new_prompt :(. I still hope for a build in LangChain functionality for something like this. Thanks for the demonstration. It is still a small and young project.

  • @areebtariq6755
    @areebtariq67552 ай бұрын

    If we are using embeddedings for this ? Where do the system store the embeddings ? Or it is as the run time that they generate those ?

  • @jamesbriggs

    @jamesbriggs

    2 ай бұрын

    Using the local index it will be rebuilt with each session - however we support Pinecone and qdrant index, which will maintain the embeddings, video on that here kzread.info/dash/bejne/o56G1K-xhJWWhJM.html

  • @crotonium
    @crotonium5 ай бұрын

    Isn't this just calculating the cosine similarity between the input query, and the mean of each route class to categorize which route it belongs to?

  • @robcz3926
    @robcz3926Ай бұрын

    man this is so much easier than langgraph routing, and it works with smaller models too

  • @jamesbriggs

    @jamesbriggs

    Ай бұрын

    yeah we use both in projects, but I'm planning to try building agents that rely wholly on semantic routes soon

  • @robcz3926

    @robcz3926

    Ай бұрын

    @@jamesbriggs looking forward to that mate🤘

  • @musifmuzammir354
    @musifmuzammir3545 ай бұрын

    Isn't this how RASA framework works?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    RASA does intent detection, and that is to some degree what we're doing here - to be honest I have not used RASA for years so I don't know where they are now, but it began to get quite dated - I unfortunately don't know enough to compare their recent versions to this, I will investigate though

  • @musifmuzammir354

    @musifmuzammir354

    5 ай бұрын

    @@jamesbriggs It still does the intent classification only. But this seems like a faster approach, finding the correct intent using embedding search.

  • @familyaccount-eb7cb

    @familyaccount-eb7cb

    2 ай бұрын

    Rasa started move away from intent based routing. Because it doesn’t handle follow up inputs well. (Input that needs previous messages context). I do wonder how semantic router handle this follow up inputs

  • @eyemazed
    @eyemazed5 ай бұрын

    i built a RAG for our inhouse project management system. it works fine when users write a prompt like "when was X topic opened and who started it" for example, because it performs vector search on "X" and then creates a context around it. however, it does not work for a prompt like "who's the newest registered user?" because that cannot be retrieved by vector search. it can however be retrieved via database. anyone knows how to solve this? given that LLM can in fact write DB queries... i'm thinking some sort of routing should be implemented for this as well

  • @towards_agi

    @towards_agi

    5 ай бұрын

    Built something similar but was using a formula 1 dataset. Used a graph db to do multi hop queries. E.g. I asked it when a particular driver won their first race. Based on the graph schema the LLM would generate a query that linked the driver and all their races. Then filters by races were the driver got 1st and returns them in ascending order. Let me know if you need assistance.

  • @sangyeonlee5417
    @sangyeonlee54175 ай бұрын

    Wow super excited, how about in case of using different language like CjK ?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    we support Cohere embedding models, and they do have multilingual support, so you would initialize our CohereEncoder using `CohereEncoder(name="embed-multilingual-v3.0")` and that comes with support for CJK languages as far as I know :)

  • @sangyeonlee5417

    @sangyeonlee5417

    5 ай бұрын

    Tanks alot i will try it. @@jamesbriggs

  • @andydataguy
    @andydataguy5 ай бұрын

    BigAI p100 whey protein 😂 love to see the sense of humor!

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    I do try

  • @zongguixie3407
    @zongguixie340717 күн бұрын

    Uvula oum😊

  • @marktucker8537
    @marktucker85373 ай бұрын

    How does Semantic Router work under the hood?

  • @Truzian
    @Truzian5 ай бұрын

    Would this every be supported in TS/JS?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    would love to build it, we do have some great TS/JS devs, so it's quite likely

  • @llaaoopp
    @llaaoopp5 ай бұрын

    How does this differ from NeMo Guardrails? I thought this pre-check of your query against a database of semantically embedded utterances is at the heart of how Guardrails functions.

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    Yes, I'm very familiar with the lib - it is what we used before semantic router and I and others from aurelioAI made a few contributions to make the lib easier to develop with and deploy in projects, but ultimately it was limiting + overly complex. To add things like dynamic routing, hybrid layers, etc (I will talk about these soon) was too difficult. We also have other upcoming features such as a topics-based conversation splitter that we believe will be key to getting the next level of performance from this type of approach. Again, implementing in guardrails would have been more complex than developing it independently, and possibly even out of scope for what they're building. Although I think guardrails is awesome, and it served as the starting point for what we're building here, I ultimately felt it better to move away from the library, that may change if nvidia decide to put more resources into it, but I haven't seen this happen yet.

  • @llaaoopp

    @llaaoopp

    5 ай бұрын

    @@jamesbriggs I completely agree, Guardrails felt a bit clunky if you wanted to actually code around it and this it honestly felt like it was not "built for the job" really, but more like a PoC. I love the direction that you guys are going with this and can't wait what other concepts you came up with around the concept of semantic routing! Cheers :)

  • @JanVansteenlandt
    @JanVansteenlandt2 ай бұрын

    What I'm wondering is how to best use this when dealing with chat history. For example if you ask a political question followed up by "please elaborate". The message itself does not mean anything, however taking the previous question into account does... is it as simple as just concatenating the last X user questions and using that as a basis for the routing input? Giving the routing input a limited "memory" but memory nonetheless.

  • @shaunpx1
    @shaunpx15 ай бұрын

    Can this work with triggering function calls when some response or simular response(utterance is detected)?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yeah 100%, I do exactly that here kzread.info/dash/bejne/gHt31qR_mt3UlMY.html

  • @Kalebryee
    @Kalebryee5 ай бұрын

    How is this different from guard-rail

  • @smoq20
    @smoq205 ай бұрын

    Is there a way to make it 100% local without relying on OpenAI or Cohere? ChromaDB?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    Fully local support in progress, doesn’t need chroma

  • @pascalshehata7648
    @pascalshehata76485 ай бұрын

    Hey, isn't it just another name for intents?

  • @avidlearner8117
    @avidlearner81175 ай бұрын

    Isn't that close to what Constitutional AI, by Anthropic, does? I can see your solution has layers of functionality that the other method doesn't have...

  • @Mr_Arun_Raj
    @Mr_Arun_Raj5 ай бұрын

    Can I use hugging face embeddings?

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yes they were added a couple days ago github.com/aurelio-labs/semantic-router/pull/90

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    can see an example notebook here github.com/aurelio-labs/semantic-router/blob/main/docs/encoders/huggingface.ipynb

  • @merefield2585
    @merefield25854 ай бұрын

    OK, but isn't this theoretically equivalent to feeding function prompts to catch certain user behaviour? I've already written an escalation function on my chatbot that responds to anger or frustration and is written as a "pure" function prompt. The advantage of that approach is I don't have to maintain another layer of logic. Presumably Open AI is using semantic similarity thresholds to trigger functions, so the approach here is not a lot different? Could you elaborate on the advantages and any disadvantages of doing this locally? On the politics issue, why not just add to the system prompt something like "you must NEVER discuss politics and decline to do so politely. "?

  • @varunmehra5

    @varunmehra5

    12 күн бұрын

    Do you mind sharing it?

  • @landerstandaert6649
    @landerstandaert66495 ай бұрын

    Similar Microsoft copilot studio

  • @seel1823
    @seel18235 ай бұрын

    Does it work with GPT4? - Ik im asking before it even starts lol

  • @jamesbriggs

    @jamesbriggs

    5 ай бұрын

    yes it can haha, it's a separate layer, so it works with anything you want it to work with :)

  • @seel1823

    @seel1823

    5 ай бұрын

    One last question @@jamesbriggs can you use it with RAG?

  • @larsbell1569
    @larsbell15695 ай бұрын

    what if it happens to contain both? dl("The rainy weather we are having today reminds me of the Prime Minister who is a damp dull man.")

  • @altered.thought
    @altered.thought4 ай бұрын

    Are there plans to support JavaScript in the future? 🙃

  • @jamesbriggs

    @jamesbriggs

    4 ай бұрын

    not planned, but would love to

  • @whiskeycalculus
    @whiskeycalculus5 ай бұрын

    Accelerate

  • @chillydoog
    @chillydoog5 ай бұрын

    No offense, but I don't really see the point of this semantic routing technique? I also don't think I fully understand what it is. why wouldn't you just use a chat GPT assistant? It seems like it's doing the same thing, but probably better. IDK feel free to correct me or help me understand with the value and this is.

  • @brytonkalyi277
    @brytonkalyi2775 ай бұрын

    ✓ I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it knows only things of the flesh such as our fleshly desires and cannot comprehend things of the spirit such as true love and eternal joy that comes from obeying God's Word. Man is a spirit and has a soul but lives in a body which is flesh. When you go to bed it is the flesh that sleeps, but your spirit never sleeps and that is why you have dreams, unless you have died in peace physically. More so, true love that endures and last is a thing of the heart. When I say 'heart', I mean 'spirit'. But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons those are things of the flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. However, true love works in conjunction with other spiritual forces such as patience and faith - in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever. To avoid sin and error which leads to the death of your body and your spirit-soul in hell fire (second death), you must make God's Word the standard for your life, not AI. If not, God will let you face AI on your own (with your own strength) and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. The prove texts can be found in the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 2, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. *HOW TO MAKE GOD'S WORD THE STANDARD FOR YOUR LIFE?* You must read your Bible slowly, attentively and repeatedly, having this in mind that Christianity is not a religion but a Love relationship. It is measured by the love you have for God and the love you have for your neighbor. Matthew 5:13 says, "You are the salt of the earth; but if the salt loses its flavor, how shall it be seasoned? It is then good for nothing but to be thrown out and trampled underfoot by men." Our spirits can only be purified while in the body (while on earth) but after death anything unpurified (unclean) cannot enter Heaven Gates. Blessed are the pure in heart, for they shall see God (Matthew 5:8). No one in his right mind can risk or even bare to put anything rotten into his body nor put the rotten thing closer to the those which are not rotten. Sin makes the heart unclean but you can ask God to forgive you, to save your soul, to cleanse you of your sin, to purify your heart by the blood of His Son, our Lord and Savior, Jesus Christ which He shed here on earth because Isaiah 53:5 says, "But He was wounded for our transgressions, He was bruised for our iniquities; the chastisement for our peace was upon Him, and by His stripes we are healed". Meditation in the Word of God is a visit to God because God is in His Word. We know God through His Word because the Word He speaks represent His heart's desires. Meditation is a thing of the heart, not a thing of the mind. Thinking is lower level while meditation is upper level. You think of your problems, your troubles but inorder to meditate, you must let go of your own will, your own desires, your own ways and let the Word you read prevail over thinking process by thinking of it more and more, until the Word gets into your blood and gains supremacy over you. That is when meditation comes - naturally without forcing yourself, turning the Word over and over in your heart. You can be having a conversation with someone while meditating in your heart - saying 'Thank you, Jesus...' over and over in your heart. But it is hard to meditate when you haven't let go of offence and past hurts. Your pain of the past, leave it for God, don't worry yourself, Jesus is alive, you can face tomorrow, He understands what you are passing through today. Begin to meditate on this prayer day and night (in all that you do), "Lord take more of me and give me more of you. Give me more of your holiness, faithfulness, obedience, self-control, purity, humility, love, goodness, kindness, joy, patience, forgiveness, wisdom, understanding, calmness, perseverance... Make me a channel of shinning light where there is darkness, a channel of pardon where there is injury, a channel of love where there is hatred, a channel of humility where there is pride..." The Word of God becomes a part of us by meditation, not by saying words but spirit prayer (prayer from the heart). When the Word becomes a part of you, it will by its very nature influence your conduct and behavior. Your bad habits, you will no longer have the urge to do them. You will think differently, dream differently, act differently and talk differently - if something does not qualify for meditation, it does not qualify for conversation. *THE BATTLE BETWEEN LIGHT AND DARKNESS (GOOD AND EVIL)* Heaven is God's throne and the dwelling place for God's angels and the saints. Hell was meant for the devil (satan) and the fallen angels. Those who torture the souls in hell are demons (unclean spirits). Man's spirit is a free moral agent. You can either yield yourself to God or to the devil because God has given us discretion. If one thinks he possesses only his own spirit, he is lying to himself and he is already in the dark. God is light while the devil is darkness. Light (Holy Spirit) and darkness (evil spirit) cannot stay together in a man's body. God is Love (Love is light) and where there is no love is hell, just as where there is no light is darkness. The one you yield yourself to, you will get his reward. The reward of righteousness to man's spirit is life (abundant life) and the reward of sin to man's spirit is death. Sin and satan are one and the same. Whatever sin can cause, satan also can cause. Sin is what gives the devil dominion or power over man's spirit. When God's Word becomes a part of you, sin power over you is broken, you become the righteousness of God through Christ Jesus. Where Jesus is, you are and when He went (to the Father), you went. In the book of John 8:42-47, Jesus said to them, “If God were your Father, you would love Me, for I proceeded forth and came from God; nor have I come of Myself, but He sent Me. Why do you not understand My speech? Because you are not able to listen to My word. You are of your father the devil, and the desires of your father you want to do. He was a murderer from the beginning, and does not stand in the truth, because there is no truth in him. When he speaks a lie, he speaks from his own resources, for he is a liar and the father of it. Which of you convicts Me of sin? And if I tell the truth, why do you not believe Me? He who is of God hears God’s words; therefore you do not hear, because you are not of God.” My prayer is, "May God bless His Word in the midst of your heart." Glory and honour be to God our Father, our Lord and Savior Jesus Christ and our Helper the Holy Spirit. Watch and pray!... Thank you for your time and may God bless you as you share this message with others.

  • @thannon72
    @thannon725 ай бұрын

    Can we please divorce ourselves from Langchain.

Келесі