Omg, didn't know about that. Definitely going to the pipeline. Thanks for sharing! About the newsletters, there was theneurondaily, rundown ai, and what was the third one please? Something Valley when you all nodded so it must be an important one 😄 BTW, The Batch from Andrew NG is also a great one.
@andresrubio2015Күн бұрын
Always inspiration
@lucindalinde4198Күн бұрын
Excellent session! Your explanations are so helpful and very much appreciated
@mohammedshuaibiqbal5469Күн бұрын
How can we track testing and benchmarking agents using AgentOps
Greg I 'd like to talk about llama3 great lectuce. 😊
@parinaztabari23322 күн бұрын
How can I access this Colab (llama_index_tracing tutorial.ipynb)? Can you share the link?
@AI-Makerspace2 күн бұрын
We've shared your comment with the presenter! We expect he'll get back with you shortly!
@leonardjin9103 күн бұрын
On a scale from 1 to John Wick how much do you love your dog? 😆🤣🤣😆
@AI-Makerspace3 күн бұрын
Watch the full video! kzread.info/dash/bejne/aqiHwZptXZiundI.html
@enceladus964 күн бұрын
Thank you thank you thank you thank you thank you
@csmac3144a5 күн бұрын
KZread quality still 720p. Really hard on the brain to follow the code examples.
@AI-Makerspace3 күн бұрын
Stay tuned! Thanks for the ping and the reminder!
@lucindalinde41987 күн бұрын
Thank you thank you thank you for this fantastic explanation of DSPy. I didn't understand it before this great webinar.❤
@AI-Makerspace7 күн бұрын
You're welcome! 🫶
@givanildogramacho4417 күн бұрын
Great point about agents, but I have a question. Whats != Agentic between other ?
@AI-Makerspace6 күн бұрын
Thanks for your question Givanildo! We do need to clarify this idea that we're playing with on distinguishing between "multi-agent" and "multiple agent," even internally. There is, admittedly, ambiguity here, and we will seek to clarify it more fully in our next multi-agent event. But we created it because we have found it to be a useful mental model. The point of making this distinction was to recognize that the complexity and dynamism of multi-agent systems varies, sometimes quite widely. For instance, multi-agent systems presented today often feel constrained, linear, and prescriptive. It many cases multi-agent systems could just as well be single agent systems created with one prompt (albeit a quite long one). Conversely, a truly complex "multi-agent" system would, in our estimation, *require* the use of multiple agents, even to the point that each LLM was fine-tuned to a specific task. Something like that - what do you think!?
@janroggisch23108 күн бұрын
Thank you very much for providing historical context, to me this is one of the most important aspects of learning about new tech, and understanding where, how, and why it fits in.
Good video but one question: Why did you choose to create the testset step-by-step yourself and not use the provided TestSetGenerator from Ragas? Was is not available back then?
@AI-Makerspace8 күн бұрын
That's right! They had just rolled it out it when we had them on for this more recent event: kzread.infoAnr1br0lLz8?si=_wIYqsL4vcVM5QDq
@mattfarmerai9 күн бұрын
Wow great video. I just found your channel, already a fan. 🦾🤖
@AI-Makerspace9 күн бұрын
Let's goooooo!
@smklearn-hy9me9 күн бұрын
Does ragas work only on openai model, which model i can use for testset genrator as critic and generative model please help me out
@AI-Makerspace9 күн бұрын
You can use any LLM that has OpenAI API compatibility. This means most closed source models, as well as open source options through certain hosting strategies (NIMs, vLLM, etc)
@smklearn-hy9me9 күн бұрын
@@AI-Makerspace i dont have openai api key Can i use models from hugging face
@smklearn-hy9me9 күн бұрын
Please help me out I want to genarate the test set data using models other than hugging face Like generate model and critic model
@sillystuff624712 күн бұрын
best video i found on learning word2vec concepts by building up from simple examples. very helpful for people who learn by doing.
@AI-Makerspace12 күн бұрын
You're speaking our language @sillystuff - we're all about learning, building, shipping, and sharing! Thanks for the comment and we're glad you found it useful!
@CoCoandMay13 күн бұрын
amazing talk~~
@andres.yodars14 күн бұрын
top
@MarinaQian14 күн бұрын
Has there been any update yet?
@micbab-vg2mu15 күн бұрын
great talk - thank you:)
@truliapro711215 күн бұрын
Very useful 👌
@therobotocracy15 күн бұрын
More more more DSPy please!
@user-zh6zn4hk1k15 күн бұрын
absolutely
@AI-Makerspace14 күн бұрын
Loud and clear guys! We'll see what we can do 🤓
@freakstiches15 күн бұрын
Nice intro dudes lol music hooked me
@rakeshkumarrout262915 күн бұрын
This is the most awaited and most useful and trendy thing in the market.
Stoked that you are diving into DSPy! Would be cool if you could do a RAG demo and show how to generate an evaluation metric from free form text responses.
@AI-Makerspace14 күн бұрын
@@seanbergman8927 we think RAG is definitely the next step here for sure! Stay tuned!
@MrTulufan19 күн бұрын
RAGAS starts at time 20:15, before which is just an overview of langchain and the RAG QA pipeline
@AI-Makerspace16 күн бұрын
Thanks for the timestamp here MrTulufan!
@rakeshkumarrout262922 күн бұрын
this is quite useful,its really commendable how you come up with trendy topics with codes.
@AI-Makerspace21 күн бұрын
Thanks Rakesh! We agree - Allan really crushed it with this one!
@RaviPrakash-dz9fm22 күн бұрын
Can anyone tell me how ragas actually calculates these numbers. Like manually I get it, but what do the algorithms or functions look like? Like how does it measure faithfulness?
@AI-Makerspace21 күн бұрын
Hey Ravi great question! We go a bit deeper into this in our more recent event with the creators! kzread.infoAnr1br0lLz8?si=UG6vRnSY9oVtAuAT We'd recommend reading through the docs and digging into the source to go EVEN deeper! e.g., docs.ragas.io/en/stable/concepts/metrics/faithfulness.html
@AI-Makerspace22 күн бұрын
The Loss Function in LLMs - Cross Entropy: colab.research.google.com/drive/1VEk0mdGYKfPezWDJ4ErajNNlr4pq2Duk?usp=sharing Event Slides: www.canva.com/design/DAGH7_m5E48/HlpvFnc2VbCsjNiiFGdYnQ/view?DAGH7_m5E48&
@user-wf4mo5kc1y24 күн бұрын
This is really great explanation. I have one query, lets say I want to improve the performance by focusing on Faithfulness or Answer Relevance, so which RAG optimization techniques I should follow to increase Faithfulness or which techniques can improve Relevance or Precision etc.
@AI-Makerspace24 күн бұрын
The answer is, unfortunately, it depends! The whole system needs to work together (from data quality, to retrieval quality, to model performance, to prompting), and it needs to work for your use case. What is the best metric to use for your use case? That also depends. It all comes down to metrics-driven development: docs.ragas.io/en/stable/concepts/metrics_driven.html , but you need to decide which direction to drive! There are some simple things to do after you set up RAG like reranking, but for any given use case the details really matter with regards to what steps you should take.
@Amruth-lv6nk24 күн бұрын
Can I use vertex model as the policy model?
@AI-Makerspace27 күн бұрын
Multi-Agent Systems - LangGraph Pilot Demo: colab.research.google.com/drive/1YDQs2RySVelF9BjCYgSq2qC_xiIVdYpN?usp=sharing Slides: www.canva.com/design/DAGHLLEXvXQ/7tm3VDBZw3YKZJHsG-C8Vw/edit?DAGHLLEXvXQ&
@BHARTIDEVI.-8828 күн бұрын
Needed more content for this and code with more complex architectures .
@AI-Makerspace27 күн бұрын
This was completed in a 30min. Lightning Lesson - so we didn't dive too deep - but we do have a separate video where we dive in a bit deeper! Check it out here: kzread.infoulTvNAXI_1E?si=-Pvd5A4KS1GKXWnG
@rakeshkumarrout262928 күн бұрын
this is quite ueful, can you help me with how to assign a name to my rag app?? can you make a video on DSPY with custom data RAG.
@AI-Makerspace27 күн бұрын
DSPy is on the horizon!
@out_and_about0828 күн бұрын
Thanks ! Where can I find the colab notebook?
@AI-Makerspace27 күн бұрын
Thanks for the reminder! colab.research.google.com/drive/1YDQs2RySVelF9BjCYgSq2qC_xiIVdYpN?usp=sharing
@sitedev28 күн бұрын
I just built a multi-agent article researching and writing team using Flowise. They’ve just added a UI for creating chatflows powered by LangGraph. No code needed - just clever prompt engineering.
@lionhuang920928 күн бұрын
Where can we download ppt file?
@AI-Makerspace28 күн бұрын
Thanks for the reminder! www.canva.com/design/DAGHLLEXvXQ/7tm3VDBZw3YKZJHsG-C8Vw/edit?DAGHLLEXvXQ&
@jdavidnorena28 күн бұрын
@@AI-Makerspace put it in the description please, in case in future this comment gets lost, btw amazing video thanks a lot for sharing !
I like the way you simplify and explain- starting with the big picture and then breaking down in to the details.❤
@koescoin810Ай бұрын
This model solved many problems to users who doesn't want to send private documents to cloud just for fine tuning or RAG. Thank's to Gradient for this great milestone.
@thomashsu5252Ай бұрын
Great context. I really rllearnd something
@AI-MakerspaceАй бұрын
Nice!
@AI-MakerspaceАй бұрын
How LLMs Choose the Next Token: colab.research.google.com/drive/1dGdMVkwlHDIitsMDWOhPTS1jwxM_5qmD?usp=sharing Event Slides: www.canva.com/design/DAGGnrrq5zg/Wij3fvn2jmXVMRjx6LrnvA/view?DAGGnrrq5zg&
@MegaClockworkDocАй бұрын
Great Work. Please consider using data which is not part of the training data as the haystack for the long context search to create a more real-world example.
@AI-MakerspaceАй бұрын
There doesn't seem to be evidence to suggest that the data being known makes this task easier - but we can definitely make that modification!
@damiangilgonzalez8011Ай бұрын
Good job!! I would like to have the code :(
@AI-MakerspaceАй бұрын
You got it Damian! github.com/bytewax/real-time-rag-workshop/tree/main/workshops/aimakerspace-2024
Пікірлер
Omg, didn't know about that. Definitely going to the pipeline. Thanks for sharing! About the newsletters, there was theneurondaily, rundown ai, and what was the third one please? Something Valley when you all nodded so it must be an important one 😄 BTW, The Batch from Andrew NG is also a great one.
Always inspiration
Excellent session! Your explanations are so helpful and very much appreciated
How can we track testing and benchmarking agents using AgentOps
AIMS-AgentOps-CrewAI-Demo: github.com/chris-alexiuk/AIMS-AgentOps-CrewAI-Demo/tree/main Event Slides: www.canva.com/design/DAGJ55plCbY/PDASCroDOh-Upiu_lICWkw/view?DAGJ55plCbY&
Greg I 'd like to talk about llama3 great lectuce. 😊
How can I access this Colab (llama_index_tracing tutorial.ipynb)? Can you share the link?
We've shared your comment with the presenter! We expect he'll get back with you shortly!
On a scale from 1 to John Wick how much do you love your dog? 😆🤣🤣😆
Watch the full video! kzread.info/dash/bejne/aqiHwZptXZiundI.html
Thank you thank you thank you thank you thank you
KZread quality still 720p. Really hard on the brain to follow the code examples.
Stay tuned! Thanks for the ping and the reminder!
Thank you thank you thank you for this fantastic explanation of DSPy. I didn't understand it before this great webinar.❤
You're welcome! 🫶
Great point about agents, but I have a question. Whats != Agentic between other ?
Thanks for your question Givanildo! We do need to clarify this idea that we're playing with on distinguishing between "multi-agent" and "multiple agent," even internally. There is, admittedly, ambiguity here, and we will seek to clarify it more fully in our next multi-agent event. But we created it because we have found it to be a useful mental model. The point of making this distinction was to recognize that the complexity and dynamism of multi-agent systems varies, sometimes quite widely. For instance, multi-agent systems presented today often feel constrained, linear, and prescriptive. It many cases multi-agent systems could just as well be single agent systems created with one prompt (albeit a quite long one). Conversely, a truly complex "multi-agent" system would, in our estimation, *require* the use of multiple agents, even to the point that each LLM was fine-tuned to a specific task. Something like that - what do you think!?
Thank you very much for providing historical context, to me this is one of the most important aspects of learning about new tech, and understanding where, how, and why it fits in.
💯we couldn't agree more!
good stuff. Thx!
AIMS-CrewAI-Demo: github.com/AI-Maker-Space/AIMS-CrewAI-Demo Event Slides: www.canva.com/design/DAGJPj_tw-c/hSR7mYwEiRqCoK_L4RRcEg/view?DAGJPj_tw-c&
Nice
Good video but one question: Why did you choose to create the testset step-by-step yourself and not use the provided TestSetGenerator from Ragas? Was is not available back then?
That's right! They had just rolled it out it when we had them on for this more recent event: kzread.infoAnr1br0lLz8?si=_wIYqsL4vcVM5QDq
Wow great video. I just found your channel, already a fan. 🦾🤖
Let's goooooo!
Does ragas work only on openai model, which model i can use for testset genrator as critic and generative model please help me out
You can use any LLM that has OpenAI API compatibility. This means most closed source models, as well as open source options through certain hosting strategies (NIMs, vLLM, etc)
@@AI-Makerspace i dont have openai api key Can i use models from hugging face
Please help me out I want to genarate the test set data using models other than hugging face Like generate model and critic model
best video i found on learning word2vec concepts by building up from simple examples. very helpful for people who learn by doing.
You're speaking our language @sillystuff - we're all about learning, building, shipping, and sharing! Thanks for the comment and we're glad you found it useful!
amazing talk~~
top
Has there been any update yet?
great talk - thank you:)
Very useful 👌
More more more DSPy please!
absolutely
Loud and clear guys! We'll see what we can do 🤓
Nice intro dudes lol music hooked me
This is the most awaited and most useful and trendy thing in the market.
DSPy - Advanced Prompt Engineering?: colab.research.google.com/drive/1Il47YSattSnWV5cfSzSD7b2rWuyv7n6O?usp=sharing Event Slides: www.canva.com/design/DAGIl1M_SYI/1nhzSuhN8YQ0uxN_wqP99g/view?DAGIl1M_SYI&
Stoked that you are diving into DSPy! Would be cool if you could do a RAG demo and show how to generate an evaluation metric from free form text responses.
@@seanbergman8927 we think RAG is definitely the next step here for sure! Stay tuned!
RAGAS starts at time 20:15, before which is just an overview of langchain and the RAG QA pipeline
Thanks for the timestamp here MrTulufan!
this is quite useful,its really commendable how you come up with trendy topics with codes.
Thanks Rakesh! We agree - Allan really crushed it with this one!
Can anyone tell me how ragas actually calculates these numbers. Like manually I get it, but what do the algorithms or functions look like? Like how does it measure faithfulness?
Hey Ravi great question! We go a bit deeper into this in our more recent event with the creators! kzread.infoAnr1br0lLz8?si=UG6vRnSY9oVtAuAT We'd recommend reading through the docs and digging into the source to go EVEN deeper! e.g., docs.ragas.io/en/stable/concepts/metrics/faithfulness.html
The Loss Function in LLMs - Cross Entropy: colab.research.google.com/drive/1VEk0mdGYKfPezWDJ4ErajNNlr4pq2Duk?usp=sharing Event Slides: www.canva.com/design/DAGH7_m5E48/HlpvFnc2VbCsjNiiFGdYnQ/view?DAGH7_m5E48&
This is really great explanation. I have one query, lets say I want to improve the performance by focusing on Faithfulness or Answer Relevance, so which RAG optimization techniques I should follow to increase Faithfulness or which techniques can improve Relevance or Precision etc.
The answer is, unfortunately, it depends! The whole system needs to work together (from data quality, to retrieval quality, to model performance, to prompting), and it needs to work for your use case. What is the best metric to use for your use case? That also depends. It all comes down to metrics-driven development: docs.ragas.io/en/stable/concepts/metrics_driven.html , but you need to decide which direction to drive! There are some simple things to do after you set up RAG like reranking, but for any given use case the details really matter with regards to what steps you should take.
Can I use vertex model as the policy model?
Multi-Agent Systems - LangGraph Pilot Demo: colab.research.google.com/drive/1YDQs2RySVelF9BjCYgSq2qC_xiIVdYpN?usp=sharing Slides: www.canva.com/design/DAGHLLEXvXQ/7tm3VDBZw3YKZJHsG-C8Vw/edit?DAGHLLEXvXQ&
Needed more content for this and code with more complex architectures .
This was completed in a 30min. Lightning Lesson - so we didn't dive too deep - but we do have a separate video where we dive in a bit deeper! Check it out here: kzread.infoulTvNAXI_1E?si=-Pvd5A4KS1GKXWnG
this is quite ueful, can you help me with how to assign a name to my rag app?? can you make a video on DSPY with custom data RAG.
DSPy is on the horizon!
Thanks ! Where can I find the colab notebook?
Thanks for the reminder! colab.research.google.com/drive/1YDQs2RySVelF9BjCYgSq2qC_xiIVdYpN?usp=sharing
I just built a multi-agent article researching and writing team using Flowise. They’ve just added a UI for creating chatflows powered by LangGraph. No code needed - just clever prompt engineering.
Where can we download ppt file?
Thanks for the reminder! www.canva.com/design/DAGHLLEXvXQ/7tm3VDBZw3YKZJHsG-C8Vw/edit?DAGHLLEXvXQ&
@@AI-Makerspace put it in the description please, in case in future this comment gets lost, btw amazing video thanks a lot for sharing !
@@jdavidnorena got it going in a comment!
Mistral Fine-Tune: colab.research.google.com/drive/1RLl-n_pIAQKldWpc-UD8MhAhHO4wzYAl?usp=sharing Event Slides: www.canva.com/design/DAGHR69OPB4/Xy3m9rj3eWTfF9uFsEDDgQ/view?DAGHR69OPB4&
I like the way you simplify and explain- starting with the big picture and then breaking down in to the details.❤
This model solved many problems to users who doesn't want to send private documents to cloud just for fine tuning or RAG. Thank's to Gradient for this great milestone.
Great context. I really rllearnd something
Nice!
How LLMs Choose the Next Token: colab.research.google.com/drive/1dGdMVkwlHDIitsMDWOhPTS1jwxM_5qmD?usp=sharing Event Slides: www.canva.com/design/DAGGnrrq5zg/Wij3fvn2jmXVMRjx6LrnvA/view?DAGGnrrq5zg&
Great Work. Please consider using data which is not part of the training data as the haystack for the long context search to create a more real-world example.
There doesn't seem to be evidence to suggest that the data being known makes this task easier - but we can definitely make that modification!
Good job!! I would like to have the code :(
You got it Damian! github.com/bytewax/real-time-rag-workshop/tree/main/workshops/aimakerspace-2024