Evaluate LLMs with Language Model Evaluation Harness
Ғылым және технология
In this tutorial, I delve into the intricacies of evaluating large language models (LLMs) using the versatile Evaluation Harness tool. Explore how to rigorously test LLMs across diverse datasets and benchmarks, including HellaSWAG, TruthfulQA, Winogrande, and more. This video features the LLaMA 3 model by Meta AI and demonstrates step-by-step how to conduct evaluations directly in a Colab notebook, offering practical insights into AI model assessment.
Don't forget to like, comment, and subscribe for more insights into the world of AI!
GitHub Repo: github.com/AIAnytime/Eval-LLMs
Join this channel to get access to perks:
/ @aianytime
To further support the channel, you can contribute via the following methods:
Bitcoin Address: 32zhmo5T9jvu8gJDGW3LTuKBM1KPMHoCsW
UPI: sonu1000raw@ybl
#openai #llm #ai
Пікірлер: 12
I love you man, ❤ You are awesome, keep uploading 😊
Thanks, great LLM tips
if i want to evaluate the LLM using custom data set is that possible using the GIT repo that you have provided here?
I LIKE THIS... nice job man !
nice! thank you for the video!
nice work
do we have to add any dataset?
Can i do it on llava model
PackageNotFoundError: No package metadata was found for bitsandbytes. I am getting this error even though bitsandbytes is installed and my cuda version is 12.1, please help me with this
What about langsmith?It does the same thing right?
How to do it on whole mmlu?
I need rag chatbot part 2 video, please release, my exam is coming