Want to connect? 💼Consulting: calendly.com/engineerprompt/consulting-call 🦾 Discord: discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: ko-fi.com/promptengineering |🔴 Join Patreon: Patreon.com/PromptEngineering

00:02 Microsoft has released the second version of the Orca model, which consists of 13 billion parameters and outperforms the 70 billion parameter model on reasoning capabilities. 01:29 Research proposes training smaller models with different techniques to optimize their performance. 03:02 Orca-2 is a 13 billion parameter model that achieves performance levels similar or better than models 5 to 10 times larger 04:32 The Orca-2 model outperforms the original Lama 270 Bill chat model. 06:05 Selecting the appropriate quantization level and model for usage 07:36 The video discusses the release of the 'Orca-2' model and its superiority over previous models. 09:12 Evolution doesn't have a goal or a direction and chicken and eggs are products of evolution 10:44 The video covers the Orca-2 release, benchmarks, and running it locally on a machine using LM studio.

My tip for future vidoes would be, don´t spend that much time on testing another version of some model. Spend more time on how specific new concepts work - e.g. grouped query attention, rotary embeddings and so on. My opinion is that that would bring more value to the community. But in general I like your videos.

wow. your channel is full of knowledge and very up to date. thank you for this video. I have started this video and waiting for the model to be setup in the local machine. I hope you set it up completely on the local system. Thank you once again.

@engineerprompt

8 ай бұрын

Thank you for your kind words. The LM Studio is running it locally. But do you mean to run it in a python environment? Will look into it if you have any specific thing in mind

@jennilthiyam1261

8 ай бұрын

@@engineerprompt Thank you for your response. Yes, I want to run through Python and not use any studio or API, that could be run from your terminal with some command, like running Llama-2 through llama.cpp in your local system with GPU. It would make the user convinced that the language model is completely set up locally, and not in some link generated by Open AI or anything. Thank you in advance.

Love the explanation, and overview

@engineerprompt

8 ай бұрын

Thank you 🙏

You are quite right. Orca-2 gave me excellent explanations for questions related to earth and its magnetic field; Mars and its loss of atmosphere; Percival Lowell, and what led him to believe that there are canals on Mars, and so on. It is really good! It's a keeper! Thank you for the tip.

Great vid, the rate of improvements in these models is truly impressive, however a wider question for me is ive yet to see a convincing practical use case for them aside from just interest/noodling etc

@LeonardLay

8 ай бұрын

If we could get a demo of how to use these to create agents that'd be awesome

Solid intro to Orca-2 and CoT prompting. Now, how do will take this information to build solutions?

@engineerprompt

8 ай бұрын

Thank you. Will be creating videos on how to use the knowledge to create useful tools

thx

How does the Orca 2 7b model compare to mistral 7b / zephyr 7b?

@engineerprompt

8 ай бұрын

I think zephyr is still better in my initial testing

Please let me know if you are going to set it up completely in the local system.

These open-source models finna be trying compete with gpt3-4, nice video.

@carkawalakhatulistiwa

8 ай бұрын

they can do this by using synthetic data created by gpt 4

@DikHi-fk1ol

8 ай бұрын

@@carkawalakhatulistiwa correct, but my guess, it's not possible to get exactly same performance.

@engineerprompt

8 ай бұрын

I agree with you but small specialized models can surpass the performance of gpt3/3.5 if they are fine tuned. I think that is the real promise of small models

@DikHi-fk1ol

8 ай бұрын

@@engineerprompt I see, You're very correct, a open-source model indeed can do this if fine-tuned properly I think, might happen soon than we think it,ll, AI is such a promising field.

I don’t understand how you know what Presets to pair with what model… can you make a video on that?

@engineerprompt

8 ай бұрын

Yes, will do. Usually the prompt template is mentioned in model cards.

@petermarin

8 ай бұрын

@@engineerprompt I'm grateful!! also, might be worth doing a full tutorial on the whole LM Studio workflow (e.g. how to find good models for use cases, what do the letters in the model name actually mean, different LM Studio options, UI explanation - e.g. I still have no clue why some chats of little tokens are 1gb+ while others are super tiny. also what is that amount of space occupied by anyways? can't be the text itself)

they can do this by using synthetic data created by gpt 4

Can it write code? You should compare it against gpt-4 and 3.5.

@Jorsten

8 ай бұрын

I've just tested it on huggingface and it can't touch gpt-4 in terms of coding.

@engineerprompt

8 ай бұрын

Yeah coding is not that great

@timjx3675

8 ай бұрын

Its pretty bad at python coding, very slow and produced bug ridden code. Compared exact same prompt with gpt3.5 - very fast and perfect code.

I like your channel 😍

@engineerprompt

8 ай бұрын

Thank you 🙏

🎯 Key Takeaways for quick navigation: 00:00 🚀 *Introduction to Orca-2* - Microsoft released the Arham Progressive learning paper in June 2023. - Orca-2 is the second version with released model weights, a 13 billion model outperforming 70 billion models. - The focus of the video is on technical details, benchmark results, and how to use Orca-2 locally. 01:12 📄 *Aram Paper Highlights* - The Aram 2 paper aims to improve reasoning abilities in small language models. - Emphasizes moving beyond imitation learning to teach models different solution strategies. - Focuses on teaching various reasoning techniques, enabling effective solution strategies for each task. 02:22 🔄 *Training Technique Comparison* - Traditional large language models often rely on imitation learning for training. - Microsoft proposes a training technique for Orca-2 involving step-by-step recall, reasoning, and generation. - Orca-2's training techniques lead to surpassing models of similar size and rivaling models 5 to 10 times larger. 03:44 🔍 *Licensing and Model Details* - Orca-2 is licensed under Microsoft Research License for research purposes. - Base Lama-2 model is licensed for commercial use. - The model weights for Orca-2 are made publicly available for research, development, and evaluation purposes. 04:26 📊 *Benchmark Results* - Orca-2 (13 billion) outperforms Lama-2 (70 billion) on reasoning capabilities. - Performance is close on the GSM 8K dataset, surpassing Lama-2 on most benchmarks. - Visual representation of benchmark results, highlighting Orca-2's performance. 06:03 ⚙️ *Running Orca-2 Locally* - Demonstrates how to run Orca-2 locally using Python and the Transformer package. - Introduces the LM Studio for testing large language models with a clean interface. - Shows the configuration settings and system prompts used for testing Orca-2. 08:31 🐔 *Chicken or Egg Prompt* - Tests Orca-2's response to the classic question: chicken or egg first. - Highlights Orca-2's arguments, including genetic information, evolution, and lack of a definitive answer. - Demonstrates the model's ability to provide thoughtful and comprehensive responses. 09:55 💼 *Financial Investment Prompt* - Queries Orca-2 about investing $1,000 in AI companies on NASDAQ. - Orca-2 provides criteria for narrowing down options and mentions companies like Nvidia, Google, Amazon, Microsoft, and Tesla. - Emphasizes the model's consideration of market size, competitive advantage, innovation, and financial performance. 11:45 🎬 *Conclusion and Future Tests* - Summarizes the content covered in the video, including the Orca-2 paper, benchmarks, and local usage. - Mentions plans for future videos testing Orca-2 on a variety of prompts and applications. - Encourages viewers to share their thoughts on the model's performance in the comment section. Made with HARPA AI

Is it multi language?

@engineerprompt

8 ай бұрын

Not sure but I would think it will still be since it’s fine tuned of llama2

@skr_8489

8 ай бұрын

@@engineerprompt I will give it a try. I like the work you are doing on your channel. Thanks.

@skr_8489

8 ай бұрын

@@engineerprompt btw - is there a hassle-free way to chat with documents using various llm models? and which ones would you recommend for multilingual workflow?

I found that the 7B model was better than the 13B model.

@petermarin

8 ай бұрын

What’s your preset? It’s only giving me garbage

@engineerprompt

8 ай бұрын

I am using the ChatML

"llmstudio is not completely open source". I'd say it's not open source at all

It's not interesting because Mistral is superior to llama2, they should release the training data instead.

@carkawalakhatulistiwa

8 ай бұрын

they can do this by using synthetic data created by gpt 4

The AI lied to you. It said it couldn’t choose amongst the 5. Why not? It chose the 5. See, AI is not ethical. It simply responds based on how it was trained by humans. This is a problem.

Bad audio

To hell with the paper just get to install...voted thumb down.

Please review uukuguy/speechless-mistral-six-in-one-7b on huggingface, it's mind blowing as a model.

@engineerprompt

8 ай бұрын

Will do, thanks

this model is performing pretty well 🥹💫 i would probably read the orca 2 research paper.. thanks for sharing with us

@engineerprompt

8 ай бұрын

I would highly recommend it.