i like the concept of basic reasoning and specialization and i guess, this could lead to generally smaller models, which are trained in specific abilities (like coding in a language, sql, summarization of text, generating text), while understanding what is needed (reasoning) based on text, images, audio, … those can be coordinated in teams (think of crew ai, langgraph or other agentic setups). there is not a huge benefit in ultimate knowledge bases, where one has to know it all (every coding language, every text type, …).

I'm excited for when you put the new Chinese SenseNova 5.0 model to the test :)

@ericsabbath

25 күн бұрын

What about Qwen? The 14B quantized version runs smoothly on free colab (T4)

The effectiveness of AI models often correlates with their size: larger models typically exhibit better reasoning capabilities. Consequently, simply increasing the number of smaller, less capable models does not necessarily compensate for their individual limitations in performance

I wish they would do benchmarks that compare performance of Mixture Of Expert LLMs vs regular old agent systems. An agent system is more easily distributed across many computers and can be more easily debugged and steered by looking at the conversation of prompts. The agent system also doesn't take a lot of new training horsepower. Overall, in the future I expect agent systems to generally dominate the MOE approach except for the fact that they are rarely directly compared in the benchmarks.

This uses 128 retail GPUs, one per model. Not cheap - but maybe great value for money? TBH this only makes sense in a multi-user system, in order to keep all those GPUs busy.

1:58 correction: there are 132 (i)phones in there Counted them😂

hmmmm... 🤔

Filtering for "Enterprise Intelligence" is grossly misleading. When you factor in all basic capabilities, including MMLU, Arc, and other tests measuring basic intelligence, knowledge and language skills, then snowflake performs horribly at most major LLM tasks and is easily beaten by the much smaller and about equally fast LLMs like Mixtral 8x7b. And it really doesn't take all that long to train an 8x7b MOE, so the added training time is much cheaper overall than the added memory resources needed during inference with a large number of client machines. This 128 expert design is basically worthless unless you're able to make it run with a much smaller memory footprint, perhaps by loading only a handful of experts into memory at a time.