Let me just say – what a time to be alive! Especially for local and open source AI! I say the local AI revolution has started because Qwen-2.5-Coder-32b has really proved that we’re nearing the point of local AI being as good as the big guys like GPT and Claude. It’s only a matter of time.
In this video, I dive into my own testing for Qwen-2.5-Coder-32b, including trying it out with oTToDev (our Bolt.new fork) and an AI agent I created specifically to test new LLMs like this. I appreciate benchmarks, don’t get me wrong, but I always want to try more practical things like this, and I wanted to share it with you too! And man, this model surpasses all my expectations and the performance of ANY other local LLM, even the larger ones!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:00 – Intro
01:42 – Model Overview + Hardware Requirements
04:22 – oTToDev with Qwen-2.5-Coder-32b
06:10 – Codellama’s Epic Failure with oTToDev
07:45 – Novita
09:43 – Testing Qwen-2.5-Coder-32b as an AI Agent
15:12 – Outro
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Give Novita a shot today – it’s honestly my new favorite platform for GPU instances and LLM inference. Not just because of our collab for this video but because of all the features I talk about in the video that I genuinely appreciate – the insanely competitive pricing, flexibility of your GPU instances, ease of setup, and the uniqueness of their available models.
Using the link below will help support me as a creator as well! 🙂
https://novita.ai/?ref=nmqzzdk&utm_source=affiliate
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Qwen-2.5-Coder-32b on Ollama (completely free!):
https://ollama.com/library/qwen2.5-coder:32b
Qwen-2.5-Coder-32b on OpenRouter (not free but about as cheap as GPT-4o-mini!):
https://openrouter.ai/qwen/qwen-2.5-coder-32b-instruct
oTToDev (our fork of Bolt.new):
https://github.com/coleam00/bolt.new-any-llm
Join the Discourse community for oTToDev:
https://thinktank.ottomator.ai
My LLM eval agent:
https://github.com/coleam00/ai-agents-masterclass/tree/main/llm-agent-evaluation-framework
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Artificial Intelligence is no doubt the future of not just software development but the whole world. And I’m on a mission to master it – focusing first on mastering AI Agents.
Join me as I push the limits of what is possible with AI. I’ll be uploading videos at least two times a week – Sundays and Wednesdays at 7:00 PM CDT! Sundays and Wednesdays are for everything AI, focusing on providing insane and practical educational value. I will also post sometimes on Fridays at 7:00 PM CDT – specifically for platform showcases – sometimes sponsored, always creative in approach!
#Novita AI, #llm, #ai chatbot, #ai chatbot, #llama
