🦥 Unsloth Roadmap + YCombinator

We're excited to bring fine-tuning to an even larger audience & so you can focus on making the models you've always dreamed of, without the complexity.

We've got a lot of new cool things coming to Unsloth slated for this year including Unsloth Studio, our long-awaited UI for fine-tuning models, and a new place where you can train, deploy & run models. Feel free to join our waitlist for Unsloth Studio and our platform.

And a huge thank you to everyone who has downloaded our models on Hugging Face as we're also celebrating our 2 million monthly downloads milestone! We're also excited to announce that Unsloth is now backed by YCombinator, and we will continue focusing on our open-source mission to make fine-tuning the best it can be!

Here's a run-through of what happened since our last update:

• Pip install Unsloth now works!

• Multi GPU is now in beta with around 20 community members testing it!

• You can now fine-tune with CSV / Excel files with multiple columns

• Microsoft's new Phi 3.5 mini is now supported and we Llamafied it

• DPO is now fixed. All chat templates now work and we fixed chat templates for Llama + Phi models

• Please update your Unsloth Colab and Kaggle notebooks - all got updated with Torch 2.4

🦥 Our Mission + Studio Update

At Unsloth, our mission is to empower users to build the highest-quality models, the right way. We're deeply committed to advancing open-source development through collaborations with incredible teams like Hugging Face and Google, ensuring proper implementation of open models. We will continue to contribute to the development of open-source models, having previously addressed key bugs in models like Google’s Gemma and Meta’s Llama.

Unsloth Studio, our first release coming soon, will encompass many highly requested features and will be able to run in Google Colab for free - or you can install it locally and use it 100% offline. Thanks to our community, we understand developer's need for integrating with existing workflows, so we're making it easier for users to incorporate their favorite open-source packages like Jan, Ollama, or VLLM, or even just exporting for inference.

And as demand for models like stable diffusion, multimodal models, and voice models grows across industries, we’re also focused on revolutionizing how these models are trained and being deployed today—just as we’ve already transformed the landscape for language model training.

Join our mission by subscribing to our newsletteror following us on social media to stay updated. P.S. We’re also hiring so if you’re eager to unsloth it, reach out to us at support@unsloth.ai—we’d love to hear from you!

💎 The Platform

Using math algorithms, low-level GPU programming (Triton, CUDA) we aim to innovate the LLM ecosystem through software, not hardware. With innovations like our Unsloth Gradient Checkpointing algorithm, which reduces training memory usage by more than 50%, we're constantly pushing the boundaries of AI. We're working on our new methodologies and algorithms for our platform which results in a 10%-40% lift in accuracy for models trained with the same controlled environments.

Our platform will have our exclusive 30x faster fine-tuning and faster inference offering along with easy peasy deployment setups plus much more. With our innovations, we aim to become the place for building custom models from scratch, offering the most accurate and accessible solutions for anyone—whether you’re a seasoned expert or just starting out.

We also understand the frustrations users experience when training & deploying models and our aim is to for you to focus on creating the models you've always dreamed of, without the hassle. Say bye bye to dataset preparation, data & chat template issues, and more! We also recognize the importance of deep customization, therefore, we're also introducing numerous components to enhance customizability without compromising accuracy or speed.

The beta version of the platform will be out by the end of this year and we'd love any feedback on the platform when we release it. In the meantime you can join our waitlist.

Thank you for reading! 😊