I sense that I don't really understand enough of your comment to know why this is important. I hope you can explain some things to me:
- Why is Qwen's default "quantization" setup "bad"
- Who is Unsloth?
- Why is his format better? What gains does a better format give? What are the downsides of a bad format?
- What is quantization? Granted, I can look up this myself, but I thought I'd ask for the full picture for other readers.
Oh hey - we're actually the 4th largest distributor of OSS AI models in GB downloads - see https://huggingface.co/unsloth
https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs is what might be helpful. You might have heard 1bit dynamic DeepSeek quants (we did that) - not all layers can be 1bit - important ones are in 8bit or 16bit, and we show it still works well.
The default Qwen "quantization" is not "bad", it's "large".
Unsloth releases lower-quality versions of the model (Qwen in this case). Think about taking a 95% quality JPEG and converting it to a 40% quality JPEG.
Models are quantized to lower quality/size so they can run on cheaper/consumer GPUs.
My LG C2 hardware isn't powerful enough to stream higher than 60hz at 1080p, if I remember correctly. It also needs a LAN cord for consistency since the tv wifi adapter is not good. Instead I put moonlight on my steam deck and plugged that into the tv.
Oh yeah I’m aware of various “plug in a thing” options, just thinking it wild be nice to have to, particularly if a single controller paired to the TV itself could operate the outer shell as well as Xbox and steam streaming.
TUnit isn't really a fair comparison, it's another test runner like xUnit/NUnit/MSTest, but with more focus on performance/extensibility, which it achieves by using .NET's new test platform :)
This impacts me quite a bit as I use this often.
I pay for Google One for the extra storage and VPN. Now the VPN is going away but my costs will stay the same.
- Why is Qwen's default "quantization" setup "bad" - Who is Unsloth? - Why is his format better? What gains does a better format give? What are the downsides of a bad format? - What is quantization? Granted, I can look up this myself, but I thought I'd ask for the full picture for other readers.