Whenever reasoning/thinking is involved, 20t/s is way too slow for most non-async tasks, yeah.
Translation, classification, whatever. If the response is 300 tokens for the reasoning and 50 tokens for the final reply, you're sitting and waiting 17,5 seconds for processing one item. In practice, you're also forgetting about prefill, prompt processing, tokenization and such. Please do share all relevant numbers :)
It is a mixture of experts model so it will run on a computer with a lot of RAM and a GPU.
Alternately, on an M3 Ultra Mac Studio with 256GB of unified memory, you can run a 4bit quant of GLM-4.6 at about 20 tokens/second. That compares to about 40 t/s for a 6bit quant of MiniMax M2. I am not sure how fast these will run if you have a Mac Studio 512GB that can load the unquantized versions of the models.
I think my link didn’t include the Javascript to choose the 512GB configuration, but it comes out to $8070, and their refurbished models are indistinguishable from new.
I think a surprising number of companies only survive because Microsoft Office gets around hostile internal IT departments and gives workers capabilities they can’t otherwise get on their locked down workstations.
It was only in 2007 that the row limit in Excel increased from 65k to one million and the column limit increased from 256 to 16k. There are better tools to work with data, but these companies’ IT departments aren’t letting users install them.
That sounds like the blundering VP/president in The Diplomat. I wonder if it's instead North Sea Oil, and Iran preferring a UK with less energy independence.
Why wouldn't an independent Scotland continue producing oil from the North Sea? It's one of the main talking points for independence: "we get to keep that money."
BTW, I had not seen that show, but if they are getting things even close to accurate once in a blue moon, then I may do so. Thanks.
If we are to take our natural place as the inventors and leaders of "The West," you know... democracy and all that boring stuff... wouldn't it be nice to have a source that is within reach, until we ween ourselves off of fossil fuels?
Another answer: with sanctions on Russian oil, and hopefully finally waking up to the fact that we cannot rely on any traditional foreign partners... yes.
Look at what China has done. They know they have no reliable sources and delivery routes for liquid fossil fuels. So, they took the reigns of their own destiny and created PV and battery production on a scale that changed the world.
Meanwhile, we in the EU, let Northvolt go under, and we closed down domestic PV production... like babes in the woods. (children wandering the the forest full of wolves)
Letting Northvolt shut down was one of the dumbest strategic things in modern European history, and that is saying a lot. I still have hope that it can somehow be resurrected.
"We" implies that the entire EU would have to be ready to support an expensive project. Surely Scotland cannot do it alone.
The question is whether this makes sense when Norway produces cheaper hydrocarbons just next door. The production sites there are every bit as vulnerable to Russian attacks as those next to Scotland.
I am curious, what do you think of the concept of European Federalism, to some extent?
It seems completely impossible, and yet somewhat inevitable. As I write this, I am sitting just a 30 minute drive into Silesia from the CZ border. In my car I receive PL, CZ, and DE radio stations.
The first I heard of the concept of a Federalized EU, it was from very excited international younger folks in Prague.
Federations are tricky to keep together. I witnessed the unraveling of Czechoslovakia as a teen and I was happy that it didn't end up in bloodshed, unlike Yugoslavia. There, guys of my age were spending their early youth killing other guys of my age speaking a mutually comprehensible language.
Don't push it, let it come naturally, if it does. Both CS and YU were built in a top-down fashion, without explicit consent of the nations involved, and this proved fatal eventually. Don't repeat the same mistake again.
I am suspicious of any claims about relative cleanliness. As with wooden vs plastic cutting boards, our intuitions are likely misleading.
To be an effective fomite the currency has to not kill the microbe, and it has to readily give up the microbe to the next recipient. Organic materials like cotton or linen seem more likely to simply absorb a viral envelope or bacterial cell wall, thereby rendering it ineffective. Furthermore, the porous nature makes it more difficult for the note to give up any microbe that isn't immediately killed before it naturally dies over time.
A brief search of the scientific literature doesn't seem to show any conclusive results, but it does seem like the relative performance is pathogen specific.
I Suppose heating fuel is cheaper, or household heat pumps are more efficient, but all of the energy consumed by running the refrigerator becomes waste heat in the same room you are trying to heat. That seems superior to the refrigerator heating a room where the waste heat isn’t useful.
Fortunately the models are increasing in efficiency about as fast as they are increasing in performance, so your homelab surprisingly doesn’t become out of date as fast as you might expect. However, I expect there will also be very capable machines like 1TB or 2TB Mac Studio M5 or M6 Ultras within a year or two.
What example tasks would you try?