Right. It is weird that this has to be explained, but let's make it clear: in the Before Times, probably anytime up to at least the late '00s, probably approximately no-one in or around free or open source software would have endorsed the idea that the legal disclaimers completely free even the most mainstream, self-publicising, broad-userbase open source projects from any moral or ethical obligation to have even the slightest concern to ensure that their software doesn't hurt or betray its non-paying users, even in the most harmful ways. And if one of the many and often vocal FOSS opponents of the time had started claiming that this is what open-source developers really believe they'd have rightly been seen as having veered off into the lunatic fringe. And that's because it's a, frankly, bonkers idea which is radically detached from normal human understanding of the social and moral role and obligations of volunteers, voluntary organisations, charitable givers or gift-givers. And also because it's a wildly counterproductive idea to put out there if you're hoping to increase FOSS adoption.
That does leave the question of why this idea has started to take off more recently. Part of the answer is certainly that Rich Hickey, disgracefully, set the ball rolling in this direction, and that many others have welcomed it as one weird trick and one pat answer for all the worsening problems of developer burnout. Unfortunately it seems hard to dismiss the idea that it's also social breakdown driven by a broader trend, as over time we move further and further from the pre-'60s "neurotic society" of people obsessed with duty and social conformity (often with oppressive or destructive results, to be sure) into the "psychopathic society" in which even people who don't themselves merit a Cluster B diagnosis have internalised narcissistic and psychopathic attitudes.
I think some of it is an unintended consequence of the backlash against bad or puzzling error dialog boxes in '80s and '90s software. A lot of people seem to have concluded that the easy solution to the challenges of error reporting was just to fail silently instead roll_safe.jpg .
Suppose that you build a childrens' slide in your front garden. You put up a big sign saying "Consider using my slide! https://neovim.io/ Here are several wonderful things about it. It's free for everyone!" in your yard. Underneath in smaller letters you add "[No liability]". You also put up noticeboard ads for your free kids' slide in neighbourhood shopping malls https://launchpad.net/ubuntu/+source/neovimhttps://wiki.archlinux.org/title/Neovim . Unfortunately, when you built the slide, you left sharp metal edges and corners sticking far up on the inside, reaching into the path of the user. No reasonably competent and diligent metalworker or slide-maker would have failed to notice these major flaws or failed to understand the serious danger they represented. Several neighbourhood children use the slide and receive serious gashes to the legs, arms or face, and have to go to the hospital. Even assuming that your no-liability small print somehow had you free and clear legally, do you believe that your behaviour would have been ethically and morally above all criticism? Do you think that "should have read the small print!" or "can't I build what I like in my own front yard?!" would have you covered? Do you think that your family and friends would agree?
It's true that LLMs "want" to be be local, but they won't shift broadly to being local until there's a sufficiently large supply of VRAM or (at least) "unified" memory from the manufacturers. (I'm also assuming here that radical regulatory changes like government bans of local models aren't going to happen.) So (AFAICS—I am no expert) the future of LLMs over the next few years comes down primarily to the nitty-gritty of how much memory fab capacity will be added and when, and to a lesser extent of what happens to future demand from LLM SaaS services (& maybe their existing stock of hardware if they get in trouble). (I'm also assuming no roughly-AGI-sized leap forward which makes the frontier models of the near future vastly more valuable than the near-fontier models of today.) For the incumbent manufacturers the high-margin business is selling to LLM SaaS providers who use VRAM efficiently, but the high-volume business is getting chips into millions of laptops which will use VRAM very inefficiently. I assume that they will want to move from high margins to high volumes as they build they physical capacity to ship higher volumes, but they seem to prefer to do it at a stately pace. Hopefully some jostling from Chinese competitors, and maybe a dropoff in demand from data centres, will speed things along.
I wouldn’t say so. Once local solutions pass a threshold of affordability consumers tend not to mind too much about their inefficient resource utilisation: see the many thousands of MacBooks which sit largely idle for most of the day. And efficient utilisation is actively against the interests of hardware manufacturers, at least while they’re not supply-constrained and looking to sell their limited supply to whomever can pay top dollar.
But high demand for LLM time isn't sufficient to keep customers at the frontier LLM SaaS providers. That demand can be satisfied locally or at non-frontier outlets, absent hardware shortages at least. The Tier 1 providers (and the would-be Tier 1s) presumably need to open up a much bigger lead in model quality, one that doesn't simply get distilled away this time, and/or continue to be protected by ongoing (or worsening!) hardware shortages. (And that's overlooking the revenue shortfalls which OpenAI and Anthropic seem to be facing already.)
That does leave the question of why this idea has started to take off more recently. Part of the answer is certainly that Rich Hickey, disgracefully, set the ball rolling in this direction, and that many others have welcomed it as one weird trick and one pat answer for all the worsening problems of developer burnout. Unfortunately it seems hard to dismiss the idea that it's also social breakdown driven by a broader trend, as over time we move further and further from the pre-'60s "neurotic society" of people obsessed with duty and social conformity (often with oppressive or destructive results, to be sure) into the "psychopathic society" in which even people who don't themselves merit a Cluster B diagnosis have internalised narcissistic and psychopathic attitudes.
reply