It's just a shell prompt. There really won't be much in the way of bugs. And it's popular. Doing a quick scan of github issues, there's no shortage of questions where some other random plugin or weird terminal emulator has strange interactions. I think the author can be forgiven for not wanting to diagnose such combinations.
> If zsh has its completion fully configured by default, there will be no need for most people to turn knobs nor will it be a "half decent shell." It'll be the best shell, if it isn't already.
The curse of backward compatibility means that zsh does not break your setup or change things on you. And there are still developers left who bear the trauma from the one time that was tried back in the early-mid 90s in the 2.x version series. Sadly that means many new features remain inactive by default, especially anything written in shell-code like command-specific completions.
If you've only compared against zsh's out-of-the-box completion then you're comparing against the fully backward compatible with how things were in 1993 state. Turn on compinit, and a half-dozen styles to enable descriptions at the very least. The content of `StartupFiles/zshrc` from zsh sources is more than enough, no need for a bloated framework like oh-my-zsh.
I'd be rather skeptical about the "fix" of updating the completion cache once a day only. Enabling oh-my-zsh already runs `compinit` and it does so after changing the function path in `fpath`. By running it again with a different `fpath` you invalidate the previous cache - meaning it builds a fresh cache twice every time you start zsh. If you use a plugin framework that already runs compinit, simply don't run it again separately. And make sure that `compinit` is only run after the final plugin or whatever has finished changing `fpath`. If you get that right, you'll only ever need to regenerate the cache when something actually updates.
Has anyone actually got this llama stuff to be usable on even moderate hardware? I find it just crashes because it doesn't find enough RAM. I've got 2G of VRAM on an AMD graphics card and 16G of system RAM and that doesn't seem to be enough. The impression I got from reading up was that it worked for most Apple stuff because the memory is unified and other than that, you need very expensive Nvidia GPUs with lots of VRAM. Are there any affordable options?
Yes. Although I suspect my definition of "moderate hardware" doesn't really match yours.
I can run 2b-14b models just fine on the CPU on my laptop (framework 13 with 32gb ram). They aren't super fast, and the 14b models have limited context length unless I run a quantized version, but they run.
If you just want generation and it doesn't need to be fast... drop the $200 for 128gb of system ram, and you can run the vast majority of the available models (up to ~70b quantized). Note - it won't be quick (expect 1-2 tokens/second, sometimes less).
If you want something faster in the "low end" range still - look at picking up a pair of Nvidia p40s (~$400) which will give you 16gb of ram and be faster for 2b to 7b models.
If you want to hit my level for "moderate", I use 2x3090 (I bought refurbed for ~$1600 a couple years ago) and they do quite a bit of work. Ex - I get ~15t/s generation for 70b 4 quant models, and 50-100t/s for 7b models. That's plenty usable for basically everything I want to run at home. They're faster than the m2 pro I was issued for work, and a good chunk cheaper (the m2 was in the 3k range).
That said - the m1/m2 macs are generally pretty zippy here, I was quite surprised at how well they perform.
Some folks claim to have success with the k80s, but I haven't tried and while 24g vram for under $100 seems nice (even if it's slow), the linux compatibility issues make me inclined to just go for the p40s right now.
I run some tasks on much older hardware (ex - willow inference runs on an old 4gb gtx 970 just fine)
So again - I'm not really sure we'd agree on moderate (I generally spend ~$1000 every 4-6 years to build a machine to play games, and the machine you're describing would match the specs for a machine I would have built 12+ years ago)
But you just need literal memory. bumping to 32gb of system ram would unlock a lot of stuff for you (at low speeds) and costs $50. Bumping to 124gb only costs a couple hundred, and lets you run basically all of them (again - slowly).
2G is pretty low and the sizes things you can get to run fast on that set up probably aren't particularly attractive. "moderate hardware" varies but you can grab a 12 GB RTX 3060 on ebay for ~$200. You can get a lot more RAM for $200 but it'll be so slow compare the the GPU I'm not sure I'd recommend it if you actually want to use things like this interactively.
If "moderate hardware" is your average office PC then it's unlikely to be very usable. Anyone with a gaming GPU from the last several years should be workable though.
I can run 7B models with Q4 quantization on a 7000 series AMD APU without GPU acceleration quite acceptably fast. This is with DDR5600 RAM which is the current roadblock for performance.
Larger models work but slow down. I do have 64GB of RAM but I think 32 could work. 16GB is pushing is, but should be possible if you don't have anything else open.
Memory requirements depend on numerous factors. 2GB VRAM is not enough for most GenAI stuff today.
This article implies that you have to use NAT with Wireguard which really isn't the case at all. Normal subnet routing works fine provided your destination hosts know to use the wireguard server as the gateway for the wireguard subnet. Just configuring a static route on the normal default router is generally enough. Certainly, there are cases where NAT is useful, for example I redirect attempts to use public DNS to my local DNS.
I've always found the fact that zsh copes with NUL characters in variables etc to be really useful. I can see why this approach makes sense for OpenBSD but they can't prevent NULs appearing in certain places like piped input.
When I was studying in the late 80s / early 90s, the text books contrasted the "old" waterfall approach with the more "modern" iterative approach. That was long before Agile or whatever had been invented and recognised that there needed to be a feedback loop.
Trying to promote agile/scrum/xp/whatever by attacking waterfall is a straw man argument in my view because it is not comparing against what came before. That's not to say that clueless managers don't still like to reinvent and impose what is essentially waterfall.
We use Oracle Linux where I work because we used to use Solaris. At one point it looked suspiciously like Redhat had explicitly removed support for some Oracle hardware from their kernel and the UEK saved us. Besides the kernel they also offer a few other extras and optional newer versions like dtrace, support for some extra filesystems and newer KVM. I get the impression that the fact that they're underdogs in the Linux market helps to keep them honest. Much of their key staff are clearly open source advocates even if Larry only cares about money. And while the Oracle support is not up to the standards of the old Sun support it really isn't a bad choice. And it'd be easy to switch away if we ever need to.
>Redhat had explicitly removed support for some Oracle hardware
Sure, if you already had Oracle as a hardware vendor then the cat is out of the bag, barn doors open w/ horse gone. After that, using their distro isn't going to increase your attack surface much, the killer is already inside the house.
