Not that dramatic of an increase actually - the M2 Max already had 400GB/s and M2 Ultra 800GB/s memory bandwidth, so the M3 Ultra's 819GB/s is just a modest bump. Though the M4's additional 146GB/s is indeed a more noticeable improvement.
For inference the bandwidth is generally not parallelized because the weights need to go through the model layer by layer. The most common model splitting method is done by assigning each GPU a subset of the LLM layers and it doesn't take much bandwidth to send model weights via PCIE to the next GPU.
My understanding is that the GPU must still load its assigned layer from VRAM into registers and L2 cache for every token, because those aren’t large enough to hold a significant portion. So naively, for a 24GB layer, you‘d need to move up to 24GB for every token.
Even though I'm using Container Proxy myself, you can almost do the same thing with the official plugin (Manage Containers -> Click any Container -> Advanced proxy settings).
Probably not exactly what you want but for proxies you can use Container proxy[0] (not affiliated), which allows you to assign different proxies added by yourself to specific containers.
I've used containers with container proxy for well over a year and it works great for segregating work traffic on my home system. I use a container for work traffic that is assigned to an SSH session to my work system that exposes a SOCKS proxy that I make sure is running.
Eventually I moved work browsing to a different browser that I set to run through that proxy explicitly, but I still have the Firefox config set up in case I don't want to launch that, or don't care to copy and paste to the work browser some times (I wish I could find a way to make links launched from Outlook/Teams to open in Chrome but other links to open in Firefox).
I'm not, but just yesterday or the day before I was thinking maybe I'd write a simple application to take the URL and make decisions what to do with it afterwards based on whatever data I decide is relevant (time of day, domain, etc). If I had access to what the source application was that would be ideal, but I doubt Windows 10 supplies that.
