It's a problem in various networking devices, in which those devices buffer too heavily rather than dropping packets. That behavior may be implemented in a combination of hardware/firmware/software. Among other things, this prevents other devices from being able to adapt to bandwidth limits (because dropped packets are the primary way to signal "slow down"), and causes high latency (because packets are sitting in deep buffers for a while).

Software such as network schedulers in Linux can work around that problem. But ultimately, it should be fixed in the various networking devices that create the problem in the first place.

It's a software issue, but many routers are sold as appliances where the average user does not choose what OS to run. Therefore, a common recommended solution to bufferbloat is to replace the router with one whose software has more modern buffer management algorithms.

OpenWRT, for example, can be flashed on many older routers and will be able to solve bufferbloat without replacing the hardware.

Bufferbloat can be a hardware or software problem. The fix is always to move the bottleneck to a device you control and can configure to not have bufferbloat at that bottleneck. For consumer hardware that means implementing AQM in software.

The buffer bloat issue happens due to bad queue management on the router immediately preceding the bottleneck link. Whether the queue management happens in "hardware" or "software" depends on the router (but I suspect it'll be software/firmware, not actual hardware).

"bottleneck link" is, of the path from the packet source to the target, whichever link is trying to exceed the available bandwidth. The problematic router (prior to that link) will receive packets faster than it can send them, so it needs to either use the queue, drop the packet, or use the ECN bits in the IP header to tell the target to tell the source to slow down. A router with bad queue management will enqueue packets until the queue is full, before picking one of the other options. This adds latency depending on the queue size; big queues on slow links cause significant latency.

With bad internet connections, the bottleneck link is often the connection between the home router/modem and the ISP's infrastructure. If upload bandwidth is fully used, bad queue management on the home router will cause latency to spike. If download bandwidth is fully used, bad queue management on the ISP's router will cause latency to spike.

With fast internet connections, the bottleneck link might be somewhere else, e.g. at the interface between two ISPs.

Because the problem only happens at the bottleneck link, it is possible to work around a bufferbloat problem in the ISP's infrastructure by introducing another artificial bottleneck in your own home network. So by using a bit of Linux software that artificially limits bandwidth, that software creates a virtual "bottleneck link" where it can control both ends, and thus "fix" the problem (as the link affected by bufferbloat will no longer be overloaded, and thus no longer use its queue). A real fix would require updating the firmware/configuration on the ISP's routers.