In modern systems everything is buffered, often way too much individually, let alone in aggregate.

If it were not for the games industry it would be much worse, they are the only force incentivizing keeping latency within somewhat acceptable bounds. Things like FreeSync can help a lot.