If you run pw-top, you might see errors accumulating. This is usually due to an underrun from the game requesting an audio quantum that’s too low.

The fix is:

    mkdir -p ~/.config/pipewire/pipewire.conf.d && echo "context.properties = {default.clock.min-quantum = 1024}" | tee ~/.config/pipewire/pipewire.conf.d/pipewire.conf

Basically, just force the quantum to be higher. Often it defaults to 64, which is around 1ms.