Thousands is nothing. A single thread can handle thousands. You don't need any of this sharding or Kubernetes stuff. I'd wager 120K connections can be handled by one fast instance.

Above that, you have basic sharding techniques like assigning each channel to an instance. This should work up to some thousands of moderately active chatters in a single channel - at which point nobody can read all the messages anyway so it's not working for non-technical reasons.

Slack is trying to solve the problem of having thousands of channels with millions of people, as well as millions of channels with thousands of people. In other words, about 6 orders of magnitude bigger than your load test.

I was limited by time and budget there. My solution is open source after all and embarrassingly parallel so can potentially handle sharding across thousands of hosts. If I remember correctly, my calculations suggested that a cluster with 2k hosts could handle 40 million concurrent users if you push a message to every user every 5 to 10 seconds with some randomness.

The per-channel limit at that rate (1 message to every users every 5 to 10 seconds) was 20K concurrent connections but the number of channels is unlimited since you can add more capacity by adding more hosts and the sharding of channels across available hosts is automatic.

The only limit was number of hosts since there is only one coordinator instance in the cluster though the cluster can continue to operate without any downtime while the coordinator instance is down. The coordinator instance only needs to be up while the cluster is scaling up or down. That said it should be able to handle up to 5k hosts, potentially even much higher.