Cloud made sense for the startup I worked for previously. If you are a startup then a $1M per year expense makes much more sense than a $5M up front purchase with 5-10 years of life - in five years you might be billionaires or you might be bankrupt and until then the Cloud was better.
If your monthly cloud bill is < $100K, you don’t need $5MM worth of hardware. That cloud spend equates to, at best, a couple of beefy servers, which (modulo AI cards) could almost certainly be had for <= $50K/ea. So for $200K, you could have a two-zone setup.
Where it does make sense in the short-term for this scenario is the experience and knowledge necessary to reliably run your own servers. If you don’t have that, you may not want to invest the time and effort to do so. But on pure cost, unless your bill is on the order of a few thousand per month, cloud will never win. It can’t; they have to make money.
Like sibling is saying it's not $5M upfront but on the order of 1 year cloud spend for large enough accounts. There are also such things as leases and loans.
One justifiable excuse is you simply don't know how much hardware you will need to buy if you're hitting hockeystick growth. That until you realize you can also go hybrid...
Or you could have rented or leased-to-own. There's hardly ever any need to actually purchase outright to get prices far below equivalent capacity in clouds. In fact, in 25 years, only one of the colo'd server setups I've worked on had any hardware purchased up-front in it.
Or where locality is critical. Like if you have a game that hits peak traffic at different times throughout the day in different regions. So, a company may not want to own hardware in multiple regions, when they would only be at peak usage for a few hours.
The variation in peak loads tends to be far smaller for most people than what they imagine, but indeed it can sometimes be cheaper. The window needs to be very short, though, to outweight the large cost differential. And you don't need to buy - you can rent.
Yes, but in practice it's even rarer for people to have loads that have multiple spikes a day, to the point it's a rounding error you can mostly ignore outside of very unusual niches. Usually the day-night cycle entirely dominates in terms of traffic variations and is already long enough to make auto-scaling unviable in terms of cost.
You're right that if the longest window is short enough to make autoscaling financially beneficial over managed hosting, then you also need to make sure that you don't regularly have other spikes that can tip things back to being unprofitable.
