>> No staged deployment {changing to} Add staged deployment
That's the thing that amazed me.
How do you regularly YOLO patches worldwide to something that runs with enough permissions to crash a system?
I don't care if this was a configuration update vs a new sensor capability -- universal rollout should never have been allowed by CrowdStrike's release team.
My speculation is they probably did stage the rollout to some extent, but didn't have a viable or fast enough feedback mechanism to let them know there was a kernel crash. That seems much more plausible than the engineers being incompetent enough to not have staged rollouts at all. Or they might have had it, but only for code rather than data.
And I believe it because for administrators there is no configuration to delay the rollout of these "content updates", you can only delay the sensor updates.
I think that is being too charitable. The problem is 100% reproducible. The machine blue-screens at boot up. What kind of staging environment allows this to go through?
My outsider's guess is that whoever was making decisions at that level was high on their own supply and decided that a staged rollout would result in time where they weren't protecting all those other machines from imminent and certain catastrophe.
I'm still awestruck that any engineer would be willing to ship code in that setup, though I guess its also possible that they were being misled about how much testing was going on at QA.
Given it's a security product, you'd want everyone protected on Day 0 that you have a new release, no?
Except on Day -1, no one was protected.
So what's magical about the day CrowdStrike decides to ship an update?
I can imagine there are possibly scenarios where mass-release would make sense (aggressive vulnerability spreading rapidly), but that can't be every day, can it?
That's the thing that amazed me.
How do you regularly YOLO patches worldwide to something that runs with enough permissions to crash a system?
I don't care if this was a configuration update vs a new sensor capability -- universal rollout should never have been allowed by CrowdStrike's release team.