That is what has surprised me. I can understand if small businesses were caught here because they lack financial resources for the infrastructure and staff, but those large corporations like airlines etc... Why don't they have a staging environment where everything goes first? I naively assumed this was established best practice due to the risk of update issues bricking your organization.
But maybe anti-malware is given a blind eye because instant updates for zero day security issues are obviously attractive.
Still, though... In hindsight it's not workable for especially anything running system drivers with liberal kernel access.
I am not surprised at all. The level of DevSecOps' skills has been falling over the last two decades as demand for their skills kept growing. Most of them would report you to HR if you suggested they use WireShark to debug a networking issue. They are useless people who came to IT because of the promise of good pay and don't know how computers, networks work.
It's automatic, no? The whole "promise" (oh sorry, the "added value proposition") of CS is that they "keep you safe" automatically! It was a content update. Meaning basically antivirus signatures ... and oops, some minor non-functional changes to the filtering kernel driver.
... well, yes, yes of course. And if I try to be serious on a late Friday night (it's almost 20:00 here), the obvious solutions is to have something like eBPF in/for the Linux kernel (which has a verifier[0]).
And security vendors should follow "secure by design" principles. Yes, I know a try-fucking-catch might be too advanced, and uh oh kernel code is hard because unwinding is costly. But guess what else is also not cheap. (Okay, I seriousness failed.) But still. This is fair and square in the "this should never happen" scenario. It's an automatically downloaded plugin or whatever. (CS can call it "content update", but von Neumann is already calling FedEx to send them a pallet of industrial grade bitchslap.) And if the plugin loader cannot gracefully fail plugin loading, then it should obviously come with the appropriate audiovisual cues[1] so sysadmins know what to expect.
Security and Compliance gets to violate all good sense, because it's just sooo important. They can run un-reviewed un-sandboxed daemons as root on every system if they really want, they can have changes pushed automatically without review or control, because "security" is just so important, and due to "compliance" you really have no choice as your company gets larger, you just have to do it. That's why, despite being obviously pretty dumb to many skilled engineers, it seems like everyone does it. No choice. Security, Compliance. So dumb ...
If I can't commit code to our app without a branch, pull requests, code review...why can the infrastructure team just send shit out willy-nilly?
"Always allow new updates" must have been checked, or someone just goes through a dashboard and blindly clicks "Approve"