There are several branches, but all of them typically run at a scale where performance costs them money. People with large server deployments (any "hyperscale" company) hire these people to save millions of dollars a year. People who do client performance attract and retain users. Those who do trading bank on being the fastest. And many other areas; these are just the big ones.
Interesting. Does it need a PHD or a specific Master? Because I feel no one needs that kind of performance at home so the only place to train is academy or industry while industry usually does not train a junior on that topic.
This type of optimization isn't something you typically study as an academic. Everyone I know that does this kind of work (these jobs definitely exist) was self-taught by experimenting in their own time because they had a passion for performance optimization. It is a rewarding specialization in that there is a quasi-objective measure of incremental progress.
Much of this optimization work requires specific knowledge of the details of the operating environment (e.g. Linux) and silicon microarchitecture (e.g. AVX-512), which can be poorly documented. You have to be comfortable doing experiments and digging into system arcana to surface properties of the system that you can't trivially google. While algorithm selection is important, doing that well is table stakes and part of the role is knowing when and why an "optimal" algorithm is worse than e.g. selectively applied brute-force.
Computational efficiency and throughput is worth a lot of money at scale but most of it is designed and implemented at the scale of a single machine. Every software engineer owns the tools required to become proficient at this. It is genuinely a rare skill even among systems programmers and is a good way to separate yourself from the crowd.
I just became an AVX2 programmer for fun. You can too, if you go to the highload.fun group chat then I or someone else will suggest blazing fast ideas for every problem that happen to be about half as fast as our own approaches.
The FAANGS certainly have people doing this kind of work.
For example, Andrei Alexandrescu has a bunch of keynote talks at CPPCon where he talks about 1-4% speed optimisations he makes to the std library at Facebook.
Various types of hyper-scale or high-performance computing where incremental improvements in throughput, efficiency, latency, and resource utilization saves millions of dollars. Hardware companies are not where you go for this, though they do offer some limited micro-optimization support for the companies that do care about this kind of thing. You want to focus on companies that spend enormous amounts of money on compute infrastructure. The appetite to invest in compute efficiency waxes and wanes with the economy. If companies can get away with throwing money at a problem they will, but that can quickly become untenable.
A closely related area is slightly bending the scaling linearity curve on e.g. big multi-core servers or scale-out systems, so that it is possible to efficiently throw hardware at problems. However, this operates from a pretty different set of theoretical principles than classic performance optimization.
Two domains that have an almost unlimited appetite for improved performance and efficiency right now due to current bottlenecks are AI and sensor processing.
