What stumbles me is information like the following [1]
> Indeed, making the much bigger model, the GPT-3 with 175 billion parameters, emitted over 550 tons of CO2e while consuming 1,287 MW hours of electricity, per computer scientist Kate Saenko. It’s the same amount of emissions as a single person taking 550 roundtrip flights between New York and San Francisco.
Specifically, I struggle with the interpretation where this is presented as significant amount. To me, training GPT-3 at the cost of about a SINGLE New York-San Francisco return flight is practically negligible. I wonder whether for example at a conference like NeurIPS the carbon cost of all the experiments is exceeded by the carbon cost of participant travel to the conference. My guess is that probably yes.
You need to look at it by sub industries in industries. That's more closer to the logical perspective people are acting on.