The term for this is federated learning. Usually it’s used to preserve privacy since a user’s data can stay on their device. I think it ends up not being efficient for the model sizes used here.