I don't even think most Americans (except those trying to do the automating) would consider it to be fair.

AI is bias automation, and reflects the data it's trained on. The vast majority of training data is biased, even against different slices of Americans. The resulting AI will be biased.