You'd need stereo fotos from a low flying plane for this. Several cities are doing this for decades already. I've built such a 3D map for my city like 25 years ago.
Are you referring to the outside seating polygons? Wouldn’t these stereo images still have a lot of noise (trees, cars, trucks, smaller non-building structures) obstructing the target areas?
