The usual life cycle of high-rise luxury housing is that buildings move somewhat downmarket as they age. The bulk of your housing stock downtown is going to be 30-40 year old buildings that can't carry the premium rents of new luxury buildings. But you won't get to that point if you don't start building now.
This is a good point and one worth making. Unfortunately neither politicians nor renting residents look that far afield to the detriment of the area and future residents.