The configuration would be saved in Flash memory and loaded at boot time. Also, you'd need to reboot the FPGA each time a new configuration would be written in.

I am not yet aware of FPGA-like chip architecture that can be reconfigured dynamically.