Exact copies aren't just about precision but also about reproducibility.

You can keep your weights in a discrete format for storage, but do inference and training in analog.

That only prevents analog copy degradation. It doesn't give you reproducibility. Reproducibility means running the same process twice with the same inputs and getting the same outputs. E.g. to later prove that something came from an LLM and not a human you could store the random seed and the input and then reproduce the output. But that only works if the network is digital.