Wait, I've done it the other way around. I have a repo directory, and I created symlinks in ~/ pointing to the dotfiles in the repo. I have a simple install script to symlink the files.

Would you suggest having the physical files in ~/ and adding their symlinks to a repo?