Do you have to figure this out? Sure, it's nice and "pure" if everything is configured through Nix but there is something to be said about being pragmatic. Personally, I just enabled nix-ld[0] and use uv to install and handle my Python versions and package dependencies. Much, much easier.
Easier and largely compatible with the rest of the world. Solving problems with "If we all switched to NixOS..." is a non-starter in most organizations.
My rule of thumb: keep a strict separation between my projects (which change constantly) and my operating system (which I set up once and periodically update). Any hard nix dependency inside the project is a failure of abstraction IMO. Collaborating with people on other operating systems isn't optional!
In practice this means using language-specific package management (uv, cargo, etc) and ignoring the nix way.
[0]: https://mynixos.com/nixpkgs/option/programs.nix-ld.enable