You're creating extra cognitive load for people. If there's one thing behavioral psychology has reliably taught us (in spite of the repeatability issues on a few findings), it is that our brains are wired to avoid cognitive load as much as possible.
The cost of putting cognitive load is many people will choose not to pay it and that cost is instead borne by your partner
Would you walk through wet mud if a grass or concrete alternative path is available? That's pretty much what your contacts are doing
Another metaphor: if someone decided not to pave their driveway, because they don't want it to get so hot in the summer, would you tell them, hey I'm not going to visit you until you pave your driveway?
In practical life, people who chose to live in less accessible places actually don't get visited much except by very close relatives & friends (I can think of 4-5 of my relatives like this). So probably proves the point :-)
The cost of putting cognitive load is many people will choose not to pay it and that cost is instead borne by your partner
Would you walk through wet mud if a grass or concrete alternative path is available? That's pretty much what your contacts are doing