I don't think you understood the author's interpretation well enough to say you disagree with it.
The problem you describe, of mapping binocular vision is an example of the topological problem described by the author. But it's not the only example: feeling nerves, for example, have the same symmetrical problem with mapping your skin sensations to the physical space, and your hearing also is "binocular" (there's actually a separate word for this, "binaural"). The visual problem you're describing is part of the topological problem. You're describing the same problem, but you're describing a part of the problem.
Where you're just wrong is on two points:
1. "When it comes to everything else, there isn't a clear benefit for having processing swapped to opposite hemispheres." Wrong. As mentioned before, binaural hearing also needs to map a 3d topology to a 2d topology from two data collection points, and skin needs to map a 3d topology from many more points (but also symmetrical). Additionally, the effect works on "outputs" as well as "inputs", mapping the 2d space to a 3d space so that you can control symmetrical tools such as your arms and legs means that the swapping is needed when sending signals outward as well.
2. "[V]isual processing benefits from it greatly, and so the rest of the nervous system goes along with it." Vision is not the evolutionary driver here. The criss-crossed neurology predates the existence of vision in our evolutionary heritage. The article mentions worms, for example: how does your hypothesis explain why their nerves crossed hemispherically given they don't have lenses and retinas?
The problem you describe, of mapping binocular vision is an example of the topological problem described by the author. But it's not the only example: feeling nerves, for example, have the same symmetrical problem with mapping your skin sensations to the physical space, and your hearing also is "binocular" (there's actually a separate word for this, "binaural"). The visual problem you're describing is part of the topological problem. You're describing the same problem, but you're describing a part of the problem.
Where you're just wrong is on two points:
1. "When it comes to everything else, there isn't a clear benefit for having processing swapped to opposite hemispheres." Wrong. As mentioned before, binaural hearing also needs to map a 3d topology to a 2d topology from two data collection points, and skin needs to map a 3d topology from many more points (but also symmetrical). Additionally, the effect works on "outputs" as well as "inputs", mapping the 2d space to a 3d space so that you can control symmetrical tools such as your arms and legs means that the swapping is needed when sending signals outward as well.
2. "[V]isual processing benefits from it greatly, and so the rest of the nervous system goes along with it." Vision is not the evolutionary driver here. The criss-crossed neurology predates the existence of vision in our evolutionary heritage. The article mentions worms, for example: how does your hypothesis explain why their nerves crossed hemispherically given they don't have lenses and retinas?