I'm doubtful about that. For one thing large amounts of left wing areas voted for Brexit including Wales as a whole. Also the Tory leadership at the time was pro-remain whereas the Labour leadership was lukewarm at best.
Yes, dividing the brexit issue as a left/right issue is very reductive. Both Labour and Tories were very much divided in half with pretty obvious internal conflicts.