They were not even in hurry. If that was important for the viewers it should be even more important to the characters. If this was not important and they do not care then the whole premise of traumatic Vecna memory was useless pile of utter garbage because memory cannot be traumatic and not important. Ever had trauma doing grocerries? Or riding a cab? And then they should not be able to escape. Because he would get them before that.

They just invent stuff that they have no idea how to explain later. Just like Lost.