I was like this but maybe not as stressful as you described. Still, I wanted to do stuff and see stuff during vacation.
After having kids, my habit changed. Now we enjoy going to local parks and walking around with no goals during vacation. This wonderful attraction? Nah we don't need to see it. If we can walk there, then maybe.
When we were planning a trip to Hawaii the first thing I researched was where all the local parks with playgrounds were. That way if we were driving and my son was getting bored or we needed to stretch our legs we knew where something was. There's usually a town, village, or city park nearby no matter where I've been. Even driving to Crater Lake through rural Oregon there were some great parks in the towns we drove through. Same with the drive out to Yosemite. This was the case up until we got within an hour or two of the park itself.
Some of the most enjoyable parts of those trips were hanging out with my family in a local park.
After having kids, my habit changed. Now we enjoy going to local parks and walking around with no goals during vacation. This wonderful attraction? Nah we don't need to see it. If we can walk there, then maybe.