I'd go further than that. Leave work at work. Don't check work email out of the office. Don't take work home: stay late if you must.

This is harder to manage if "the office" is actually your home but it's important to make it work.

You can't really help where your brain wanders in your downtime, but there's no need to encourage it by doing work in your downtime.