I'll second Griffiths for E&M, Quantum, and Particle. I think everyone that has gone through a physics undergrad will swear by Griffiths. He is just such a delight.
But I'll suggest Marion and Thornton [0] for Classical. With Griffiths for E&M and Thornton for Classical you should be able to get a really good grasp on most of "physics". I'll also say that these series are upper division for undergrads. Given OP's background I think these would be fine starting points. If not, start with Halliday[1].
For grad level, the gold standard is Goldstein for Classical [2] and Jackson for Electrodynamics[3].
These types of books will give you a very strong base in physics and should enable you to branch out. Given your ML background you may be very interested in thermodynamics and statistical mechanics.
I'll also add an "out there" idea. Sign up for physics classes at your local community college if they offer labs. I say this as someone who went Physics -> CS and is doing HPC + ML. Labs really stress and force you to do analysis. Lower division labs won't ask too much, but if you go in with this intent I think you can get a lot out of it. I'm sure that if you did these lower div labs and talked to a professor at your local uni they would allow you to sit in on upper division labs (you would NEED to show that you are serious first, because these labs can be dangerous! Happy to help you learn doesn't mean happy to babysit and make sure you don't electrocute yourself or making sure you don't take a laser to the eye. TAing undergrad labs I saw enough people get electrocuted, including myself... more than once).
But I'll suggest Marion and Thornton [0] for Classical. With Griffiths for E&M and Thornton for Classical you should be able to get a really good grasp on most of "physics". I'll also say that these series are upper division for undergrads. Given OP's background I think these would be fine starting points. If not, start with Halliday[1].
For grad level, the gold standard is Goldstein for Classical [2] and Jackson for Electrodynamics[3].
These types of books will give you a very strong base in physics and should enable you to branch out. Given your ML background you may be very interested in thermodynamics and statistical mechanics.
I'll also add an "out there" idea. Sign up for physics classes at your local community college if they offer labs. I say this as someone who went Physics -> CS and is doing HPC + ML. Labs really stress and force you to do analysis. Lower division labs won't ask too much, but if you go in with this intent I think you can get a lot out of it. I'm sure that if you did these lower div labs and talked to a professor at your local uni they would allow you to sit in on upper division labs (you would NEED to show that you are serious first, because these labs can be dangerous! Happy to help you learn doesn't mean happy to babysit and make sure you don't electrocute yourself or making sure you don't take a laser to the eye. TAing undergrad labs I saw enough people get electrocuted, including myself... more than once).
[0] https://smile.amazon.com/Classical-Dynamics-Particles-System...
[1] https://www.smile.amazon.com/Fundamentals-Physics-10th-David...
[2] https://smile.amazon.com/Classical-Mechanics-Goldstein-Poole...
[3] https://smile.amazon.com/Classical-Electrodynamics-Third-Dav...