Funding everything with taxes concentrated power in a government that then becomes a honeypot to be abused by the wealthy. I absolutely do not think that is a good way forward.
It doesn't have to be either/or. You can have a government that's only big enough to keep corporations in check (e.g. by actively busting anything that gets too large), but no bigger than that.
It's reasonable to worry about the wealthy abusing their position in government, but there should be separate fixes for that. It's not like we worry about the power-hungry trying to take over the military in modern societies.
Also, I don't believe in high taxes across the board and funding everything with the government. I primarily want the state to stop arms races. It's not like I want them to take over shoe manufacturing, for example.