I think if anyone really doubts this they should spend some time programming something directly on top of Xorg to convince themselves that when the ball of mud is gone there will be at least a bit more sanity.
I haven't used the raw stream protocol directly, but if you mean Xlib, I actually find it more usable than most toolkits. (Although a big part of that is being able to use select(), whereas most toolkits force you to do everything in callbacks while the toolkit drives the main loop.)
