I think the annoyance with TikZ is twofold: (1) it tries to do a really hard thing (create a picture with text in a human writable way), (2) it is used infrequently enough that it’s hard to learn through occasional use.
That said, nobody makes you use TikZ, fire up Inkscape and do it wysiwyg.
I mean it is one of the few packages that can actually manage to annoy LaTeX fans, which is really saying something.