People who would get offended if you called their kid ugly are not the kind of people you'd want to spend time with? That's basically every person on earth.
Thinking that something is not beautiful is not the same thing as thinking that something is ugly. And there is a pretty obvious (though not to you, apparently) difference between being asked that question and me responding honestly, and me simply remarking, unprovoked, "boy, that kid sure looks pretty damn ugly, don't he?"
The kind of people I want to spend my time around are those who prefer honest questions and honest answers, as opposed to those simply soliciting me for a bit of verbal fellatio.