I think it's only possible to up vote comments after a day or so, and no longer down vote. So when you say something controversial, the first hours you get a lot of both up- and down votes. But after a while, you only get the up votes.