One problem I see with this: Some people are going to see buttons that are identical in style to Twitter follow buttons, and assume that clicking it will "Watch" the repo.

For some subset of the population, they'll either not notice they didn't actually Watch the repo (and hence be a lost Watcher for your project), or they'll notice, be annoyed, and assume its a bug.