Face detection still being an open problem in computer vision area. While very good algorithms exist, they still fail on some cases or are not fast enough to run in real time. Our implementation is based on Viola-Jones algorithm, one of the most used algorithms for this problem (and also used by OpenCV). This algorithm can be used to track several types of objects just by changing the training data that is used as input. For the face detector, we used training data generated by OpenCV, but we don't know which faces were used to generate it. Perhaps this issue is a matter of changing the training data, or maybe this is a case where the algorithm is not robust enough to handle. We hope we can fix this problem soon, but we still don't know how (suggestions and contributions are appreciated ^.^)

The Viola-Jones technique is still fairly competitive for face detection performance, IF you train it with a lot of data (and of course good data, which means with all the types of variations you expect to see). However, the pre-trained classifier provided with OpenCV is terrible (unless they've updated it recently).

Fortunately, OpenCV provides functions to train your own classifier using your own data, so if you can get good data, and are willing to wait for the week or two it'll take to process, you can get much better performance using the same technique.

Finding sufficient training data (labeled faces) is more tricky. For an automated approach, you could try bootstrapping it by simply running a more state-of-the-art detector (such as on Facebook, Google, etc.) and feedings the outputs into the training function (you'll need to do some work to actually get the outputs programmatically, perhaps through undocumented APIs).

Another approach is to use an existing face dataset. One possibility is the FDDB [1] although it's kind of small (5171 faces). Unfortunately, most other datasets I know of in the face community are either unrealistic images (e.g., only "mugshot"-style) or were detected using OpenCV (and thus won't help you get better).

The final approach is to label images yourself, which would require time and money (e.g., to do it on Amazon Mechanical Turk).

If you do decide to retrain classifiers, it's essential that you augment your data in various ways, such as by mirroring images (and labels) left/right, generating slightly rotated versions of faces, and at slightly different sizes. This augmentation can GREATLY improve the performance of your detector.

[1] http://vis-www.cs.umass.edu/fddb/index.html

Thanks for this great help, apu! A new issue was created on github (https://github.com/eduardolundgren/tracking.js/issues/88) to register the suggestions =)

"While very good algorithms exist ... or are not fast enough to run in real time."

As a research scientist at a company whose main product is face recognition software I have to disagree with this part of the statement.

Are they relevant to this implementation?

What I mean is this is a client-side javascript library, so it is meant to work in the browser on who knows how powerful hardware.

There may be facial recognition libraries that work very well in real time, but how powerful of hardware is required?

And the other big question, are they proprietary algorithms or open-source?

That's really good to hear! The face tracker was one of the first modules to be implemented on tracking.js and, back then, we found Viola-Jones to be the most promising algorithm to make face detection (besides being the one used by our main reference, OpenCV). The whole library was refactored for this release, but we didn't change the algorithm and perhaps it is already out of date. We would be grateful if you could help us by sending papers about better algorithms (and reference implementations?).

I also cannot get it to work. The little bit of instructions there are make no sense so I don't know if I'm doing something wrong or if it's just broken on my computer.

Doesn't work for me either. It does nothing.