I have trained FisherFaces algorithm with about 50 images for two individuals. It does recognize them accurately, but it also recognizes others as them.
I've tried to use the threshold value as an indicator but unfortenately I get also accurate predictions (small distance) on these samples also.
In order to try and battle with this issue I've added some pre processing:
- decreased the face area crop on both training and test images
- brightened the test images by x scalar (30 was the best value)
I have also considered training a new category with the "unknown" label with random people as the training images.