face recognition and skin or similarity problems

edit

This is a followup of my previous question but:

I have 4 trained persons in my database (200 pics each). 1 is of colored (brown) skin type.

If i show a unknown person of similar skin type, it matches, with the same accuracy as a regular good match (no elevation in eigenvalue, so threshold won't do much good. I get the same when i have an trained old person in my database, it will match against a picture shown of a ..old person.

How do i protect against this? Do i need to expand my own trained database more get an increase in eigenvalues? Or do i need to resort to more advanced recognition techniques?

Thank you for reading!