In a face database containing multiple images per subject, how do we determine the face image which is most suited for face recognition?

Question

Let us imagine a face database with several subjects, each subject having multiple face images. How do we determine which is the best face suitable for face recognition purposes?

Answer 1

This should not matter that much as long as you do enough preprocessing?

But one could use e.g. the Facenet architecture to extract the embedding vector for each face of the same subject.

Then you could compute the covariance matrix and perform PCA on it. This would give you the most significant features with which you can decide which face to take.

Alternatively, you could do something in the direction of the eigenfaces: https://en.wikipedia.org/wiki/Eigenface.