What's in a Face? Metric Learning for Face Characterization

Please reload

Omry Sendik, Dani Lischinski and Daniel Cohen-Or

Tel Aviv University

We present a method for determining which facial parts (mouth, nose, etc.) best characterize an individual, given a set of that individual's portraits. We introduce a novel distinctiveness analysis of a set of portraits, which leverages the deep features extracted by a pre-trained face recognition CNN and a hair segmentation FCN, in the context of a weakly supervised metric learning scheme. Our analysis enables the generation of a polarized class activation map (PCAM) for an individual's portrait via a transformation that localizes and amplifies the discriminative regions of the deep feature maps extracted by the aforementioned networks.
A user study that we conducted shows that there is a surprisingly good agreement between the face parts that users indicate as characteristic and the face parts automatically selected by our method.
We demonstrate a few applications of our method, including determining the most and the least representative portraits among a set of portraits of an individual, and the creation of  facial hybrids: portraits that combine the characteristic recognizable facial features of two individuals. Our face characterization analysis is also effective for ranking portraits in order to find an individual's look-alikes (Doppelgangers).

Please reload


title={What’s in a face? Metric Learning for Face Characterization}, author={Sendik, Omry and Lischinski, Dani and Cohen-Or, Daniel}, booktitle={Eurographics 2019, the 40th Annual Conference of the European Association for Computer Graphics}, pages={13}, year={2019}}