Face recognition systems are ubiquitous today — yet we still do not fully understand how they actually make their decisions. Do they recognise a person by the shape of their face, or by its surface texture? This seemingly simple question has significant consequences for the robustness, fairness, and trustworthiness of deployed biometric systems.
This work addresses the question in a systematic and reproducible way. Using synthetic face pairs with independently controlled 3D geometry and surface texture, the study examines which signal modern face recognition models rely on more heavily.
Method
The experiments use controlled synthetic face pairs in which shape and texture are independently swapped between identities. By comparing model responses to faces with correct texture but incorrect shape against faces with correct shape but incorrect texture, the study directly quantifies the degree of reliance on each signal. Four state-of-the-art face recognition models are included. To determine which facial regions contribute most to the decisions, spatial analysis using occlusion-based heatmaps was applied — separately for shape-driven and texture-driven decisions.
Results
The evaluated models show a tendency to favour texture over shape — a finding consistent with results from general object recognition research, now examined systematically for models specialised in personal identity. Spatial analysis reveals that texture-based recognition relies more heavily on the periocular region, while shape-based recognition is distributed more evenly across the face.
Why it matters
Understanding what information drives model predictions is essential for fairness auditing and system accountability. This work provides a reproducible framework for quantifying and visualising how deep networks process facial identity signals — and paves the way towards building more trustworthy biometric systems.
The paper will be presented IEEE International Conference on Automatic Face and Gesture Recognition (FG 2026) — one of the premier conferences in face recognition research.
Filip Plesko, Tomáš Goldmann, Kamil Malinka, Petr Hanáček Shape vs. Texture: Influence of Geometric Structure and Surface Patterns on Face Recognition IEEE FG 2026 fg2026.ieee-biometrics.org Supported by FIT, Brno University of Technology.
