Impacts of facial image distortion on verification accuracy in the context of the Entry/Exit System

Abstract

Face recognition is more and more utilized in the border checks in the external borders of the Schengen area. The whole ecosystem of border checks conducted in the European Union is under a great renewal. One of the first components to be operational will be the Entry/Exit System (EES) which will record the facial images of third country nationals at their first entry to the Schengen area to be used for biometric verification for their subsequent entries and exits to and from the Schengen area.

Mobile devices are already used in border checks done by border authorities worldwide. Led by European Border and Coast Guard Agency (Frontex), there has been official interest to utilize mobile devices for the pre-registration of the passenger into the EES done by the passengers themselves. In close-up photographs taken with a mobile phone's camera, such as "selfies", the camera may cause distortion of the facial characteristics in the image.

This study presents the results of the feasibility of the mobile phones’ cameras for capturing facial images for biometric verification in the context of the EES. The influence of the close distance facial images on the verification accuracy of the facial images is studied as well.

The quality of the facial images is assessed by specific OFIQ (open-source facial image quality) toolset and the verification of the facial images captured for the study are done in DeepFace, which utilizes ten different state-of-the-art face recognition models.

The images have been collected in the evaluation study and the analysis of the data are done following the quantitative research method. Before the evaluation study, the related legislation and standards are presented and analysed, and the principles of the face recognition technology are presented.

The measurement of the magnification distortion revealed that mobile phones’ cameras are slightly more robust against magnification distortion than the system cameras, especially when capturing images from short camera to subject distances. Cosine distances between the facial image pairs were compared and the results were analysed between the camera types. Based on the results of this study, mobile phones’ cameras are capable to capture facial images for biometric verification.