Mobai (together with Steinbeis Transfer Centre BISE) have been awarded a contract with European Union Agency for the Operational Management of Large-Scale IT Systems in the Area of Freedom, Security and Justice (eu-LISA) to generate representative artificial data for fingerprints (FP) and facial images (FI), and the provisioning of biometric data quality requirements for the curation of test datasets. eu-LISA has started the implementation of the European Entry-Exit Systems’ biometric components. During its use, the European Entry-Exit System will capture and verify biometric data of travellers to the Schengen area at all border crossing points as well as by visa and law enforcement authorities. The Steinbeis Network is dedicated to the transfer of academic findings and knowledge into applications. Steinbeis Transfer Centre BISE is focused on biometrics under the directorship of Prof. Dr. Christoph Busch at Hochschule Darmstadt. Mobai AS is a spin-off company from the Norwegian University of Science and Technology (NTNU) commercializing technology developed at the Norwegian Biometrics laboratory. In this contract Mobai collaborates with experts from NTNU and Paris Lodron University of Salzburg (PLUS) to ensure expert quality within the field of face and fingerprint biometrics. The European Union Agency for the Operational Management of Large-Scale IT Systems in the Area of Freedom, Security and Justice (eu-LISA), operates large-scale IT systems for the implementation of the asylum, border management and migration policies of the EU. Details: This contract covers the need of evaluating large scale synthetically generated biometric (fingerprint and face) databases with the intention to understand to which extent they could be used for biometric performance (accuracy) tests as well as solutions for tailoring of these datasets to make them fit for purpose. The main hypothesis to contrast is to which extent synthetic samples could replace the need for real images for biometric performance tests of the large scale IT systems. eu-LISA is responsible for the operational management of three large scale IT systems, namely Visa Information System (VIS), EURODAC and the Schengen Information System (SIS II). The Agency currently manages Eurodac, the second generation of the Schengen Information System (SIS II) and the Visa Information System (VIS). Further to these, eu-LISA is developing the Entry/Exit System (EES), the European Travel Information Authorization System (ETIAS) and the European Criminal Records Information System - Third-Country Nationals (ECRIS-TCN). Background and objectives: As part of the Smart Borders program, according to Regulation (EU) 2017/2226 of the European Parliament and of the Council establishing an Entry/Exit System (EES). While the three systems currently operated and maintained by eu-LISA (VIS, SISII and EURODAC) include biometric operations based on fingerprint data, EES is a unique project due to its large size, very high performance requirements and the combined use of fingerprints and facial images on all external border crossing points of the Schengen Area. Therefore pre-go-live biometric performance testing requires changes to the old testing methods as well as new datasets that are representative enough to the future Production environment in terms of both data characteristics and size. The outcome of this [...]
About Brage StrandThis author has not yet filled in any details.
So far Brage Strand has created 2 blog entries.
The National Institute of Standards and Technology (NIST, part of U.S. Department of Commerce) conducts large scale testing of face recognition software in their Face Recognition Vendor Test (FRVT). One of the core tests is the FRVT 1:1 Verification, where algorithms are evaluated on multiple large-scale datasets in how good they perform on 1:1 facial verification. What is 1:1 verification? It is the scenario where a person claims to be a certain individual, such as the owner of a specific passport or a specific user requesting access to a system with facial biometrics. The system then performs the 1:1 facial verification by capturing a new face sample and compares it against the reference face sample stored in the passport document or in the system. For example, the automatic border gate compares the new image against the facial image from your passport, or your phone compares a new selfie against the stored data when you unlock it. In NIST FRVT 1:1 the evaluation is conducted on a large series of individuals in datasets with images collected from Visa, Mugshot, Wild and Border control scenarios. Below you can see some of the sample photos from the NIST datasets: In Mobai we are very proud of being a top contender in the NIST FRVT 1:1, and especially for the results on the Mugshot and Visa-Border datasets. The Visa-Border dataset is one of the more interesting scenarios since the comparisons consist of images from a Visa-document, such as a passport and then that is compared to actual border crossing images. This test is a good representation for border control or (self-service) identity verification use cases. The Mobai results can be found at https://pages.nist.gov/frvt/reportcards/11/mobai_001.html The test results are illustrated in graphs showing the relationship between False non-match rate (FNMR) and False match rate (FMR). The FMR indicates the proportion of comparisons between two different persons that is falsely considered as a match. The FNMR indicates the proportion of comparisons between the same person that is falsely considered to not match. You can think of FMR as the security dimension; how large is the risk that a person will be able to successfully generate match as someone else, and FNMR as the usability dimension; what is the likelihood that a person will not be considered a match when a comparison is performed against their own reference image. Above you can see the results for Mobai on the Visa-Border datasets, where we obtain 0,0093 FNMR @ FMR 1e-06, which is extremely low and only 0,006 behind the leader. To rephrase the results a bit: when the security dimension (FMR) is fixed at a level where only 1 person out of 1 000 000 is wrongly matched against another person in the dataset, then our system wrongly rejects the correct person approximately 1 time out of 100 attempts. If we compare this result to human performance in face recognition, we see that the results are significantly better in automated systems, as studies show the FMR or the chance [...]