Here I will presents the combination of a new sensor technology based on ultrasounds [3] and a popular deep-learning architecture (the "siamese neural network" [2]) to solve a problem of fingerprint verification. Proposed initially in the 1990's for signature verification [2] and concurrently applied to optical fingerprint verification [1], this architecture has received lately a lot of attention in computer vision. Although it has not yet been adopted by the fingerprint verification community, as shown in this paper, it presents distinctive qualitative advantages over competing methods and its performances can be boosted with appropriate pre-processing, making it attractive for mobile applications.
Biometrics play an increasingly important role in security to ensure privacy and identity verification, as evidenced by the increasing prevalence of fingerprint sensors on mobile devices. However, much remains to be done to improve the accuracy of verification, both in terms of false negatives (due in part to poor image quality when fingers are wet or dirty) and in terms of false positives due to the ease of forgery. The oldest types of optical sensors are easily fooled with a simple photo-copy of a genuine fingerprint. Capacitive sensors are harder to fool (though a simple 3-d printed finger can deceive them) but they get confused by wet fingers. The recently developed ultrasound fingerprint reader [3] addresses the shortcomings of older technologies. It permits reading both the epidermis (surface) and the dermis (below the surface) of the finger. This makes it extremely hard to counterfeit a fingerprint. Furthermore, the technology is insensitive to humidity and dirt on fingers.
The task of fingerprint verification is different from that of classifying or recognizing fingerprints. One or several templates of the genuine fingerprints are usually stored on the device (e.g. smart phone) or in a central repository. The identification procedure requires that the user registers first fingerprint samples; at utilization time, the user (or an attacker) provides examples of fingerprints to be matched with the recorded templates. The verification device carries out a comparison and the result is ``accept" or ``reject". Typically, the matching mechanism returns a score. Varying a threshold on the matching score allows us to monitor the trade-off between false acceptance rate and false rejection rate.
The siamese network deep-learning architecture is a method for computing a matching score with a neural network. Both sides of the bottom layers are identical and preprocess data in an identical way (note the use of shared weights), using several layers of convolutions and pooling. The top layers compute the matching score.
Biometrics play an increasingly important role in security to ensure privacy and identity verification, as evidenced by the increasing prevalence of fingerprint sensors on mobile devices. However, much remains to be done to improve the accuracy of verification, both in terms of false negatives (due in part to poor image quality when fingers are wet or dirty) and in terms of false positives due to the ease of forgery. The oldest types of optical sensors are easily fooled with a simple photo-copy of a genuine fingerprint. Capacitive sensors are harder to fool (though a simple 3-d printed finger can deceive them) but they get confused by wet fingers. The recently developed ultrasound fingerprint reader [3] addresses the shortcomings of older technologies. It permits reading both the epidermis (surface) and the dermis (below the surface) of the finger. This makes it extremely hard to counterfeit a fingerprint. Furthermore, the technology is insensitive to humidity and dirt on fingers.
The task of fingerprint verification is different from that of classifying or recognizing fingerprints. One or several templates of the genuine fingerprints are usually stored on the device (e.g. smart phone) or in a central repository. The identification procedure requires that the user registers first fingerprint samples; at utilization time, the user (or an attacker) provides examples of fingerprints to be matched with the recorded templates. The verification device carries out a comparison and the result is ``accept" or ``reject". Typically, the matching mechanism returns a score. Varying a threshold on the matching score allows us to monitor the trade-off between false acceptance rate and false rejection rate.
The siamese network deep-learning architecture is a method for computing a matching score with a neural network. Both sides of the bottom layers are identical and preprocess data in an identical way (note the use of shared weights), using several layers of convolutions and pooling. The top layers compute the matching score.
Compared to the most prevalent fingerprint verification techniques, which are based on minutiae detection, the siamese network is particularly adequate for the use case we are interested in. Minutiae are points of interest in a fingerprint, such as bifurcations (a ridge splitting into two) and ridge endings. Because methods based on minutiae are very sensitive to occlusions, they do not perform well with the typical sensor areas used in fingerprint verification for smart phone, which do not exceed 1/3 of the total fingerprint area. In contrast, methods based on feature maps, such as convolutional neural networks, are more robust to occlusions. This is one of the motivations towards using the (convolutional) siamese networks, but there is another one. In our scenario, the genuine fingerprint templates may eventually be compromised. Therefore, it is advisable not to store the fingerprints themselves, but a coded representation. The upper layer internal representation of the network offers such coded representation, relatively difficult to invert, and easy to replace if compromised (one just needs to use an alternative version of the siamese network, trained with different initial conditions, and ask the user to provide new samples of fingerprints). This is very much like providing a new password if a password in compromised.
Before inputing the fingerprint images to the network, the images are preprocessed using standard signal processing steps, including: (1) Delimitation of fingerprints and application of enhancement: median filter, contrast enhancement, and other noise reduction; (2) Alignment of the two fingerprints (angle and position); (3) Creation of local orientation, frequency, and variance maps. The maps thus computed are provided to the network. Although the convolutional network is, in principle, capable of computing such maps, the preprocessing steps saves some training effort. To further reduce computational needs, the input maps are reduced to 67-by-67 pixels. In the network, the fingerprints first go through a convolutional layer with 32 filters of size 7-by-7 and moving with a stride of 3, then after ReLU and max pool layer, the output will go through a a convolutional layer with 64 filters of size 5-by-5 and moving with a stride of 1, the output of which go through ReLU, max pool layer and also a sigmoid function. The output from sigmoid function is a size 3-by-3-by-64 structure and the difference between the values of the two sides of the Siamese network should be high when the inputs are not the same fingerprint and low when the inputs are the same fingerprint. The difference is then flattened and goes through two fully connected layers followed by a sigmoid function layer each, and they shrink the size to 1024 and 256, respectively. The final layer is a readout layer and is valued at 0 or 1, indicate non-matching or matching fingerprints, rounded from previous sigmoid layer’s output. The implementation was done in python with TensorFlow to accelerate the computation if Cuda is supported.
To assess our new method, we have assembled a large database of thousands of fingerprints from various public sources, which can be used as training data. Such signatures have been collected with an optical device. Using a model of the ultrasound sensor, we can make them to resemble closely to ultrasound images captured by the fingerprint sensor, of which we have sample images. With a subset of the available data we obtained promising preliminary results that compare favorably to other methods. More testing is under way and will be reported at the workshop.
To assess our new method, we have assembled a large database of thousands of fingerprints from various public sources, which can be used as training data. Such signatures have been collected with an optical device. Using a model of the ultrasound sensor, we can make them to resemble closely to ultrasound images captured by the fingerprint sensor, of which we have sample images. With a subset of the available data we obtained promising preliminary results that compare favorably to other methods. More testing is under way and will be reported at the workshop.
Reference
[1] Pierre Baldi and Yves Chauvin, Neural Networks for Fingerprint Recognition. Neural Comp. 5(3), 1993.
[2] Jane Bromley et al, Signature Verification using a ``Siamese" Time Delay Neural Network. In NIPS 1994.
[3] Hao-Yen Tang et al, 11.2 3D ultrasonic fingerprint sensor-on-a-chip, in IEEE conf. ISSCC Proc. 2016.
[1] Pierre Baldi and Yves Chauvin, Neural Networks for Fingerprint Recognition. Neural Comp. 5(3), 1993.
[2] Jane Bromley et al, Signature Verification using a ``Siamese" Time Delay Neural Network. In NIPS 1994.
[3] Hao-Yen Tang et al, 11.2 3D ultrasonic fingerprint sensor-on-a-chip, in IEEE conf. ISSCC Proc. 2016.