Face recognition¶

Face recognition is the process of identifying or verifying a person’s identity using their facial features. The id3 Face SDK implements face recognition through a complete pipeline consisting of three main steps:

Face detection: Locating and extracting faces from an image (see Face detection)
Face encoding: Converting facial features into a compact digital representation called a FaceTemplate
Face matching: Comparing face templates to verify identity or identify individuals

This tutorial covers the face encoding and matching steps required to build a complete face recognition system.

Face encoding¶

Face encoding is the process of extracting distinctive facial features from a DetectedFace Class and converting them into a mathematical representation. The output is a FaceTemplate Class that can be stored and later used for comparison.

Important

Face encoding requires an Face Encoder AI model to run. See AI Models for details.

def extract_template(image: id3.Image) -> id3.FaceTemplate:
    """
    face_encoder: id3.FaceEncoder = ... # Previously initialized encoder module
    """
    detected_face = detect_face(image)
    return face_encoder.create_template(image, detected_face)

Face template¶

A face template is a compact biometric representation of a face, typically 140 bytes or 264 bytes depending on the encoding model used. Face templates offer several advantages:

Compact storage: Requires minimal space compared to storing full face images
Privacy-preserving: Cannot be reversed to reconstruct the original face image
Fast comparison: Enables rapid matching against large databases
Portable: Can be exported to formats compatible with id3 Face Match-on-Card solutions

Encoding quality¶

Starting from version 10, face templates contain a quality score, indicating how well the template extracted will perform for recognition. Low scores reflect a bad image quality which will likely increase the risk a non-match. Quality score should help you :

Reject poor-quality enrollments before storing them
Prompt users to provide better images during enrollment
Select highest-quality frame when live-tracking

Hint

For optimal recognition accuracy: - Ensure good lighting conditions during capture - Maintain a ideal minimum interocular distance (IOD) of 64 pixels - Capture frontal faces with minimal pose variation (yaw < ±45°, pitch < ±30°) - Avoid occlusions like sunglasses, masks, or hair covering the face

Face matching¶

Face matching (also called face comparison) is the process of comparing two face templates to determine if they belong to the same person. The FaceMatcher module compares templates and returns a similarity score, representing the degree of similarity between two templates:

Higher scores: Greater similarity, more likely to be the same person
Lower scores: Less similarity, more likely to be different people

Understanding the trade-off¶

The decision threshold is a critical parameter that determines when two templates are considered a match. By setting this threshold, you control the balance between security and usability in your system. The decision threshold directly impacts two error rates:

False Match Rate (FMR): The probability that the system incorrectly accepts an impostor as a genuine user
False Non-Match Rate (FNMR): The probability that the system incorrectly rejects a genuine user

These rates have an inverse relationship:

Higher threshold: Better security (lower FMR) but more genuine users rejected (higher FNMR)
Lower threshold: Better user experience (lower FNMR) but more impostors accepted (higher FMR)

Threshold selection table¶

Choose a threshold based on your required security level:

False Match Rate	Decision threshold
1.0%	2000
0.1%	3000
0.01%	4000
0.001%	5000
0.0001%	6000
0.00001%	7000
0.000001%	8000
0.000001%	9000
0.0000001%	10000

Threshold recommendations by use case¶

Consumer applications (mobile unlock, photo tagging): 2000-3000 (FMR 0.1%-1%)
Enterprise access control (building entry, workstation login): 4000-5000 (FMR 0.001%-0.01%)
Financial transactions (payment authorization): 6000-7000 (FMR 0.00001%-0.0001%)
Law enforcement (criminal identification): 8000-10000 (FMR ≤ 0.000001%)

Warning

These threshold are calibrated for 1-to-1 identification. When performing N matches to search a user in database, the False-Match risk will accumulate and cause security issues ! See 1-to-N Face search for a correct 1-to-N implementation

Hint

To obtain a value between 0 and 100 (for a more understandable user feedback), we recommend applying the following formula to the matching score (S):

\(S' = MIN(100, S * 8 / 250)\)

Which model and template should I use in which scenario ?¶

Depending on your use-case and your available hardware, we provide different AI models. From version 10, all templates extracted by these models or compatible and can be cross matched. We suggest the following matching scenarios:

Enrolment	Probe	Scenario
10A	10A	Maximize performance. Recommended for all desktop use.
10B	10B	Best performance tradeoff for mobile/embedded devices.
10A	10B	Slightly better performance than 10B vs 10B. Recommended for scenarios where enrolment is performed on desktop, and verification on mobile, but keeping the possibility for best performance verification (adjudication, highest security use-cases, etc).
10A_Compact	10B	Identical to 10A vs 10B, except that when only matching with smaller templates, 10A template can be optimized for size. Recommended if you never plan to verify against another 10A template.
10C	10C	Use for realtime tracking in FaceTracker. NOT RECOMMENDED FOR SECURITY DECISIONS.
10A/10B	10C	Use for realtime tracking, to identify a realtime probe in a pre-enrolled database of 10A/10B templates. NOT RECOMMENDED FOR SECURITY DECISIONS.

Important

Using legacy 9.x models, templates are not cross-matchable. Only templates with identical format can be matched

Complete face recognition example¶

The following example demonstrates a complete face recognition workflow including detection, encoding, and matching:

import id3face as id3

# Initialization
id3.FaceLibrary.load_model("models", id3.FaceModel.FACE_DETECTOR_4B, id3.ProcessingUnit.CPU)
id3.FaceLibrary.load_model("models", id3.FaceModel.FACE_ENCODER_10B, id3.ProcessingUnit.CPU)

face_detector = id3.FaceDetector(thread_count=2)
face_encoder  = id3.FaceEncoder(model=id3.FaceModel.FACE_ENCODER_10B)
face_matcher  = id3.FaceMatcher()

# Placeholder functions
def load_reference_template(user_name: int) -> id3.FaceTemplate:
    return id3.FaceTemplate.from_file(f"data/ref_{user_name}_10A.bin")
def capture_image() -> id3.Image:
    return id3.Image.from_file("data/placeholder.jpg", id3.PixelFormat.BGR_24_BITS)

# Computation functions
def detect_face(image: id3.Image) -> id3.DetectedFace:
    image_downscaled = image.clone()
    ratio = image_downscaled.downscale(256)
    detected_face_list = face_detector.detect_faces(image_downscaled)
    face = detected_face_list.get_largest_face()
    face.rescale(1/ratio)
    return face

def extract_template(image: id3.Image) -> id3.FaceTemplate:
    """
    face_encoder: id3.FaceEncoder = ... # Previously initialized encoder module
    """
    detected_face = detect_face(image)
    return face_encoder.create_template(image, detected_face)


# Application loop
while True:

    # Capture image and extract template
    try:
        image = capture_image()
        face_template = extract_template(image)
    except id3.FaceException as ex:
        print(ex)
        continue
    
    # Check on quality before continuing
    if face_template.quality < 60:
        print("Face quality too low. Please try again.")
        continue
    
    # Retrieve which user to compare with (get identifier, read ID Document, form, etc)
    user_name = input("What's your name? ")
    reference_template = load_reference_template(user_name)

    # Compute match and take decision
    score = face_matcher.compare_templates(face_template, reference_template)
    if score >= id3.FaceMatcherThreshold.FMR10000:
        print("Access granted.")
        break
    else:
        print("Access denied. You do not match with " + user_name)