# Overview

The Demographic Estimation Library is a portable C++ library to estimate the persons' demographic characteristics (age, gedner, etc).

The SDK also has wrappers in the following languages:

• Python
• C# / .NET

• Version 1.0.0 (24 Jan 2024)
  • Initial release

The SDK doesn't have any special hardware requirement:

• CPU: No special requirement, any modern 64 bit capable CPU (x86-64 with AVX, ARM8) is supported
• GPU: No special requirement
• RAM: 2 GB of available RAM required
• Camera: No special requirement, minimum resolution: 640x480.

The SDK is regularly tested on the following Operating Systems:

• Windows 10
• Ubuntu 22.4
• Mac OS 12 Monterey
• iOS 14
• Android 12

While the SDK is released under a proprietary license, the following Open-Source projects where used in it with their respective licenses:

• OpenCV - 3 clause BSD
• Tensorflow - Apache License 2.0
• Protobuf - 3 clause BSD
• zlib - zlib license
• minizip-ng - zlib license
• stlab - Boost Software License 1.0
• docopt.cpp - MIT License
• pybind11 - 3 clause BSD
• fmtlib - MIT License

The public C++ API hides all the implementation details from the user, and it only depends on the C++17 Standard Library. It also provides a binary compatible interface, making it possible to change the underlying implementation without the need of recompilation of the user code.

Extract the SDK contents, include the headers from the include folder and link libDemographicEstimationLibrary to your C++ project.

The python version of the SDK can be installed with pip:

$ pip install realeyes.demographic_estimation

The .NET version of the SDK can be installed via NuGet:

$ dotnet add package Realeyes.DemographicEstimation

The main entry point of this library is the del::DemographicEstimator class.

After a estimator object was constructed, the user can call the del::DemographicEstimator::detectFaces() function to get the faces for a frame of a video or other frame source.

The resulting del::Face objects can be used to call del::DemographicEstimator::estimate() to get the classifiers estimations of the face. The results returned in del::Output structs.

del::DemographicEstimator::detectFaces() and del::DemographicEstimator::estimate() has two versions both are non-blocking async calls, one is returning std::future the other is calling the callback on completion. After one call a subsequent call is possible without waiting for the result.

For the frame data, the user must construct a del::ImageHeader object and pass that to del::DemographicEstimator::detectFaces(). The frame data must outlive this object since it is a non-owning view of the data, but it only needs to be valid during the del::DemographicEstimator::detectFaces() call, the library will copy the frame data internally.

The following example shows the basic usage of the library using OpenCV for loading images and feeding them to the estimator:

#include "demographicestimator.h"

#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>

#include <iostream>

int main()
{
   del::DemographicEstimator estimator("model/model.realZ");

   cv::Mat image = cv::imread("image.jpg");

   std::vector<del::Face> faces = estimator.detectFaces({image.ptr(), image.cols, image.rows, static_cast<int>(image.step1()), del::ImageFormat::BGR}).get();

   for (const del::Face& face: faces) {
       auto results = estimator.estimate(face).get();
       del::BoundingBox bbox = face.boundingBox();
       std::cout << "User at (" << bbox.x << ", " << bbox.y << ", " << bbox.width << ", " << bbox.height << ") has the following estimated characteristics:" << std::endl;

       for (const del::Output& result: results) {
           std::cout << "  " << result.name << " = ";
           if (result.type == del::OutputType::AGE || result.type == del::OutputType::AGE_UNCERTAINTY)
               std::cout << std::get<float>(result.value) << std::endl;
           else
               std::cout << ((std::get<del::Gender>(result.value) == del::Gender::FEMALE) ? "FEMALE" : "MALE") << std::endl;
       }
       std::cout << std::endl;
   }
   return 0;
}

The main entry point of this library is the realeyes.demographic_estimation.DemographicEstimator class.

After a estimator object was constructed, the user can call the realeyes.demographic_estimation.DemographicEstimator.detect_faces() function to get the faces for a frame of a video or other frame source.

The resulting realeyes.demographic_estimation.Face objects can be used to call realeyes.demographic_estimation.DemographicEstimator.estimate() to get the classifiers estimations of the face.

The following example shows the basic usage of the library using OpenCV for loading images and feeding them to the estimator:

import realeyes.demographic_estimation as del
import cv2

estimator = del.DemographicEstimator('model/model.realZ')

image = cv2.imread('image.jpg')[:, :, ::-1]  # opencv reads BGR we need RGB

faces = estimator.detect_faces(image)

for face in faces:
    print(f'User at {face.bounding_box()} has the following estimated characteristics:')
    results = estimator.estimate(face)
    for result in results:
        print(f'  {result.name} = {result.value} ')

The main entry point of this library is the DemographicEstimator class.

After an estimator object is constructed, you can call the DetectFacesAsync() method to detect faces in a frame. The method returns a Task<FaceList> allowing for asynchronous, non-blocking operation.

The resulting Face objects can be used to call EstimateAsync() to get demographic estimations. Both methods support concurrent execution - you can start multiple operations in parallel without waiting for results.

The following example demonstrates parallel processing of multiple frames using the async interface:

using Realeyes.DemographicEstimation;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        using var estimator = new DemographicEstimator("model/model.realZ");

        // Load an image (example using raw byte array)
        byte[] imageData = LoadImageData("image.jpg");
        var imageHeader = new ImageHeader(imageData, 1920, 1080, 1920 * 3, ImageFormat.RGB);

        // Detect faces asynchronously
        await using var faces = await estimator.DetectFacesAsync(imageHeader);

        Console.WriteLine($"Detected {faces.Count} face(s)");

        // Start parallel demographic estimation for all faces
        var estimationTasks = faces.Select(face => estimator.EstimateAsync(face)).ToList();

        // Check concurrent calculations in flight
        Console.WriteLine($"Running {estimator.ConcurrentCalculations} concurrent calculations");

        // Wait for all estimations to complete
        var demographics = await Task.WhenAll(estimationTasks);

        // Process results
        for (int i = 0; i < faces.Count; i++)
        {
            var face = faces[i];
            var result = demographics[i];

            Console.WriteLine($"Face at ({face.BoundingBox.X}, {face.BoundingBox.Y}, " +
                            $"{face.BoundingBox.Width}, {face.BoundingBox.Height}):");

            if (result.Age.HasValue)
                Console.WriteLine($"  Age: {result.Age.Value:F1}");

            if (result.Gender.HasValue)
                Console.WriteLine($"  Gender: {result.Gender.Value}");

            if (result.AgeUncertainty.HasValue)
                Console.WriteLine($"  Age Uncertainty: {result.AgeUncertainty.Value:F2}");
        }
    }

    static byte[] LoadImageData(string path)
    {
        // Implementation depends on your image loading library
        throw new NotImplementedException();
    }
}

The Face objects consist of the following members:

• bounding_box: Bounding box of the detected face (left, top, width height).
• confidence: Confidence of the detection ([0,1] more is better).

• landmarks: 5 landmarks from the face:

  1. left eye
  2. right eye
  3. nose (tip)
  4. left mouth corner
  5. right mouth corner

  

It is possible to calculate the embedding of a face which was detected with a different library. One can create Face object by specifying the source image and the landmarks.