nxp/deepface-emotion-imx

Deepface Emotion

Introduction

Deepface-emotion is a simple Convolutional Neural Network (CNN) for classification of facial expressions into 7 emotion categories. We use the pretrained weights from the DeepFace library.

Note: This is not a face detection model. Faces must be detected and cropped before being passed to this model.

Key Features

• Lightweight CNN optimized for edge deployment
• 7 emotion classes: angry, disgust, fear, happy, sad, surprise, neutral
• Grayscale 48x48 input - low memory footprint
• int8 quantized for accelerated NPU inference on NXP i.MX platforms

Model Description

This repository contains the Deepface emotion model quantized to int8 and compiled for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

• Base Model: serengil/deepface (facial_expression_model_weights.h5)
• Original Model Authors: Sefik Ilkin Serengil
• Original License: MIT
• Modified by: NXP

Modifications

• Quantization: Converted from Keras float32 to TensorFlow Lite int8 (uint8 input, float32 output)
• Compilation: Compiled for i.MX 93 with Ethos-U65 NPU using Vela (lf-6.18.20_2.0.0 and lf-6.18.37_2.1.0)
• Conversion: Converted for i.MX 95 and i.MX 952 with Neutron NPU using eIQ Neutron SDK 3.1.3 and 3.2.1

Model Information

Information	Value
Input shape	Grayscale face crop (1, 48, 48, 1), dtype uint8
Input example	(Image from FER2013 dataset [1])
Output shape	Vector of probabilities shape (1, 7), dtype float32
Output labels	angry, disgust, fear, happy, sad, surprise, neutral
Output example	[[0., 0., 0., 0.996, 0., 0., 0.]] -> Recognized emotion: happy
FLOPS	58.5 MOPS
Parameters	23,497,424
Source framework	TensorFlow / Keras

Version and Changelog

Initial release of quantized int8 model with i.MX 93, i.MX 95, and i.MX 952 compiled variants.

Tested Configurations

The quantized int8 model has been tested on i.MX 8M, i.MX 93, i.MX 95 and i.MX 952 using benchmark-model (see i.MX Machine Learning User Guide).

Training and Evaluation

The model was trained and evaluated on the FER2013 dataset [1]. The original training procedure is described here.

Accuracy on FER2013 PrivateTest split

Evaluation was performed on all 3,589 FER2013 PrivateTest images using the evaluate.py script included in this repository. The FER2013 dataset is a challenging benchmark; a score around 57% is considered strong for this split.

Model	Accuracy
Deepface emotion float32 (Keras)	56.5%
Deepface emotion int8 (TFLite)	56.56%

Conversion/Quantization

The model is converted directly from Keras to TensorFlow Lite int8 using the TFLite converter. The representative calibration dataset is 100 synthetic Gaussian-noise images spanning the 0-255 range (normalized to [0, 1]); no external dataset is required for quantization. This avoids the licensing restrictions of the FER2013 dataset while yielding int8 accuracy within a fraction of a percentage point of the float32 model. The input type is uint8 and the output type is float32.

Download and Run

To recreate the TFLite model (quantized int8 with uint8 input and float32 output), simply run:

bash recipe.sh

The recipe uses a synthetic Gaussian-noise representative dataset for calibration, so no external dataset or Kaggle account is required.

The Vela tag and eIQ Neutron SDK version can be passed as arguments (defaulting to lf-6.18.20_2.0.0 and 3.1.3 respectively):

bash recipe.sh lf-6.18.37_2.1.0 3.2.1

The TFLite model for i.MX 8M Plus is at original_model/emotion_uint8_float32.tflite. The Vela-compiled models for i.MX 93 are in imx93/lf-6.18.20_2.0.0/ and imx93/lf-6.18.37_2.1.0/. The Neutron-converted models for i.MX 95 and i.MX 952 are in imx95/{3.1.3,3.2.1}/ and imx952/{3.1.3,3.2.1}/.

An example of how to use the model is in example.py:

python example.py -m original_model/emotion_uint8_float32.tflite -i example_input.jpg

Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each platform.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/serengil/deepface/

[1] Goodfellow, Ian J., et al. "Challenges in representation learning: A report on three machine learning contests." International Conference on Neural Information Processing. Springer, Berlin, Heidelberg, 2013.