Micro-expression recognition with small sample size by transferring long-term convolutional neural network

Abstract

n

Micro-expression is one of important clues for detecting lies. Its most outstanding characteristics include short duration and low intensity of movement. Therefore, video clips of high spatial-temporal resolution are much more desired than still images to provide sufficient details. On the other hand, owing to the difficulties to collect and encode micro-expression data, it is small sample size. In this paper, we use only 560 micro-expression video clips to evaluate the proposed network model: Transferring Long-term Convolutional Neural Network (TLCNN). TLCNN uses Deep CNN to extract features from each frame of micro-expression video clips, then feeds them to Long Short Term Memory (LSTM) which learn the temporal sequence information of micro-expression. Due to the small sample size of micro-expression data, TLCNN uses two steps of transfer learning: (1) transferring from expression data and (2) transferring from single frame of micro-expression video clips, which can be regarded as “big data”. Evaluation on 560 micro-expression video clips collected from three spontaneous databases is performed. The results show that the proposed TLCNN is better than some state-of-the-art algorithms.

Authors:
Wang Su-Jing, Li Bing-Jun, Liu Yong-Jin, Yan Wen-Jing, Ou Xinyu, Huang Xiaohua, Xu Feng, Fu Xiaolan

Publication type:
A1 Journal article – refereed

Place of publication:

Keywords:
convolutional neural network, Deep learning, micro-expression, transferring learning

Published:

Full citation:
Su-Jing Wang, Bing-Jun Li, Yong-Jin Liu, Wen-Jing Yan, Xinyu Ou, Xiaohua Huang, Feng Xu, Xiaolan Fu, Micro-expression recognition with small sample size by transferring long-term convolutional neural network, Neurocomputing, Volume 312, 2018, Pages 251-262, ISSN 0925-2312, https://doi.org/10.1016/j.neucom.2018.05.107.

DOI:
https://doi.org/10.1016/j.neucom.2018.05.107

Read the publication here:
http://urn.fi/urn:nbn:fi-fe2019042913501