Time-series clustering with jointly learning deep representations, clusters and temporal boundaries

Clustering and segmentation of temporal data is an important task across several fields, with prominent applications in computer vision and machine learning such as face and gesture segmentation. Several related methods have been proposed in literature, focusing on learning temporal boundaries and clusters, with recent works focusing on learning deep representations for clustering. However, none of the proposed methods is suitable for jointly learning segments, clusters, as well as representations. In this paper, we propose the first methodology that simultaneously discovers suitable deep representations, as well as clusters and temporal boundaries, with the clustering process providing supervisory cues for updating temporal boundaries and training the proposed deep learning architecture. We demonstrate the power of the proposed approach on a human motion segmentation task using the CMU-MMAC database. Our method provides the best results with respect to normalized mutual information compared to other clustering algorithms.

Tzirakis Panagiotis, Nicolaou Mihalis A., Schuller Björn, Zafeiriou Stefanos

Publication type:
A4 Article in conference proceedings

Place of publication:
14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019, 14-18 May 2019, Lille, France

6G Publication


Full citation:
P. Tzirakis, M. A. Nicolaou, B. Schuller and S. Zafeiriou, “Time-series Clustering with Jointly Learning Deep Representations, Clusters and Temporal Boundaries,” 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France, 2019, pp. 1-5. doi: 10.1109/FG.2019.8756618


Read the publication here: