Speech Communication

Volume 99, May 2018, Pages 27-46
Speech Communication

Automatic segmentation of speech articulators from real-time midsagittal MRI based on supervised learning


Automatic articulators’ segmentation is based on a modified Active Shape Model (mASM).

A large corpus of real-time MRI midsagittal images was recorded for a French speaker.

Articulatory contours were manually segmented on a small subset of the corpus.

The mASM trained on these image and contour data was used for automatic segmentation.

Overall Mean Sum of Distances of the automatic segmentation error was 0.55 mm.


Speech production mechanisms can be characterized at a peripheral level by both their acoustic and articulatory traces along time. Researchers have thus developed very large efforts to measure articulation. Thanks to the spectacular progress accomplished in the last decade, real-time Magnetic Resonance Imaging (RT-MRI) offers nowadays the advantages of frame rates closer than before to those achieved by electromagnetic articulography or ultrasound echography while providing very detailed geometrical information about the whole vocal tract. RT-MRI has thus become inescapable for the study of speech articulators’ movements. However, making efficient use of large sets of images to characterize and model speech tasks implies the development of automatic methods to segment the articulators from these images with sufficient accuracy. The present article describes our approach to develop, based on supervised machine learning techniques, an automatic segmentation method that offers various useful features such as (1) capability of dealing with individual articulators independently, (2) ensuring hard palate, jaw and hyoid bone to be adequately tracked as rigid structures, (3) delivering contours for a full set of articulators, including the epiglottis and the back of the larynx, which partly reflects the vocal fold abduction / adduction state, (4) dealing more explicitly and thus more accurately with contact between articulators, and (5) reaching an accuracy better than one millimeter.

The main contributions of this work are the following. We have recorded the first large database of high quality RT-MRI midsagittal images for a French speaker. We have manually segmented the main speech articulators (jaw, lips, tongue, velum, hyoid, larynx, etc.) for a small training set of about 60 images selected by hierarchical clustering to represent the whole corpus as faithfully as possible. We have used these data to train various image and contour models for developing automatic articulatory segmentation methods. The first method, based on Multiple Linear Regression, allows to predict the contour coordinates from the image pixel intensities with a Mean Sum of Distances (MSD) segmentation error over all articulators of 0.91 mm, computed with a Leave-One-Out Cross Validation procedure on the training set. Another method, based on Shape Particle Filtering, reaches an MSD error of 0.66 mm. Finally the modified version of Active Shape Models (mASM) explored in this study gives an MSD error of a mere 0.55 mm (0.68 mm for the tongue). These results demonstrate that this mASM approach performs better than state-of-the-art methods, though at the cost of the manual segmentation of the training set. The same method used on other MRI data leads to similar errors, which testifies to its robustness. The large quantity of contour data that can be obtained with this automatic segmentation method opens the way to various fruitful perspectives in speech: establishing more elaborate articulatory models, analyzing more finely coarticulation and articulatory variability or invariance, implementing machine learning methods for articulatory speaker normalization or adaptation, or illustrating adequate or prototypical articulatory gestures for application in the domains of speech therapy and of second language pronunciation training.


Real-time MRI
Speech articulation
Articulator segmentation
Multiple Linear Regression
Active Shape Models
Shape Particle Filtering


Active Appearance Models
Active Shape Models
ElectroMagnetic Articulography
frames per second (also images per second)
Histogram of Oriented Gradients
modified Active Shape Models
Multiple Linear Regression
Magnetic Resonance Imaging
Mean Sum of Distances
Principal Components Analysis
Root Mean Square
Region of Interest
real time MRI
Shape Particle Filtering
Speed-Up Robust Features

This article presents an extension of work published in Labrunie et al. (2016b)