Explaining smartphone-based acoustic data in bipolar disorder: : Semi-supervised fuzzy clustering and relative linguistic summaries

Smartphones enable to collect large data streams about phone calls that, once combined with Computational Intelligence techniques, bring great potential for improving the monitoring of patients with mental illnesses. However, the acoustic data streams recorded in uncontrolled environments are dynamically changing due to various sources of uncertainty. In addition, such acoustic data are usually difficult to interpret by psychiatrists. Within this study, we propose an approach based on Linguistic Summaries with Fuzzy Clustering (LS-FC) aiming at the development of human-consistent and easily interpretable summaries about relations between acoustic data and mental state of a patient affected by Bipolar Disorder, e.g., Most calls in the state of hypomania have low loudness compared to the state of euthymia [T = 1]. To capture the dynamics of acoustic data streams, we apply a dynamic incremental semi-supervised fuzzy clustering that synthesizes data into clusters. These clusters are represented by prototypes which are used for the construction of the membership functions describing linguistic terms e.g., low loudness, and then, linguistic summaries. The main contribution of this paper is the incorporation of information about clusters’ prototypes in the generation of linguistic summaries. The primary goal of this research is explainability. The semi-supervised learning algorithm is used mainly for deriving clusters and building improved linguistic summaries. Numerical results indicate that linguistic summaries provide intuitive and clear information about voice features in a patient’s affective state and they are consistent with clinical observation. In particular, during most calls in hypomania/mania both the quality of the patient’s voice and the dynamics of change in the spectrum signal reflected in spectral flux are low compared to euthymia. The proposed approach enables to summarize large data streams into meaningful descriptions that, although relatively simple, offer information granules that are very intuitive for clinicians and are promising to support the smartphone-based monitoring of bipolar disorder patients to inform about the potential change of mental state.