Browsing by Author "Chen, Wenwan"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    AmbianceCount: An Objective Social Ambiance Measure from Unconstrained Day-long Audio Recordings
    (2020-12-07) Chen, Wenwan; Sabharwal, Ashutosh
    Measuring social ambiance in unconstrained environments is of significant importance in mental health due to the association between sociability and psychological outcome. However, it has been challenging to quantify social ambiance since existing objective methods fail to capture the transient ambiance patterns in unconstrained environments. In this thesis, I present AmbianceCount, an automatic and objective method that extracts social ambiance from unconstrained audio recordings by estimating the number of concurrent speakers. AmbianceCount consists of a supervised deep neural network (DNN) embedding extractor to differentiate speech mixtures, and a scoring system for estimation and improving generalization. The performance of Am- bianceCount is compared with baseline and evaluated on several synthesized datasets. Lastly, I utilize AmbianceCount to evaluate data from a sociability pilot, with audio data from depression and psychosis patients as well as age-matched healthy controls. Our analysis shows that extracted social ambiance patterns are significantly different across three groups. Besides, it is observed that captured social ambiance patterns are associated with psychometric and personality scores, which is consistent with clinical diagnosis.
  • Thumbnail Image
    ItemEmbargo
    Objective Speech-Based Sociability Measure for Mental Health Assessment
    (2024-04-17) Chen, Wenwan; Sabharwal, Ashutosh
    Sociability measures play a pivotal role in mental health assessment due to their associations with mood and symptoms of mental disorders. The conventional approach for assessing sociability relies on self-reports, often biased and error-prone. To address these limitations, I propose Ambiance-aware Social Interaction Measure (ASIM), an objective, fine-grained and comprehensive sociability measure. ASIM captures both individual social interactions and social ambiance – the environmental elements impacting social engagements – through the analysis of unconstrained audio data. Specifically, ASIM is structured as an 8-dimensional vector, with each element originating from proxies of social ambiance and social interactions. The number of concurrent speakers is proposed as a proxy for social ambiance, while a dedicated target speaker detection algorithm is devised to capture individual social interactions. To suit diverse usage scenarios, I present both offline processing and on-device processing solutions. In the offline processing scenario, concurrent speaker count benefits from the knowledge of large-scale self-supervised representations through model fine-tuning, and target speaker detection incorporates unsupervised source separation as a pre-processing step. For on-device processing, the concurrent speaker count is compressed to 5% of its original size to facilitate device-side implementation. Additionally, an Android application has been developed to support data collection, processing, and real-time feedback. Furthermore, target speaker detection leverages the enroll-aware attention statistic pooling method to effectively eliminate interfering speakers and enhance the model's robustness. ASIM's performance is evaluated using benchmark datasets and Rice University campus data, followed by an in-depth error analysis to understand its limitations and utilization. ASIM's utility is further substantiated through application to a clinical dataset. Results show a significant correlation between sociability patterns extracted by ASIM and self-reported metrics. Additionally, ASIM proves effective in predicting self-reported mood and sociability, showcasing its potential as an instrumental tool in both mental health research and clinical settings.
  • Thumbnail Image
    Item
    Privacy-Preserving Social Ambiance Measure From Free-Living Speech Associates With Chronic Depressive and Psychotic Disorders
    (Frontiers, 2021) Chen, Wenwan; Sabharwal, Ashutosh; Taylor, Erica; Patel, Ankit B.; Moukaddam, Nidal
    A social interaction consists of contributions by the individual, the environment and the interaction between the two. Ideally, to enable effective assessment and interventions for social isolation, an issue inherent to depressive and psychotic illnesses, the isolation must be identified in real-time and at an individual level. However, research addressing sociability deficits is largely focused on determining loneliness, rather than isolation, and lacks focus on the richness of the social environment the individual revolves in. In this paper, We describe the development of an automated, objective and privacy-preserving Social Ambiance Measure (SAM) that converts unconstrained audio recordings collected from wrist-worn audio-bands into four levels, ranging from none to active. The ambiance levels are based on the number of simultaneous speakers, which is a proxy for overall social activity in the environment. Results show that social ambiance patterns and time spent at each ambiance level differed between participants with depressive or psychotic disorders and healthy controls. Individuals with depression/psychosis spent less time in diverse environments and less time in moderate/active ambiance levels. Moreover, social ambiance patterns are found associated with the severity of self-reported depression, anxiety symptoms and personality traits. The results in this paper suggest that objectively measured social ambiance can be used as a marker of sociability, and holds potential to be leveraged to better understand social isolation and develop effective interventions for sociability challenges, thus improving mental health outcomes.