naa a22 4500 510739733 CHVBK 20180411083024.0 cr unu---uuuuu 180411e20131101xx s 000 0 eng 10.1007/s12193-013-0123-2 doi (NATIONALLICENCE)springer-10.1007/s12193-013-0123-2 Multimodal assistive technologies for depression diagnosis and monitoring [Elektronische Daten] [Jyoti Joshi, Roland Goecke, Sharifa Alghowinem, Abhinav Dhall, Michael Wagner, Julien Epps, Gordon Parker, Michael Breakspear] Depression is a severe mental health disorder with high societal costs. Current clinical practice depends almost exclusively on self-report and clinical opinion, risking a range of subjective biases. The long-term goal of our research is to develop assistive technologies to support clinicians and sufferers in the diagnosis and monitoring of treatment progress in a timely and easily accessible format. In the first phase, we aim to develop a diagnostic aid using affective sensing approaches. This paper describes the progress to date and proposes a novel multimodal framework comprising of audio-video fusion for depression diagnosis. We exploit the proposition that the auditory and visual human communication complement each other, which is well-known in auditory-visual speech processing; we investigate this hypothesis for depression analysis. For the video data analysis, intra-facial muscle movements and the movements of the head and shoulders are analysed by computing spatio-temporal interest points. In addition, various audio features (fundamental frequency f0, loudness, intensity and mel-frequency cepstral coefficients) are computed. Next, a bag of visual features and a bag of audio features are generated separately. In this study, we compare fusion methods at feature level, score level and decision level. Experiments are performed on an age and gender matched clinical dataset of 30 patients and 30 healthy controls. The results from the multimodal experiments show the proposed framework's effectiveness in depression analysis. OpenInterface Association, 2013 Depression analysis nationallicence Multimodal nationallicence LBP-TOP nationallicence Bag of words nationallicence Joshi Jyoti University of Canberra, ACT 2601, Australia aut Goecke Roland Australian National University, ACT 0200, Australia aut Alghowinem Sharifa Australian National University, ACT 0200, Australia aut Dhall Abhinav Australian National University, ACT 0200, Australia aut Wagner Michael University of Canberra, ACT 2601, Australia aut Epps Julien University of New South Wales, Kensington, Australia aut Parker Gordon University of New South Wales, Kensington, Australia aut Breakspear Michael Queensland Institute of Medical Research, Brisbane, Australia aut Journal on Multimodal User Interfaces Springer Berlin Heidelberg 7/3(2013-11-01), 217-228 1783-7677 7:3<217 2013 7 12193 https://doi.org/10.1007/s12193-013-0123-2 text/html Onlinezugriff via DOI 1 research-article jats NATIONALLICENCE 856 40 https://doi.org/10.1007/s12193-013-0123-2 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Joshi Jyoti University of Canberra, ACT 2601, Australia aut NATIONALLICENCE 700 1- Goecke Roland Australian National University, ACT 0200, Australia aut NATIONALLICENCE 700 1- Alghowinem Sharifa Australian National University, ACT 0200, Australia aut NATIONALLICENCE 700 1- Dhall Abhinav Australian National University, ACT 0200, Australia aut NATIONALLICENCE 700 1- Wagner Michael University of Canberra, ACT 2601, Australia aut NATIONALLICENCE 700 1- Epps Julien University of New South Wales, Kensington, Australia aut NATIONALLICENCE 700 1- Parker Gordon University of New South Wales, Kensington, Australia aut NATIONALLICENCE 700 1- Breakspear Michael Queensland Institute of Medical Research, Brisbane, Australia aut NATIONALLICENCE 773 0- Journal on Multimodal User Interfaces Springer Berlin Heidelberg 7/3(2013-11-01), 217-228 1783-7677 7:3<217 2013 7 12193 Metadata rights reserved Springer special CC-BY-NC licence nationallicence BK010053 XK010053 XK010000 NATIONALLICENCE NATIONALLICENCE NL-springer