Welcome to CogVox
EXPLOITING VOICE ASSISTANT SYSTEMS FOR EARLY DETECTION OF COGNITIVE DECLINE
We are an interdisciplinary research team composed of computer scientists, geriatric medicine physician, neuropsychologist, and computational linguist, aiming to develop a low-cost, passive, and practical cognitive assessment method using Voice Assistant Systems (VAS) for early detection of cognitive decline. Our research project is funded by the National Science Foundation and the National Institute of Health National Institute on Aging 2019-2023.
- (Related) A comprehensive systematic review on mobile applications to support dementia patientsThe increasing number of patients living with dementia requires effective support measures. Easier access to mobile devices can provide the follow-up and monitoring of patients with dementia. The literature presents several studies on the use of mobile applications to assist patients, health professionals, and caregivers, considering the different knowledge areas, and scientific and technological contributions.Continue reading “(Related) A comprehensive systematic review on mobile applications to support dementia patients”
- Early Detection of Cognitive Decline using Voice Assistant Commands (ICASSP 2023)Eli Kurtz, Youxiang Zhu, Tiffany Driesse, Bang Tran, John A. Batsis, Robert M. Roth, and Xiaohui Liang Early detection of Alzheimer’s Disease and Related Dementias (ADRD) is critical in treating the progression of the disease. Previous studies have shown that ADRD can be detected and classified using machine learning models trained on samples of spontaneousContinue reading “Early Detection of Cognitive Decline using Voice Assistant Commands (ICASSP 2023)”
- Exploiting Relevance of Speech to Sleepiness Detection via Attention Mechanism (ICC 2023)Bang Tran, Youxiang Zhu, James W. Schwoebel, and Xiaohui Liang Excessive sleepiness in critical tasks and jobs can lead to adverse outcomes, such as work accidents and car crashes. Detecting and monitoring sleepiness levels can prevent these adverse events from happening. In this paper, we propose an attention-based sleepiness detection method using HuBERT embeddings andContinue reading “Exploiting Relevance of Speech to Sleepiness Detection via Attention Mechanism (ICC 2023)”
- More Data, Better Accuracy? An Empirical Study on Few-shot Speech-based Dementia Detection (SoCal 2022)Youxiang Zhu and Xiaohui Liang Speech-based dementia detection faces small data problems and weak data-label correlation problems. Existing works still used cross-validation or fixed training testing split as evaluation protocols on the small data, which may overestimate the performance. We propose a new evaluation protocol under the few-shot learning setting. Based on our evaluation results,Continue reading “More Data, Better Accuracy? An Empirical Study on Few-shot Speech-based Dementia Detection (SoCal 2022)”
- Dr. Liang delivered a Keynote Speech in IEEE International Conference on Universal VillageDr. Liang delivered a keynote speech on “Exploiting Human Speech for Early Detection of Cognitive Decline” at IEEE International Conference on Universal Village on Oct. 24, 2022.
- (Related) Should Alexa diagnose Alzheimer’s?: Legal and ethical issues with at-home consumer devicesVoice-based AI-powered digital assistants, such as Alexa, Siri, and Google Assistant, present an exciting opportunity to translate healthcare from the hospital to the home. But building a digital, medical panopticon can raise many legal and ethical challenges if not designed and implemented thoughtfully. This paper highlights the benefits and explores some of the challenges ofContinue reading “(Related) Should Alexa diagnose Alzheimer’s?: Legal and ethical issues with at-home consumer devices”
- Towards Interpretability of Speech Pause in Dementia Detection using Adversarial Learning (INTERSPEECH 2022)Youxiang Zhu, Xiaohui Liang, John A. Batsis (University of North Carolina), Robert M. Roth (Dartmouth) Abstract: Detecting dementia using human speech is promising but faces a limited data challenge. While recent research has shown general pretrained models (e.g., BERT) can be applied to improve dementia detection, the pretrained model can hardly be fine-tuned with theContinue reading “Towards Interpretability of Speech Pause in Dementia Detection using Adversarial Learning (INTERSPEECH 2022)”
- AGS Presidential Poster AwardOur poster “Older Adults experience and response to Voice Assistant Systems” was chosen as the best poster in the category of Geriatric Syndromes (descriptive research on the mechanisms, natural history or management of major geriatric syndromes) at the American Geriatrics Society (AGS) 2022.
- Phase II updateWe have successfully recruited 32 older adults for our Phase II evaluation. 13 participants (5 healthy, 8 mild cognitive impairment) chose Pathway 1, i.e., performing seven sessions once every three months + home data tracking 19 participants (10 healthy, 9 mild cognitive impairment) chose Pathway 2, i.e., performing seven sessions only once every three monthsContinue reading “Phase II update”
- Patient Perceptions of Using Voice-Based Dietary Assessment Tools Among Older AdultsTiffany M. Driesse, Xiaohui Liang, Michael Fowler, Jing Yuan, and John A. Batsis Background: Poor diet among older adults is a risk factor for developing multiple chronic diseases. Dietary recall comprises an important component in intervention research and clinical care. Commonly used tools include the web-based automated self-administered 24-hour assessment (ASA-24). Yet voice assistant (VAS)Continue reading “Patient Perceptions of Using Voice-Based Dietary Assessment Tools Among Older Adults”
- Domain-aware Intermediate Pretraining for Dementia Detection with Limited DataYouxiang Zhu, Xiaohui Liang, John A. Batsis, and Robert M. Roth Detecting dementia using human speech is promising but faces a limited data challenge. While recent research has shown general pretrained models (e.g., BERT) can be applied to improve dementia detection, the pretrained model can hardly be fine-tuned with the available small dementia dataset asContinue reading “Domain-aware Intermediate Pretraining for Dementia Detection with Limited Data”
- Two papers accepted by “IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP)” 2022Our two papers have been accepted by IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) 2022 Towards Interpretability of Speech Pause in Dementia Detection using Adversarial Learning, Youxiang Zhu, Bang Tran, Xiaohui Liang, John A. Batsis (University of North Carolina), Robert M. Roth (Dartmouth) Speech Tasks Relevant to Sleepiness Determined with Deep Transfer Learning, BangContinue reading “Two papers accepted by “IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP)” 2022″
- A Bibliograph of Speech-based Dementia DetectionCheck here for a list of Dementia Detection Papers with Keywords https://github.com/billzyx/awesome-dementia-detection The list is maintained by Youxiang Zhu.
- Speech Tasks Relevant to Sleepiness Determined with Deep Transfer LearningBang Tran, Youxiang Zhu, Xiaohui Liang, James W. Schwoebel, Lindsay A. Warrenburg Excessive sleepiness in attention-critical contexts can lead to adverse events, such as car crashes. Detecting and monitoring sleepiness can help prevent these adverse events from happening. In this paper, we use the Voiceome dataset to extract speech from 1,828 participants to develop aContinue reading “Speech Tasks Relevant to Sleepiness Determined with Deep Transfer Learning”
- Towards Interpretability of Speech Pause in Dementia Detection using Adversarial LearningYouxiang Zhu, Bang Tran, Xiaohui Liang, John A. Batsis, Robert M. Roth Speech pause is an effective biomarker in dementia detection. Recent deep learning models have exploited speech pauses to achieve highly accurate dementia detection, but have not exploited the interpretability of speech pauses, i.e., what and how positions and lengths of speech pauses affectContinue reading “Towards Interpretability of Speech Pause in Dementia Detection using Adversarial Learning”
- “Evaluating Voice-Assistant Commands for Dementia Detection” has been accepted by Computer Speech & LanguageXiaohui Liang, John A. Batsis, Youxiang Zhu, Tiffany M. Driesse, Robert M. Roth, David Kotz, and Brian MacWhinney Early detection of cognitive decline involved in Alzheimer’s Disease and Related Dementias (ADRD) in older adults living alone is essential for developing, planning, and initiating interventions and support systems to improve users’ everyday function and quality ofContinue reading ““Evaluating Voice-Assistant Commands for Dementia Detection” has been accepted by Computer Speech & Language”
- Dr. Liang is organizing Symposium on e-Health at IEEE International Conference on Communications (ICC) 2022https://icc2022.ieee-icc.org/ The e-Health track provides an opportunity to bring together healthcare professionals, researchers, scientists, engineers, academics, and students from all around the world to share their experience and latest advances on new technologies and systems development in different healthcare and medicine applications. In particular, the e-Health track of the SAC symposium will focus on theContinue reading “Dr. Liang is organizing Symposium on e-Health at IEEE International Conference on Communications (ICC) 2022”
- “WavBERT: Exploiting Semantic and Non-semantic Speech using Wav2vec and BERT for Dementia Detection” has been accepted by INTERSPEECH 2021Youxiang Zhu, Abdelrahman Obyat, Xiaohui Liang, John A. Batsis, and Robert M. Roth In this paper, we exploit semantic and non-semantic information from patient’s speech data using Wav2vec and Bidirectional En-coder Representations from Transformers (BERT) for dementia detection. We first propose a basic WavBERT model by extracting semantic information from speech data using Wav2vec, andContinue reading ““WavBERT: Exploiting Semantic and Non-semantic Speech using Wav2vec and BERT for Dementia Detection” has been accepted by INTERSPEECH 2021″
- Collaborative Research with SondeHealthXiaohui’s group and SondeHealth will collaborate on research related to vocal biomarkers and mental health disorders. Thanks to Jim Schwoebel, Vice President of Data and Research at SondeHealth, for this collaboration opportunity and the access to the voice dataset collected over thousands of users at Sonde Health. Check more about SondeHealth at https://www.sondehealth.com/ Check more about SurveylexContinue reading “Collaborative Research with SondeHealth”
- Privacy Concerns Among Older Adults Using Voice Assistant SystemsHillary Spangler, Tiffany Driesse, Robert Roth, Xiaohui Liang, John Batsis, David Kotz Voice Assistant Systems (VAS) are software platforms that complete various tasks using voice commands (e.g., Amazon Alexa), with increasing usage by older adults. It is unknown whether older adults have significant privacy concerns with VAS. 55 participants were evaluated from ambulatory practice sitesContinue reading “Privacy Concerns Among Older Adults Using Voice Assistant Systems”
- (Related) Speech for predicting car accidents and medical Alexa2021/2 Using Speech Data From Interactions With a Voice Assistant to Predict the Risk of Future Accidents for Older Drivers: Prospective Cohort Study 2021/4 Can Medical Alexas Make Us Healthier?
- The ADReSSo 2021 ChallengeWe have participated in the ADReSSo 2021 challenge, and submitted “WavBERT: Exploiting Semantic and Non-semantic Speech using Wav2vec and BERT for Dementia Detection,” Youxiang Zhu, Abdelrahman Obyat, Xiaohui Liang, John Batsis and Robert Roth ADReSSo 2021: http://www.homepages.ed.ac.uk/sluzfil/ADReSSo-2021/
- Group meeting on March 19, 2021Meeting using Zoom Attendees: Dr. Liang, Dr. Batsis, Dr. Roth, Dr. Kotz, Dr. MacWhinney, Tiffany, Youxiang We have discussed the project progress, including: We have collected 11 VAS data from onsite in-lab evaluation and 56 VAS data from virtual in-lab evaluation. We plan to recruit patients with dementia from our DH site for the restContinue reading “Group meeting on March 19, 2021”
- (Related) News of Smart Speaker2021/3. Proof-of-concept system turns smart speakers into contactless heart rhythm monitors 2021/2 Efficacy of Smart Speaker–Based Metamemory Training in Older Adults: Case-Control Cohort Study 2020/11. Amazon releases new Alexa features allowing families to monitor seniors living alone 2020/8. Amazon Alexa & Oral-B ink deal for voice-integrated toothbrush system 2018/6. In-Depth: Voice, independence-focused technologies drive agingContinue reading “(Related) News of Smart Speaker”
- Exploring Deep Transfer Learning Techniques for Alzheimer’s Dementia DetectionYouxiang Zhu, Xiaohui Liang, John A. Batsis, and Robert M. Roth Examination of speech datasets for detecting dementia, collected via various speech tasks, has revealed links between speech and cognitive abilities. However, the speech dataset available for this research is extremely limited because the collection process of speech and baseline data from patients with dementia in clinical settingsContinue reading “Exploring Deep Transfer Learning Techniques for Alzheimer’s Dementia Detection”
- Research Update on March 8We have recruited a total of 61 patients, including 30 Healthy Controls (HC), 31 Mild Cognitive Impairments (MCI), and 1 Dementia. Our team recently have finished the following works. Xiaohui Liang, John Batsis, Youxiang Zhu, Tiffany Driesse, Robert Roth, David Kotz, and Brian MacWhinney, “Evaluating Voice-Assistant Commands for Dementia Detection.” Youxiang Zhu, Xiaohui Liang, JohnContinue reading “Research Update on March 8”
- Medical Monitor Meeting on Oct. 26, 2020Meeting using Zoom Attendees: Dr. Liang, Dr. Batsis, Dr. Roth, Dr. Stark, Tiffany Driesse, Dr. Alvin McKelvy (NIH), and Dr. Yuan Luo (NIH) We have discussed the medical monitor report, including: Project Organizational Chart Purpose of the Study Projected Timetable and Schedule Study Status Recruitment and Participant Status Safety Assessments for all participants
- Team meeting on Sept. 11, 2020Meeting using Zoom Attendees: Dr. Liang, Dr. Batsis, Dr. Roth, Dr. MacWhinney, Dr. Kotz, Youxiang We have discussed: Year-1 progress Year-2 plan and timeline IRB submission at UNC Recruitment at UNC in late September/early October Meeting with Safety Officer in the late October Progress on technical ideas General discussion on data collection and analysis
- Dr. John A. Batsis moved to UNCOur team member Dr. John A. Batsis has started his new position at the University of North Carolina (UNC) at Chapel Hill on September 1, 2020. We plan to finish the in-lab evaluation at UNC in year 2, i.e., before May 30, 2021, and start the in-home evaluation in January 2021. John A. Batsis, MD, FACP, AGSF,Continue reading “Dr. John A. Batsis moved to UNC”
- The ADReSS 2020 ChallengeWe have participated in the ADReSS 2020 challenge. Check our recent paper “Exploiting Fully Convolutional Network and Visualization Techniques on Spontaneous Speech for Dementia Detection” at https://arxiv.org/abs/2008.07052.
- Dr. Aleksandra C. Stark joined our teamIn May 2020, Dr. Aleksandra C. Stark joined our team as the safety officer. She will perform data and safety monitoring activities in our study.
- In-lab evaluation at DHWe began a very successful recruitment in March 2020. We enrolled 14 participants and had 11 participants onsite for data collection from March 2-12. Our initial goal for the recruitment process was to have the in-lab recruitment (n=90) finished by summer 2020, which was about six-month earlier than our plan in the proposal. However, theContinue reading “In-lab evaluation at DH”
- PressJanuary – Februray, 2020 – Our project has been reported by EurekAlert! “New research utilizes voice assistant systems for early detection of cognitive decline.” Seacoastonline “Can Alexa help doctors detect the onset of dementia?“ CBS Boston “New England Researchers Hope Voice Assistants Can Spot Signs Of Dementia.“ AI-in-Healthcare “Can Alexa spot signs of cognitive decline?“Continue reading “Press”
- Flyer and Tear-off
- ProjectSeptember 17, 2019 – Dr. Liang (PI) from University of Massachusetts Boston and Dr. Batsis and Dr. Roth (Co-Is) from Dartmouth-Hitchcock receive a four-year, nearly $1.2 million NIH/NIA R01 grant “SCH: INT: Collaborative Research: Exploiting Voice Assistant Systems for Early Detection of Cognitive Decline” 2019-2023.
CogVox 