USF Health co-leads landmark project using voice, AI to diagnose disease

USF Health is leading a landmark research project to use a patient’s voice and artificial intelligence technology to diagnose diseases.

In collaboration with co-lead Weill Cornell Medicine, USF Health will oversee a team of researchers from 12 institutions in the U.S. and Canada on the development of a large, open-source database of human voices that will help doctors diagnose neurological diseases such as Alzheimer’s and Parkinson’s, along with voice, psychological, respiratory and pediatric speech disorders.

The project is one of just four to receive funding from the National Institutes of Health’s new Bridge2AI initiative, which seeks to develop artificial intelligence as a next-generation clinical tool. Dr. Yaël Bensoussan, the project’s principal investigator and an assistant professor of otolaryngology at the USF Health Morsani College of Medicine, says the research will establish voice as a biomarker similar to the genomic information from chromosomes or the images from a CT scan.

“We know voice can provide information on different diseases,” Bensoussan says. “That is really interesting because it’s very easy to collect. It’s super cheap to collect; we can record voices very easily. Most importantly, there is no risk to the patient. It’s not like getting a blood draw or getting radiation from a CT scan. By working with different types of populations, we know as clinicians by hearing somebody’s voice that this person sounds like he has Parkinson’s. There are some diseases that we have known for a long time that the voice changes or the speech changes, like Alzheimer’s, like Parkinson's, like vocal cord cancer. But there are also diseases where we don’t necessarily think a voice change is related. An example is cystic fibrosis, which is a disease that can cause lung issues and also nasal issues. Because the nose is blocked, they can sound very nasal.”

With the Bridge2AI funding, the “Voice as a Biomarker for Health” project will build a 
large, diverse, identity-protected database to train AI algorithms on how to identify voice characteristics associated with a specific disease. Depending on annual congressional appropriations for the NIH, the project is planned to last four years with funding of up to $14 million.

“It’s the first big project with voice as a biomarker that they are funding at that level,” Bensoussan says. “Its importance is to build a solid infrastructure that will last a long time that people can add to.“Another goal of the program is to teach people how to use these databases. It’s really kind of the first step for generations to come to be able to use the databases.

Since the voice, similar to fingerprints or genetic information, can serve as a bio-identifier even without a patient’s name, the project will also establish a legal and ethical framework for the use of the database. That’s important at a time when AI technology has advanced to the point it can mimic individual voices.

“We know technology is moving very fast,” Bensoussan says. “When you think about it, the technology is amazing. But the technology moved faster than the ethical and legal rules. That’s also one of our goals with this project is to determine what do we do with voice data and how do we tell researchers to use it.”

Because of those considerations and the technology involved, the project has brought together a diverse team that includes doctors, lawyers, ethicists, engineers and acoustic technicians. Bensoussan says that is a positive step. 

“It’s really important that people stop working in silos,” she says. “Often, doctors work with doctors and engineers work with engineers. Everybody kind of works on their own projects. This group is really special because we get to collaborate and work with one another.”

For more information on the NIH program go to Bridge2AI.