Anand Munje interviews entrepreneurs, academics and artists on his YouTube channel.
He communicates with them even though his cerebral palsy makes it difficult to talk, but this is made possible thanks to the Aihearu software developed by his brothers Arun and Amit Munje.
It’s pronounced “I hear you” and takes Anand’s words and transcribes them on-screen during his show Anand’s World.
The brothers integrated Aihearu into Zoom’s video conferencing, and three dots appeared on the screen as the software processed Anand’s voice and then displayed it as text.
“Talking to people with my own voice without Amit’s help is new for me,” he said in the episode where he interviews his siblings. Amit often acted as Anand’s interpreter.
Anand prepares and practices his questions for his channel. This allows the software to fine-tune the vocabulary it uses to interpret your voice.
CBC News met with Arun at his home in the Ottawa suburb of Kanata for a video interview with Anand in India.
Unlike the way CBC normally conducts interviews, Anand took some questions from the day before so he could prepare answers as he did for the program.
“The best part is that it can recognize my voice,” he said in one of his prepared answers. “I hope this app can be used by millions of speech-impaired people. I hope others can speak independently.”
We also tried the “open mode” to answer some questions that were not given to him in advance.
“I’m so nervous,” he said via software in open mode. “This is the first time I’ve used open mode.”
Arun, who previously worked at a teleconferencing startup and founded Aiheru as a side project, explained their “open mode” attempt to match Anand’s words with a massive word dataset.
“This is a completely open vocabulary. So some errors or more errors may occur,” Arun said. “Doesn’t know the context [of] What are we talking about?”
Anand Munje speaks using the Aihearu interface integrated with Zoom. The software displays his words and shows an ellipsis as it processes his speech. (Zoom in)
Open mode made some mistakes and Arun had to help interpret, a sign of work to be done. For example, “it is 22:00” became “this is a poem”.
Anand admits that he sometimes finds it difficult to manage the software’s vocabulary.
But while there may be difficulties with spontaneous long conversations, Anand can use it for regular activities such as managing smart home devices.
Arun said AI allowed them to create and improve the model that recognized Anand’s speech faster, rather than having to “hand-feed” Anand’s voice patterns into the program.
He said his goal was to help his brother express himself, but he hopes Aiheru can help people with different speech disorders and even thick accents.
“What we did was to get Anand to use it so we get first-hand experience of all the problems and how we can overcome them,” Arun said.
‘Live a more fulfilling life’
Claire Davies says people with speech disorders need more options to express themselves individually and participate in society.
“The more things there are, the more we can enable other people to interact with people with speech impairments,” said Davies, who directs the Assistive Technology Creation and Design Lab at Queen’s University.
“This allows them to find jobs, engage in social circles, work with co-workers, and actually live more fulfilling lives.”
Davies said it was especially valuable that the technology came from the family’s lived experiences and their understanding of how Anand expressed himself.
“They know best how to interpret that information, which allows the software to more effectively learn what is actually being said,” he said.
He said his work could also benefit other users.
Claire Davies, an associate professor at Queen’s University, says people with speech disorders need more options to express themselves. (Submitted/Claire Davies)
In open mode, Anand was able to answer a question about how he felt about his siblings’ hard work.
“I am so grateful and proud of my brother[s],” Anand said via the app.
Arun was momentarily surprised that the software produced this result.
“I’m impressed too when I see you get caught [phrases] it’s in on mode because nothing is being fed to it.”
Arun said that through a charity, they tested Aihearu with a handful of people with speech disorders, but each user had to set it up manually.
“The next step will be to make it easily available to everyone, so anyone can download and use it.”