Everyone deserves to be heard, but for millions of Americans who rely on assistive devices to communicate, being heard is still not a given.  

Alternative and augmentative communication (AAC) technologies are an essential part of day-to-day life for people with conditions that limit communication, such as autism, cerebral palsy or aphasia, but the often primitive technology has struggled to keep up with the needs of its users. 

Stephen MacNeil, assistant professor of computer and information sciences at the College of Science and Technology (CST), is hoping to change that.  

“Existing AAC devices are really primitive,” said MacNeil, who joined Temple in 2021 and conducts research out of Temple’s Human-Computer Interaction Lab. “You might have a device that looks like a keyboard where each key represents a word, and you press those words to communicate. But users have to sift through all of these words, and some of them are nested away in folders, so they need to know which folder to open and which option to choose to communicate. It’s just really inconvenient.” 

MacNeil is leading the technology and engineering development efforts for a new Rehabilitation Engineering Research Center (RERC), a multisite center based out of the University of Arkansas. RERC recently received $5 million from the National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR), to advance AAC technologies by making them more intuitive, personalized and accessible.  

$1.5 million of the funding will directly support the work of MacNeil and his co-primary investigators, Slobodan Vucetic, professor of computer science and director of Temple’s Center for Hybrid Intelligence, and Eduard Dragut, professor of computer and information sciences at CST. 

AAC devices range from low-tech, such as picture boards, to high-tech, such as speech generating devices, or iPads and other devices that offer communication options that a user might need. The technology often has limitations, including that device capabilities don’t evolve as a user grows and develops. 

“Many users are stuck with the default options, even as they become better communicators,” MacNeil said. “And these devices are hard to configure, so a lot of people end up abandoning them.”  

MacNeil and his team are working to make AAC devices more aware of a user’s environment and use that context to provide relevant communication options, since a user may need different options when they’re in the bathroom than they do in the kitchen, or when playing with a sibling, for example. 

An important aspect of the research, MacNeil said, is working directly with AAC users. His team regularly consults with an advisory committee made up of people who use AAC devices, which helps the researchers better understand the challenges and potential use cases for the technology. 

“‘Nothing about us without us’ is an important mantra in AAC and accessibility research,” MacNeil said. “There needs to be more participation from people with disabilities in accessibility research, and right now that just doesn’t happen enough.” 

MacNeil’s experience with codesign and participatory design has helped him emphasize input from AAC users throughout the research. The research team also brings an extensive background in developing artificial intelligence solutions that solve real problems, and they are incorporating a suite of AI applications in their work.  

One use of AI involves processing information from Bluetooth beacons to help locate an AAC user within a space. MacNeil is working alongside Dragut to incorporate these capabilities into the devices. 

“This way we can identify if the user is near the books, or the puzzles, or some other activity, and as they move into these different activity spaces, we can change the communication options that they receive,” he said. 

The team is similarly working to incorporate AI computer vision, so that as a user moves an AAC device throughout a space, it recognizes objects and provides relevant communication options using that information. The researchers are also using AI speech recognition to provide communication options based on speech that the AAC user is hearing.  

“There’s a lot of critique right now that AI is very techno solutionist, and that AI developers are building technologies that no one is asking for, perhaps even creating more problems than solutions,” MacNeil said. “A big part of my work is very responsibly and very carefully developing AI solutions that work for people. I don’t design for these communities. I am working with them as partners.” 

The NIDILRR funding will support the research for the next five years. During that period, MacNeil expects the work to involve more than a dozen graduate researchers, and up to 40 undergraduate researchers. 

The researchers hope to develop AAC tools that can be integrated into existing mainstream and commercial products, but MacNeil is also excited about the promise of developing new and open-source devices. 

“In cases where we can’t integrate our work into commercial products, we can create those products ourselves, release them to the public, and offer open-source tools that people can adopt and use,” he said. “We have a big opportunity over this five-year period to change how AAC technology is used in the world.”