Project Title
Challenge
Tools
My Role
Project Time
Vocalex: Socially inclusive project design
Create an integrated aided Augmentative & Alternative Communication (AAC) tool
Figma, Miro, Descript (AI voice generation), GSuite
Product Manager, UX Designer
1 month
As a group, we were tasked with creating a product/tool that would promote social inclusivity by prioritizing the design for a small section of the population that could later be expanded for use by other groups. The goal was to challenge how we thought about what it meant to be an "average human", how "average" is a misleading term, and by designing for the 'average' person our designs may not really benefit anyone as planned.
Ideation
The idea for our product, Vocalex, came from one of our team member's volunteer experience where he worked with adults that needed various levels of assistance in accomplishing daily tasks. Through our colleague we observed frustrations with Augmentative & Alternative Communication (AAC).
Observed User Difficulties
Through our colleague's observations we identified the following challenges with AAC devices:
-
Users can feel more excluded from interactions with others due to speed of and type of communication
-
Receivers of communication taking over AAC device from the user when they are frustrated with the speed of communication
-
Occasionally users had been ignored when trying to use their AAC devices
-
-
AAC devices can be quite complex and difficult to navigate creating labored communication that can feel impersonal
-
Difficultly finding correct thoughts
-
Lack of personaliztion
-
-
High costs of hardware/software make accessibility challenging for aided devices
Goals
To create Vocalex, an AAC application, for mobile devices/tablets that integrates a wearable speaker and supports more personalized and efficient communication strategies by:
-
Offering AI powered text-to-speech personalization through
-
Speech generation either through saved samples of the user's voice or from a library of synthetic voice options
-
-
Providing mood modulation of voice
-
Emotive selection
-
Expressive volume control (e.g. whispering)
-
-
Saving personal stories, jokes, frequently used phrases, interjections etc.
-
Incorporating a wearable speaker to reinforce the link that the AAC device is an extension of the user
-
Increasing speed of communication by:
-
Syncing AAC and calendar to provide contextual clues for predictive text
-
Accessing saved phrases on the application and through quick phrases saved on programable buttons on the wearable speaker
-
Rough Draft
Chalkboard draft of early ideas based on observed user challenges, goals, and bibliographic research.
The sketch to the right is an initial attempt to understand how aided AAC devices might work and how to incorporate identified goals for Vocalex.
Feature Brainstorm
Drafting of the personalization portion of Vocalex in tandem with predictive text. Here we considered using emojis to express emotive qualities of sound in verbalized communication, however we identified that emojis would need further personalization if used to change the sound of a voice since the meaning of emojis can vary between cultures, age groups, and even friends.
Emotive & Contextual Ideas
User Flow
After the rough sketches on the chalkboard we drafted a user flow in FigJam to understand and outline the pages we would need to create for the User Interface UI of Vocalex.
User Flow Mapping
Challenges
Based on several brainstorming sessions a few concerns stood out at the outset of the project:
-
Hardware
-
Battery longevity for both AAC processing and cellular network switches
-
Internal device memory
-
External wearable speaker
-
Speaker volume
-
Battery life
-
Visibility as an extension of user
-
-
-
Software
-
Voice generation
-
Training of user
-
Compatibility with user devices
-
Full functionality may be confined by areas with robust WiFi or cellular service
-
-
Cost
-
Approval from European Medical Agency (EMA) for use
-
Digital literacy
-
Training
-
Social interactions
-
Visual contact reduced with AAC in use
-
Communication speed may remain delayed
-
Potential for creating group of exclusive users
-
Acceptance of out-group members to users of AAC devices may be difficult
-
Design
Based on the challenges and our ideas from research of available tools an initial design began to take shape through consideration of our audience and the context in which Vocalex would be used in.
Audience
Based on our idea, we kept a specific user in mind when considering design features. Users considered during the original design were:
-
18 - 30 years old
-
Digitally literate
-
Experiencing difficulty with speech production (e.g. Broca's Aphasia)
Context
Low-Fidelity
Each group member drafted up their ideas and we discussed the merits of each mock-up and voted on which portions made sense to incorporate in to the next design stage.
Focus on key iconography needed to communicate complex aspects of text-to speech as imagined:
-
Wearable speaker
-
Battery
-
Connectivity
-
-
Speech controls
-
Play
-
Pause
-
-
Sharing voice notes & TTY integration
-
Emotive emphasis for speech
-
Settings for Vocalex
High-Fidelity
AAC Vocalex app and speaker with walkthroughs include:
-
Onboarding walkthrough
-
Wearable Speaker draft
-
Voice and speaker settings
These prototype slides are the first in an iterative process and were revised after intial review.
.jpg)
.jpg)
.jpg)
User Journey
With this specific audience in mind, we considered the scenarios where this device may be used in order to consider specific environmental challenges. Scenarios considered were those such as:
-
Work
-
University
-
Post Office
-
Bus
-
Bar
There are different needs for communication in these scenarios that would need a change in volume, tone, cadence, and expressiveness in order to carry out daily tasks.
Intro pages for account login and account type.
UI for vocalized speech options for personalization, context predictive texts, and external speaker options.
The voice setup option encourages further personalization of Vocalex with the creation of a synthetic voice based on user added voice files. Additionally, there are sliders for varying the emotional expression in speed, pitch, and word cadence for each selected emotion. There is an option to listen to the results of the changes for each emotion.
To assist with context based predictive text, a link to a calendar was considered as Vocalex should offer a place to save necessary details for a specific speaking partner/appointment to speed up communication or for frequent speaking partners offer commonly used phrases or topics.
An external speaker was drafted to offer buttons with saved quick phrases to help the user get the attention of their intended speaking partner and further define the AAC device as an extension of them, their de facto voice.
.jpg)
.jpg)
After initial review of the Vocalex app and headset we created a user journey to check that the design had met some of the basic goals outlined at the beginning of the project.
The user journey was defined as a person who:
-
18-30 years old age range
-
Needs to communicate in university scenarios
-
Currently has Broca's Aphasia and is using Vocalex as a tool during recovery
-
Does not plan to learn sign language but still needs to be able to communicate
This demonstration used Descript (AI voice generation) that was a model trained on the vocal recordings of one of our colleagues to demonstrate how this feature would work.
Descript provided a voice model output, as can heard by clicking the play icon, but due to the free nature of the model, "jabber" is used in place of Vocalex as an auditory watermark.
The idea is to have three pre-saved messages on the wearable speaker in order to have a quick way to grab the attention of the target of communication and initiate a conversation.
When play is pressed on a message there is highlight that scrolls over the words as the audio is played. Additionally, there are backward and forward options due to the nature of communication where people often talk over one another and it is important to be able to articulate the full message, even if interrupted.
Icon & Login Screens
Vocalex Sign-in & Account Types
Vocalex Settings
Contextual Calendar Idea
Wearable Speaker Draft
Vocalex Screen Walkthrough
Start of Vocalex Demo
Vocalex Demo: Settings
Vocalex: Voice Model Sample
Vocalex: Wearable Speaker Idea
Vocalex: Playback of Message
Vocalex: Favorite Phrases
Vocalex: Voice Note Playback
Financials
As an exercise in developing a product we also researched potential financial costs to gain a fuller picture of potential costs and timeline for development. To understand the costs and time required, we utilized a gantt chart to ensure we considered a wide variety of needs to develop Vocalex.
Limitations
Due to the scope of this project there were some limitations that need further investigation. The identified limitations fall into either the development of Vocalex or the experience.
Development
-
Competencies in specific hardware and engineering areas are limited amongst the group skill set and outside help is needed to complete a functional prototype
-
Hardware
-
Smartphone/tablet may vary among users
-
Standard minimum requirements need to be established
-
Battery life
-
Wearable speaker volume and positioning needs further development so that the wearer projects the sound and is not deafened by it
Experience
-
Eye contact may continue to be reduced with AAC as user searches for correct phrase or types out dialogue
-
Communication speed may remain delayed due to text-to-speech processing
-
Potential for creating group of exclusive users
-
Acceptance of out-group members to users of AAC devices may be difficult
Future Work
As our time on the project came to a close there were some items marked for further improvements:
-
Speech recognition for frequent conversation partners to improve personalized predictive responses
-
Addition of Stories feature to promote auto-bibliographic sharing
-
Context based predictions
-
Personalization of wearable speaker
-
Testing to determine outcomes of speed and personalization of communication
While this project focused on inclusive design, development, and testing is still required to validate our research.
