Loading...

Accessibility Lab

We design, build, and deploy interactive systems to enhance accessibility.

About us

We are a research lab in the University of Michigan campus focusing on building interactive systems to address pressing accessibility challenges. We employ a diverse team of people from multidisciplinary backgrounds—including engineers, designers, architects, healthcare specialists, sociologists, and psychologists—allowing us to tackle accessibility problems holistically. Our projects undergo the full design, development, and evaluation cycle—starting from understanding a problem through a multi stakeholder perspective, to building an end-to-end usable solution to address the problem, and, finally, deploying and studying our solution over extended use periods in the field. Due to this holistic focus, we're able to achieve immediate real-world impact; our research work has been publicly released (e.g., one system has been used by over 100,000 disabled people) and has directly impacted products at leading tech companies such as Microsoft, Google, and Apple.

Currently, we are pushing boundaries in the following three research areas, with generous support from National Institutes of Health (NIH), Google, and Michigan Medicine:

1. Interactive AI for Accessibility. Involving end-users in the AI model training and personalization pipeline can increase their reliability, flexibility, and scalability. However, developing interfaces for people with limited sensory abilities to interact with AI is challenging. For example, how can deaf and hard of hearing people, who have trouble hearing sounds themselves, record sounds to train a sound recognition model? Or, how can blind people access the correctness and reliability of an image classification model? We're prototyping interfaces that will help Deaf/Disabled people to record training data, access the quality of their samples, train an AI model, and assess its correctness all by themselves.

2. Next-Generation Hearables. What capabilities should next generation of hearables (e.g., earbuds, hearing aids) support and how can innovations in AI sound recognition technology help? Who can benefit from the enchanced capabilities (e.g., deaf and hard of hearing people, people with autism, blind people, even non-disabled people) and what are the promising interfaces to deliver these enhancements? Our lab is inventing novel algorithms, system pipelines, and interfaces to support novel and exciting interactions with the audio environment previously only possible in the realm of science fiction.

3. Accessibility Toolkits for AR/VR Devices. XR devices are poised to dominate every facet of our lives; however, accessibility remains a challenge. Indeed, accessibilty has always been an afterhought when designing modern technlogies, leading to sub-par and inaccessible user experiences. Fortunately, since XR is still emerging, we have a great oppportunity to push accesibility into this medium from the start. Our lab is building and studying software toolkits for AR and VR to easily integrate accessibility into their apps.

We're continuously recruiting. If you are interested in working in these areas, please apply.

Recent News

Jan 23: Our Masters student, Jeremy Huang, has been accepted to UMich CSE PhD program. That's two good news for Jeremy this month (the CHI paper being the first). Congrats, Jeremy!
Jan 19: Our paper detailing our brand new human-AI collaborative approach for sound recognition has been accepted to CHI 2024! We can't wait to present our work in Hawaii later this year!
Nov 10: Professor Dhruv Jain invited to give a talk on accessiblity research in the Introduction to HCI class at the University of Michigan.
Oct 24: SoundWatch receives the best student paper nominee at ASSETS 2023! Congrats, Jeremy and team!
Aug 28: We welcome a new PhD student, Xinyun Cao, into our lab. Welcome Xinyun!
Aug 17: New funding notice! Our NIH funding proposal on "Developing Patient Education Materials to Address the Needs of Patients with Sensory Disabilities" has been accepted!
Aug 10: Professor Dhruv Jain invited for a talk on Sound Accessibility at Google.
Jun 30: Two papers from our lab, SoundWatch field study and AdaptiveSound, accepted to ASSETS 2023!
Apr 19: Professor Dhruv Jain awarded theGoogle Research Scholar Award.
Mar 16: Professor Dhruv Jain elected as the inaugral ACM SIGCHI VP for Accessibility!
Feb 14: Professor Dhruv Jain honored with the SIGCHI Outstanding Dissertation Award.
Jan 30: Professor Dhruv Jain honored with the William Chan Memorial Dissertation Award.

Our Team

Headshot of Dhruv Jain
Dhruv "DJ" Jain

Dhruv "DJ" Jain

Assistant Professor, Computer Science & Engineering (Lab head)
Headshot of Xinyun Cao
Xinyun Cao

Xinyun Cao

PhD Student, Computer Science & Engineering
Headshot of Jeremy Huang
Jeremy Huang

Jeremy Huang

Graduate Research Assistant, Computer Science & Engineering
Headshot of Liang-Yuan Wu
Liang-Yuan Wu

Liang-Yuan Wu

MS Student, Computer Science & Engineering
Headshot of Andy Jin
Andy Jin

Andy Jin

Undergraduate Student, Computer Science & Engineering
Headshot of Yuni Park
Yuni Park

Yuni Park

Undergraduate Research Assistant, Computer Science & Engineering
Headshot of Hriday Chhabria
Hriday Chhabria

Hriday Chhabria

Undergraduate Student, Computer Science & Engineering
Headshot of Reyna Wood
Reyna Wood

Reyna Wood

Undergraduate Student, Computer Science & Engineering
Headshot of Rue-Chei Chang
Rue-Chei Chang

Rue-Chei Chang

PhD Student, Computer Science & Engineering
Headshot of Anhong Guo
Anhong Guo

Anhong Guo

Assistant Professor, Computer Science & Engineering (Collaborator)
Headshot of Xinyue Chen
Xinyue Chen

Xinyue Chen

PhD Student, Computer Science & Engineering
Headshot of Xu Wang
Xu Wang

Xu Wang

Assistant Professor, Computer Science & Engineering (Collaborator)
Headshot of Elijah Bouma-Sims
Elijah Bouma-Sims

Elijah Bouma-Sims

PhD Student, Carnegie Mellon University (Collaborator)
Headshot of Lorrie Faith Cranor
Lorrie Cranor

Lorrie Cranor

Professor, Carnegie Mellon University (Collaborator)
Headshot of Michael M. McKee
Michael M. McKee

Michael M. McKee

Associate Professor, Michigan Medicine (Collaborator)

Alumni

Headshot of Emily Tsai
Emily Tsai

Emily Tsai

Masters Student, School of Information
Headshot of Mansanjam Kaur
Mansanjam Kaur

Mansanjam Kaur

Masters Student, School of Information
Headshot of Yifan Zhu
Yifan Zhu

Yifan Zhu

Masters Student, Computer Science & Engineering
Headshot of Andrew Dailey
Andrew Dailey

Andrew Dailey

Undergraduate Student, Computer Science & Engineering

Publications

We publish our research work in the most prestigious venues for human-computer interaction and accessibility research including CHI, UIST, and ASSETS. We have published over 25 high-impact peer-reviewed articles, of which eight have been honored with awards!

A user is wearing a smartwatch in front of water running down a sink. The smartwatch displays the identified sound as 'water running' with a classification confidence of 83%.
AWARD

SoundWatch Field Study

Real-World Feasibility of Sound Recognition
(Best paper honorable mention)
ASSETS 2023: PAPER | CODE
A close up shot of a person attending a 10-person video conference on a laptop.
AWARD

Classes Taught by DJ

EECS 495: Accessible Computing

This upper-level undergraduate class serves as an introduction to accessibility for undergraduate studdents and uses a curriculum designed by Professor Dhruv Jain. Students learn essential concepts related to accessibiity, disability theory, and user-centric design, and contribute to a studio-style team project in collaboration with clients with a disability and relevant stakeholders we recruit. This intense 14-week class requires working in teams to lead a full scale end-to-end accessibility project from its conceptualization, to design, to implementation, and evaluation. The goal is to reach a level of proficiency comparable to that of a well-launched employee team in a computing industry. Often, projects terminate in real-world deployments and app releases.

Read more →

EECS 598: Advanced Accessibility

This graduate-level class focuses on advances topics in accessibility including disabilty theory, user-research, and their impact on technology. Includes guest lectures by esteemed researchers and practioners in the field of accessibility.

Read more →

Talks

Sound Sensing for Deaf and Hard of Hearing Users

Navigating Graduate School with a Disability

Deep Learning for Sound Awareness on SmartWatches

Field Study of a Tactile Sound Awareness Device

First slide of the talk. A scene of a kitchen in the background with the talk title: Field Deployment of a Smarthome Sound Awareness System for Deaf and Hard of Hearing Users

Field Deployment of a In-Home Sound Awareness System

First slide of the talk. Shows DJ riding on a camel in a desert. The title of the talk reads: Autoethography of a Hard of Hearing Traveler

Autoethnography of a hard of hearing traveler

First slide of the talk. A person claps in front of a tablet interface that visaulizes the clapping sound using a pulsating bubble. The title reads: Exploring Sound Awareness in the Home for People who are Deaf or Hard of Hearing

Exploring sound awareness in the home

First slide of the talk with an image of an ear doning a hearing aid. The title reads: Deaf and Hard of Hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies​

Online Survey of Wearable Sound Awareness

First slide of the talk showing a person walking and talking with another person. The first person is wearing a HoloLens which shows ​real-time captions in Augmented Reality. Title is Towards Accessible Conversations in a Mobile Context for People who are Deaf and Hard of Hearing.

Towards accessible conversations in a mobile context

First slide of the talk showing a rocky beach with waves crashing over the beach. Talk title reads: Immersive Scuba Diving Simulator Using Virtual Reality

Immersive scuba diving simulator using virtual reality​

First slide of the talk showing a round table conversation with a person wearing a Google Glass. The directions of the active speakers in the conversation are visualized as arrows on the Glass. Talk title is Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing.

HMD Visualizations to Support Sound Awareness

Videos

Lab Openings

Prospective PhD students: We are recruiting PhD students with the following set of skills: (1) those with prior HCI and user study experience to lead projects focused on Deaf/disabled population, or (2) those with prior applied ML experience around sound and audio, but who also can build HCI systems and run independent HCI focused projects (through conducting user studies, collecting data, and evaluating end-to-end systems). If you think you fit any of these skills, please email DJ at profdj [at] umich [dot] edu with a brief justification of your skill set (e.g., through relevant research experience), a list of projects from the lab you are interested in, and your CV.

Undergraduates/Masters students: Please complete this form and we will get back to you!

Potential PostDocs: Please email DJ with your reasoning for pursuing a post-doc position, a draft of your dissertation, and your CV.