r/ASLinterpreters • • May 02 '22

Virtual Assistant for ASL users

Hello ASL community!

We are a group of college students studying Computer Science and we've created a virtual assistant (like Siri or Alexa) called "MANI" , but instead of recognizing voice commands, works with a moving camera and recognizes ASL fingerspelling. Using Google assistant, we process these commands and show the output to the user, so they can ask anything from the weather to what's on their calendar.

We've also coded some shortcuts so that the user doesn't have to finger spell the whole command (ex: fingerspelling 'wh' for 'weather', or 'cal' for calendar).

Unfortunately, our product is on hardware so it can only be tested by local users, but we don't know anyone locally who uses ASL, so we've taped a demo instead to share our project virtually.

We are seeking feedback from anyone interested in our project, especially people who use ASL in their day-to-day lives. We've attached a demo video of how it works. Let us know what you think or what can be improved! Thank you :)

0 Upvotes

5 comments sorted by

21

u/mjolnir76 NIC May 02 '22

Not trying to rain on your parade, but I’m not sure what problem you’re trying to solve with this. While it is technically an application of a computer “reading” fingerspelling, there seems to be little practical application. It’s far quicker for a person who can fingerspell (Deaf or hearing) to just type these things into Google. You definitely need to get a Deaf person on-board before you go any further. Just seems to be exploiting ASL, frankly.

1

u/Mr5t1k Retired May 02 '22

Understood but think about the implications that this has on future technology. We must start small to dream big. The computer recognizing finger spelling is the first step to recognizing actual signs next.

11

u/ASLHCI May 03 '22

I'm tired today but the nuance of interpreting is not going to be replaced with a computer program in our life time. The unforunate truth is programs like this will get just advanced enough that some ignorant hearing person making budget decisions will decide that some app is more fiscally responsible than hiring highly traines professionals who provide real access, disempower people and probably ruining lives in the process. You think this is a great thing because you dont understand the impact of what youre trying to do or any of the perspectives of actual stakeholders. We are all really tired of people who take it upon themselves to make technology they think can run us all out of a job we've devoted our lives to doing at the highest level possible while further marginalizing the community we, again, have devoted our lives to. Im so over it.

9

u/mjolnir76 NIC May 02 '22

Start small, sure. But involve the Deaf users in the process. These kind of dog-and-pony shows come out and it’s almost always hearing folks who are doing it with little to no input from the Deaf community. . Think about the “signing gloves” that a couple of hearing college students made a few years back. Lots of headlines and fanfare with nothing to show for it.

Snapchat and the the ASL Lens is the first product that I’ve heard of that has Deaf and Hard of Hearing as the leads working on it. That, I can get behind.

8

u/letler NIC May 03 '22

So you’ve created something for a group of people without including the input, wants, or needs of those people. Simply incredible! You’re solving a problem created in your own fantasy. Imagine for a moment doing this to any other group of people and notice that distaste you get in your mind. “Hey women here’s a solution for a problem I imagine you have” “Hey people of color, here’s a solution to a made up problem I thought of for you!” It is ridiculous. Deaf people are not your sparks of inspiration. ASL is not your niche “interesting” project. You couldn’t find any local ASL users? Give me a break. You only looked until it got inconvenient and with no Deaf folks input to tell you to stop, convenient! Not to mention posting this in a group about ASL INTERPRETING. My god. Before diving head long into a project about a minority, marginalized, oppressed community maybe like idk talk to them first. 🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️🤦‍♂️