“MEET THE KENYAN ENGINEER WHO CREATED GLOVES THAT TURN SIGN LANGUAGE INTO AUDIBLE SPEECH” screamed the all-caps headline. My first thought was, “How wonderful, I simply must write about this.” It didn’t take long before I discovered that this technological advancement for the hearing impaired is the subject of heated controversy.
Members of the Deaf community pointed out that devices created to improve their accessibility require them to accommodate people who can hear and not the other way around. The signer wears the gloves, carries the computers, and modifies the rate of signing to compensate for the machine’s limitations. The hearing people are in fact accommodated, not the Deaf.
Another problem with gloves that translate American Sign Language (ASL) into speech or text is that they overlook key parts of the ASL grammar including lifted or lowered eyebrows, a shift in the orientation of the signer’s torso, or a mouth movement.
The glove solution fails to take into account that facial expressions and the coarticulation that occurs when the hand shape in one sign anticipates the shape or location of the following sign.
Back in the 1980s, scientists began playing around with human gesture-based communication with the new computers that were opening vast new vistas of data-crunching research.
In 1983, Bell Labs engineer Gary Grimes invented a data entry glove programmed for ASL speakers. It consisted of a cloth glove onto which was sewn numerous touch, bend, and inertial sensors positioned to recognize the Single Hand Manual Alphabet for the American Deaf.
The circuitry was hard-wired to recognize 80 unique combinations of sensor readings to output a subset of the 96 printable ASCII characters. Grimes’ glove was never put into actual use or developed commercially.
Five years later, in 1988, Stanford University researchers James Kramer and Larry Leifer unveiled the first glove designed to facilitate interactions between deaf and non-deaf people. It was called the CyberGlove and came as part of a system that cost $3,500 – plus the glove.
The Stanford team was working on translating ASL into spoken English. Their CyberGlove consists of a custom-made cloth glove with up to 22 thin foil strain gauges sewn into the fabric to sense finger and wrist bending. Analog signals were converted electronically into a digital stream that can be read by a computer’s standard serial port.
A 3-space tracker can be mounted on the glove to get hand position in space. This device did become commercially available from Virtual Technologies.
The average American in the 1980s was not tuned in to these geeky developments meant to help the hearing impaired. All that changed in 2001 when a 17-year-old Colorado high school student named Ryan Patterson invented the American Sign Language Translator.
The system includes a soft, leather glove outfitted with ten sensors that a signer wears on one hand. A small computer associates each hand position with a corresponding letter. Finger-spelling ASL words translated each letter through a processing unit that turns the signal into a clearly visible, digital letter on a small liquid crystal display. The translation time for each letter was less than half a second and the device was customizable: users could “train” the system to recognize their individual signing style.
The teen’s invention won the Grand Award in the 2001 Intel International Science and Engineering Fair (named best of category in engineering) and placed first in the individual category at the 2001 Siemens Westinghouse Science & Technology Competition.
On March 11, 2002, Patterson received top honors and a $100,000 scholarship at Intel’s Science Talent Search, a competition often termed “the junior Nobel prize.”
After that, a variety of similar devices splashed across the media. None of them left their ivory academic towers for commercial production and distribution:
A glove with 15 flex sensors in the fingers invented by Ukrainian Team QuadSquad claimed first prize and $25,000 in the 2012 Microsoft Imagine Cup, a student technology competition.
Cornell students came up with a glove in 2014 that helps people with hearing disabilities by identifying and translating the user’s signs into spoken English.
In 2015, two Mexican researchers developed a glove to translate text and sign languages to facilitate conversation between deaf and mute people and those not conversant with sign language.
The same year, Saudi designer and media artist Hadeel Ayoub introduced her SignLanguageGlove that “translates sign language into speech in real-time” using a “smart” data glove.
In 2016, University of Washington undergraduates Thomas Pryor and Navid Azodi won the Lemelson-MIT Student Prize for a pair of gloves called SignAloud that recognized rudimentary ASL signs and was touted by NPR, Discover, Bustle, and other media sources.
Lance Forshay directs the ASL program at the University of Washington and watched this parade of related technologies get media attention and academic awards, including cash grants. Finally, he had to speak out:
“Initially, I didn’t want to deal with [SignAloud] because this has been a repeated phenomenon or fad. I was surprised and felt somehow betrayed because they obviously didn’t check with the Deaf community or even check with ASL program teachers to make sure that they are representing our language appropriately.”
From the Department of More of the Same, making the media rounds now is the story of 25-year-old Roy Allela, an engineer and inventor from Kenya who, in 2019, invented “Sign-IO” gloves that can translate signed hand movements to audible speech.
Rhodes Scholar and Ph.D. student at Emory University Rachel Kolb, born deaf, voiced the Deaf community’s concerns about all high-tech signer-based communications systems:
“ASL gloves are mainly created/designed to serve hearing people. The concept of the gloves is to render ASL intelligible to hearing people who don’t know how to sign but this misses and utterly overlooks so many of the communication difficulties and frustrations that Deaf people can already face.”
Are you listening?