Speech to Text in the Classroom

The issue

francispam_mobile_biology_dsc7794-copy

Enabling students who are deaf/hard of hearing (DHH) enrolled in mainstream classes to have access to communication and to learn the information presented in classes is a critical task. The process is not an easy one, because DHH students have diverse communication needs. In order to provide each of these students an opportunity to learn and participate, support services need to be individually tailored. An example of a service that meets the needs of some students is real-time text, also called real-time captioning. The provider of the service, who is often in the classroom next to the DHH student(s), produces text as it is being spoken by the teacher or other students and displays it on a portable device so that the student can have access and understand what is taking place in the classroom.

What we know

In the two common speech-to-text service options, the provider uses either (a) a standard typing (QWERTY) keyboard or (b) a stenographic machine. A little used option is (c) automatic speech recognition.

These services may be provided in the classroom, or remotely. If remotely, the speaker typically wears a Bluetooth microphone and the spoken message is delivered using either a cellular phone or Voice over Internet Protocol (VoIP) via a cellular or broadband Internet connection (i.e., Skype, Google Hangout). These services also support display of text on diverse devices (standard laptops, smartphones, etc.).

Typing-based services (C-Print, Typewell) use a standard QWERTY keyboard. A trained service provider uses the word-abbreviation feature in the system’s software to speed up typing. The software transforms abbreviations into full words on the computer screen. In addition, providers learn strategies for identifying important points and for condensing information. These systems cannot provide word-for-word transcription because they cannot keep up with the speed of speech; however, the systems do capture almost all of the meaning of what is being said.

Research studies have compared comprehension and retention of information after viewing a lecture with C-Print speech-to-text support and after viewing a comparable lecture with interpreter support. Students retained more or an equal amount of information with C-Print as they did with interpreter support.

With a steno-based system, a trained stenographer uses a 24-key machine to encode spoken words phonetically into a computer that converts it into the language’s text (e.g., English) and displays it in real time. The stenographer depresses several keys simultaneously instead of sequentially as in conventional typing. Generally, the text is produced verbatim.

Research has found that DHH students demonstrate significantly better comprehension of a spoken message with real-time text technology than when only following the video and audio of the message. When student lecture comprehension with stenographic, C-Print, and interpreting services was compared in a research study, no real differences were found among the three services.

What we don’t know

With respect to what we do not know, one lingering question pertains to what information students acquire when teachers use visual materials, such as PowerPoint slides, and the student uses real-time text for communication access.  If students pay full attention to the text display, they may miss essential information in the visuals; if they pay full attention to the visuals they may miss crucial information in the text. A second question is whether automatic speech recognition (ASR) is approaching a level where it can produce real-time text that provides effective communication access without the assistance of a service provider.

Implications

Research supports the idea that some DHH students prefer real-time text instead of sign language interpreters or note takers as a means of acquiring information in the classroom. Other students prefer an interpreter. It is—or at least should be—an individual student’s choice that educators involved in provision of communication access (Disability Support Services Provider, Teacher of the Deaf, etc.) must work with in order to provide DHH students with fair and appropriate educational opportunities. If real-time text is the system of choice, educators will need to make arrangements to ensure that the service is effective. For the service to be beneficial, they need to work with teachers to make sure that the provider receives materials in advance, that the provider is able to distribute a copy of the saved text to the DHH student after class, and so on.

Posted on Jan 11, 2016 by
Michael S. Stinson
msserd {at} rit.edu
and
Pamela Francis
pggncp {at} rit.edu
National Technical Institute for the Deaf
Rochester Institute of Technology

Further reading