Speechreading is sometimes confused with lip-reading. But it is more; it uses the expression of the speakers face, movement of the limbs, his posture and even gait. It is used by the deaf and severely hearing-impaired people. But many use it along natural signing or sign systems.
Speechreading can improve speech recognition; investigations show that 15-20% of the information transferred in dialoques is conveyed by the visual information. This is of importance in e.g. videoconferencing. Knowledge of speech reading including lip and facial formation significantly improve the understandability of speaking avatars in games and on websites.
Together with international market & research partners we investigated high-speed AVSR (audio-visual speech recognition) as a means of improving speech recognition. Interesting applications are:
- communication by whispering in public spaces were silence is required,
- more realistic speaking avatars,
- long-range speak recognition e.g. through analysis of telescopic images in military and sports,
- surveillance and entry recognition
In the webshop are examples of high-speed camera registrations of the mouth-shape taken from 3 directions when uttering vowel-consonant-vowel combinations. The full registrations are available as are the full registrations of 10 representative Dutch sentences comprising several phoneme combinations.