|
Lip Reading and Visemes
DefinitionsLip Reading
Viseme
We are just on the cusp of an age of visemes in virtual environments. Lip-synching technologies have advanced to the point where, given an input text stream, and knowing the language of said stream - how to pronounce the phonemes - a computer program can animate a virtual face with a compatible muscle structure, at the same time as it is converting text to speech. Likewise, hearing an audio stream, once the phonemes are identified from the audio track alone, visemes can be reproduced in a virtual avatar. This puts us in a unique position, where spoken language and written language can be represented equally, within a virtual environment, at least as far as visual appearance is concerned, if not yet audibly. However, this is why knowing the language is important, especially for written text. Different languages produce different phonemes, as the syllables themselves require different muscle movements to pronounce. One study found for example, 'lip-rounding' in French speakers, and prominent tongue movements in Arabic speakers. Thus, with text especially, it becomes necessary to know the language used, before beginning translation. There are several ways to do this. One of the simplest, is simply to tell the system before typing begins, what language is being used. Another, is to have the system analyse each word before it acts. This is fraught with difficulty, as often words which are pronounced with different phonemes in different languages - and thus with different visemes - look identical as text. ReferencesLip-reading computer picks out your language Staff Comments
|
|
||||||||||||||||||||||||||||||||||||
|