UNIT 4: CRITICAL EVALUATION OF COMPUTER BASED
TECHNOLOGY
Table of contents
1.0 Introduction
2.0 Objectives
3.0 Main Content
3.1 Multi-Sensory Systems
3.2 Multimodal and Multi-media systems:
3.2.1
Speech
3.2.2 Problems in Speech Recognition.
3.2. 3 Speech Related Human-Interaction Technologies.
3. 3 Sounds
3.4 Recognition and Gestures
3.5 Devices for the Elderly and Disabled
4.0 Conclusion
5.0 Summary
6.0 Tutor Marked Assignment
7.0 Further Reading/References
1.0 Introduction
This unit briefly describes some uncommon technologies associated with human computer
interaction. These are innovations that improve upon the user interface, particularly those
innovations benefiting the disabled. Technologies such as the phonetic typewriter, the ear cons, the
auditory icons, the recognition and gesture devices for the disabled and the elderly are described.
2.0 Objectives
By the end of this unit, you should be able to:
• Describe multi-modal, multi-media and multi-sensory systems
• Appreciate the speech and the Phonetic typewriter interfaces
• Understand the Ear cons and Auditory Icons as important components of multi-modal
systems
• Know that Recognition and Gestures Devices are essential for the Elderly and Disabled
3.0 Main Content
3.1 Multi-Sensory Systems
Here, more than one sensory channel are involved in interaction as in sounds, text, hypertext,
1
, animation, video, gestures and vision.
They are used in a range of applications particularly for users with special needs and virtual reality.
The components of Multi-Sensory systems are: Speech, Non-speech sounds, Handwriting, together
with their applications and principles.
Usable Senses
The five senses: sight, sound, touch, taste and smell are used by us every day and each is
important on its own Together, they provide a fuller interaction with the natural world
We can ideally utilise the Computers to use all the available senses but this becomes practically
impossible because computers rarely offer such a rich interaction.
We can use the sight, sound, and sometimes the touch senses, but we cannot yet use the taste and
smell.
3.2 Multi-modal and Multi-media systems:
Multi-modal systems
These use more than one sense (or mode) of interaction as in visual and aural senses. For
example, a text processor may speak the words as well as echo them to the screen
Multi-media systems
These use a number of different media to communicate information. For example, a computer-
based teaching system may use video, animation, text and still images; different media all using
the visual mode of interaction. These may also use sounds, both speech and non-speech. Of
course two or more media now use different modes.
3.2.1 Speech
Human beings have a great and natural mastery of speech which makes it difficult to appreciate the
complexities but it is an easy medium for communication
Simple terminologies used to describe speech:
The structure of speech is called phonemes and there are 40 of them. The phonemes as basic
atomic units that sound slightly different depending on the context they are in, the larger units of
phonemes are the Allophones. Allophones are the sounds in the language between 120 and 1 30
and are formed into morphemes. The morphemes are the smallest units of language that have
meaning.Prosody is the alteration in tone and quality. They are also variations in emphasis, stress,
pauses and pitch. They impart more meaning to sentences.Co-articulation is the effect of context
on the sound. It transforms the phonemes into allophones. Syntax is the term used for the
structure of sentences while semantics is the collective term used for the meaning of sentences.
3.2.2 Problems in Speech Recognition.
Different people speak differently because accent, intonation, stress, idiom, volume, etc. differ. The
syntax of semantically similar sentences may also vary while background noises can interfere.
People often "ummm....." and "errr ....... " but words are not enough - semantics are also needed. It
requires
intelligence to understand a sentence because context of the utterance often has to be known as
well as information about the subject and speaker. For example, even if "Errr I, um, don't like this" is
recognised, it
2
TECHNOLOGY
Table of contents
1.0 Introduction
2.0 Objectives
3.0 Main Content
3.1 Multi-Sensory Systems
3.2 Multimodal and Multi-media systems:
3.2.1
Speech
3.2.2 Problems in Speech Recognition.
3.2. 3 Speech Related Human-Interaction Technologies.
3. 3 Sounds
3.4 Recognition and Gestures
3.5 Devices for the Elderly and Disabled
4.0 Conclusion
5.0 Summary
6.0 Tutor Marked Assignment
7.0 Further Reading/References
1.0 Introduction
This unit briefly describes some uncommon technologies associated with human computer
interaction. These are innovations that improve upon the user interface, particularly those
innovations benefiting the disabled. Technologies such as the phonetic typewriter, the ear cons, the
auditory icons, the recognition and gesture devices for the disabled and the elderly are described.
2.0 Objectives
By the end of this unit, you should be able to:
• Describe multi-modal, multi-media and multi-sensory systems
• Appreciate the speech and the Phonetic typewriter interfaces
• Understand the Ear cons and Auditory Icons as important components of multi-modal
systems
• Know that Recognition and Gestures Devices are essential for the Elderly and Disabled
3.0 Main Content
3.1 Multi-Sensory Systems
Here, more than one sensory channel are involved in interaction as in sounds, text, hypertext,
1
, animation, video, gestures and vision.
They are used in a range of applications particularly for users with special needs and virtual reality.
The components of Multi-Sensory systems are: Speech, Non-speech sounds, Handwriting, together
with their applications and principles.
Usable Senses
The five senses: sight, sound, touch, taste and smell are used by us every day and each is
important on its own Together, they provide a fuller interaction with the natural world
We can ideally utilise the Computers to use all the available senses but this becomes practically
impossible because computers rarely offer such a rich interaction.
We can use the sight, sound, and sometimes the touch senses, but we cannot yet use the taste and
smell.
3.2 Multi-modal and Multi-media systems:
Multi-modal systems
These use more than one sense (or mode) of interaction as in visual and aural senses. For
example, a text processor may speak the words as well as echo them to the screen
Multi-media systems
These use a number of different media to communicate information. For example, a computer-
based teaching system may use video, animation, text and still images; different media all using
the visual mode of interaction. These may also use sounds, both speech and non-speech. Of
course two or more media now use different modes.
3.2.1 Speech
Human beings have a great and natural mastery of speech which makes it difficult to appreciate the
complexities but it is an easy medium for communication
Simple terminologies used to describe speech:
The structure of speech is called phonemes and there are 40 of them. The phonemes as basic
atomic units that sound slightly different depending on the context they are in, the larger units of
phonemes are the Allophones. Allophones are the sounds in the language between 120 and 1 30
and are formed into morphemes. The morphemes are the smallest units of language that have
meaning.Prosody is the alteration in tone and quality. They are also variations in emphasis, stress,
pauses and pitch. They impart more meaning to sentences.Co-articulation is the effect of context
on the sound. It transforms the phonemes into allophones. Syntax is the term used for the
structure of sentences while semantics is the collective term used for the meaning of sentences.
3.2.2 Problems in Speech Recognition.
Different people speak differently because accent, intonation, stress, idiom, volume, etc. differ. The
syntax of semantically similar sentences may also vary while background noises can interfere.
People often "ummm....." and "errr ....... " but words are not enough - semantics are also needed. It
requires
intelligence to understand a sentence because context of the utterance often has to be known as
well as information about the subject and speaker. For example, even if "Errr I, um, don't like this" is
recognised, it
2
