Multimedia and Virtual Reality: Designing Multisensory User Interfaces
This is a Printer Friendly Article on the Virtual Worldlets Network.
Return to the web-view version.
Product Information
By Alistair Sutcliffe
Produced By Lawrence Erlbaum; illustrated edition
Additional Information


  Buy From Amazon.com
  Buy From Amazon.co.uk



Right from the preface, Multimedia and Virtual Reality identifies itself as something of an academic book, with a relatively dry prose. It has a great deal of fascinating content, carefully examined and presented, yet the language used, does assume a certain level of vocabulary.

The author sees this book as a bridge between human computer interfaces, and software engineering. He seems to believe that these two fields are two separate disciplines requiring reuniting.

Likewise, the book?s focus, for all its depth, is surprisingly narrow, not too bothered with VR technology or history, and more focussed on general interfaces for sensory immersion, regardless of how they are constructed.

In other words, this work focuses on the user?s perception of the virtual space, and general means to bring that about, rather than specific types of interface technology.

An example from the book, describing current means of VR, serves as a demonstration for the prose that fills the pages.

?
Whole body representations may be displayed, although they are controlled
by simple interactive devices (e.g., joystick, 3D space mouse). User
representations or avatars can be operated by a command language for
movement and change of facial expression, but the need to control the avatar
by keyboard or menu commands makes interaction complex and impairs
the sense of immersion, so most collaborative VE Web spaces use simple
controls for a full presence representation with typed text or speech communication.
?

From this point, the book goes on to describe other potential modalities for interfacing with VRs and the need for sensory immersion. No-one could possibly describe this book as easy reading, but, the content is all there.

Contents



Preface xi

1 Background and Usability Concepts l

Design Problems 3

Architectures and Devices 6

Multimedia Architectures 6

Virtual Reality Architectures 9

Augmented Reality 12

Tangible User Interfaces 12

Definitions and Terminology 14

Physical Media Formats and Storage 16

Logical Media Definitions 18

Summary 23

2 Cognitive Psychology for Multimedia Information Processing 24

Perception and Modalities 24

Vision 24

Audio Modality: Hearing and Speech 31

Proprioception 34

Haptic Sense: Touch 34

Olfaction and Gustation 35

Sensory Integration and Motor Coordination 36

Comprehension and Cognitive Models 37

Memory 42

Working Memory 42

Long-Term Memory 43

Organization of Memory 45

Thinking and Problem Solving 49

Mental Models 50

Levels of Reasoning 51

Attention 53

Motivation and Arousal 55

Emotion 58

Stress and Fatigue 59

Principles for Multisensory User Interface Design 59

General Multimedia Principles 60

Principles for Virtual Reality Applications 61

Conventional HCI Principles 62

Summary 63

3 Models of Interaction 65

Modeling Multisensory Interaction 67

Models of Multimedia Interaction 68

Multimedia Conversations 71

Cognitive Resources 75

Interaction Models 82

Task-Action Model 82

System Initiative Model 85

Exploration and Navigation 87

Context Switching 88

Design Features 90

Integrating Resources and Action 91

Multimedia Interaction 93

Virtual Reality Walkthrough 94

Navigation-Exploration Cycle 97

System Initiative Cycle 99

Error Diagnosis and Recovery 103

Summary 106

4 Multimedia User Interface Design l08

Design Method Overview 109

Design Approach and Technology 111

Requirements and User Modeling 113

User Characteristics 113

Domain Modeling 115

Task and Information Modeling 117

Information Types 119

Information Analysis 122

Media Selection and Combination 124

Media Selection Guidelines 126

Aesthetic Motivations 134

Presentation Scripting 138

Concurrent or Sequential Presentation 140

Window Management 140

Navigation and Control 143

Hypermedia Dialogues 143

Navigation Controls 148

Specifying Media Controls 148

Designing the Reading or Viewing Sequence 148

Media Integration and Design for Attention 149

Salience Effects in Single Media 153

Case Study Example 156

Summary 158

5 Designing Virtual Environments 16O

Introduction 161

Development Approach 162

Task and Requirements Analysis for Virtual Environments 164

Task and Domain Models 167

Business Park Exploration : Case Study Example 172

Virtual Environment Design 173

Selecting Modalities and Interactive Devices 178

Pointing and Selecting 180

Speech in Virtual Worlds 181

Haptic Interaction 181

Selecting Communication Modalities 184

Designing the User Presence 185

Designing the Presence of Others 188

Business Park Example 192

Dialogue and Controls 193

Design of Interactive Agents 193

Viewpoint Controls 196

Navigation and Movement 197

Adding Interaction Support 199

Summary 200

6 Evaluating Multisensory User Interfaces 202

Introduction 203

Benchmark Evaluation 203

Diagnostic Evaluation 204

Data Analysis and Classification of Usability Errors 206

Eyetracking Analyses 209

Assessing Attractiveness 210

Assessing Multimedia Web Sites 213

Case Study: Web Site Evaluation 215

Evaluating Virtual Reality Applications 221

Expert Evaluation for Virtual Reality 222

Heuristic Evaluation 223

Walkthrough Evaluation 225

Diagnostic Evaluation 234

Summary 240

7 Applications, Architectures, and Advances 241

Educational Applications 241

Learning Environments 242

Individual-Level Learning 243

Social-Level Learning 243

Education Software Technology 247

Scripted Presentations and Hypermedia 247

Interactive Microworlds and Active Engagement 250

Virtual Prototyping 254

Requirements Analysis With Virtual Environments 255

Designing Virtual Prototypes 256

Scenario-Based Requirements Analysis 259

Future Trends 260

Intelligent Multisensory User Interfaces 261

Frontiers for Multisensory Interaction 262

Ubiquitous Multisensory Experience 265

Summary 267

Appendixes 269

A-Multimedia Design Guidelines From ISO 14915, Part 3 269

B-Generalized Design Properties 275

General Multimedia GDPs 276

General Virtual World GDPs 278

Task-Action Cycle 278

Navigation and Exploration Cycle 285

Responsive Action Cycle 290

References 295

Author Index 3O7

Subject Index 311

About the Author 333