Interaction Design for 3D User Interfaces
The World of Modern Input Devices for Research, Applications, and Game Development
Preview
Book Description
In this new era of computing, where the iPhone, iPad, Xbox Kinect, and similar devices have changed the way to interact with computers, many questions have risen about how modern input devices can be used for a more intuitive user interaction. Interaction Design for 3D User Interfaces: The World of Modern Input Devices for Research, Applications, and Game Development addresses this paradigm shift by looking at user interfaces from an input perspective.
The book is divided into four parts:
- Theory of input devices and user interfaces, with an emphasis on multi-touch interaction
- Advanced topics on reducing noise on input devices using Kalman Filters
- A collection of hands-on approaches that allows the reader to gain experience with some devices
- A case study examining speech as input
Most of the chapters contain exercises that provide practical experience to enhance knowledge of the material in the related chapter. With its hands-on approach and the affordability of the required hardware, this book is an excellent flexible resource for both the novice and the expert in 3D user input device development. Researchers and practitioners will gain a much deeper understanding about user input devices and user interfaces. Game developers and software designers will find new techniques to improve their products by adding intuitive user interaction mechanisms to their games and applications. In addition to the resources provided in the book, its companion website, http://3DInputBook.com, provides additional resources, which include: additional exercises and project ideas, additional chapters, source code, and class instructors’ resources. The additional resources are provided to keep helping you with new research and new technology as it becomes available to help you stay up to date.
Table of Contents
THEORY
Introduction
The Vision
Human–Computer Interaction
Definitions
Further Reading
Input Interfaces
Introduction
Input Technologies
User Input Interfaces
Input Devices
Input Recognition
Virtual Devices
Input Taxonomies
Further Reading
Output Interfaces and Displays
3D Output Interfaces
Displays
Further Reading
Computer Graphics
Computer Graphics
Further Reading
3D Interaction
Introduction
3D Manipulation
Further Reading
3D Navigation
3D Travel
Wayfinding
3D Navigation: User Studies
Further Reading
Descriptive and Predictive Models
Introduction
Predictive Models
Descriptive Models
Further Reading
Multi-Touch
Introduction
Hardware
Multi-Touch and Its Applications
Figures of Large TabletTop Displays
Further Reading
Multi-Touch for Stereoscopic Displays
Understanding 3D Touch
Touching Parallaxes
Multi-Touch Above the Tabletop
Interaction with Virtual Shadows
Perceptual Illusions for 3D Touch Interaction
Pen and Multi-Touch Modeling and Recognition
Introduction
The Dollar Family
Proton++ and More
FETOUCH
Using Multi-Touch with PetriNets
Background
PeNTa: Petri Nets
Further Reading
Eye Gaze Tracking as Input in Human–Computer Interaction
Principle of Operation
Post-Processing of POG Data: Fixation Identification
Emerging Uses of EGT in HCI: Affective Sensing
Further Reading
Brain–Computer Interfaces: Considerations for the Next Frontier in Interactive Graphics and Games
Frances Lucretia Van Scoy
Introduction
Neuroscience Research
Implications of EEG and fMRI-Based Research for the Brain–Computer Interface
Neuroheadsets
A Simple Approach to Recognizing Specific Brain Activities Using Low-End Neuroheadsets and Simple Clustering Techniques
Evidence of Feasibility of Using EEG Data to Recognize Active Brain Regions
Conclusion
For Further Reading
ADVANCED TOPICS
Math for 3D Input
Steven P. Landers and David Rieksts
Introduction
Axis Conventions
Vectors
Matrices
Axis Angle Rotations
Two Vector Orientation
Calibration of Three Axis Sensors
Smoothing
Further Reading
Introduction to Digital Signal Processing
Introduction
What Is a Signal?
Classification of Signals
Applications of Digital Signal Processing
Noise
Signal Energy and Power
Mathematical Representation of Elementary Signals
Sampling Theorem
Nyquist–Shannon Theorem
Aliasing
Quantization
Fourier Analysis
Fast Fourier Transform
z-Transform
Convolution
Further Reading
Three Dimensional Rotations
Introduction
Three Dimensional Rotation
Coordinate Systems
Euler Angles
Quaternions
Further Reading
MEMS Inertial Sensors and Magnetic Sensors
Introduction
Inertial Sensors
MEMS Inertial Sensor Errors
Magnetometers
MEMS Magnetometer Errors
Further Reading
Kalman Filters
Introduction
Least Squares Estimator
Kalman Filter
Discrete Kalman Filter
Extended Kalman Filter
Further Reading
Quaternions and Sensor Fusion
Introduction
Quaternion-Based Kalman Filter
Quaternion-Based Extended Kalman Filter
Conversion between Euler and Quaternion
Further Reading
HANDS-ON
Hands-On: Inertial Sensors for 3D Input
Paul W. Yost
Introduction
Motion Sensing and Motion Capture
Types of Motion Sensing Technology
Inertial Sensor Configurations for Input
Hands-On: YEI 3-Space Sensors
Hands-On: YEI Prio for Whole-Body Input
Further Reading
Simple Hands-On Project with Unity 3D and Oculus Rift
Nonnarit O-larnnithipong
Installation and System Requirements
Getting Started
Creating Game Scene
Lighting, Camera and Skybox
GameObject and Basic Action Script
Graphic User Interface (GUI)
Oculus Rift Integration for Unity
Further Reading
Hands-On Approach with Leap Motion
Frank E. Hernandez
What Is Leap Motion
Installation
Hands-On Mini-Project
Further Reading
Hands-On Approach with Kinect Sensor v2
Frank E. Hernandez
What Is the Kinect Sensor
Installation
Hands-On Mini-Project
Further Reading
Creating Home-Brew Devices with Arduino Microcontrollers
Sudarat Tangnimitchok
Microcontroller
Analog Sensor
Serial Communication
Hands-On Project: Ultrasonic Proximity Sensor
Autonomous Bicycle with Gyroscope Sensor
Panuwat Janwattanapong and Mercedes Cabrerizo
Introduction
AU Self-Balancing Bicycle (AUSB)
Data Processing
System Implementation and Results
Conclusion
Further Reading
Exercise
Input Implementation Details
Input Devices
Multi-Touch Implementation
Working with a 3D Graphics Engine: OGRE
ECHoSS: Experiment Module
Further Reading
CASE STUDY: SPEECH AS INPUT
Multimodal Human-Like Conversational Interfaces
Ugan Yasavur and Christine Lisetti
Dialogue Management Overview
Dialogue Management in Health Dialogue Systems
Task-Based Spoken Dialog Systems
Embodied Conversational Agents
Brief Interventions for Alcohol Problems
Conclusion
Adaptive Dialogue Systems for Health
Ugan Yasavur and Christine Lisetti
Approach
Reinforcement Learning Background
Markov Decision Processes
Modeling World with Interconnected MDPs
Agent and Dialogue Strategy Learning
Reward Function Design
Speech Recognition and Language Model
Dialog Corpus
Conclusion
CONTENTS OF ONLINE SUPPLEMENTARY MATERIAL
The World of Gesture Recognition
Input Frameworks and Development Considerations for Input Technologies
Hands-On: Multi-Touch and Pen Using Microsoft Windows 8 and Windows 10 with Microsoft Surface Pro
Hands-On: Intel RealSense SDK.
APPENDICES
Displays
Jorge Chernicharo
Fixed Displays
Portable Displays
Hybrid Systems
Creating Your Own Virtual Reality Headset
Karrel Muller
Introduction
Google Cardboard
Author(s)
Biography
Francisco R. Ortega, PhD, is a postdoctorate research fellow at Florida International University, Miami, where he received his PhD in computer science. He is the current director of the Open Human-Interface Device Laboratory at Florida International University (http://openhid.com). He was a member of the Digital Signal Processing Laboratory at FIU, and has over 17 years of experience in software development and systems integration. His interests are in 3D user interfaces, input devices, human–computer interaction, 3D navigation, and input modeling, among other interests. He has multiple publications in journals, lecture notes, and conference proceedings.
Fatemeh Abyarjoo, PhD, received her PhD in Electrical engineering from Florida International University, Miami, where she was also a research assistant in the Digital Signal Processing Laboratory, focusing on sensor fusion for human motion tracking. She is currently a Fraud Risk Data Scientist, focusing on financial data analyzing. Her research interests are data mining, data analysis, statistical modeling, sensor fusion and wearable devices. She is a former Open Science Data Cloud PIRE National Science Foundation Fellow.
Armando Barreto, PhD, is a faculty member of the Electrical and Computer Engineering Department at Florida International University, Miami, as well as the director of FIU’s Digital Signal Processing Laboratory. He earned his PhD in electrical engineering from the University of Florida, Gainesville. His work has focused on applying DSP techniques to the facilitation of human–computer interactions, particularly for the benefit of individuals with disabilities. He has developed human–computer interfaces based on the processing of signals and has developed a system that adds spatialized sounds to the icons in a computer interface to facilitate access by individuals with "low vision." He is a senior member of the Institute of Electrical and Electronics Engineers and the Association for Computing Machinery.
Naphtali Rishe, PhD, is Eminent Chair Professor of Computer Science at Florida International University, Miami. He has authored three books on database design and geography and has edited five books on database management and high performance computing. He holds four US patents on database querying, semantic database performance, Internet data extraction, and computer medicine. He has also authored 300 papers in journals and proceedings on databases, software engineering, Geographic Information Systems, Internet, and life sciences. His TerraFly project—a 50-terabyte database of aerial imagery and Web-based GIS—has been extensively covered by worldwide press.
Malek Adjouadi, PhD, is a professor with the Department of Electrical and Computer Engineering at Florida International University, Miami. He received his PhD from the Electrical Engineering Department at The University of Florida, Gainesville. He is the founding director of the Center for Advanced Technology and Education funded by the National Science Foundation. His earlier work on computer vision to help persons with blindness led to his testimony to the US Senate on the committee of Veterans Affairs on the subject of technology to help persons with disabilities. His research interests are in image and signal processing with applications in neuroscience and assistive technology research.