Feedforward Modalities in Remote Pointing
Date: 2012-06-20 02:30 PM – 04:00 PM
Last modified: 2012-04-27
Abstract
Remote pointing devices like the Wii remote and Smart TV remote controllers have a wide range of applications and are becoming more important for the manipulation of and interactions with information on a distant display due to wide-screen display and wireless technology in electric home appliances as well as the most advanced computing technology of virtual environment and augmented reality. Because remote pointing devices are used without external support, however, muscular tremors and motional disparity between the display and motor space can result in usability problems of mouse jitters and instability. In the present study, we propose a solution using feedforward technology with multisensory stimulation in which a user is provided with predictive information while approaching a target. Using a psychophysical experiment for user behavioral effectiveness and a survey for subjective rating for user satisfaction and difficulty, the feedforward technique was found to be more effective than was typical feedback. Also, the modality variations in feedforward were discussed. In particular, the feedforward with auditory and tactile stimulation was effective in user experience. The findings can be used to improve user interfaces for remote pointing controllers.
References
Akamatsu, M., MacKenzie, I. S., & Hasbroucq, T. (1995). A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics, 38, 816-827.
Akamatsu, M., & MacKenzie, I. S. (1996). Movement characteristics using a mouse with tactile and force feedback. International Journal Human-Computer Studies, 45(4), 483-493.
Beaudouin-Lafon, M., & Conversy, S. (1996). Auditory illusions for audio feedback. Proceedings of the International Conference Companion on Human Factors in Computing Systems : Common Ground (CHI ’96), 299-300.
Blatter, K., Graw, P., Münch, M., Knoblauch, V., Wirz-Justice, A., & Cajochen, C. (2006). Gender and age differences in psychomotor vigilance performance under differential sleep pressure conditions. Behavioural Brain Research, 168, 312-317.
Bowman, D. A., Chen, J., Wingrave, C. A., Lucas, J., Ray, A., Polys, N. F., Li, Q., Haciahmetoglu, Y., Kim, J. S., Kim, S., Boehringer, S., & Ni, T. (2006). New directions in 3D user interfaces. International Journal of Virtual Reality, 5(2), 3-14.
Brewster, S. (1998). The design of sonically-enhanced widgets. Interacting with Computers, 11, 211-235.
Brewster, S., & Brown, L. M. (2004). Tactons: structured tactile messages for non-visual information display. Proceedings of the fifth conference on Australasian user interface (AUIC ’04), 28, 15-23.
Calvert, G. A., Spence, C., & Stein, B. E. (2004). Handbook of multisensory processes, MIT Press.
Charness, N., & Bosman, E. A. (1994). Age-related changes in perceptual and psychomotor performance: implications for engineering design. Experimental Aging Research, 20(1), 45-59.
Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48(9), 1129-1150.
Edwards, G. W. (2000). Performance and Usability of Force Feedback and Auditory Substitutions in a Virtual Environment Manipulation Task. Unpublished master's thesis. Virginia Polytechnic Institute and State University, Blacksburg, Virginia.
Forlines, C., & Balakrishnan, R. (2008). Evaluating tactile feedback and direct vs. indirect stylus input in pointing and crossing selection tasks. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (CHI '08), 1563-1572.
Fraser, J., & Gutwin, C. (2000). The Effects of Feedback on Targeting Performance in Visually Stressed Conditions. Proceedings of Graphic Interface Conference ’00, 19-26.
Fukawa, T., & Yoshida, S. (1988). Sex difference in susceptibility to delayed auditory feedback in oral reading tasks. Shinrigaku Kenkyu, 59(3), 144-150.
Grunwald, M. (2008). Human haptic perception. Boston: Birkhauser.
Halpern, D. F. (2000). Sex Differences in Cognitive Abilities, Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Hayne, S., Pendergast, M., & Greenberg, S. (1993). Gesturing through cursors: implementing multiple pointers in group support systems. Proceedings of the 26th Hawaii International Conference on System Sciences, 4, 4-12.
Ho. C., Reed, N., & Spence, C. (2007). Multisensory In-Car Warning Signals for Collision Avoidance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(8), 1107-1114.
Ho, C., Santangelo, V., & Spence, C. (2009). Multisensory warning signals : when spatial correspondence matters. Experimental Brain Research, 195(2), 261-272.
Hopp-Levine, P. J., Smith, C. A. P., Clegg, B. A., & Heggestad, E. D. (2006). Tactile interruption management: Tactile cues as task-switching reminders. Cognition, Technology & Work, 8, 137-145.
Hsu, S. H. Huang, C. C., Tsuang, Y. H., & Sun, J. S. (1999). Effects of age and gender on remote pointing performance and their design implications. International Journal of Industrial Ergonomics, 23(5-6), 461-471.
Ives, J. C., Kroll, W. P., & Bultman, L. L. (1993). Rapid movement kinematic and EMG control characteristics in males and females. Research Quarterly for Exercise and Sport, 64, 274-283.
Jacko, J. A., Scott, I. U., Sainfort, F., Barnard, L., Edwards, P. J., & Emery, V. K. (2003). Older adults and visual impairment: What do exposure times and accuracy tell us about performance gains associated with multimodal feedback?, CHI Letters 5(1), 33-40.
Jenkins, W. L., & Connor, M. B. (1949). Some design factors in making settings on a linear scale. Journal of Applied Psychology, 33(4), 395-409.
Jenness, J. W., Lattanzio, R. J., O'Toole, M., Taylor, N., & Pax, C. (2002). Effects of manual versus voice-activated dialing during simulated driving. Perceptual and Motor Skills, 94, 363-379.
Keogh, J., Morrison, S., & Barrett, R. (2004). Augmented visual feedback increases finger tremor during postural pointing. Experimental Brain Research, 159, 467–477.
Lamble, D., Kauranen, T., Laasko, M., & Summala, H. (1999). Cognitive load and detection thresholds in car following situations: Safety implications for using mobile (cellular) telephones while driving. Accident Analysis and Prevention, 31, 617-623.
Langdon, P. M., Keates, S., Clarkson, P. J., & Robinson, P. (2000) Using haptic feedback to enhance computer interaction for motion-impaired users. Proceedings of the 3rd International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT 2000), 23-25.
Lawther, A., & Griffin, M. J. (1987). Prediction of the incidence of motion sickness from the magnitude, frequency, and duration of vertical oscillation. Journal of the Acoustical Society of America, 82(3), 957-966.
MacKenzie, I. S., & Jusoh, S. (2001). An Evaluation of Two Input Devices for Remote Pointing. Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction (EHCI '01), 235-250.
Mithal, A. K., & Douglas, S. A. (1996). Differences in movement microstructure of the mouse and the finger-controlled isometric joystick. Proceedings of the SIGCHI conference on Human factors in computing systems: common ground (CHI '96), 300-307.
Poupyrev, I., Weghorst, S., Billinghurst, M., & Ichikawa, T. (1997). A framework and testbed for studying manipulation techniques for immersive VR. Proceedings of the ACM symposium on Virtual reality software and technology (VRST '97), 21-28.
Prestopnik, J. L., & Roskos-Ewoldsen, B. (1999). The Relations Among Wayfinding Strategy Use, Sense of Direction, Sex, Familiarity, and Wayfinding Ability. Journal of Environmental Psychology, 20(2), 177-191.
Schiff, W., & Oldaka, R., (1990). Accuracy of Judging Time to Arrival: Effects of Modality, Trajectory, and Gender. Journal of Experimental Psychology: Human Perception and Performance, 16(2), 303-316.
Sharit, J., & Czaja, S. J. (1994). Ageing, computer-based task performance, and stress: issues and challenges. Ergonomics, 37, 559-577.
Sivak, M. (1996). The information that drivers use: Is it indeed 90% visual?. Perception, 25, 1081-1089.
Spence, C., & Driver, J. (1997). Cross-modal links in attention between audition, vision, and touch: Implications for interface design. International Journal of Cognitive Ergonomics, 1, 351–373.
Sreedharan, S., Zurita, E. S., & Plimmer, B. (2007). 3D input for 3D worlds. Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces (OZCHI '07). 227-230.
Stein, R. B., & Lee, R. G. (1989). Tremor and Clonus. In V. B. Brooks, Editor, Handbook of Physiology (10th ed. Ed.), The Nervous System II, Washington, DC: American Psychological Association.
Tan, D. S., Czerwinski, M. P., & Robertson, G. G. (2006). Large Displays Enhance Optical Flow Cues and Narrow the Gender Gap in 3-D Virtual Navigation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2), 318-333.
Vaillancourt, E., & Newell, K. M. (2000). Amplitude changes in the 8-12, 20-25, and 40 Hz oscillations in finger tremor. Clinical neurophysiology, 111(10), 1792-1801.
Viaud-Delmon, I., Ivanenko, Y. P., Berthoz, A., & Jouvent, R. (1998). Sex, Lies and Virtual Reality. Nature Neuroscience, 1, 15-16.
Vitense, H., Jacko, J., & Emery, V. (2003). Multimodal feedback: An assessment of performance and mental workload. Ergonomics, 46, 68-87.
Voyer, D., Voyer, S., & Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117, 250-270.
Wolf, C. G., & Rhyne, J. R. (1993). Gesturing with shared drawing tools. Proceedings of INTERACT '93 and CHI '93 conference companion on Human factors in computing systems (CHI '93), 137-138.
Worden, A., Walker, N., Bharat, K., & Hudson, S. (1997). Making computers easier for older adults to use: area cursors and sticky icons. Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '97), 266-271.
Akamatsu, M., & MacKenzie, I. S. (1996). Movement characteristics using a mouse with tactile and force feedback. International Journal Human-Computer Studies, 45(4), 483-493.
Beaudouin-Lafon, M., & Conversy, S. (1996). Auditory illusions for audio feedback. Proceedings of the International Conference Companion on Human Factors in Computing Systems : Common Ground (CHI ’96), 299-300.
Blatter, K., Graw, P., Münch, M., Knoblauch, V., Wirz-Justice, A., & Cajochen, C. (2006). Gender and age differences in psychomotor vigilance performance under differential sleep pressure conditions. Behavioural Brain Research, 168, 312-317.
Bowman, D. A., Chen, J., Wingrave, C. A., Lucas, J., Ray, A., Polys, N. F., Li, Q., Haciahmetoglu, Y., Kim, J. S., Kim, S., Boehringer, S., & Ni, T. (2006). New directions in 3D user interfaces. International Journal of Virtual Reality, 5(2), 3-14.
Brewster, S. (1998). The design of sonically-enhanced widgets. Interacting with Computers, 11, 211-235.
Brewster, S., & Brown, L. M. (2004). Tactons: structured tactile messages for non-visual information display. Proceedings of the fifth conference on Australasian user interface (AUIC ’04), 28, 15-23.
Calvert, G. A., Spence, C., & Stein, B. E. (2004). Handbook of multisensory processes, MIT Press.
Charness, N., & Bosman, E. A. (1994). Age-related changes in perceptual and psychomotor performance: implications for engineering design. Experimental Aging Research, 20(1), 45-59.
Cockburn, A., & Brewster, S. (2005). Multimodal feedback for the acquisition of small targets. Ergonomics, 48(9), 1129-1150.
Edwards, G. W. (2000). Performance and Usability of Force Feedback and Auditory Substitutions in a Virtual Environment Manipulation Task. Unpublished master's thesis. Virginia Polytechnic Institute and State University, Blacksburg, Virginia.
Forlines, C., & Balakrishnan, R. (2008). Evaluating tactile feedback and direct vs. indirect stylus input in pointing and crossing selection tasks. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (CHI '08), 1563-1572.
Fraser, J., & Gutwin, C. (2000). The Effects of Feedback on Targeting Performance in Visually Stressed Conditions. Proceedings of Graphic Interface Conference ’00, 19-26.
Fukawa, T., & Yoshida, S. (1988). Sex difference in susceptibility to delayed auditory feedback in oral reading tasks. Shinrigaku Kenkyu, 59(3), 144-150.
Grunwald, M. (2008). Human haptic perception. Boston: Birkhauser.
Halpern, D. F. (2000). Sex Differences in Cognitive Abilities, Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Hayne, S., Pendergast, M., & Greenberg, S. (1993). Gesturing through cursors: implementing multiple pointers in group support systems. Proceedings of the 26th Hawaii International Conference on System Sciences, 4, 4-12.
Ho. C., Reed, N., & Spence, C. (2007). Multisensory In-Car Warning Signals for Collision Avoidance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(8), 1107-1114.
Ho, C., Santangelo, V., & Spence, C. (2009). Multisensory warning signals : when spatial correspondence matters. Experimental Brain Research, 195(2), 261-272.
Hopp-Levine, P. J., Smith, C. A. P., Clegg, B. A., & Heggestad, E. D. (2006). Tactile interruption management: Tactile cues as task-switching reminders. Cognition, Technology & Work, 8, 137-145.
Hsu, S. H. Huang, C. C., Tsuang, Y. H., & Sun, J. S. (1999). Effects of age and gender on remote pointing performance and their design implications. International Journal of Industrial Ergonomics, 23(5-6), 461-471.
Ives, J. C., Kroll, W. P., & Bultman, L. L. (1993). Rapid movement kinematic and EMG control characteristics in males and females. Research Quarterly for Exercise and Sport, 64, 274-283.
Jacko, J. A., Scott, I. U., Sainfort, F., Barnard, L., Edwards, P. J., & Emery, V. K. (2003). Older adults and visual impairment: What do exposure times and accuracy tell us about performance gains associated with multimodal feedback?, CHI Letters 5(1), 33-40.
Jenkins, W. L., & Connor, M. B. (1949). Some design factors in making settings on a linear scale. Journal of Applied Psychology, 33(4), 395-409.
Jenness, J. W., Lattanzio, R. J., O'Toole, M., Taylor, N., & Pax, C. (2002). Effects of manual versus voice-activated dialing during simulated driving. Perceptual and Motor Skills, 94, 363-379.
Keogh, J., Morrison, S., & Barrett, R. (2004). Augmented visual feedback increases finger tremor during postural pointing. Experimental Brain Research, 159, 467–477.
Lamble, D., Kauranen, T., Laasko, M., & Summala, H. (1999). Cognitive load and detection thresholds in car following situations: Safety implications for using mobile (cellular) telephones while driving. Accident Analysis and Prevention, 31, 617-623.
Langdon, P. M., Keates, S., Clarkson, P. J., & Robinson, P. (2000) Using haptic feedback to enhance computer interaction for motion-impaired users. Proceedings of the 3rd International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT 2000), 23-25.
Lawther, A., & Griffin, M. J. (1987). Prediction of the incidence of motion sickness from the magnitude, frequency, and duration of vertical oscillation. Journal of the Acoustical Society of America, 82(3), 957-966.
MacKenzie, I. S., & Jusoh, S. (2001). An Evaluation of Two Input Devices for Remote Pointing. Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction (EHCI '01), 235-250.
Mithal, A. K., & Douglas, S. A. (1996). Differences in movement microstructure of the mouse and the finger-controlled isometric joystick. Proceedings of the SIGCHI conference on Human factors in computing systems: common ground (CHI '96), 300-307.
Poupyrev, I., Weghorst, S., Billinghurst, M., & Ichikawa, T. (1997). A framework and testbed for studying manipulation techniques for immersive VR. Proceedings of the ACM symposium on Virtual reality software and technology (VRST '97), 21-28.
Prestopnik, J. L., & Roskos-Ewoldsen, B. (1999). The Relations Among Wayfinding Strategy Use, Sense of Direction, Sex, Familiarity, and Wayfinding Ability. Journal of Environmental Psychology, 20(2), 177-191.
Schiff, W., & Oldaka, R., (1990). Accuracy of Judging Time to Arrival: Effects of Modality, Trajectory, and Gender. Journal of Experimental Psychology: Human Perception and Performance, 16(2), 303-316.
Sharit, J., & Czaja, S. J. (1994). Ageing, computer-based task performance, and stress: issues and challenges. Ergonomics, 37, 559-577.
Sivak, M. (1996). The information that drivers use: Is it indeed 90% visual?. Perception, 25, 1081-1089.
Spence, C., & Driver, J. (1997). Cross-modal links in attention between audition, vision, and touch: Implications for interface design. International Journal of Cognitive Ergonomics, 1, 351–373.
Sreedharan, S., Zurita, E. S., & Plimmer, B. (2007). 3D input for 3D worlds. Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces (OZCHI '07). 227-230.
Stein, R. B., & Lee, R. G. (1989). Tremor and Clonus. In V. B. Brooks, Editor, Handbook of Physiology (10th ed. Ed.), The Nervous System II, Washington, DC: American Psychological Association.
Tan, D. S., Czerwinski, M. P., & Robertson, G. G. (2006). Large Displays Enhance Optical Flow Cues and Narrow the Gender Gap in 3-D Virtual Navigation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2), 318-333.
Vaillancourt, E., & Newell, K. M. (2000). Amplitude changes in the 8-12, 20-25, and 40 Hz oscillations in finger tremor. Clinical neurophysiology, 111(10), 1792-1801.
Viaud-Delmon, I., Ivanenko, Y. P., Berthoz, A., & Jouvent, R. (1998). Sex, Lies and Virtual Reality. Nature Neuroscience, 1, 15-16.
Vitense, H., Jacko, J., & Emery, V. (2003). Multimodal feedback: An assessment of performance and mental workload. Ergonomics, 46, 68-87.
Voyer, D., Voyer, S., & Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117, 250-270.
Wolf, C. G., & Rhyne, J. R. (1993). Gesturing with shared drawing tools. Proceedings of INTERACT '93 and CHI '93 conference companion on Human factors in computing systems (CHI '93), 137-138.
Worden, A., Walker, N., Bharat, K., & Hudson, S. (1997). Making computers easier for older adults to use: area cursors and sticky icons. Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '97), 266-271.