NAVIG: Guidance system for the visually impaired using virtual augmented reality

Finding ones way to an unknown destination, navigating complex routes, finding inanimate objects; these are all tasks that can be challenging for the visually impaired. The project NAVIG (Navigation Assisted by artificial VIsion and GNSS) is directed towards increasing the autonomy of visually impaired users in known and unknown environments, exterior and interior, large scale and small scale, through a combination of a Global Navigation Satellite System (GNSS) and rapid visual recognition with which the precise position of the user can be determined. Relying on geographical databases and visually identified objects, the user is guided to his or her desired destination through spatialized semantic audio rendering, always maintained in the head-centered reference frame. This paper presents the overall project design and architecture of the NAVIG system. In addition, details of the new type of detection and localization device are presented in relation to guidance directives developed through participative design with potential users and educators for the visually impaired. A fundamental concept in this project is the belief that this type of assistive device is able to solve one of the major problems faced by the visually impaired: their difficulty in localizing specific objects.

[1]  J. O'Regan,et al.  Learning to Perceive with a Visuo — Auditory Substitution System: Localisation and Object Recognition with ‘The Voice’ , 2007, Perception.

[2]  Christophe Jouffrais,et al.  Route selection algorithm for Blind pedestrian , 2010, ICCAS 2010.

[3]  Christophe Jouffrais,et al.  Artificial Vision for the Blind: a Bio-Inspired Algorithm for Objects and Obstacles Detection , 2010, Int. J. Image Graph..

[4]  Cindy Cappelle,et al.  Multi-sensors data fusion using Dynamic Bayesian Network for robotised vehicle geo-localisation , 2008, 2008 11th International Conference on Information Fusion.

[5]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[6]  Anke M. Brock,et al.  Méthodes et outils de conception participative avec des utilisateurs non-voyants , 2010, IHM.

[7]  Bruce N. Walker,et al.  Navigation performance in a virtual environment with bonephones , 2005 .

[8]  Adam C. Winstanley,et al.  Spatial Characteristics of Walking Areas for Pedestrian Navigation , 2009, 2009 Third International Conference on Multimedia and Ubiquitous Engineering.

[9]  J. Fletcher.  Spatial Representation in Blind Children. 1: Development Compared to Sighted Children , 1980 .

[10]  Roberta L. Klatzky,et al.  Personal guidance system for the visually impaired , 1994, ASSETS.

[11]  Bettina Pressl,et al.  ODILIA - A Mobility Concept for the Visually Impaired , 2008, ICCHP.

[12]  Romina Kühn,et al.  Mobility Impaired Pedestrians Are Not Cars: Requirements for the Annotation of Geographical Data , 2008, ICCHP.

[13]  B. L. Bentzen,et al.  Audible Signage as a Wayfinding Aid: Verbal Landmark versus Talking Signs , 1995 .

[14]  Xu Liu,et al.  A camera phone based currency reader for the visually impaired , 2008, Assets '08.

[15]  Brian F. G. Katz,et al.  The effect of spatialization in a data sonification exploration task. , 2008 .

[16]  Brian F. G. Katz,et al.  Audio haptic feedbacks for an acquisition task in a multi-target context , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[17]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[18]  Christophe Jouffrais,et al.  Fusion of Artificial Vision and GPS to Improve Blind Pedestrian Positioning , 2011, 2011 4th IFIP International Conference on New Technologies, Mobility and Security.

[19]  Gert Jan Gelderblom,et al.  Inventory of Electronic Mobility Aids for Persons with Visual Impairments: A Literature Review , 2008 .

[20]  Malika Auvray,et al.  Perception With Compensatory Devices: From Sensory Substitution to Sensorimotor Extension , 2009, Cogn. Sci..

[21]  Mehdi Ammi,et al.  Multisensory VR interaction for protein-docking in the CoRSAIRe project , 2009, Virtual Reality.

[22]  Brian F. G. Katz,et al.  Spatial Audio Applied to Research with the Blind , 2011 .

[23]  Gerhard Weber,et al.  A New Approach for Pedestrian Navigation for Mobility Impaired Users Based on Multimodal Annotation of Geographical Data , 2007, HCI.

[24]  Bernard Oriola,et al.  Designing an assistive device for the blind based on object localization and augmented auditory reality , 2008, Assets '08.

[25]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[26]  Matthijs L. Noordzij,et al.  The influence of visual experience on the ability to form spatial mental models based on route and survey descriptions , 2006, Cognition.

[27]  S. Thorpe,et al.  NAVIG : Navigation Assisted by Artificial Vision and GNSS , 2010 .

[28]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[29]  Florian Dramas,et al.  NAVIG : An Object Localization System for the Blind , 2010 .

[30]  H. B. Mitchell,et al.  Multi-Sensor Data Fusion: An Introduction , 2007 .

[31]  Mathieu Gallay,et al.  Navigation assistance for blind pedestrians: guidelines for the design of devices and implications for spatial cognition* , 2013 .

[32]  Roberta L. Klatzky,et al.  A Geographical Information System for a GPS Based Personal Guidance System , 1998, Int. J. Geogr. Inf. Sci..

[33]  Bernard Oriola,et al.  Designing a Virtual Environment Framework for Improving Guidance for the Visually Impaired , 2012 .

[34]  Hugh F. Durrant-Whyte,et al.  Sensor Models and Multisensor Integration , 1988, Int. J. Robotics Res..

[35]  Christophe Jouffrais,et al.  Assistive device for the blind based on object recognition: an application to identify currency bills , 2009, Assets '09.

[36]  Thomas Ertl,et al.  Design and development of an indoor navigation and object identification system for the blind , 2004, ACM SIGACCESS Access. Comput..

[37]  Philippe Truillet,et al.  Ivy: un bus logiciel au service du développement de prototypes de systèmes interactifs , 2002, IHM '02.

[38]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[39]  Mehdi Ammi,et al.  Multisensory VR exploration for computer fluid dynamics in the CoRSAIRe project , 2009, Virtual Reality.