Arabic Sign Language Recognition an Image-Based Approach

In this paper we propose an image based system for Arabic sign language recognition. A Gaussian skin color model is used to detect the signer's face. The centroid of the detected face is then used as a reference to track the hands movement using region growing from the sequence of images comprising the signs. A number of features are then selected from the detected hand regions across the sequence of images. The recognition stage is performed using a hidden Markov model. The proposed system achieved a recognition accuracy of about 93% for a data set of 300 signs with leave one out method.

[1]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[2]  Kazuyoshi Yoshino,et al.  Recognition of Japanese sign language from image sequence using color combination , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[3]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[4]  Shan Lu,et al.  Recognition of local features for camera-based sign language recognition system , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[5]  Cheng-Chang Lien,et al.  Sign language recognition using 3-D Hopfield neural network , 1995, Proceedings., International Conference on Image Processing.

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Hermann Hienz,et al.  Relevant features for video-based continuous sign language recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[8]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[9]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[10]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  T. Kobayashi,et al.  Partly-hidden Markov model and its application to gesture recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.