More Web Proxy on the site http://driver.im/

research-article

Trigger Detection System for American Sign Language using Deep Convolutional Neural Networks

Authors:

Debasrita Chakraborty,

Deepankar Garg,

Jonathan H. ChanAuthors Info & Claims

IAIT '18: Proceedings of the 10th International Conference on Advances in Information Technology

Article No.: 4, Pages 1 - 6

https://doi.org/10.1145/3291280.3291783

Published: 10 December 2018 Publication History

Abstract

Automatic trigger-word detection in speech is a well known technology nowadays. However, for people who are incapable of speech or are in some silence zone, such voice activated trigger detection systems find no use. We have developed a trigger detection system using the 24 static hand gestures of the American Sign Language (ASL). Our model is primarily based on Deep Convolutional Neural Network (Deep CNN) as they are capable of capturing interesting visual features at each hidden layer. We aim at constructing a customisable switch that can turn 'on' if it finds a given trigger gesture in any video that it receives and stays 'off' if it does not. The model was trained on images of various hand gestures in a multi-class classification setting. This allows the user to choose a custom trigger gesture for oneself. To test the efficiency of such a model in the trigger detection process, we have made 7,000 videos (each 10s long) consisting of random images from the test set which were never shown to the model during the training process. It is experimentally shown that such a system has a better performance than the other state-of-the art techniques used in static hand gesture image recognition tasks. This approach also finds real-time application and can be applied to develop small scale devices which trigger any particular response by capturing the gestures made by the people.

References

[1]

F. Ge and Y. Yan. 2017. Deep Neural Network based Wake-Up-Word Speech Recognition with Two-Stage Detection. In International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2761--2765.

[2]

A. Ghosh, D. Chakraborty, and A. Law. 2018. Artificial Intelligence in Internet of Things. CAAI Transactions on Intelligence Technology (October 2018). http://digital-library.theiet.org/content/journals/10.1049/trit.2018.1008

[3]

A. Ghosh, N. R. Pal, and S. K. Pal. 1993. Self-Organization for Object Extraction using a Multilayer Neural Network and Fuzziness Measures. IEEE Transactions on Fuzzy Systems 1, 1 (1993), 54--68.

Digital Library

[4]

I. Goodfellow, Y. Bengio, and A. Courville. 2016. Deep Learning. MIT Press. https://books.google.co.in/books?id=Np9SDQAAQBAJ

Digital Library

[5]

J. Huang, W. Zhou, H. Li, and W. Li. 2015. Sign Language Recognition using 3D Convolutional Neural Networks. In IEEE International Conference on Multimedia and Expo (ICME), 2015. IEEE, 1--6.

[6]

Kaggle. 2017. Sign Language MNIST: Drop-In Replacement for MNIST for Hand Gesture Recognition Tasks. https://www.kaggle.com/datamunge/sign-language-mnist

[7]

H. L. Lane and F. Grosjean. 2017. Recent Perspectives on American Sign Language. Psychology Press. https://books.google.co.in/books?id=Fcg3DwAAQBAJ

[8]

M. Lech, B. Kostek, and A. Czyżewski. 2013. Examining Classifiers Applied to Static Hand Gesture Recognition in Novel Sound Mixing System. In Multimedia and Internet Systems: Theory and Practice. Springer, 77--86.

[9]

A. M. Lindahl. 2016. Speech Recognition Wake-Up of a Handheld Portable Electronic Device. US Patent 9,245,527.

[10]

P. Molchanov, S. Gupta, K. Kim, and J. Kautz. 2015. Hand Gesture Recognition with 3D Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1--7.

[11]

K. R. Murthy and A. Ghosh. 2012. An Efficient Illumination Invariant Face Recognition Technique using Two Dimensional Linear Discriminant Analysis. In Recent Advances in Information Technology (RAIT), 2012 1st International Conference on. IEEE, 62--67.

[12]

G. Naithani, J. Kivinummi, T. Virtanen, O. Tammela, M. J. Peltola, and J. M. Leppänen. 2018. Automatic Segmentation of Infant Cry Signals using Hidden Markov Models. EURASIP Journal on Audio, Speech, and Music Processing 2018, 1 (2018), 1.

Digital Library

[13]

S. Nowozin, P. Kohli, and J. D. J. Shotton. 2017. Gesture Detection and Recognition. US Patent 9,619,035.

[14]

E. Ohn-Bar and M. M. Trivedi. 2014. Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations. IEEE Transactions on Intelligent Transportation Systems 15, 6 (2014), 2368--2377.

[15]

S. H. K. Parthasarathi, B. Hoffmeister, B. King, and R. Maas. 2017. Anchored Speech Detection and Speech Recognition. US Patent App. 15/196,228.

[16]

L. Pigou, S. Dieleman, P.J. Kindermans, and B. Schrauwen. 2014. Sign Language Recognition using Convolutional Neural Networks. In Workshop at the European Conference on Computer Vision. Springer, 572--578.

[17]

J. L. Raheja, A. Mishra, and A. Chaudhary. 2016. Indian Sign Language Recognition using SVM. Pattern Recognition and Image Analysis 26, 2 (2016), 434--441.

Digital Library

[18]

S. S. Rautaray and A. Agrawal. 2015. Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intelligence Review 43, 1 (2015), 1--54.

Digital Library

[19]

S.W. Salvador, J. P.Lilly, F. V. Weber, J. P. Adams, and R. P.Thomas. 2016. Wake Word Evaluation. US Patent 9,275,637.

[20]

D. Yu and L. Deng. 2014. Automatic Speech Recognition: A Deep Learning Approach. Springer London. https://books.google.co.in/books?id=rUBTBQAAQBAJ

Digital Library

Cited By

Hoshino YRathnayake NLinh Dang TRathnayake U(2024)Cascaded-ANFIS and Its Successful Real-World ApplicationsFuzzy Logic Controllers and Applications10.5772/intechopen.1006491Online publication date: 28-Aug-2024
https://doi.org/10.5772/intechopen.1006491
Ramirez SL. Ramírez MTito EApaza H(2024)Prototype App Mobile for Real Time American Sign Language Recognition Based on Deep LearningIntelligent Systems and Applications10.1007/978-3-031-47724-9_14(203-211)Online publication date: 19-Apr-2024
https://doi.org/10.1007/978-3-031-47724-9_14
Boggaram ABoggaram ASharma ASrinivasa Ramanujan AR. B(2022)Sign Language Translation SystemsInternational Journal of Software Science and Computational Intelligence10.4018/IJSSCI.31144814:1(1-33)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.4018/IJSSCI.311448
Show More Cited By

Index Terms

Trigger Detection System for American Sign Language using Deep Convolutional Neural Networks
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

American and russian sign language dactyl recognition
PETRA '19: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

Sign languages are the main way for people from deaf community to communicate with other people. In this paper, we have compared several real-time sign language dactyl recognition systems using deep convolutional neural networks. Our system is able to ...
Convergence of deep convolutional neural networks
Abstract
Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep networks with the Rectified ...
I-vectors and Deep Convolutional Neural Networks for Language Identification in Clean and Reverberant Environments
Computational Linguistics and Intelligent Text Processing
Abstract
In the current study, a method for automatic language identification based on deep convolutional neural networks (DCNN) and the i-vector paradigm is proposed. Convolutional neural networks (CNN) have been successfully applied to image ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IAIT '18: Proceedings of the 10th International Conference on Advances in Information Technology

December 2018

145 pages

ISBN:9781450365680

DOI:10.1145/3291280

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

KMUTT: King Mongkut's University of Technology Thonburi

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 December 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IAIT 2018

IAIT 2018: The 10th International Conference on Advances in Information Technology

December 10 - 13, 2018

Bangkok, Thailand

Acceptance Rates

IAIT '18 Paper Acceptance Rate 20 of 47 submissions, 43%;

Overall Acceptance Rate 20 of 47 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
233
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hoshino YRathnayake NLinh Dang TRathnayake U(2024)Cascaded-ANFIS and Its Successful Real-World ApplicationsFuzzy Logic Controllers and Applications10.5772/intechopen.1006491Online publication date: 28-Aug-2024
https://doi.org/10.5772/intechopen.1006491
Ramirez SL. Ramírez MTito EApaza H(2024)Prototype App Mobile for Real Time American Sign Language Recognition Based on Deep LearningIntelligent Systems and Applications10.1007/978-3-031-47724-9_14(203-211)Online publication date: 19-Apr-2024
https://doi.org/10.1007/978-3-031-47724-9_14
Boggaram ABoggaram ASharma ASrinivasa Ramanujan AR. B(2022)Sign Language Translation SystemsInternational Journal of Software Science and Computational Intelligence10.4018/IJSSCI.31144814:1(1-33)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.4018/IJSSCI.311448
Bhushan SAlshehri MKeshta IChakraverti ARajpurohit JAbugabah A(2022)An Experimental Analysis of Various Machine Learning Algorithms for Hand Gesture RecognitionElectronics10.3390/electronics1106096811:6(968)Online publication date: 21-Mar-2022
https://doi.org/10.3390/electronics11060968
Rathnayake NLinh Dang THoshino Y(2022)Designing and Implementation of Novel Ensemble model based on ANFIS and Gradient Boosting methods for Hand Gestures ClassificationProceedings of the 11th International Symposium on Information and Communication Technology10.1145/3568562.3568598(283-289)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1145/3568562.3568598
Goswami DLaw AChakraborty DDey A(2020)Convolutional neural network for prediction of COVID-19 from chest X-ray imagesCSBio '20: Proceedings of the Eleventh International Conference on Computational Systems-Biology and Bioinformatics10.1145/3429210.3429219(104-105)Online publication date: 19-Nov-2020
https://dl.acm.org/doi/10.1145/3429210.3429219
Hasan MSrizon ASayeed AHasan M(2020)Classification of American Sign Language by Applying a Transfer Learned Deep Convolutional Neural Network2020 23rd International Conference on Computer and Information Technology (ICCIT)10.1109/ICCIT51783.2020.9392703(1-6)Online publication date: 19-Dec-2020
https://doi.org/10.1109/ICCIT51783.2020.9392703
Hasan MSrizon ASayeed AHasan M(2020)Classification of Sign Language Characters by Applying a Deep Convolutional Neural Network2020 2nd International Conference on Advanced Information and Communication Technology (ICAICT)10.1109/ICAICT51780.2020.9333456(434-438)Online publication date: 28-Nov-2020
https://doi.org/10.1109/ICAICT51780.2020.9333456
Luqman HEl-Alfy EBinMakhashen G(2020)Joint space representation and recognition of sign language fingerspelling using Gabor filter and convolutional neural networkMultimedia Tools and Applications10.1007/s11042-020-09994-080:7(10213-10234)Online publication date: 18-Nov-2020
https://doi.org/10.1007/s11042-020-09994-0
Makarov IVeldyaykin NChertkov MPokoev AMakedon F(2019)American and russian sign language dactyl recognitionProceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3316782.3316786(204-210)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3316782.3316786
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten