基于关键动作双重转移概率的连续手语语句识别算法

计算机科学 ›› 2019, Vol. 46 ›› Issue (11A): 299-302.

基于关键动作双重转移概率的连续手语语句识别算法

李晨¹, 黄元元¹, 胡作进²

(南京航空航天大学计算机科学与技术学院南京210016)¹;
(南京特殊教育师范学院数学与信息科学学院南京210038)²

出版日期:2019-11-10 发布日期:2019-11-20
作者简介:李晨(1995-),女,硕士生,主要研究方向为模式识别、图像处理;黄元元(1975-),女,博士,副教授,主要研究方向为多媒体技术、图像处理、模式识别;胡作进(1965-),男,博士,教授,主要研究方向为数据处理、机器学习,E-mail:805861040@qq.com。
基金资助:
本文受江苏省“双创” 项目资助。

Continuous Sign Language Sentence Recognition Based on Double Transfer Probability of Key Actions

LI Chen¹, HUANG Yuan-yuan¹, HU Zuo-jin²

(College of Computer Science and Technology,Nanjing University of Aeronautics & Astronautics,Nanjing 210016,China)¹;
(College of Math and Information Science,Nanjing Normal University of Special Education,Nanjing 210038,China)²

Online:2019-11-10 Published:2019-11-20

摘要/Abstract

摘要： 目前,连续手语识别的最大难点在于如何对其中包含的词汇进行有效分割。本文将关键动作看作手语的基元,提出了一种基于关键动作双重转移概率的连续手语识别算法。在获得连续手语基元序列的前提下,根据相邻基元的词内及词间转移关系,可以有效地寻找到词汇边界,从而对基元序列做分割,并逐一识别出各基元分组的候选词汇。最后,根据不同基元分组的候选词汇间的转移概率,计算出对应合成句子的概率,并按照最大概率原则输出连续手语的最终识别结果。该算法容易实现,执行效率高,经实验验证其可以面向非特定人群。

关键词: 关键动作, 手语语句, 转移概率

Abstract: At present,the most difficult problem in continuous sign language recognition is how to split out the words effectively.In this paper,key actions were regarded as the basic units of sign language and an algorithm based on double transfer probability of key actions was proposed.After acquiring the sequence of basic units from continuous sign language,the boundaries of words can be effectively found by judging the intra-word and inter-word transfer relations of all adjacent basic units.Then the sequence of basic units are segmented by these boundaries and the candidate words of each group of basic units can be identified.Finally,according to the transfer probabilities between candidate words of different groups,the probability of corresponding synthetic sentence is calculated and then the final recognition result is output by the principle of maximum probability.The algorithm is easy to implement and has high execution efficiency.It can be applied to non-specific population through experimental verification.

Key words: Key actions, Sign language sentence, Transfer probability

中图分类号:

TP391.41

李晨, 黄元元, 胡作进. 基于关键动作双重转移概率的连续手语语句识别算法[J]. 计算机科学, 2019, 46(11A): 299-302. https://doi.org/

LI Chen, HUANG Yuan-yuan, HU Zuo-jin. Continuous Sign Language Sentence Recognition Based on Double Transfer Probability of Key Actions[J]. Computer Science, 2019, 46(11A): 299-302. https://doi.org/

参考文献

[1]STARNER T,PENTLAND A.Visual Recognition of American Sign Language Using Hidden Markov Models[C]∥International Workshop on Automatic Face & Gesture Recognition.1995:189-194.
[2]KONG W W,RANGANATH S.Automatic Hand TrajectorySegmentation and Phoneme Transcription for Sign Language[C]∥IEEE International Conference on Automatic Face & Gesture Recognition.IEEE,2008:1-6.
[3]KONG W W,RANGANATH S.Towards Subject Independent Continuous Sign Language Recognition[J].Pattern Recognition,2014,47(3):1294-1308.
[4]YANG H D,SCLAROFF S,LEE S W.Sign Language Spotting with a Threshold Model Based on Conditional Random Fields[M].IEEE Computer Society,2009.
[5]YANG R,SARKAR S,LOEDING B.Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2010,32(3):462-477.
[6]KOLLER O,ZARGARAN S,NEY H,et al.Deep Sign:Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs[J].International Journal of Computer Vision,2018,126(12):1311-1325.
[7]方高林,高文,陈熙霖,等.基于SRN/HMM的非特定人连续手语识别系统[J].软件学报,2002,13(11):2169-2175.
[8]FANG G,GAO W,ZHAO D.Large-Vocabulary ContinuousSign Language Recognition Based on Transition-Movement Models[J].IEEE Transactions on Systems,Man,and Cyberne-tics－Part A:Systems and Humans,2007,37(1):1-9.
[9]YANG W,TAO J,YE Z.Continuous sign language recognition using level building based on fast hidden Markov model[J].Pattern Recognition Letters,2016,78(C):28-35.
[10]XU X X,HUANG Y Y,HU Z J.Research on Continuous Sign Language Sentence Recognition Algorithm Based on Weighted Key-Frame[J].Journal of Advanced Computational Intelligence and Intelligent Informatics,2018,22(4):483-490.
[11]HUANG J,ZHOU W,ZHANG Q,et al.Video-based Sign Language Recognition without Temporal Segmentation[C]∥32nd AAAI Conference on Artificial Intelligence.2018:2257-2264
[12]中国残疾人联合会教育就业部,中国聋人协会.中国手语[M].北京:华夏出版社,2003.
[13]LI S R,HUANG Y Y,HU Z J,et al.Key Frame Detection Algorithm based on Dynamic Sign Language Video for the Non Specific Population[J].International Journal of Signal Proces-sing,Image Processing and Pattern Recognition,2015,8(12):135-148.
[14]SHI M M,HUANG Y Y,HU Z J.Dynamic Sign Language Recognition Algorithm Using Weighted Gesture Units[J].Journal of Information and Computational Science,2015,12(15):5611-5621.
[15]LIANG W L,HUANG Y Y,HU Z J.Real-Time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy[J].International Journal of Signal Processing,Image Processing and Pattern Recognition,2017,10(7):21-34.

相关文章 7

[1]	李昇智, 乔建忠, 林树宽. 一种基于用户移动行为相似性的位置预测方法 Location Prediction Method Based on Similarity of Users Moving Behavior 计算机科学, 2018, 45(12): 288-292. https://doi.org/10.11896／j.issn.1002-137X.2018.12.046
[2]	徐鑫鑫, 黄元元, 胡作进. 连续复杂手语中关键动作的提取算法 Extraction Algorithm of Key Actions in Continuous and Complex Sign Language 计算机科学, 2018, 45(11A): 189-193.
[3]	郭鑫鹏,黄元元,胡作进. 基于关键帧的连续手语语句识别算法研究 Research on Continuous Sign Language Sentence Recognition Algorithm Based on Key Frame 计算机科学, 2017, 44(Z11): 178-183. https://doi.org/10.11896/j.issn.1002-137X.2017.11A.037
[4]	郭艳卿,赵锐,孔祥维,付海燕,蒋金平. 基于事件要素加权的新闻摘要提取方法 News-summarization Extraction Method Based on Weighted Event Elements Strategy 计算机科学, 2016, 43(1): 237-241. https://doi.org/10.11896/j.issn.1002-137X.2016.01.051
[5]	王芳,李昆鹏,袁明新. 一种人工势场导向的蚁群路径规划算法 AntColony Algorithm Based on Optimization of Potential Field Method for Path Planning 计算机科学, 2014, 41(Z11): 47-50.
[6]	高晓东,杨亚涛,李子臣. SHA-3置换函数的差分转移概率分析 Differential Transition Probability Analysis of SHA-3Permutation Function 计算机科学, 2014, 41(3): 159-162.
[7]	王冠军,王茂励,赵莹. 基于马尔可夫决策模型的测试向量排序新方法 Research on Novel Test Vector Ordering Approach Based on Markov Decision Processes 计算机科学, 2010, 37(5): 287-290.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed