Abstract
Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available—for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance.
Similar content being viewed by others
References
Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440
Ahad MAR, Tan JK, Kim H, Ishikawa S (2010) Motion history image: its variants and applications. Mach Vis Appl
Aqmar MR, Shinoda K, Furui S (2012) Robust gait-based person identification against walking speed variations. IEICE Trans Inf Syst E95.D(2):668–676
Aristidou A, Cameron J, Lasenby J (2008) Real-time estimation of missing markers in human motion capture. Proc ICBBE
Bariska A (2007) Recovering periodically spaced missing samples. IEEE Signal Process Mag 24(6):127–129
Bashir K, Xiang T, Gong S (2009) Gait representation using flow fields. Proc Br Mach Vis Conf
Bazin AI (2006) On probabilistic methods for object description and classification. Doctoral Dissertation, Univ. of Southampton
BenAbdelkader C, Cutler R, Davis L (2001) Stride and cadence as a biometric in automatic person identification and verification. Proc Face Gesture Recognit
BenAbdelkader C, Cutler R, Davis L (2002) View-invariant estimation of height and stride for gait recognition. Proc Workshop Biom Authentication (BIOMET)
BenAbdelkader C, Cutler R, Davis L (2002) View-invariant estimation of height and stride for gait recognition. Proc Int ECCV 2002 Workshop Biom Authentication
BenAbdelkader C, Cutler R, Nanda H, Davis L (2001) Eigengait: motion-based recognition of people using image self similarity. Proc 3rd Int Conf Audio Video-Based Biom Person Authentication (AVBPA)
Birchfield S (1997) An elliptical head tracker. Proc Asilomar Conf Signals Syst Comput
Bissacco A, Soatto S (2009) Hybrid dynamical models of human motion for the recognition of human gaits. Int J Comput Vis 85(1):101–114
Bobick AF, Johnson AY (2001) A multi-view method for gait recognition using static body parameters. Proc 3rd Int Conf Audio-Video-Based Biom Person Authentication (AVBPA)
Bobick AF, Johnson AY (2001) Gait recognition using static, activity specific parameters. Proc. 2001 IEEE Conf Comput Vis Pattern Recog (CVPR)
Bowyer K, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Comput Vis Image Underst 101(1):1–15
Bradski GR (1998) Computer video face tracking for use in a perceptual user interface. Intel Technol J Q2
Bregler C, Malik J (1998) Tracking people with twists and exponential maps. Proc 1998 IEEE Conf Comput Vision Pattern Recognit (CVPR)
Caselles V, Coll B (1996) Snakes in movement. SIAM J Numer Anal 33(2):445–456
Cham TJ, Rehg JM (1999) A multiple hypothesis approach to figure tracking. Proc 1999 IEEE Conf Comput Vision Pattern Recognit (CVPR)
Chen C, Liang J, Zhao H, Hua H, Tian J (2009) Frame difference energy image for gait recognition with incomplete silhouettes. Pattern Recognit Lett 30(11):977–984
Chen C, Zhang J, Fleischer R (2010) Distance approximating dimension reduction of riemannian manifolds. IEEE Trans Syst Man Cybern B 40(1):208–217
Chowdhury AR, Kale A, Chellappa R (2003) Video synthesis of arbitrary views for approximately planar scenes. Proc 2003 IEEE Int Conf Acoust Speech Signal Process (ICASSP)
Colombo C, Del Bimbo A (1999) Generalized bounds for time to collision from first-order image motion. Proc 7th Int Conf Comput Vis (ICCV)
Comaniciu D. Ramesh V, Meer P (2000) Real-time tracking of non-rigid objects using mean shift. Proc 2000 IEEE Conf Comput Vis Pattern Recognit (CVPR)
Cunado D, Nixon MS, Carter JN (1997) Using gait as a biometric, via phase-weighted magnitude spectra. Proc 1st Int Conf Audio- Video-Based Biom Person Authentication (AVBPA)
Curwen R, Blake A (1992) Dynamic contours: real-time active splines. In: Blake A, Yuille A (eds) Active vision. MIT Press, pp 39–58
Cutler R, Davis L (2000) Robust real-time periodic motion detection, analysis and applications. IEEE Trans Pattern Anal Mach Intell 13(2):129–155
Cutting JE (1978) A program to generate synthetic walkers as dynamic point-light displays. Behav Res Methods Instrum 10(1):191–194
Cutting JE, Kozlowski LT (1977) Recognizing friends by their walk: gait perception without familiarity cues. Bull Psychon Soc 9(5):353–356
Cutting JE, Proffitt DR (1981) Gait perception as an example of how we may perceive events. In: Walk R, Pick HL (eds) Intersensory perception and sensory integration. Plenum, New York
Cutting JE, Proffitt DR, Kozlowski LT (1978) A biomechanical invariant for gait perception. J Exp Psychol Hum Percept Perform 4(3):353–372
Czyz J, Vandendorpe L (2002) Evaluation of LDA-based face verification with respect to available computational resources. Proc 2nd Int Workshop Pattern Recognit Inf Syst (PRIS)
Dagan E, Mano O, Stein GP, Shashua A (2004) Forward collision warning with a single camera. Proc Intell Veh Symp (IV)
Davies JW, Bobick AF (1997) The representation and recognition of action using temporal templates. Proc 1997 IEEE Conf Comput Vis Pattern Recognit (CVPR)
DiFranco DE, Cham TJ, Rehg JM (2001) Recovery of 3D articulated motion from 2D correspondences. Proc 2001 IEEE Conf Comput Vision Pattern Recognit (CVPR)
Dubuisson MP, Lakshmanan S, Jain AK (1996) Vehicle segmentation and classification using deformable templates. IEEE Trans Pattern Anal Mach Intell 18:293–308
Eldar YC, Oppenheim AV (2000) Filterbank reconstruction of bandlimited signals from nonuniformand generalized samples. IEEE Trans Signal Process 48(10):2864–2875
Esquef PAA, Välimäki V, Roth K, Kauppinen I (2003) Interpolation of long gaps in audio signals using the warped burg’s method. Proc 6th Int Conf Digit Audio Effects (DAFx)
Feichtinger HG, Grochenig K (1994) Theory and practice of irregular sampling. In: Benedetto JJ, Frazier MW (eds) Wavelets: mathematics and applications. CRC Press, Boca Raton, pp 305–363
Ferreira PJSG (1994) Noniterative and faster iterative methods for interpolation and extrapolation. IEEE Trans Signal Process 42(11):3278–3282
Ferreira PJSG (2001) Iterative and noniterative recovery of missing samples for 1-D band-limited signals. In: Marvasti FA (ed) Sampling theory and practice. Plenum Publishing Corporation
Foster JP, Nixon MS, Preugel-Bennett A (2001) New area based metrics for gait recognition. Proc 3rd Int Conf Audio-Video-Based Biom Person Authentication (AVBPA)
Fujiyoshi H, Lipton AJ (2004) Real-time human motion analysis by image skeletonization. IEICE Trans Inf Syst E87-D(1):113–120
Gabriel PF, Verly JG, Piater JH, Genon A (2003) The state of the art in multiple object tracking under occlusion in video sequences. Proc Adv Concepts Intell Vis Syst (ACIVS)
Gavrila DM, Davis LS (1996) Tracking of humans in action: a 3D model-based approach. ARPA Image Underst Workshop
Geng X, Wang L, Li M, Wu Q, Smith-Miles K (2007) Distance-driven fusion of gait and face for human identification in video. Proc Image Vis Comput
Gerchberg RW (1974) Super-resolution through error energy reduction. J Mod Opt 21(9):709–720
Gu J, Ding X, Wang S, Wu Y (2010) Action and gait recognition from recovered 3-d human joints. IEEE Trans Syst Man Cybern B 40(4):1021–1033
Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(2)
Ho M-F, Chen K-Z, Huang C-L (2009) Gait analysis for human walking paths and identities recognition. Proc Int Conf Multimedia Expo (ICME) 1054–1057
Hogg D (1983) Model-based vision: a program to see a walking person. Comput Vis Graph Image Process 1(1):5–20
Howe NR, Leventon ME, Freeman WT (1999) Bayesian reconstruction of 3D motion from single-camera video. Adv Neural Inf Process Syst 12
Hu M, Wang Y, Zhang Z, Wang Y (2010) Combining spatial and temporal information for gait based gender classification. Proc IEEE/IAPR Int Conf Pattern Recognit 3679–3682
Hu M, Wang Y, Zhang Z, Zhang D (2011) Multi-view multi-stance gait identification. Proc IEEE Int Conf Image Process
Jain AK, Ross A, Nandakumar K (2011) Introduction to biometrics. Springer
Jean F, Albu AB, Bergevin R (2009) Towards view-invariant gait modeling: computing view-normalized body part trajectories. Pattern Recognit 42(11):2936–2949
Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Psychophys 14(2):201–211
Johnson AY, Bobick AF (2001) A multi-view method for gait recognition. Proc 3rd Int Conf Audio Video-Based Biom Person Authentication (AVBPA)
Kale A, Chowdhury AR, Chellapa R (2003) Toward a view invariant gait recognition algorithm. Proc Int Conf Acoust Speech Signal Process (ICASSP)
Kale A, Cuntoor N, Yegnanarayana B, Rajagopalan AN, Chellapa R (2004) Gait-based human identification using appearance matching. Optical and digital techniques for information security. Springer-Verlag
Kale A, Rajagopalan AN, Cuntoor N, Krueger V (2002) Gait-based recognition of humans using continuous HMMs. Proc 5th IEEE Int Conf Autom Face Gesture Recognit (AFGR)
Kale A, Rajagopalan AN, Sundaresan A, Cuntoor N, Roychowdhury A, Krueger V (2004) Identification of humans using gait. IEEE Trans Image Process 13:1163–1173
Kale A, Roy-Chowdhury AK (2004) Fusion of gait and face for human identification. Proc 2004 IEEE Int Conf Acoust Speech Signal Process (ICASSP 2004)
Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis 1:321–332
Kellokumpu V, Zhao G, Li SZ, Pietikainen M (2009) Dynamic texture based gait recognition. Proc IAPR/IEEE Int Conf Biometrics 1000–1009
Khan JI, Guo Z, Oh W (2001) Motion based object tracking in MPEG-2 video stream for perceptual region discrimination rate transcoding. Proc 9th ACM Int Conf Multimedia
Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239
Koller D, Weber J, Malik J (1994) Robust multiple car tracking with occlusion reasoning. Proc 2nd Eur Conf Comput Vis (ECCV)
Kwon KS, Park SH, Kim EY, Kim HJ (2007) Human shape tracking for gait recognition using active contours with mean shift. Proc Int Conf Human-Comput Interact 690–699
Lee L (2002) Gait dynamics for recognition and classification (2002) Proc 5th IEEE Int Conf Autom Face Gesture Recognit (AFGR)
Lee TKM, Belkhatir M, Lee PA, Sanei S (2008) Fronto-normal gait incorporating accurate practical looming compensation. Proc 19th Int Conf Pattern Recognit (ICPR 2008)
Lee TKM, Loe KF, Lee PA, Sanei S (2007) A comparison of the basic temporal features of fronto-normal and fronto-parallel gait. DSP 2007, Cardiff, UK, July, pp 1–4
Lee TKM, Ranganath S, Sanei S (2006) Frontal view-based gate identification incorporating the largest Lyaponuv exponents. ICASSP 2006, Toulouse, France, May, 15–19
Leventon ME, Freeman WT (1998) Bayesian estimation of 3D human motion from an image sequence. Tech Rep 98-06, Mitsubishi Electric Research Lab
Little JL, Boyd JE (1998) Shape of motion and the perception of human gaits. Proc 1998 IEEE Conf Comput Vis Pattern Recognit (CVPR)
Liu G, McMillan L (2006) Estimation of missing markers in human motion capture. Vis Comput 22(9–11):1432–2315
Liu Z, Sarkar S (2007) Outdoor recognition at a distance by fusing gait and face. Image Vis Comput 25(6):817–832
Lu J, Zhang E (2007) Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion. Pattern Recognit Lett 90(7):2401–2411
Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. Proc 7th Int Joint Conf Artif Intell (IJCAI)
Makihara Y, Mannami H, Tsuji A, Hossain MA, Sugiura K, Mori A, Yagi Y (2012) The OU-ISIR gait database comprising the treadmill dataset. IPSJ Trans Comput Vis Appl 4:53–62
Marr DC (1976) Early processing of visual information. Phil Trans R Soc Lond B275:483–524
Marr D (1978) Representing visual information—a computational approach. In: Hanson AR, Riseman EM (eds) Computer vision systems. Academic Press
Marr D, Nishihara HK (1978) Representation and recognition of three dimensional shapes. Proc R Soc Lond B 269–294
McCormick J, Blake A (2000) A probabilistic exclusion principle for tracking multiple objects. Int J Comput Vis 39(1):57–71
Metaxas D, Terzopoulos D (1993) Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans Pattern Anal Mach Intell 15(6):580–591
Moeslund TB. Granum E (2000) 3D human pose estimation using 2D data and an alternate phase space representation. Proc IEEE Workshop Hum Model Anal Synth (HuMAns)
Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Elsevier Comp Vision Image Underst 104(2-3):90–126
Morris DD, Rehg JM (1998) Singularity analysis for articulated object tracking. Proc 1998 IEEE Conf Comput Vis Pattern Recog (CVPR)
Murase H, Sakai R (1996) Moving object recognition in eigenspace representation: gait analysis and lip reading. Pattern Recognit Lett 17:155–162
Ning HZ, Wang L, Hu WM, Tan TN (2002) Articulated model based people tracking using motion models. Proc. 4th IEEE Int Conf Multimodal Interfaces
Ning HZ, Wang L, Hu WM, Tan TN (2004) Model-based tracking of human walking in monocular image sequences. Image Vision Comput 22:429–441
Nixon MS, Carter JN, Nash JM, Huang PS, Cunado D, Stevenage SV (1999) Automatic gait recognition. Biometrics: personal identification in networked society. Kluwer Academic Publishers
Niyogi SA, Adelson EH (1994) Analyzing and recognizing walking figures in XYT. Proc 1994 IEEE Conf Comput Vis Pattern Recognit (CVPR)
Niyogi SA, Adelson EH (1994) Analyzing gait with spatiotemporal surfaces. Proc Workshop Non-Rigid Motion Articulated Objects
Nizami IF, Hong S, Lee H, Lee B, Kim E (2010) Automatic gait recognition based on probabilistic approach. Int J Imaging Syst Technol 20(4):400–408
Ortega-Garcia J, Bousono-Crespo C (2005) Report on existing biometric databases. BioSecure Deliverable D1.1.1, European Commision
Phillips PJ, Moon HJ, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104
Phillips PJ, Sarkar S, Robledo I, Grother P, Bowyer K (2002) Baseline results for the challenge problem of HumanID using gait analysis. Proc 5th IEEE Int Conf Automatic Face Gesture Recog (AFGR)
Phillips PJ, Sarkar S, Robledo I, Grother P, Bowyer K (2002) Baseline results for the challenge problem of human ID using gait analysis. Proc 5th IEEE Int Conf Autom Face Gesture Recog (AFGR)
Ran Y, Weiss I, Zheng Q, Davis LS (2007) Pedestrian detection via periodic motion analysis. Int J Comput Vis 2(71):143–160
Ran Y, Zheng Q, Chellappa R, Strat TM (2010) Applications of a simple characterization of human gait in surveillance. IEEE Trans Syst Man Cybern B 40(4):1009–1020
Raviv D (2000) The visual looming navigation cue: a unified approach. Comput Vis Image Underst 79:331–363
Rohr K (1994) Towards model-based recognition of human movements in image sequences. Comput Vis Graph Image Process 59(1):94–115
Ross A, Nandakumar K, Jain AK (2006) Handbook of multibiometrics. Springer, New York
Sarkar S, Phillips PJ, Liu Z, Vega IR, Grother P, Bowyer KW (2005) The human id gait challenge problem: data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27(2):162–177
Souppa A. www.music.miami.edu/programs/Mue/Research/asouppa/chapter3.htm
Tanawongsuwan R, Bobick AF (2001) Gait recognition from time-normalized joint-angle trajectories in the walking plane. Proc 2001 IEEE Conf Comput Vis Pattern Recog (CVPR)
Tanawongsuwan R, Bobick A (2004) Modelling the effects of walking speed on appearance-based gait recognition. In: IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp 783–790
Taycher L, Fisher JW, Darrell T (2002) Recovering articulated model topology from observed motion. Proc IEEE Workshop Stat Methods Video Process (SMVP)
Tomasi C, Kanade T (1992) Shape and motion from image streams under orthography: a factorization method. Int J Comput Vis 9:137–154
Trivinoa G, Alvarez-Alvareza A, Bailadorb G (2010) Application of the computational theory of perceptions to human gait pattern recognition. Pattern Recognit 43(7):2572–2581
Troje NF (2002) Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. J Vis 2:371–387
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86
Venkat I, DeWilde P (2011) Robust gait recognition by learning and exploiting sub-gait characteristics. Int J Comput Vis 91(1):7–23
Verhoeven G (2007) Did the digital (R)evolution change the concept of focal length? AARGNEWS 34:30–35
Vezzetti E, Marcolin F (2012) Geometry-based 3D face morphology analysis: soft-tissue landmark formalization. Multimedia Tools Appl J
Vio R, Strohmer T, Wamsteker W (2000) On the reconstruction of irregularly sampled time series. Publ Astron Soc Pac 112:74–90
Wachter S, Nagel HH (1997) Tracking of persons in monocular image sequences. Proc IEEE Nonrigid Articulated Motion Workshop
Wang L, Hu WM, Tan TN (2002) A new attempt to gait-based human identification. Proc 16th Conf Pattern Recog (ICPR)
Wang L, Ning HZ, Hu WM, Tan TN (2002) Gait recognition based on procrustes shape analysis. Proc 9th IEEE Int Conf Image Process (ICIP)
Wang L, Tan T (2003) Silhouette analysis-based gait recognition for human identification. IEEE Trans PAMI 25(12):1505–1518
Wang L, Tan T, Ning H, Hu W (2003) Silhouette analysis-based gait recognition for human identification. IEEE Trans Pattern Anal Mach Intell 25(12):1505–1518
Whittle MW (2007) Gait analysis: an introduction, 4th edn. Butterworth-Heinemann, Philadelphia, p 139
Yang Y, Levine M (1992) The background primal sketch an approach for tracking moving objects. Mach Vis Appl 5:17–34
Yu S, Tan T, Huang K, Jia K, Wu X (2009) A study on gait-based gender classification. IEEE Trans Image Process 18(8):1905–1910
Yu SQ, Tan DL, Tan TN (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. Proc 18th Int Conf Pattern Recognit (ICPR)
Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. Proc 18th Int Conf Pattern Recognit 441–444
Yu S, Tan D, Tan T (Jan. 2006) Modelling the effect of view variation on appearance-based gait recognition. In Proc. of the 7’th Asian Conference on Computer Vision (ACCV06). Hyderabad, India
Zhang X, Fan G (2010) Dual gait generative models for human motion estimation from a single camera. IEEE Trans Syst Man Cybern B 40(4):1034–1049
Zhang E, Zhao Y, Xiong W (2010) Active energy image plus 2DLPP for gait recognition. Signal Process 90(7):2295–2302
Zhou X, Bhanu B (2006) Integrating face and gait for human recognition at a distance in video. IEEE Trans Syst Man Cybern B 37(5):1119–1137
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, T.K.M., Belkhatir, M. & Sanei, S. A comprehensive review of past and present vision-based techniques for gait recognition. Multimed Tools Appl 72, 2833–2869 (2014). https://doi.org/10.1007/s11042-013-1574-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-013-1574-x