WO2022076245A1 - Visible and far infrared camera imaging scheme for detecting elevated body temperature - Google Patents

Abstract

A fast face capture system and process for identifying an individual as the individual walks through a designated area is described. A set of raw images is streamed from a long wavelength infrared camera and visible light spectrum camera for detecting individuals entering the designated area. The temperature of the individual is detected based on evaluating an area in a thermal image bound by an ROI. Preferably, the ROI is first detected in a visible light image and mapped onto the thermal image of the individual. Also described herein is a method and system that computes a transformation matrix or mapping between image points from a visible light camera and that of a long wavelength infrared camera image. A novel calibration target includes patterns that are simultaneously detectable in both the visible light and long infrared wavelength range.

Description

VISIBLE AND FAR INFRARED CAMERA IMAGING SCHEME FOR DETECTING

ELEVATED BODY TEMPERATURE

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This claims priority to provisional application number 63/087,725, filed October 5, 2020, entitled “CALIBRATION TARGET FOR VISIBLE LIGHT AND FAR INFRARED CAMERAS AND RELATED METHODS”, and to provisional application number 63/087,743, filed October 5, 2020, entitled “VISIBLE AND FAR INFRARED CAMERA IMAGING SCHEME FOR DETECTING ELEVATED BODY TEMPERATURE”.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] This invention relates to face recognition systems, and particularly, to face recognition systems for capturing the image of a face of a moving person.

[0004] 2. Description of the Related Art

[0005] Face recognition systems are gaining popularity for identifying or confirming a person’s identity in a number of different venues such as, for example, security check points. A real time image of the candidate’s face is submitted for matching with a previously- acquired confirmed (or validated) image of the person’ s face. In a typical arrangement, the candidate poses or holds still for the camera, and an image of the candidate’s face is acquired. Other examples of face matching systems are described in the US Patent No. 7,643,671.

[0006] More recently, and in view of the Covid-19 virus pandemic, detection of elevated body temperature is desirable for purposes of screening for individuals that may be carrying a contagious virus.

[0007] Accordingly, there is a need for improved systems that address the above challenges.

SUMMARY OF THE INVENTION

[0008] A face recognition system for identifying and detecting temperature of a person as the person is moving through a designated area includes a camera set comprising a visible light spectrum (VIS) camera and a camera capable of imaging the long wavelength infrared (LWIR) region of the spectrum. The cameras are aimed at the designated area; and at least one processor is operable to perform face tracking of the person as the person moves through the designated area based on receiving a plurality of consecutive images from the VIS camera. The processor is further operable to perform image selecting by selecting at least one candidate image for matching from the plurality of consecutive images being tracked based on an image quality metric, time elapsed, and an image quality count.

[0009] Images from both the VIS and the LWIR cameras are simultaneously streamed and synchronized. In embodiments, camera frame synchronization is provided by 1) image timestamps, 2) frame capture of one camera triggering the capture of the other, or 3) both cameras are triggered by a global time reference signal.

[0010] In embodiments, the processor computes the temperature of the person based on evaluating an area in the thermal image bound by a region of interest (ROI). Preferably, the ROI is first detected in a VIS image and mapped onto the thermal image of the individual. Consequently, a trained classifier for processing the thermal image is not necessary.

[0011] In embodiments, the processor is further operable to perform image selecting using a trained classifier.

[0012] In embodiments, the system further includes a face matching engine, and the processor is further operable to send the at least one candidate image to the face matching engine, and wherein the image quality count is adjusted with each candidate image sent to the face matching engine.

[0013] In embodiments, the processor is operable to continue face tracking and image selecting for the person until the image quality count reaches a maximum count value. In embodiments, the maximum count value is less than 5.

[0014] In embodiments, the system further comprises a guidance feature to corral the person walking through the designated region. In embodiments, the guidance feature is a handrail. In some embodiments, the guidance features is presented by a display or an augmented reality projector using projection mapping techniques.

[0015] In embodiments, the system further comprises a display, and the processor and display are operable to show on the display live images of the face of the person during face tracking.

[0016] In embodiments the processor is operable to superimpose graphics on the live images enclosing the face during face tracking.

[0017] In embodiments, the processor is operable to perform a transformation on the live images during the face tracking to encourage the person to look at the display thereby obtaining a higher quality image.

[0018] In embodiments, the transformation is selected from the group consisting of blurring (blurred around subject’s face); cropping (face cropped image); configure face to line image; configure face cartoon image; unsharp masking; noise suppression; illumination adjustment; high dynamic range; rotation (roll) correction; emoji-type face representation; and animal-type face representation.

[0019] In embodiments, the system further comprises a housing enclosing the processor, camera and display.

[0020] In embodiments, the system further comprises display ring, and wherein the display ring is operable to change visually based on the images of the person walking through the designated area.

[0021] In embodiments, the processor is further operable to monitor the time elapsed for face tracking of the person, and terminate face tracking for the person after the time elapsed reaches a maximum time elapsed. In embodiments, the maximum time elapsed is equal to or greater than 2 seconds.

[0022] In embodiments, the image quality metric is selected from the group consisting of face size (Fsize) or inter pupillary distance (IPD), Yaw Pitch Roll (YPR), Laplacian variance, or a frequency-based focus metric.

[0023] In embodiments, the processor is operable to determine instructions for person moving based on the plurality of consecutive images. In embodiments, the processor is operable to determine when the person has exited the designated area.

[0024] In embodiments, the processor determines the person has exited the designated area when a face size (F_Size) of the person is greater than a maximum face size (F_max) and the person is tracked outside the field of view of the camera.

[0025] In embodiments, the system further comprises a display, and is operable to indicate on the display for a next person to enter the designated area. In embodiments, the instructions direct said person to enter a second area for further identification.

[0026] In embodiments, the system further comprises a remote server, and wherein the face matching engine is located on the remote server. The candidate image that meets the criteria for being stored as the maximum quality image is sent from the local memory or processor to a remote storage such as a server. The data may be sent wirelessly or otherwise. [0027] In embodiments, the face matching engine interrogates the at least one candidate image of the person to confirm the identity of the person.

[0028] In embodiments, the processor is operable to monitor an enrollment state of the system corresponding to a total number of persons whose identity has been confirmed by the face matching engine.

[0029] In embodiments, the processor is further operable to perform image selecting based on a cue arising from the person walking. The cue can be visual-based. Examples of cues include, without limitation, a badge, article of clothing, band, flag, sign, and gesture. [0030] In embodiments, the system further comprises a visual privacy warning feature to direct the person into the designated area for face capture and alternatively to a face capture exclusion area.

[0031] In embodiments, the invention includes a face recognition method for identifying individuals based on a maximum quality image of the face of the individual as each individual moves through a designated area. The method comprises: (a) streaming images from a VIS and the LWIR camera for each individual entering the designated area; (b) face detecting by searching the streamed images for a face until a face of an individual is detected; (c) face tracking; and (d) maximizing image quality by repeating the face tracking step for each individual if the elapsed time is within a maximum time and the quality select count is within a threshold count range.

[0032] In embodiments, the images from both the VIS and the LWIR cameras are simultaneously streamed and synchronized.

[0033] In embodiments, camera frame synchronization is provided by 1) image timestamps, 2) frame capture of one camera triggering the capture of the other, or 3) both cameras are triggered by a global time reference signal.

[0034] In embodiments, the method further computes the temperature of the person based on evaluating an area in the thermal image bound by a region of interest (ROI). Preferably, the ROI is first detected in the VIS image and mapped onto the thermal image of the individual.

[0035] In embodiments, the step of face tracking is performed by: (i) assigning a tracking ID and a quality select count to the individual; (ii) tracking the face of the individual to obtain at least one candidate image of the face of the individual as the individual moves through the designated area; (iii) timing the individual during tracking for an elapsed time; (iv) storing as the maximum quality image the at least one candidate image for face matching if an image quality metric is within a threshold quality range and higher than that of a previously-stored quality image; and (v) adjusting the quality select count for the individual based on whether the at least one candidate image was stored as the maximum quality image.

[0036] In embodiments, the step of storing is carried out remotely from the designated area. The candidate image that meets the criteria for being stored as the maximum quality image is sent from a local memory or processor to a remote storage such as a server. The data may be sent wirelessly or otherwise.

[0037] In embodiments, the method further comprises, subsequent to the step of maximizing image quality, face matching the maximum quality image with a validated image of the face of the individual.

[0038] In embodiments, the step of maximizing is based on the quality image count being less than or equal to 5, and optionally less than or equal to 3.

[0039] In embodiments, the method comprises displaying a live stream of images of the individual being tracked during the tracking step.

[0040] In embodiments, the method comprises superimposing graphics on the live stream of images of the individual being tracked to encourage the individual to look at the camera. [0041] In embodiments, the method comprises terminating the tracking step for the individual when the elapsed time is equal to or greater than 2 seconds.

[0042] In embodiments, the method further comprises displaying an instruction to enter the designated area until tracking is commenced for a next individual.

[0043] In embodiments, at least three (3) candidate images are generated during the step of tracking.

[0044] In embodiments, the quality metric being selected from the group consisting of face size or interpupillary distance, Yaw Pitch Roll, and Laplacian variance.

[0045] In embodiments, the invention includes a face recognition method for identifying individuals based on a maximum quality image of the face of the individual as each individual moves through a designated area. The method comprises: (a) streaming images from at least one camera for each individual entering the designated area; (b) face detecting by searching the streamed images for a face until a face of an individual is detected; (c) assigning a tracking ID and a quality select count to the individual; (d) commencing a face tracking timer for the individual; (e) tracking in real time the face of the individual to obtain a current candidate image of the face of the individual as the individual moves through the designated area; (f) delegating the current candidate image as the maximum quality image for face matching if certain criteria are met; (g) maintaining a quality select count for the individual corresponding to a total number of current candidate images delegated; and (h) continuously updating the maximum quality image for the individual by repeating steps (e) - (g) so long as the quality select count is less than a threshold count and an elapsed tracking time measured by the face tracking timer is less than a maximum time.

[0046] In embodiments, the step of delegating is performed if (i) an image quality metric of the current candidate image is within a threshold quality range and (ii) in the event the maximum quality image had been previously delegated, the current candidate image has a higher quality rating than the previously delegated maximum quality image. [0047] In embodiments, the step of delegating is carried out by saving the delegated candidate image in a storage located remote to the designated area. The candidate image that meets the criteria for being stored as the maximum quality image is sent from the local memory or processor to a remote storage such as a server. The data may be sent wirelessly or otherwise.

[0048] In embodiments, subsequent to the step of updating, the method further includes face matching the delegated candidate image with a validated image of the face of the individual.

[0049] Calibration

[0050] One embodiment of the present invention is a method for calibrating a set of different types of cameras. The method comprises the steps of providing a calibration target that has a known pattern which is detectable in each of the cameras. The method further comprises generating at least one image set by simultaneously taking an image of the target from each of the different types of cameras and computing a transformation matrix based on the image sets.

[0051] In embodiments of the invention, the types of cameras include a thermal camera such as a far infrared camera and a visible light camera such as a CMOS Sensor.

[0052] In embodiments of the invention, the method further comprises computing calibration correction parameters (intrinsic and extrinsic camera parameters) based on the set of images, and correcting the image sets. Examples of calibration parameters include optical distortion. The correction parameters can be computed prior to computing the transformation matrix, or optionally, simultaneously.

[0053] In embodiments of the invention, at least 100, and more preferably 200 to 300, image sets are captured by the cameras. In embodiments, images sets are captured of the target at a plurality of different distances from the cameras. -

[0054] In embodiments of the invention, a calibration target includes a plurality of different types of discrete units or areas, having known dimensions, and detectable in each of the camera types.

[0055] In embodiments of the invention, the calibration target includes a first set of units that is detectable by a visible light camera (sensing radiation in the range between 400nm to lOOOnm) and a thermal or long wavelength infra-red camera (sensing radiation in the range between lOOOnm to 14,000nm). In a particular embodiment of the invention, the first set of units is made of a white insulating material such as a white foam. [0056] In embodiments of the invention, the calibration target further includes a second set of units that is also detectable by the visible light camera and the thermal camera. In a particular embodiment of the invention, each of the units in the second set comprises a heating element or source, a thermal conducting layer, and a thin black absorbing cover with very low emissivity. When the heating element or source is activated, the temperature of each of the units in the second set is elevated to a greater temperature than that of the first set of units. The intensity of light from the second set of units (namely, the black squares) is relatively higher than that of the first set of units (namely, the white squares). The difference in temperature between the first and second units provides good contrast from which to compute a transformation matrix for the camera images.

[0057] In a particular embodiment of the invention, when the environment is at room temperature, the temperature of each of the second high-temp units is raised to about 37C and the temperature of each of the first low-temp/insulating units is raised to about 32C.

[0058] In another embodiment of the invention, when the environment is at room temperature, the ratio of the temperature of the high-temp units to that of the low-temp units ranges from 1.1 to 2.

[0059] In embodiments, the first and second units are arranged in an alternating pattern.

[0060] In embodiments, a system comprises a calibration target, a plurality of different types of cameras, and a processor operable to compute a transformation matrix based on captured images of the target from the plurality of different types of cameras.

[0061] The description, objects and advantages of embodiments of the present invention will become apparent from the detailed description to follow, together with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0062] FIGS. 1 A- ID are sequential illustrations of a face capture system in accordance with an embodiment of the invention for capturing the image of a candidate’s face as the candidate is walking;

[0063] FIG. 2 is a flow chart of a face capture process in accordance with an embodiment of the invention;

[0064] FIG. 3 is a block diagram of a face capture system in accordance with an embodiment of the invention;

[0065] FIGS. 4 A, 4B are flow charts illustrating a face capture process in accordance with an embodiment of the invention; [0066] FIG. 5 is an illustration of various screen shots of a display in accordance with an embodiment of the invention;

[0067] FIG. 6 is another flow chart of a face capture process in accordance with an embodiment of the invention for carrying out an action based on face matching;

[0068] FIG. 7 is an illustration of a face capture system in accordance with an embodiment of the invention for carrying out an electrical or mechanical action based on face matching;

[0069] FIG. 8 is a flow chart of a face capture process in accordance with an embodiment of the invention for detecting body temperature;

[0070] FIG. 9 is an illustration of a combined visible light and thermal image in accordance with an embodiment of the invention for detecting body temperature;

[0071] FIG. 10 is a front view of a face recognition device including a plurality of different types of cameras; and

[0072] FIG. 11 is side view of the face recognition device shown in FIG. 9 .

[0073] Figures 12-14 are various views of a calibration target in accordance with one embodiment of the invention; and

[0074] Figures 15-16 are visible light and thermal images, respectively, of the calibration target shown in Figure 12.

DETAILED DESCRIPTION OF THE INVENTION

[0075] Before the present invention is described in detail, it is to be understood that this invention is not limited to particular variations set forth herein as various changes or modifications may be made to the invention described and equivalents may be substituted without departing from the spirit and scope of the invention. As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process act(s) or step(s) to the objective(s), spirit or scope of the present invention.

[0076] Methods recited herein may be carried out in any order of the recited events which is logically possible, as well as the recited order of events. Furthermore, where a range of values is provided, it is understood that every intervening value, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the invention. Also, it is contemplated that any optional feature of the inventive variations described may be set forth and claimed independently, or in combination with any one or more of the features described herein.

[0077] All existing subject matter mentioned herein (e.g., publications, patents, patent applications and hardware) is incorporated by reference herein in its entirety except insofar as the subject matter may conflict with that of the present invention (in which case what is present herein shall prevail).

[0078] Described herein is a fast face capture system and related methods.

[0079] FACE CAPTURE OVERVIEW

[0080] With reference to FIGS. 1A-1D a face capture system 10 in accordance with an embodiment of the invention is illustrated for capturing the image of a candidate’ s face as the candidate 20 is walking through region (R). The system 10 is shown having a face recognition device 30 and a pair of beam sensors 40, 42, each of which is triggered when the person crosses its beam. As discussed further herein, the recognition device 30 can include a plurality of cameras for obtaining images of the candidate 20 and a display to provide instructions to the candidate for walking through the various regions R0, Rl, R2, and R3. [0081] Initially, and with reference to FIG. 1A, the candidate 20 is shown waiting in region R0. The recognition device 30 is operable to exclude images of the face of the candidate while the candidate is in region R0. In embodiments, the recognition device 30 excludes faces having a size less than a threshold size. In a particular embodiment, the recognition device excludes a face detected with less than 150 pixels. Naturally, if no faces are detected, no images need be excluded, and no images are submitted for matching. The length of R0 may vary depending on the application. An exemplary length of R0 is greater than 10 feet.

[0082] With reference to FIG. IB, candidate 20 is shown walking through region Rl. In this face tracking STAGE, the recognition device 30 is operable to detect and track the face of the candidate 20. In embodiments, the recognition device 30 detects a face having a size within a first predetermined range and can start a face detected timer (to) for the candidate. In a particular embodiment, the first predetermined range at which to start tracking is 150 to 180 pixels. In embodiments, the determination of whether to submit a detected image is based on additional constraints including but not limited to yaw, pitch, and roll (YPR), and the Laplacian variance (LV), blur or sharpness factor. In a preferred embodiment, in addition to the above described size constraints, a candidate image is submitted for matching if the LV across the face is greater than 8, and the max(YPR) is less than 10. [0083] Additionally, the length of R1 may vary depending on the application. An exemplary length OF R1 ranges from 3 to 10 feet.

[0084] With reference to FIG. 1C, the candidate 20 is shown walking through region R2. In this image QUALITY assurance stage, the recognition device 30 is operable to continue tracking the face of the candidate 20 and optionally submits one or more images for matching based on a quality assurance algorithm, discussed further herein.

[0085] In embodiments, if a real-time image is detected while the candidate is walking through region R2, AND the real-time image is better than a previously submitted image (or if no previous image was submitted), the real-time image will be submitted for matching, replacing any previously submitted images for the candidate. In this manner, the system can continuously update or improve the quality of captured images to be submitted. Without intending to being bound to theory, the latter acquired image is deemed a higher quality than a former acquired image based on the assumption that the candidate is walking towards the camera and the face in the latter image is larger than the face in the previous image. Of course, this assumption is not true if the candidate is not walking towards the camera, or no longer within the field of view. Optionally, additional quality metrics are applied to further screen for high quality images during this stage including, for example, and without limitation, face size, sharpness, and YPR. In embodiments, the recognition device 30 evaluates and screens for faces having a size between 180 to 500 pixels. In a preferred embodiment, a candidate image is submitted for matching during this stage if, in addition to the above described size constraints, the LV is greater than 8, and the max(YPR) is less than 10.

[0086] Additionally, the length of R2 may vary depending on the application. An exemplary length of R2 ranges from 0 to 3 feet.

[0087] With reference to FIG. ID, the candidate 20 is shown walking downstream of the recognition device in region R3. In this exiting or departure stage, the system is operable to (a) accurately determine whether the candidate being tracked has exited the quality assurance region R2 and (b) restart the process for the next candidate. In embodiments, if the previously tracked candidate is no longer within the camera’s field of view for 5 or more frames, and/or optionally the timer for the candidate is greater than 5 seconds, the candidate shall be considered to have exited the region R2.

[0088] ADDITIONALLY, R3 is any distance behind the recognition device..

[0089] As MENTIONED above, the system 10 can include beam sensors 40, 42 which serve to identify whether the candidate has entered and exited region R2, as well as the time elapsed for the candidate to move through region R2. However, the invention is not intended to be so limited.

[0090] In embodiments of the invention, at least one image of the candidate’s face is submitted for MATCHING. The quality of the image is optimized based on a quality assurance algorithm described herein. The time to track and capture and submit an image of the candidate is fast and in embodiments, the time is less than about 3 seconds, and more preferably less than 1 seconds. An advantage of the invention is to track and capture the image without the person slowing down, or stopping. In embodiments, as discussed further herein, the system is adapted to provide a message direction to the person in real time. For example, the display may instruct the person to avoid stopping and continue in a specific direction, or to proceed to another location such as a seat or gate.

[0091] ADDITIONALLY, in embodiments, the quality assurance stage corresponding to region R2 is performed in less than or equal to 1 seconds.

[0092] Optionally, the number of images sent or submitted for matching is limited to a maximum count. In EMBODIMENTS, the maximum count of submitted images per candidate ranges from 2 to 10, and in some embodiments less than or equal to 5, and most preferable less than or equal to 3. Limiting the number of submitted images per candidate serves to increase the speed of the system so more candidates can walk through the region R. [0093] FIG. 2 is a flowchart of a process 100 for submitting images of a face of a candidate for matching in accordance with an embodiment of the invention. To facilitate understanding of the process 100, and the performance of exemplary steps of the process, reference is also made to the components and functionality shown in the face capture system 210 shown in FIG. 3.

[0094] Step 110 states to stream images from multiple cameras. In a preferred embodiment, one or more cameras and sensors 202 are enclosed in the recognition device 210 shown in FIG. 3. The types of cameras employed in embodiments of the invention may vary widely and include, without limitation, cameras operable in the visible light spectrum (e.g., RGB) or thermal spectrum (e.g., near or far IR). Examples of cameras, include without limitation, Leopard Imaging CMOS camera, model number LI-USB30-AR023ZWDRB (Freemont, California) as well as the Infrared Cameras Inc. ICI 8640 S-Series infrared cameras (Beaumont, Texas). The sensors and cameras may comprise their own image processing software 204. The cameras are preferably positioned downstream of the candidates, and aimed at the designated region (R). [0095] With reference again to FIG. 2, step 120 states to search for faces and optionally other objects within the images. This step can be carried out by computer hardware 220 executing one or more software modules or engines 230. Examples of hardware includes processors 222 (e.g., CPU, GPU, or AIA), data storage 224, memory 226, and various image and graphics processing units 228.

[0096] A detection tracking and recognition engine or module 232 searches for faces and optionally other objects as the candidate walks towards the recognition device. A wide range of face and object detection and tracking algorithms may be employed on the system 210 by the processor 220. Non-limiting examples of suitable face and object detection and tracking algorithms include: the dlib face detector and the JunshengFu/tracking-with-Extended- Kalman-Filter. The dlib face detector is stated to employ a Histogram of Oriented Gradients (HOG) feature combined with a linear classifier, an image pyramid, and sliding window detection scheme.

[0097] Additionally, a user interface or human factor layer 240 is shown in the system 210 of FIG. 3. In embodiments, a subject viewable display 242 assists in directing the person to walk in the designated area during the proper time, as well as to look towards the camera. Optionally, FED 244 such as a LED light ring surrounding the display is indicative of direction or visually changes based on position of the subject. As described further herein, other human factors can be included in the system including guide rails 246 and virtual or augmented reality type graphics to assist in guiding the candidate through the region and to look in the direction of the cameras.

[0098] With reference again to FIG. 2, step 130 states to select maximum quality human image in capture range. This step can be performed by the face recognition device 210 employing an image quality or quality assurance module 234, described further herein. The output of the quality assurance module 234 is a best or optimum image of the face of the candidate as he is walking through the designated region.

[0099] Step 140 states to submit for matching. This step submits an optimum image from step 130 to be matched with a pre-acquired (and validated) image of the person to be identified. A matching engine (not shown) can be included in the face recognition device 210, or a remote server 300, in which case a communication interface 250 is available to send the candidate’s optimum image to the remote server. A server processor 302, data storage 304, and memory 306 are operable to rapidly determine whether the difference between the candidate image and a pre-acquired stored image is acceptable to confirm the person’s identity. Examples of suitable matching engines 308 include, without limitation, the Algorithms evaluated by the NIST Face Recognition Vendor Test (FRVT) .

[00100] Machine learning algorithms and inferencing engines 308 can be incorporated into the server 300 or device 210 for increasing the accuracy and efficiency of the above described steps, particularly, for increasing the accuracy and efficiency of face detection and matching. Examples of such algorithms include, without limitation, the algorithms evaluated by the National Institute of Standards and Technology (NIST) Face Recognition Vendor Test (FRVT).

[00101] Additionally, the system may be operable to recognize cues for categorizing individuals into groups (e.g., tour group, team, military, etc.) as well as to recognize an individual’s clear intention (namely, acceptance) of being tracked for face matching. Cues may be visual-, audio-, or electromagnetic -based. Non-limiting examples of cues include badges, clothing, wrist bands, tags, RFID, voice, gestures including hand or face gesture, etc. [00102] FIG. 4A is a flow chart illustrating a process 400 to guide a candidate through a designated area and capture an optimized image of her face.

[00103] Step 410 states to scan or stream raw images of the candidate. Simultaneously, and with reference to step 412, instructions are displayed for the candidate to ‘enter’ the designated area or region.

[00104] Step 414 immediately interrogates the stream of images for a person’s face based on a minimum image quality metric. In embodiments, the initial raw images are considered acceptable if the inter pupillary distance (IPD) is at least 150, and each YPR value is less than 10.

[00105] Additionally, in preferred embodiments, a trained classifier is used to determine whether a face is present. Examples of trained classifiers include, for example: the dlib face detector. Id.

[00106] NO FACE DETECTED

[00107] In the event a face is not detected at step 414 (e.g., candidate falls or otherwise drops out of the FOV) or the image fails to pass the minimum quality threshold, the method proceeds to step 418.

[00108] Step 418 states to determine whether a tracking ID exists for the candidate. [00109] If a tracking ID does not exist (e.g., the candidate is new), the process simply proceeds to step 412 described above. The candidate is instructed by the display to ‘enter’ (412), and the live stream of images (step 410) is interrogated for a face having a minimum level of quality (step 414). [00110] If a tracking ID exists for the candidate (e.g., the candidate was being tracked but has fallen or leaned over to pick up a belonging), then the method proceeds to step 420. In embodiments, step 420 stops the current tracking ID, resets the timer, and resets a quality select counter (QSC), discussed further herein in connection with FIG. 4B. Following resetting the tracking ID, QSC, and timer, the candidate is instructed by the display to ‘enter’ (step 412), and the live stream of images (step 410) is interrogated for a face having a minimum level of quality (step 414).

[00111] FACE DETECTED

[00112] In the event a face is detected and passes the minimum quality threshold at step 414, the method proceeds to step 416 for tracking.

[00113] Step 416 states to determine whether a tracking ID exists. If not, the process proceeds to step 440 and a tracking ID is assigned. Face tracking is commenced and the display simultaneously shows the stream of images with a graphic to indicate face tracking has commenced. In embodiments, the face is outlined or enclosed by a boundary that is overlaid with the image (450).

[00114] If, at step 416, a tracking ID already exists, then the process continues tracking the candidate, and the display indicates same.

[00115] In either case, the process 400 proceeds to a quality assurance phase 500, discussed in detail in connection with FIG. 4B.

[00116] Output from the quality enhancement engine 500 is interrogated at step 560 for whether the process should be (a) restarted for a new candidate, or (b) continued for the same candidate. As described further herein, thresholds for determining whether to continue or restart can be based on time elapsed, the number of images submitted for matching, candidate is outside the field of view, etc. In preferred embodiments, the process is restarted if the time elapsed is greater or equal to 10 seconds, more preferably 5 seconds, and in embodiments, 3 seconds. In another preferred embodiment, the process is restarted if 3 images of a candidate have been submitted for matching, discussed further below in connection with FIG. 4B.

[00117] If it is determined to restart the process for a new candidate, step 420 stops the current tracking ID, resets the timer, and resets a quality select counter (QSC), discussed further herein in connection with FIG. 4B. After the tracking ID, timer, and QSC have been reset, the process proceeds to step 412 for the new candidate, and the face tracking is commenced according to the steps described above.

[00118] QUALITY SELECT ENGINE [00119] As stated herein, in embodiments, a quality assurance or enhancement engine improves the accuracy and efficiency of face capture. FIG. 4B is a flow chart illustrating a quality assurance process 500 to obtain a maximum quality image for submitting in accordance with an embodiment of the invention. According to step 510, a quality select counter (QSC) value associated with the image is interrogated. The next step of the method is determined and based on the value of the QSC.

[00120] For example, according to the embodiment shown in FIG. 4B, if the QSC is zero (0), the method proceeds to step 514, and a quality metric is compared to a threshold value or range to evaluate whether the image is considered a ‘pass’ or ‘no pass’. Examples of quality metrics include the face size, number of pixels of the face or object, time elapsed since tracking began, distance from the camera to the candidate, YPR, LP, and sharpness characteristics. If the image is considered a ‘pass’, the process proceeds to step 518 and the image is submitted for matching and the QSC is advanced by one (1). If the image is considered a ‘no pass’, the process proceeds directly to step 560 to determine whether to proceed to obtain more raw images 410 from the sensor or to proceed to step 420 to restart a new tracking candidate ID, restart the QSC, and reset the timer.

[00121] If the QSC is one (1), the method proceeds to step 520, which applies the quality metric threshold to evaluate whether the image is considered a ‘pass’ or ‘no pass’ as described above. If the image is considered a ‘pass’, the process proceeds to step 522 and the image is submitted (replacing the previously submitted or stored image) and the QSC is advanced by one (1). If the image is considered a ‘no pass’, the process proceeds directly to step 560 to determine whether to proceed to obtain more raw images 410 from the sensor or to proceed to step 420 to restart a new tracking candidate ID, restart the QSC, and reset the timer.

[00122] The quality enhancement process 500 can continue as described above until the QSC equals a NMAX, at which point the image capture process for the instant candidate is terminated, and the process is restarted for a new candidate, and proceeds to step 420 in FIG. 4A. In embodiments, the quality enhancement process continues until the QSC (e.g., NMAX) reaches 10, more preferably 2-5, and in one embodiment, NMAX is three (3).

[00123] Additionally, although FIG. 4B describes to submit or send the “pass” images, in other embodiments, the “pass” images are stored or saved and only a final image is ultimately submitted or sent for matching. In embodiments, determination of whether an image is a ‘pass’ or ‘sent’ can also be based on whether the instant image has a higher quality rating than the previous stored image. In embodiments, the instant image is only sent for matching (or stored) if it is better in quality than the previously stored image. The quality rating may be based on comparing YPR, blur, size, etc. However, as a practical matter, and as described herein, as the person moves towards the camera, the size of the face becomes larger in the image and the image quality shall generally be better. It follows that the image quality generally increases with time elapsed for the applications described herein. Thus, a wide range of techniques are operable to quickly maximize the image quality by selectively updating the stored or sent images with higher quality images.

[00124] FIG. 5 illustrates consecutive screen shots on a display 600 of a face recognition device 610 as a candidate 620 walks through various regions (R0, Rl, R2, and R3) in accordance with embodiments of the invention.

[00125] As described above, during stage R0, the face recognition device 610 can continuously scan for people to detect. An instruction to ‘walk’ or ‘enter’ may be displayed on the face recognition device, or as shown in FIG. 5, a stream of raw images is sent to the display 600 including the person 620. As described above, the system simultaneously runs object detection software to detect the person 620 as she is moving.

[00126] The next screen shot shown in FIG. 5 is display tracking 630. Particularly, as the person 620 crosses into the region (Rl), the face detection engine begins to track her face. FIG. 5 shows an optional graphic 630 overlaid on the image, enclosing her face as it is being tracked.

[00127] The next screen shot shown in FIG. 5 is display recognition 640. The process described above for submitting one or more optimal images has been carried out, and her face has been matched. Optionally, a stream of images may be frozen showing the largest face shot.

[00128] The next screen shot shown in FIG. 5 is guidance 650 as the person moves through area R2 and into R3. By this time in the process, her face has been matched to confirm her identity, and the display instructs her to proceed to her seat 6C. Although the display guidance shown in FIG. 5 is directed to a seat such as an airplane seat, the invention is not so limited except where recited in the appended claims. The instructions and guidance being displayed may vary widely in subject matter, application, text, graphics, and icons. It may be audio, visual, or and combinations thereof. Nonlimiting exemplary applications include transportation vehicles for people, arena venues, stadiums, border checkpoints, public parks, shopping malls, etc. Non-limiting exemplary text includes stop, go, pass, no pass, alert, etc. Non-limiting exemplary icons and graphics include signs to stop, go, pass, no pass, alert, traffic light, etc. Indeed, a wide range of subject matter, application, text, graphics may be displayed on the face recognition device 610. [00129] FIGS. 6-7 illustrate a process 700 for taking an action (e.g., unlocking a door) based on matching the submitted image with a confirmed image.

[00130] Initially, raw images are streamed from the cameras and the images are searched for a face (steps 710, 720).

[00131] The face is detected and tracked including applying a quality enhancement process to select and submit the face with a highest quality (steps 730, 740).

[00132] The image is matched using an image matching engine (step 740). Exemplary algorithms for image matching include, for example, the Algorithms evaluated by the National Institute of Standards and Technology (NIST) Face Recognition Vendor Test (FRVT).

[00133] After the match is confirmed, an action is taken such as to provide instructions to the person, notify another, or perform an electro- or mechanical physical action such as unlocking a door (step 750). Indeed, the system may send instructions or a signal to a controller to open a door to a hotel room, personal home, car, etc.

[00134] Additionally, in embodiments of the invention, enrollment, entry, or egress of confirmed individuals may be monitored by a state of system module. The system counts the number of people entering the designated area; maintains an image of each person entering the designated area; and maintains the person’s ID, and more preferably, an anonymized ID of each person entering the designated area. The system further monitors whether a person has left the designated area such that at any time, the system tracks the total number of people in the designated area. The designated areas may be located in various types of facilities, stations, or vehicles including, without limitation, cruise ships, trains, buses, subways, arenas, airports, office buildings, and schools.

[00135] Additionally, with reference to FIG. 7, a timer can monitor time elapsed as the person walks through each area. In embodiments, the timer commences (to) for the person as she enters the detection phase (step 720). The system is configured to recognize or match her face by time (ti), and to physically unlock the door by time (t2). In embodiments, and by using components and steps as described herein, time (ti) ranges from 1 to 10 seconds, and more preferably is 2-5 seconds, and in one embodiment is 2 seconds. In embodiments, and by using components and steps as described herein, time (t2) ranges from 1 to 30 seconds, and more preferably is 5-10 seconds, and in one embodiment is 10 seconds.

[00136] HUMAN FACTORS

[00137] Embodiments of the invention improve image capture using various human factors. Various human factors, whether an aspect of the face recognition device itself or a part of the overall system serve to optimize face capture and particularly, to increase the probability of obtaining a clear image of the person’s face.

[00138] In a particular embodiment, the system includes guard rails, barriers, lines, or graphics to mark or physically limit the region (R) in which the person walks. Patterns and marks may be physically applied to the environment using paint, stickers, handrails, etc., or by an augmented reality projection system. Such augmented reality devices can remove unwanted features in the field of view and accentuate desired features in the field of view, optionally based on feedback from the camera. In one embodiment, arrows are projected along the floor to indicate the direction and path through the region. In another embodiment, handrails corral the person through a region which forces the individual to optimally approach the camera such that camera can obtain a clear view of the individual’s face.

[00139] Notwithstanding the above, another difficulty is to encourage the subject to look in the direction of the camera. In embodiments, the display is used to assist the system to capture an image of the subject’s face sufficient for recognition. A variety of operations (e.g., image transformations) can be employed to enhance human factors on the device display in accordance with the invention. The images may be presented to the candidate, the device operator, or both.

[00140] Examples of suitable image transformation include, without limitation, blurring (blurred around subject’s face); cropping (face cropped image); configure face to line image; or configure face cartoon image. All of these images could be created and displayed in realtime to the subject or to the operator.

[00141] In embodiments, other types of transformations can include: unsharp masking; blurring; noise suppression; illumination adjustment; high dynamic range; rotation (roll) correction; emoji-type face representation; and animal-type face representation.

[00142] In embodiments, facial landmarks are used to define where to apply the various possible transformations. The dynamic facial landmarks could also be used as a way to provide display privacy. In place of the subject’s face, one could display the points or the connected points of the facial landmarks. The lines connecting the facial landmark points could be smoothed to provide a smooth outline of dynamic facial features. The landmarks could be used to animate the emoji or animal face representations. Visual feedback could be provided to show the user head rotation, eyes open or closed, mouth open or closed.

[00143] The images displayed may be different from those images saved for face recognition. These transformations may be performed by image processing hardware (ISP) on the camera board or on the main processor board. [00144] BODY TEMPERATURE DETECTION

[00145] With reference to FIG. 8, a method 800 for detecting body temperature of the individual is described in accordance with embodiments of the invention.

[00146] Step 810 states to stream images from the cameras where at least one camera streams visible light images and another streams thermal (e.g., far LWIR) images of the individual. With reference to FIG. 10, cameras may be arranged in a camera unit or device 900. The camera unit 900 is similar to the face recognition device 30 described above except it additionally shows LWIR camera 920 in addition to RGB camera sensors 930, 932. An enclosure 910, which holds the electronics and cameras, also features a display 940 to show instructions, status, and data. Optionally, the display is touchscreen operable and can act as a user input device.

[00147] Returning to FIG. 8, step 820 states to search for the face and other objects in the visible light images. For example, using the processor described above, the face and other landmarks such as the eyes and tear ducts may be identified in the visible light images. In preferred embodiments, a trained classifier is used to determine whether a face and other objects, e.g., a facial landmarks such as the eyes, are present. Examples of suitable trained classifiers for detecting the facial landmarks include, without limitation, Gizatdinova Yulia, Surakka Veikko, Feature-Based Detection of Facial Landmarks from Neutral and Expressive Facial Images, 2006 IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: 28.

[00148] Step 830 states to define a region of interest (ROI) in the visible light image based on the features identified from step 820. Examples of ROIs include, without limitation, rectangles, squares, circles, or another typically 2-D shape enclosing the target feature such as the eyes or tear ducts. Without intending to being bound to theory, the tear ducts are associated with body temperature and in embodiments, the ROI encompasses the tear ducts. [00149] Step 840 states to transform or map the region of interest (ROI) from the visible light spectrum (VIS) image to the thermal image. In embodiments, and with reference to FIG. 9, a ROI is shown registered on a thermal image of a person’ s face. The ROI is defined by two horizontal lines 850, 852 shown above and below the eyes, respectively.

[00150] Examples of techniques to transform/register image points from one image to another are described in, without limitation, (1) Zhang, Z. “A Flexible New Technique for Camera Calibration”. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 22, No. 11, 2000, pp. 1330-1334; (2) Heikkila, J, and O. Silven. “A Four-step Camera Calibration Procedure with Implicit Image Correction.” IEEE International Conference on Computer Vision and Pattern Recognition. 1997; and (3) co-assigned US Patent Application No. 63087725, by Skoda et al., filed October 5, 2020, Entitled “CALIBRATION TARGET FOR VISIBLE LIGHT AND FAR INFRARED CAMERAS AND RELATED METHODS”, each of which is incorporated by reference herein in its entirety for all purposes.

[00151] Step 850 states to determine the temperature of the ROI based on the thermal image.

[00152] Step 850 may be performed according to a number of different ways. For example, in embodiments, a histogram is computed for the pixels bound by the ROI in the thermal image. Preferably, the pixel(s) having the highest real value can be identified as the subject temperature pixel. Optionally, if noise is present, the noise is removed from the histogram prior to identifying the subject temperature pixel. Noise may be removed by, for example, removing the brightest pixels according to a threshold percent. The threshold percent may range from 1-5%, and preferably is about 1%.

[00153] Optionally, the statistical uncertainty in the subject’s temperature is computed. One technique for evaluating the statistical uncertainty is to initially determine the location of the computed subject temperature pixel, and then compute the standard deviation of pixel values in the neighborhood of the subject temperature pixel. Examples of neighborhood pixels include, without limitation, a 3x3 or 5x5 pixel matrixes/neighborhoods.

[00154] Optionally, and with reference again to FIG. 9, the temperature and standard deviation of a temperature reference 846 is computed. An example of a temperature reference is a heating element having a known temperature positioned in view of the thermal camera when the subject is also in the field of view. In FIG. 9, the temperature reference is a blackbody source hanging from the ceiling such as 12V 4" Blackbody for Gain Cal & Supplemental FFC from FLIR Systems, Inc. (Goleta, California). The standard deviation of the subject and reference may be combined. A technique to combine these two values is “in quadrature:” given by the equation tot_uncertainty = sqrt(stdl^A2 + std2^A2). However, the invention is not so limited. Other techniques may be employed to combine the values.

[00155] Optionally, after the temperature of the ROI of the person is determined, it may be matched with the identity of the person as obtained by, for example, the face recognition scheme described herein. In a particular embodiment, the subject temperature is taken in the region near the tear ducts. This region can be representative of the person’s body temperature and provides a quick, immediate temperature for a given individual. When placed at an entrance to a door or barrier or room, this information may be input to determine an action as described above in connection with FIGS. 6-7. Examples of actions can include actuating a door to open or close, actuating a lock to unlock or lock, sending an alert or alarm to another, etc. Indeed, embodiments of the invention include sending a notice or alert to another when a person’s body temperature is elevated beyond a threshold temperature which could be indicative of the person having a virus or infection.

[00156] Embodiments of the invention also include storing or associating a person’s ID with his/her temperature as a health or biometric record. Embodiments of the invention analyze the individual’s biometric record over time and identify change and a rate of change in an individual’s temperature which can be indicative of a symptom of disease or infection. [00157] CALIBRATION TARGET FOR VISIBLE LIGHT AND FAR INFRARED CAMERAS

[00158] Background

[00159] Mapping regions of interest or landmarks from an image of one camera type to an image of another camera type requires, amongst other things, correcting optical distortion inherent to each camera as well as determining a transformation matrix to map image points from one camera image to image points in the other camera image.

[00160] Calibration Target

[00161] With reference to Figures 12-14, a planar square-shaped calibration target 1000 is shown in accordance with one embodiment of the present invention. The front of the calibration target 1000 includes a plurality of different colored units 1010, 1020. The two types of units shown in Figure 12 are white units 1010 and black units 1020.

[00162] In Figure 12, the individual 2D geometric shapes are square-shaped and arranged in an alternating or checkboard-like pattern. In an embodiment, the square units are 1 by 1 inches in size. However, it is to be understood the shape and size of the individual 2D geometrical units may vary as well as the pattern and overall profile of the target calibration plate. The invention is not intended to be limited except as where recited in any appended claims.

[00163] Figure 13 shows a rear view of the calibration target 1000. The body or substrate 1030 is shown having a plurality of spaced apart cavities 1040 in registration (namely, aligned) with each of the black units 1020. With reference to Figure 14, which is cross section of Figure 13 taken along line 14-14, each cavity 1040 is adapted to receive and house a heating element 1032. A non-limiting example of heating element or source is an aluminum-housed resistor (2.5 Ohms, 10W, 1%, p/n: UAL10-2R5F8) manufactured by Riedon (Alhambra, California). [00164] The heating elements 1032 are activated such that, as described further herein, the pattern shown in Figure 12 is detectable by both the visible light and thermal camera. Particularly, each heating element 1032 is arranged to correspond to a black unit 1020 with a white thermally insulating unit 1010 disposed between each of the black units.

[00165] An exemplary material for the white insulating material is an insulating foam such as but not limited to white polystyrene foam. The thickness of the white insulating material may vary. In embodiments, the thickness of the white foam ranges from 0.1 to 0.5 inches and in one embodiment, is about 0.2 inches thick.

[00166] The configuration of the black units may vary. In the embodiment shown in Figures 12-14, each black unit 1020 features several components arranged in layers. The front or cover layer is a thin (e.g., without limitation, 0.02 inch thick) layer that can absorb light such as a black flocking optical material. An exemplary material is light absorbing black-out paper.

[00167] Each black unit also features a thermal conducting layer serving to conduct heat from the heating element/source throughout the entire square shaped black unit. A suitable material for the thermally conducting layer is copper. Additionally, each black square 1020 is insulated by the surrounding white foam squares 1010, creating a clear thermal boundary. We have found that by using this construction, and as described further herein, each of the individual units 1010, 1020 are well defined in both the visible light and LWIR wavelength ranges. Figures 15 and 16 show respectively exemplary visible light and LWIR images of the heat activated calibration target 1000 taken simultaneously. Clear contrast between the individual units is evidenced in both images.

[00168] Calibration Method

[00169] In another embodiment of the invention, a method to calibrate a thermal and visible light camera includes providing and locating a calibration target (e.g., target 1000 shown in FIGS. 12-14) at a first distance from the cameras.

[00170] In embodiments, one hundred or more image pairs are obtained at three or more different distances. Image quality constraints (e.g., sharpness, comer detection, etc.) are selected and applied to discard any image pairs if one of the images in the pair has poor quality. An example of an image pair comprising a visible light spectrum (VIS) image and a far infrared (FIR) image of the heat activated calibration target 1000 taken simultaneously is shown in Figures 15 and 16, respectively. [00171] Next, a candidate geometry is detected of the calibration target 1000 based on the image pairs captured. This step may be performed by applying a corner-finding algorithm (e.g., implementing the Harris Corner Detector) available in the OpenCV library.

[00172] Next, a lens distortion parameter is computed for each of the cameras based on the known geometry and the detected (candidate) geometry of the calibration target 1000.

[00173] Next, corrected thermal and visible light images are computed.

[00174] Next, a global camera coordinate transformation matrix is computed based on the corrected thermal and visible light images. This step may be carried out based on, for example, the techniques described in (1) Camera Calibration and 3D Reconstruction, at https://docs.opencv.org/master/d9/db7/tutorial_py_table_of_contents_calib3d.html; (2) Zhang, Z. “A Flexible New Technique for Camera Calibration”. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 22, No. 11, 2000, pp. 1330-1334; and/or (3) Heikkila, J, and O. Silven. “A Four-step Camera Calibration Procedure with Implicit Image Correction.” IEEE International Conference on Computer Vision and Pattern Recognition. 1997; and (4). Alexander Duda and Udo Frese. Accurate detection and localization of checkerboard corners for calibration. In 29th British Machine Vision Conference. British Machine Vision Conference (BMVC-29), September 3-6, Newcastle, United Kingdom. BMVA Press, 2018.

[00175] In embodiments, the steps are preferably carried out by one or more programmed processor(s) or processing frameworks.

[00176] Subsequent to calibration, the images may be used for a wide range of applications including but not limited to mapping or registering one landmark, ROI, or feature from one type of image to another.

[00177] The above described method and system for detecting temperature has a number of meaningful advantages over use of thermal imaging alone to detect a person’ s temperature not the least of which is speed and efficiency. The speed and efficiency arises because the processors executing trained face detection classifiers for visible light cameras are mature, robust, and can accurately and quickly identify landmarks for evaluation. In contrast, little or underwhelming classifiers are available for thermal image analysis. Consequently, the approach described in embodiments of the invention set forth herein offer tremendous advantage in speed and robustness in a contact-less body temperature detection.

[00178] Although a number of embodiments have been disclosed above, it is to be understood that other modifications and variations can be made to the disclosed embodiments without departing from the subject invention.

Claims

CLAIMS A face recognition system for identifying a person and detecting their body temperature as the person is moving through a designated area, the system comprising: a visible light camera and thermal camera aimed at the designated area; and at least one processor operable to perform: face tracking of the person as the person moves through the designated area based on receiving a plurality of consecutive images from each of the cameras; image selecting by selecting at least one candidate image for matching from the plurality of consecutive images being tracked by the visible light camera; defining a ROI from the selected at least one candidate image; mapping the ROI onto one of the plurality of consecutive images tracked by the thermal camera to create a mapped thermal image; and computing the temperature of the person based on the mapped thermal image. The system of claim 1 , wherein the processor is further operable to perform image selecting using a trained classifier. The system of claim 1, further comprising a face matching engine, and wherein the processor is further operable to send the at least one candidate image to the face matching engine, and wherein the image quality count is adjusted with each candidate image sent to the face matching engine. The system of claim 3, wherein the processor is operable to continue face tracking and image selecting for the person until the image quality count reaches a maximum count value. The system of claim 4, wherein the maximum count value is less than 5. The system of claim 1 , further comprising a guidance feature to corral the person walking through the designated region. The system of claim 6, wherein the guidance feature is a handrail.

24 The system of claim 6, wherein the guidance feature is presented by a display or a virtual projector. The system of claim 1, further comprising a display, and the processor and display are operable to show on the display live images of the face of the person during face tracking. The system of claim 9, wherein the processor is operable to superimpose graphics on the live images enclosing the face during face tracking. The system of claim 9, wherein the processor is operable to perform a transformation on the live images during the face tracking to encourage the person to look at the display thereby obtaining a higher quality image. The system of claim 11, wherein the transformation is selected from the group consisting of blurring (blurred around subject’s face); cropping (face cropped image); configure face to line image; configure face cartoon image; unsharp masking; noise suppression; illumination adjustment; high dynamic range; rotation (roll) correction; emoji-type face representation; and animal-type face representation. The system of claim 9, further comprising a housing enclosing the processor, camera and display. The system of claim 9, further comprising display ring, and wherein the display ring is operable to change visually based on the images of the person walking through the designated area. The system of claim 1, wherein the processor is further operable to monitor the time elapsed for face tracking of the person, and terminate face tracking for the person after the time elapsed reaches a maximum time elapsed. The system of claim 15, wherein the maximum time elapsed is equal to or greater than 2 seconds. The system of claim 1, wherein the image quality metric is selected from the group consisting of face size (F_s), Yaw Pitch Roll (YPR), and Laplacian variance (LP). The system of claim 1, wherein the processor is operable to determine instructions for person moving based on the plurality of consecutive images. The system of claim 18, wherein the processor is operable to determine when the person has exited the designated area. The system of claim 19, wherein the processor determines the person has exited the designated area when a face size (F_Size) of the person is greater than a maximum face size (Fmax) and the person is tracked outside the field of view of the camera. The system of claim 20, further comprising a display, and indicating on the display for a next person to enter the designated area. The system of claim 18, wherein the instructions direct said person to enter a second area for further identification. The system of claim 3, further comprising a remote server, and wherein the face matching engine is located on the remote server. The system of claim 23, wherein the face matching engine interrogates the at least one candidate image of the person to confirm the identity of the person. The system of claim 24, wherein the processor is operable to monitor an enrollment state of the system corresponding to a total number of persons whose identify has been confirmed by the face matching engine. The system of claim 1 , wherein the processor is further operable to perform image selecting based on a cue arising from the person walking. The system of claim 24, wherein the cue is visual-based. The system of claim 25, wherein the cue is selected from the group consisting of a badge, article of clothing, band, flag, sign, and gesture. The system of claim 1 , further comprising a visual privacy warning feature to direct the person into the designated area for face capture or to a face capture exclusion area. The system of claim 4, further comprising wherein the maximum count value is less than 3. A face recognition system for identifying and detecting the temperature of a person as the person moves through a designated area, the system comprising: a face detection engine for detecting a face of the person when the person enters the designated area and based on a first sequence of images generated by a camera set as the person enters the designated area; a tracking engine for tracking the face of the person as the person moves through the designated area and based on a second sequence of images generated by the camera set as the person traverses the designated area; and a quality select engine for selecting a maximum quality image for each person traversing the designated area. The system of claim 31, further comprising a face matching engine for matching the maximum quality image with a validated image of the person. The system of claim 32, further comprising a display module for displaying tracking of the person in real time. The system of claim 33, further comprising a guidance feature to corral the person towards the camera set as the person traverses the designated area. The system of claim 33, further comprising a housing enclosing at least one camera of the camera set, detecting engine, quality select engine, tracking engine, and display module.

27 The system of claim 31, further comprising a ROI detection engine for detecting at least one ROI on the face of the person. The system of claim 36, wherein the camera set includes at least one visible light camera and at least one infrared camera, wherein the ROI is detected in an image of the visible light camera. The system of claim 37, further comprising a temperature detection engine for transforming the ROI from the visible light image to a thermal image from the infrared camera, and for computing a temperature of the person based on evaluating the area on the thermal image bound by the ROI. The system of claim 38, wherein the infrared camera is a long range wavelength infrared camera. A face recognition system for identifying and detecting the temperature of a person as the person moves through a designated area, the system comprising a processor operable to transform a ROI detected in a visible light image to a thermal image of the individual; and determine temperature of the individual based on evaluating the area in the thermal image bound by the ROI. A face recognition method for identifying individuals based on a maximum quality image of the face of the individual as each individual moves through a designated area, the method comprising:

(a) streaming images from at least one camera for each individual entering the designated area;

(b) face detecting by searching the streamed images for a face until a face of an individual is detected;

(c) transforming a ROI detected in a visible light image to a thermal image of the individual; and

(d) determining temperature of the individual based on evaluating the area in the thermal image bound by the ROI.

28 The method of claim 41, further comprising face tracking by:

(i) assigning a tracking ID and a quality select count to the individual;

(ii) tracking the face of the individual to obtain at least one candidate image of the face of the individual as the individual moves through the designated area;

(iii) timing the individual during tracking for an elapsed time;

(iv) storing as the maximum quality image the at least one candidate image for face matching if an image quality metric is within a threshold quality range and higher than that of a previously-stored quality image; and

(v) adjusting the quality select count for the individual based on whether the at least one candidate image was stored as the maximum quality image; and

(vi) maximizing image quality by repeating the face tracking step for each individual if the elapsed time is within a maximum time and the quality select count is within a threshold count range. The method of claim 41, wherein the step of storing is carried out remotely from the designated area. The method of claim 41, further comprising, subsequent to the step of maximizing image quality, face matching the maximum quality image with a validated image of the face of the individual. The method of claim 41, wherein the step of maximizing is based on the quality image count being less than or equal to 5, and optionally less than or equal to 3. The method of claim 41, comprising displaying a live stream of images of the individual being tracked during the tracking step. The method of claim 47, comprising superimposing graphics on the live stream of images of the individual being tracked to encourage the individual to look at the camera.

29 The method of claim 48, comprising terminating the tracking step for the individual when the elapsed time is equal to or greater than 2 seconds. The method of claim 49, further comprising displaying an instruction to enter the designated area until tracking is commenced for a next individual. The method of claim 41, wherein at least three (3) candidate images are generated during the step of tracking. The method of claim 41, wherein the quality metric being selected from the group consisting of face size (F_s), Yaw Pitch Roll (YPR), and Laplacian variance (LP). A face recognition method for identifying individuals based on a maximum quality image of the face of the individual as each individual moves through a designated area, the method comprising:

(a) streaming images from at least one camera for each individual entering the designated area;

(b) face detecting by searching the streamed images for a face until a face of an individual is detected;

(c) assigning a tracking ID and a quality select count to the individual;

(d) commencing a face tracking timer for the individual;

(e) tracking in real time the face of the individual to obtain a current candidate image of the face of the individual as the individual moves through the designated area;

(f) delegating the current candidate image as the maximum quality image for face matching if an image quality metric of the current candidate image is within a threshold quality range and, if the maximum quality image had been previously delegated, the current candidate image has a higher quality rating than the previously delegated maximum quality image;

(g) maintaining a quality select count for the individual corresponding to a total number of current candidate images delegated;

(h) continuously updating the maximum quality image for the individual by repeating steps (e) - (g) so long as the quality select count is less than a threshold

30 count and an elapsed tracking time measured by the face tracking timer is less than a maximum time;

(i) transforming a ROI detected in a visible light image to a thermal image of the individual; and

(j) determining temperature of the individual based on evaluating the area in the thermal image bound by the ROI.

53. The method of claim 53, wherein the step of delegating is carried out by saving the delegated candidate image in a storage located remote to the designated area.

54. The method of claim 53, further comprising, subsequent to the step of updating, face matching the delegated candidate image with a validated image of the face of the individual.

55. A method for calibrating a thermal LWIR and visible light camera as described herein.

56. A calibration target as described herein.

57. A system for calibrating a thermal LWIR and visible light camera comprising: a camera unit and calibration target as described herein.

58. A calibration method comprising computing a transformation matrix for mapping image points between a long wavelength infrared (LWIR) camera and image points of a visible light spectrum (VIS) camera based on a plurality of image pairs, wherein each of the image pairs comprises a LWIR image and VIS image of a target taken simultaneously.

59. The method of claim 58, further comprising correcting the image pairs for optical defects prior to the step of computing.

60. The method of claim 58, further comprising providing the target.

61. The method of claim 60, wherein the target comprises a plurality of types of units.

31 The method of claim 61, further comprising activating at least one of the types of units, creating contrast between the different types of units in the LWIR region of the spectrum prior to taking the image pairs. The method of claim 62, wherein the activating is performed by heating. A method as recited in any one of the above claims 41-54, wherein the visible light images are obtained by a visible light spectrum (VIS) camera and the thermal images are obtained by a long wavelength infrared (LWIR) camera, and the method further comprising calibrating the VIS camera with the LWIR thermal camera. A system as recited in any one of the above claims 1-40, further comprising a calibration target as described herein for calibrating a VIS camera and a LWIR camera.

32