CN113111806B - Method and system for target recognition - Google Patents

Abstract

The embodiments of this specification disclose a target recognition method and system. The target recognition method includes: determining an illumination sequence, the illumination sequence is used to determine multiple colors of multiple illuminations emitted by a terminal when illuminating a target object; acquiring multiple target images based on the terminal, the shooting time of the multiple target images has a corresponding relationship with the irradiation time of the multiple illuminations; and determining the authenticity of the multiple target images based on the illumination sequence and the multiple target images.

Description

Method and system for target recognition

Technical Field

The present invention relates to the field of image processing technology, and in particular to a method and system for target recognition.

Background Art

Target recognition is a biometric technology based on the target acquired by the image acquisition device. For example, face recognition technology, which targets human faces, is widely used in application scenarios such as permission verification and identity verification. In order to ensure the security of target recognition, it is necessary to determine the authenticity of the target image.

Therefore, it is desirable to provide a method and system for target recognition, which can determine the authenticity of a target image.

Summary of the invention

One of the embodiments of the present specification provides a target recognition method, which includes: determining a lighting sequence, the lighting sequence is used to determine multiple colors of multiple lightings that illuminate a target object; acquiring multiple target images, the shooting time of the multiple target images having a corresponding relationship with the exposure time of the multiple lightings; and determining the authenticity of the multiple target images based on the lighting sequence and the multiple target images.

One of the embodiments of the present specification provides a target recognition system, which includes: a determination module, used to determine a lighting sequence, the lighting sequence is used to determine multiple colors of multiple lightings that illuminate a target object; an acquisition module, used to acquire multiple target images, the shooting time of the multiple target images has a corresponding relationship with the irradiation time of the multiple lightings; a verification module, used to determine the authenticity of the multiple target images based on the lighting sequence and the multiple target images.

One of the embodiments of this specification provides a target recognition device, including a processor, wherein the processor is used to execute the target recognition method disclosed in this specification.

One of the embodiments of the present specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the target recognition method disclosed in the present specification.

BRIEF DESCRIPTION OF THE DRAWINGS

This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

FIG1 is a schematic diagram of an application scenario of a target recognition system according to some embodiments of this specification;

FIG2 is an exemplary flow chart of a target recognition method according to some embodiments of this specification;

FIG3 is a schematic diagram of an illumination sequence according to some embodiments of the present specification;

FIG4 is another schematic diagram of an illumination sequence according to some embodiments of the present specification;

FIG5 is an exemplary flow chart of acquiring multiple target images according to some embodiments of the present specification;

FIG6 is a schematic diagram of texture replacement according to some embodiments of the present specification;

FIG7 is an exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification;

FIG8 is another exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification;

FIG. 9 is another exemplary flowchart of determining the authenticity of multiple target images according to some embodiments of this specification.

FIG10 is a schematic diagram of the structure of a first verification model according to some embodiments of this specification;

FIG11 is another exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification;

FIG12 is another exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification;

FIG13 is a schematic diagram of the structure of a second verification model according to some embodiments of this specification;

FIG. 14 is another exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification; and

FIG. 15 is a schematic diagram of the structure of a third verification model according to some embodiments of the present specification.

DETAILED DESCRIPTION

In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

It should be understood that the "system", "device", "unit" and/or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and/or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

Target recognition is a technology that performs biometric identification based on the target object acquired by the image acquisition device. In some embodiments, the target object can be a face, fingerprint, palm print, pupil, etc. In some embodiments, target recognition can be applied to authority verification. For example, access control authority authentication and account payment authority authentication. In some embodiments, target recognition can also be used for identity verification. For example, employee attendance authentication and personal registration identity security authentication. As an example only, target recognition can be based on matching the target image acquired in real time by the image acquisition device with the pre-acquired biometric features to verify the target identity.

However, the image acquisition device may be attacked or hijacked, and the attacker can upload a fake target image to pass the identity authentication. For example, attacker A can directly upload the face image of user B after attacking or hijacking the image acquisition device. The target recognition system performs face recognition based on the face image of user B and the pre-acquired face biometric features of user B, thereby passing the identity authentication of user B.

Therefore, in order to ensure the security of target recognition, it is necessary to determine the authenticity of the target image, that is, to determine whether the target image is captured in real time by the image acquisition device during the target recognition process.

Fig. 1 is a schematic diagram of an application scenario of a target recognition system according to some embodiments of this specification. As shown in Fig. 1 , a target recognition system 100 may include a processing device 110 , a network 120 , a terminal 130 , and a storage device 140 .

The processing device 110 may be used to process data and/or information from at least one component of the target recognition system 100 and/or an external data source (e.g., a cloud data center). For example, the processing device 110 may determine an illumination sequence, obtain multiple target images, and determine the authenticity of multiple target images. For another example, the processing device 110 may pre-process multiple initial images obtained from the terminal 130 (e.g., replace textures, etc.) to obtain multiple target images. During the processing, the processing device 110 may obtain data (e.g., instructions) from other components of the target recognition system 100 (e.g., storage device 140 and/or terminal 130) directly or through the network 120 and/or send the processed data to the other components for storage or display.

In some embodiments, the processing device 110 may be a single server or a server group. The server group may be centralized or distributed (e.g., the processing device 110 may be a distributed system). In some embodiments, the processing device 110 may be local or remote. In some embodiments, the processing device 110 may be implemented on a cloud platform or provided in a virtual manner. By way of example only, a cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.

The network 120 can connect the components of the system and/or connect the system with external parts. The network 120 enables communication between the components of the target identification system 100 and between the target identification system 100 and the external parts, and promotes the exchange of data and/or information. In some embodiments, the network 120 can be any one or more of a wired network or a wireless network. For example, the network 120 can include a cable network, an optical fiber network, a telecommunications network, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network (ZigBee), a near field communication (NFC), a bus within a device, a line within a device, a cable connection, etc. or any combination thereof. In some embodiments, the network connection between the various parts of the target identification system 100 can adopt one of the above methods or multiple methods. In some embodiments, the network 120 can be a point-to-point, shared, central, etc., various topological structures or a combination of multiple topological structures. In some embodiments, the network 120 can include one or more network access points. For example, network 120 may include wired or wireless network access points, such as base stations and/or network switching points 120-1, 120-2, ..., through which one or more components of target identification system 100 may connect to network 120 to exchange data and/or information.

The terminal 130 refers to one or more terminal devices or software used by the user. In some embodiments, the terminal 130 may include an image acquisition device 131 (e.g., a camera, a still camera), which may capture a target object and obtain multiple target images. In some embodiments, when the image acquisition device 131 captures the target object, the terminal 130 (e.g., the screen and/or other light emitting components of the terminal 130) may sequentially emit multiple colors of light in the illumination sequence to illuminate the target object. In some embodiments, the terminal 130 may communicate with the processing device 110 via the network 120 and send the captured multiple target images to the processing device 110. In some embodiments, the terminal 130 may be a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, other devices with input and/or output functions, etc., or any combination thereof. The above examples are only used to illustrate the extensiveness of the types of the terminal 130 and are not intended to limit their scope.

The storage device 140 can be used to store data (such as an illumination sequence, multiple initial images or multiple target images, etc.) and/or instructions. The storage device 140 may include one or more storage components, each of which may be an independent device or part of another device. In some embodiments, the storage device 140 may include a random access memory (RAM), a read-only memory (ROM), a large capacity storage device, a removable memory, a volatile read-write memory, etc., or any combination thereof. Exemplarily, the large capacity storage device may include a magnetic disk, an optical disk, a solid-state disk, etc. In some embodiments, the storage device 140 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof. In some embodiments, the storage device 140 may be integrated or included in one or more other components of the target recognition system 100 (e.g., the processing device 110, the terminal 130, or other possible components).

In some embodiments, the target recognition system 100 may include a determination module, an acquisition module, and a verification module.

The determination module may be used to determine a lighting sequence, where the lighting sequence is used to determine a plurality of colors of a plurality of lightings for illuminating the target object.

The acquisition module can be used to acquire multiple target images, and the shooting time of the multiple target images has a corresponding relationship with the irradiation time of multiple light illuminations.

In some embodiments, the acquisition module may be used to acquire multiple initial images, and pre-process the multiple initial images to obtain multiple target images.

In some embodiments, the acquisition module can be used to acquire a color verification model, which is a machine learning model with preset parameters. For example, the preset parameters of the color verification model are acquired through training.

The verification module can be used to determine the authenticity of multiple target images based on the illumination sequence and the multiple target images.

In some embodiments, the plurality of colors include at least one reference color and at least one verification color. The relationship between the at least one reference color and the at least one verification color can be various. For example, each of the at least one verification color is determined based on at least a portion of the color of the at least one reference color, and for another example, one or more of the at least one reference color is the same as one or more of the at least one verification color.

In some embodiments, the plurality of target images include at least one verification image and at least one reference image, each of the at least one verification image corresponds to one of the at least one verification color, and each of the plurality of reference images corresponds to one of the at least one reference color.

In some embodiments, the verification module can be used to determine the color of the illumination when at least one verification image was taken based on at least one reference image, and determine the authenticity of multiple target images based on the illumination sequence and the color of the illumination when at least one verification image was taken.

In some embodiments, the verification module may be used to determine the authenticity of the plurality of target images based on a first image sequence determined based on the plurality of target images and a second image sequence determined based on the plurality of color template images, wherein the plurality of color template images are generated based on the illumination sequence.

In some embodiments, the verification module can be used to determine the authenticity of multiple target images based on a first color relationship between at least one reference image and at least one verification image, a second color relationship between at least one reference color and at least one verification color, and based on the first color relationship and the second color relationship.

In some embodiments, the verification module can be used to determine the authenticity of multiple target images based on the illumination sequence and the color verification model. For example, the verification model processes the multiple target images based on the color verification model to obtain processing results, and determines the authenticity of the multiple target images in combination with the processing results and the illumination sequence.

For more detailed descriptions of the determination module, acquisition module and verification module, please refer to Figures 2-15, which will not be repeated here.

It should be noted that the above description of the target identification system and its modules is only for the convenience of description, and this specification cannot be limited to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the various modules, or form a subsystem connected with other modules without deviating from this principle. In some embodiments, the determination module, acquisition module and verification module disclosed in Figure 1 can be different modules in a system, or a module can implement the functions of two or more modules mentioned above. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the scope of protection of this specification.

In some embodiments, the method for performing target recognition by the processing device of the target recognition system 100 may include: determining a lighting sequence, wherein the lighting sequence is used to determine multiple colors of multiple lightings that illuminate the target object; acquiring multiple target images, wherein the shooting time of the multiple target images corresponds to the exposure time of the multiple lightings; and determining the authenticity of the multiple target images based on the lighting sequence and the multiple target images.

In some embodiments, the processing device acquiring multiple target images may include: acquiring multiple initial images, the multiple initial images including a first initial image and a second initial image; replacing the texture of the second initial image with the texture of the first initial image to generate a processed second initial image; and using the processed second initial image as one of the multiple target images.

In some embodiments, the processing device replaces the texture of the second initial image with the texture of the first initial image to generate a processed second initial image, which may include: based on a color migration algorithm, migrating the color of the illumination when the second initial image was captured to the first initial image to obtain the processed second initial image.

In some embodiments, the plurality of colors include at least one reference color and at least one verification color, each of the at least one verification color being determined based on at least a portion of the at least one reference color. In some embodiments, one or more of the at least one reference color is the same as one or more of the at least one verification color.

In some embodiments, the multiple target images include at least one verification image and at least one reference image, each of the at least one verification image corresponds to one of the at least one verification colors, and each of the at least one reference image corresponds to one of the at least one reference colors. The processing device determines the authenticity of the multiple target images based on the illumination sequence and the multiple target images, which may include: extracting a reference color feature of the at least one reference image and a verification color feature of the at least one verification image; for each of the at least one verification image, determining the color of illumination when the verification image was captured based on the verification color feature of the verification image and the reference color feature of the at least one reference image; and determining the authenticity of the multiple target images based on the illumination sequence and the color of illumination when the at least one verification image was captured.

In some embodiments, the processing device determines the authenticity of the multiple target images based on the illumination sequence and the multiple target images, including: determining a first image sequence based on the multiple target images; determining a second image sequence based on a plurality of color template images, wherein the plurality of color template images are generated based on the illumination sequence; processing the first image sequence based on a first extraction layer to extract first feature information of the first image sequence; processing the second image sequence based on a second extraction layer to extract second feature information of the second image sequence; and processing the first feature information and the second feature information based on a discriminant layer to determine the authenticity of the multiple target images, wherein the first extraction layer, the second extraction layer and the discriminant layer are machine learning models with preset parameters, and the first extraction layer and the second extraction layer share parameters.

In some embodiments, the preset parameters of the first extraction layer, the second extraction layer and the discrimination layer are obtained through an end-to-end training method.

In some embodiments, the multiple colors include at least one reference color and at least one verification color, the multiple target images include at least one reference image and at least one verification image, each of the at least one reference image corresponds to one of the at least one reference colors, and each of the at least one verification image corresponds to one of the at least one verification colors. The processing device determines the authenticity of the multiple target images based on the illumination sequence and the multiple target images, including: extracting reference color features of each of the at least one reference image and verification color features of each of the at least one verification image; for each of the at least one reference image, based on the reference color features of the reference image and the verification color features of each verification image, determining a first color relationship between the reference image and each verification image; for each of the at least one reference color, determining a second color relationship between the reference color and each of the verification colors; and determining the authenticity of the multiple target images based on the at least one first color relationship and the at least one second color relationship.

In some embodiments, the processing device determines the authenticity of the multiple target images based on the illumination sequence and the multiple target images, including: obtaining a color verification model, where the color verification model is a machine learning model with preset parameters; and based on the illumination sequence, processing the multiple target images using the color verification model to determine the authenticity of the multiple target images.

FIG2 is an exemplary flow chart of a target recognition method according to some embodiments of the present specification. As shown in FIG2, process 200 includes the following steps:

Step 210, determining an illumination sequence. The illumination sequence is used to determine multiple colors of multiple illuminations for illuminating a target object. In some embodiments, step 210 may be performed by a determination module.

The target object refers to the object that needs to be identified. For example, the target object can be a specific body part of the user, such as the face, fingerprint, palm print or pupil. In some embodiments, the target object refers to the face of the user who needs identity authentication and/or authority authentication. For example, in the online car-hailing application scenario, the platform needs to verify whether the driver who accepts the order is a registered driver user reviewed by the platform, then the target object is the driver's face. For another example, in the face payment application scenario, the payment system needs to verify the payment authority of the payee, then the target object is the face of the payee.

To perform target recognition on the target object, the terminal may be instructed to emit the illumination sequence. The illumination sequence includes a plurality of illuminations for illuminating the target object. The colors of different illuminations in the illumination sequence may be the same or different. In some embodiments, the plurality of illuminations include at least two illuminations of different colors, that is, the plurality of illuminations have multiple colors.

As used herein, "determining an illumination sequence" refers to determining information of each illumination in the multiple illuminations contained in the illumination sequence, such as color information, illumination time, etc. The color information of the multiple illuminations in the illumination sequence can be represented in the same or different ways. For example, the color information of the multiple illuminations can be represented by color categories. For example, the colors of the multiple illuminations in the illumination sequence can be represented as red, yellow, green, purple, cyan, blue, and red. For another example, the color information of the multiple illuminations can be represented by color parameters. For example, the colors of the multiple illuminations in the illumination sequence can be represented as RGB (255, 0, 0), RGB (255, 255, 0), RGB (0, 255, 0), RGB (255, 0, 255), RGB (0, 255, 255), RGB (0, 0, 255). In some embodiments, the illumination sequence may also be referred to as a color sequence, which contains the color information of the multiple illuminations.

The illumination time of multiple illuminations in the illumination sequence may include the start time, end time, duration, etc. or any combination thereof for each illumination plan irradiating the target object. For example, the start time of red light irradiating the target object is 14:00, and the start time of green light irradiating the target object is 14:02. For another example, the duration of red light and green light irradiating the target object is 0.1 second. In some embodiments, the duration of different illuminations irradiating the target object may be the same or different. The illumination time may be represented by other methods, which will not be described in detail here.

In some embodiments, the terminal may emit multiple illuminations in sequence in a specific order. In some embodiments, the terminal may emit illumination through a light-emitting element. The light-emitting element may include a light-emitting element built into the terminal, such as a screen, an LED lamp, etc. The light-emitting element may also include an external light-emitting element. For example, an external LED lamp, a light-emitting diode, etc. In some embodiments, when the terminal is hijacked or attacked, the terminal may receive an instruction to emit illumination, but will not actually emit illumination. For more details about the illumination sequence, please refer to Figures 3 and 4 and their related descriptions, which will not be repeated here.

In some embodiments, the terminal or processing device (e.g., determination module) may generate a lighting sequence randomly or based on a preset rule. For example, the terminal or processing device may randomly extract a plurality of colors from a color library to generate a lighting sequence. In some embodiments, the lighting sequence may be set by a user at the terminal, determined according to a default setting of the target recognition system 100, or determined by a processing device through data analysis (e.g., using a determination model), etc. In some embodiments, the terminal or storage device may store the lighting sequence. Accordingly, the acquisition module may acquire the lighting sequence from the terminal or storage device via a network.

Step 220, acquiring multiple target images. In some embodiments, step 220 may be performed by an acquisition module.

The multiple target images are images used for target recognition. The formats of the multiple target images may include Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), Graphics Interchange Format (GIF), Kodak Flash PiX (FPX), Digital Imaging and Communications in Medicine (DICOM), etc. The multiple target images may be two-dimensional (2D, two-dimensional) images or three-dimensional (3D, three-dimensional) images.

In some embodiments, the acquisition module may acquire the multiple target images based on the terminal. For example, the acquisition module may send an acquisition instruction to the terminal via a network, and then receive the multiple target images sent by the terminal via the network. Alternatively, the terminal may send the multiple target images to a storage device for storage, and the acquisition module may acquire the multiple target images from the storage device. The target image may or may not contain the target object.

The target image may be taken by the image acquisition device of the terminal, or may be determined based on data uploaded by the user (e.g., video or image). For example, during the verification of the target object, the target recognition system 100 will give the terminal a sequence of illumination. When the terminal is not hijacked or attacked, the terminal may emit the multiple illuminations in sequence according to the illumination sequence. When the terminal emits one of the multiple illuminations, its image acquisition device may be instructed to capture one or more images during the illumination time of the illumination. Alternatively, the image acquisition device of the terminal may be instructed to capture a video during the entire illumination period of the multiple illuminations. The terminal or other computing device (e.g., processing device 110) may capture one or more images captured during the illumination time of each illumination from the video according to the illumination time of each illumination. One or more images captured by the terminal during the illumination time of each illumination may be used as the multiple target images. At this time, the multiple target images are real images of the target object captured when being illuminated by the multiple illuminations. It can be understood that there is a corresponding relationship between the illumination time of the multiple illuminations and the shooting time of the multiple target images. If one image is collected within the illumination time of a single illumination, the correspondence is one-to-one; if multiple images are collected within the illumination time of a single illumination, the correspondence is one-to-many.

When the terminal is hijacked, the hijacker can upload images or videos through the terminal device. The uploaded images or videos may contain specific body parts of the target object or other users, and/or other objects. . The uploaded images or videos may be historical images or videos taken by the terminal or other terminals, or synthesized images or videos. The terminal or other computing device (such as the processing device 110) may determine the multiple target images based on the uploaded images or videos. For example, the hijacked terminal may extract one or more images corresponding to each illumination from the uploaded images or videos according to the illumination order and/or illumination duration of each illumination in the illumination sequence. As an example only, the illumination sequence contains five illuminations arranged in sequence, and the hijacker may upload five target images through the terminal device. The terminal or other computing device will determine the target image corresponding to each illumination in the five illuminations according to the order in which the five target images are uploaded. For another example, the illumination time of the five illuminations in the illumination sequence is 0.5 seconds respectively, and the hijacker may upload a video of 2.5 seconds through the terminal. The terminal or other computing device can divide the uploaded video into five segments of 0-0.5 seconds, 0.5-1 seconds, 1-1.5 seconds, 1.5-2 seconds and 2-2.5 seconds, and capture a target image in each video segment. The five target images captured from the video correspond to the five illuminations in sequence. At this time, the multiple target images are false images uploaded by the hijacker, rather than real images taken when the target object is illuminated by the multiple illuminations. In some embodiments, if the target image is uploaded by the hijacker through the terminal, the upload time of the target image or the time it is shot in the video can be regarded as its shooting time. It can be understood that when the terminal is hijacked, there is also a corresponding relationship between the illumination time of multiple illuminations and the shooting time of multiple target images.

For each of the multiple target images, the determination module may use the color of the illumination corresponding to the illumination time in the illumination sequence and the shooting time of the target image as the color corresponding to the target image. Specifically, if the illumination time of the illumination corresponds to the shooting time of one or more target images, the color of the illumination is used as the color corresponding to the one or more target images. It can be understood that when the terminal is not hijacked or attacked, the colors corresponding to the multiple target images should be the same as the multiple colors of the multiple illuminations in the illumination sequence. For example, the multiple colors of the multiple illuminations in the illumination sequence are "red, yellow, blue, green, purple, and red". When the terminal is not hijacked or attacked, the colors corresponding to the multiple target images obtained by the terminal should also be "red, yellow, blue, green, purple, and red". When the terminal is hijacked or attacked, the colors corresponding to the multiple target images may be different from the multiple colors of the multiple illuminations in the illumination sequence.

In some embodiments, the acquisition module may acquire multiple initial images from the terminal and pre-process the multiple initial images to acquire the multiple target images. The multiple initial images may be taken by the terminal or uploaded by the hijacker through the terminal.

It is understandable that there is a corresponding relationship between the shooting time of the multiple initial images and the irradiation time of the multiple illuminations. If the multiple target images are obtained based on the preprocessing of the multiple initial images, the corresponding relationship between the shooting time of the multiple target images and the irradiation time of the multiple illuminations actually reflects the corresponding relationship between the shooting time of the multiple initial images corresponding to the multiple target images and the irradiation time of the multiple illuminations, and the color of the illumination when the target image is shot actually reflects the color of the illumination when the initial image corresponding to the target image is shot.

In some embodiments, preprocessing may include texture uniformization. The texture of an image refers to the grayscale distribution of elements (such as pixels) in the image and their surrounding spatial neighborhoods. It is understandable that if the multiple initial images are taken by a terminal, the textures of the multiple initial images may be different due to the possible changes in the distance, angle, and background of the terminal and the target object during acquisition. Texture uniformization can make the textures of the multiple initial images the same or substantially the same, reduce the interference of texture features, and thus improve the efficiency and accuracy of target recognition.

In some embodiments, the acquisition module can achieve texture uniformization through texture replacement. Texture replacement refers to replacing the texture of all initial images with the texture of a specified image. In some embodiments, the specified image can be one of multiple initial images, that is, the acquisition module can replace the texture of other initial images with the texture of one of the multiple initial images to achieve texture uniformization. Alternatively, the specified image can be an image of the target object other than the multiple initial images. For example, an image of the target object taken in the past and stored in a storage device. For a specific description of texture replacement, please refer to Figure 5 and its related description, which will not be repeated here.

In some embodiments, the acquisition module can achieve texture consistency processing by removing the background, correcting the shooting angle, etc. For example, taking the target object as the target face, the parts other than the face in the multiple initial images are removed, and then the angle of the face in the remaining part is corrected to a preset angle (for example, facing the image acquisition device, etc.). As an example only, the removal of the background can be achieved by identifying the facial contour of each of the multiple initial images through image recognition technology, and then removing the part other than the facial contour. Angle correction can be achieved by a correction algorithm (for example, a face correction algorithm) or a model. In some embodiments, the acquisition module can also achieve texture consistency processing in other ways, which are not limited here.

In some embodiments, preprocessing may also include image screening, image denoising, image enhancement, etc.

Image screening may include screening out images that do not include the target object or a specific body part of the user. The screened object may be an initial image captured by the terminal, or an image obtained by other preprocessing (e.g., texture uniformization) of the initial image. For example, the acquisition module may match the features of the initial image with the features of the image containing the target object, and screen out images that do not include the target object from the multiple initial images.

Image denoising may include removing interference information in the image. Interference information in the image not only reduces the quality of the image, but also affects the color features extracted based on the image. In some embodiments, the acquisition module may implement image denoising through a median filter, a machine learning model, etc.

Image enhancement can add missing information in the image. Missing information in the image will cause the image to be blurred and will also affect the color features extracted based on the image. For example, image enhancement can adjust the brightness, contrast, saturation, hue, etc. of the image to increase its clarity, reduce noise, etc. In some embodiments, the acquisition module can achieve image enhancement through a smoothing filter, a median filter, etc.

Similar to image screening, the object of image denoising or image enhancement can be the original image, or the image obtained after other preprocessing of the initial image.

In some embodiments, the preprocessing may also include other operations, which are not limited here. In some embodiments, the object recognition system 100 may further include a preprocessing module for preprocessing the initial image.

Step 230, based on the illumination sequence and the multiple target images, determine the authenticity of the multiple target images. In some embodiments, step 230 can be performed by a verification module.

The authenticity of multiple target images can reflect whether the multiple target images are images captured by the target object under illumination of multiple colors. For example, when the terminal is not hijacked or attacked, its light-emitting element can emit multiple colors of light, and its image acquisition device can record or take pictures of the target object to obtain the target image. At this time, the target image is authentic. For another example, when the terminal is hijacked or attacked, the target image is obtained based on the image or video uploaded by the attacker. At this time, the target image is not authentic.

If multiple target images are obtained based on preprocessing of multiple initial images, the authenticity of the multiple target images can also be called the authenticity of the multiple initial images, which can reflect whether the multiple initial images corresponding to the multiple target images are images of the target object captured under illumination of multiple colors. For example, when the terminal is not hijacked or attacked, the initial image and the target image are authentic. For another example, when the terminal is hijacked or attacked, the initial image and the target image are not authentic. For the convenience of description, the authenticity of multiple target images and the authenticity of multiple initial images are collectively referred to as the authenticity of multiple target images below.

The authenticity of the target image can be used to determine whether the image acquisition device of the terminal has been hijacked by an attacker. For example, if at least one of the multiple target images is not authentic, it means that the image acquisition device has been hijacked. For another example, if more than a preset number of the multiple target images are not authentic, it means that the image acquisition device has been hijacked.

In some embodiments, the verification module can determine the authenticity of multiple target images based on the color features and illumination sequences of the multiple target images. For more details on determining the authenticity of the target images based on the color features of the target images, please refer to FIG. 7 and its related description, which will not be repeated here.

The color features of an image refer to information related to the color of the image. The color of the image includes the color of the illumination when the image is captured, the color of the object in the image, the color of the background in the image, etc. In some embodiments, the color features may include deep features and/or complex features extracted by a neural network.

Color features can be represented in a variety of ways. In some embodiments, color features can be represented based on the color values of each pixel in the image in the color space. Color space is a mathematical model that uses a set of numerical values to describe color, and each numerical value in the set of numerical values can represent the color value of the color feature on each color channel in the color space. In some embodiments, the color space can be represented as a vector space, and each dimension of the vector space represents a color channel of the color space. Color features can be represented by vectors in the vector space. In some embodiments, the color space may include but is not limited to RGB color space, Lαβ color space, LMS color space, HSV color space, YCrCb color space, and HSL color space. It can be understood that different color spaces contain different color channels. For example, the RGB color space contains a red channel R, a green channel G, and a blue channel B, and the color features can be represented by the color values of each pixel in the image on the red channel R, the green channel G, and the blue channel B, respectively.

In some embodiments, the color feature can be represented by other means (e.g., color histogram, color moment, color set, etc.). For example, the color values of each pixel in the image in the color space are counted by histogram to generate a histogram representing the color feature. For another example, a specific operation (e.g., mean, square difference, etc.) is performed on the color values of each pixel in the image in the color space, and the result of the specific operation is used to represent the color feature of the image.

In some embodiments, the verification module can extract color features of multiple target images through a color feature extraction algorithm and/or a color verification model (or part thereof). The color feature extraction algorithm includes: color histogram, color moment, color set, etc. For example, the verification module can obtain a color histogram by statistically calculating the gradient histogram based on the color values of each pixel in the image in each color channel of the color space. For another example, the verification module can divide the image into multiple regions, and use the set of binary indexes of multiple regions established by the color values of each pixel in the image in each color channel of the color space to determine the color set of the image. For more details on extracting color features based on the color verification model, see Figures 10, 13 and 15 and their related descriptions.

In some embodiments, the verification module can process multiple target images based on the illumination sequence and using a color verification model to determine the authenticity of the multiple target images, wherein the color verification model is a machine learning model with preset parameters. For determining the authenticity of the target image based on the color verification model, please refer to FIG. 8 and its related description, which will not be repeated here.

In some embodiments of the present specification, the target recognition system 100 will provide a terminal with a light illumination sequence, and obtain a target image corresponding to multiple illuminations in the illumination sequence from the terminal. By identifying the color of the illumination when the target image is captured, the processing device can determine whether the target image or its corresponding initial image is an image captured when the target object is illuminated by the illumination sequence, and further determine whether the terminal has been hijacked or attacked. It can be understood that without the attacker knowing the illumination sequence, it is difficult for the color of the illumination when the uploaded image or the image in the uploaded video is captured to be the same as the colors of the multiple illuminations in the illumination sequence. Even if the types of colors are the same, the position order of each color is difficult to be the same. The method disclosed in this specification can increase the difficulty of the attacker's attack and ensure the security of target recognition.

FIG. 3 is a schematic diagram of an illumination sequence according to some embodiments of the present specification.

As mentioned above, the illumination sequence may include multiple illuminations. In some embodiments, the colors of the multiple illuminations in the illumination sequence may be completely the same, completely different, or partially the same. For example, the colors of the multiple illuminations are all red. For another example, at least two of the multiple illuminations have different colors, that is, the multiple illuminations have multiple colors. In some embodiments, the multiple colors include white. In some embodiments, the multiple colors include red, blue, and green.

Multiple lights in the lighting sequence are arranged in a specific order. As shown in Figure 3, lighting sequence a includes four lights, namely red light, white light, blue light, and green light, arranged in sequence; lighting sequence b includes four lights, namely white light, blue light, red light, and green light, arranged in sequence; lighting sequence c includes four lights, namely red light, white light, blue light, and white light, arranged in sequence; and lighting sequence d includes four lights, namely red light, white light, white light, and blue light, arranged in sequence. The colors of multiple lights in lighting sequence a and lighting sequence b are the same, but the arrangement order is different. Similarly, the colors of multiple lights in lighting sequence c and lighting sequence d are the same, but the arrangement order is different. In addition, the colors of the four lights in lighting sequence a and b are completely different, and the colors of two lights in lighting sequence c and d are the same.

FIG. 4 is another schematic diagram of an illumination sequence according to some embodiments of the present specification.

In some embodiments, the multiple colors of illumination in the illumination sequence may include at least one reference color and at least one verification color. The verification color is a color among multiple colors that is directly used to verify the authenticity of the image. The reference color is a color among multiple colors that is used to assist the verification color in determining the authenticity of the target image. For example, the target image corresponding to the reference color (also called the reference image) can be used to determine the color of illumination when the target image corresponding to the verification color (also called the verification image) is photographed. Further, the verification module can determine the authenticity of multiple target images based on the color of illumination when the verification image is photographed. For another example, the target image corresponding to the reference color (also called the reference image) can be used to determine the first color relationship of the target image corresponding to the verification color (also called the verification image). Further, the verification module can determine the authenticity of multiple target images based on the first color relationship. As shown in FIG. 4 , the illumination sequence e includes illumination of multiple reference colors "red light, green light, blue light", and illumination of multiple verification colors "yellow light, purple light... cyan light"; the illumination sequence f includes illumination of multiple reference colors "red light, white light... blue light", and illumination of multiple verification colors "red light... green light".

In some embodiments, there are multiple verification colors. The multiple verification colors can be exactly the same. For example, the verification colors can be red, red, red, and red. Alternatively, the multiple verification colors can also be completely different. For example, the verification colors can be red, yellow, blue, green, and purple. Alternatively, the multiple verification colors can also be partially the same. For example, the verification colors can be yellow, green, purple, yellow, and red. Similar to the verification colors, in some embodiments, there are multiple reference colors, and the multiple reference colors can be exactly the same, completely different, or partially the same. In some embodiments, the verification colors can include only one color, such as green.

In some embodiments, the at least one reference color and the at least one verification color can be determined according to the default setting of the target recognition system 100, manually set by the user, or determined by the determination module. For example, the determination module can randomly select the reference color and the verification color. As an example only, the determination module can randomly select some colors from a plurality of colors as the at least one reference color, and the remaining colors as the at least one verification color. In some embodiments, the determination module can determine the at least one reference color and the at least one verification color based on preset rules. The preset rules can be rules about the relationship between verification colors, the relationship between reference colors, and/or the relationship between verification colors and reference colors. For example, the preset rule is that the verification color can be generated based on the fusion of the reference colors.

In some embodiments, each of the at least one verification color can be determined based on at least a portion of the colors in at least one reference color. For example, the verification color can be obtained by merging at least a portion of the colors in at least one reference color. In some embodiments, at least one reference color can include a base color or primary color in a color space. For example, the at least one reference color can include the three primary colors of the RGB space, namely "red, green, and blue". As shown in Figure 4, multiple verification colors "yellow, purple...cyan" in the illumination sequence e can be determined based on the three reference colors "red, green, and blue". For example, "yellow" can be obtained by merging the reference colors "red, green, and blue" based on a first ratio, and "purple" can be obtained by merging the reference colors "red, green, and blue" based on a second ratio.

In some embodiments, one or more of at least one reference color is the same as one or more of at least one verification color. At least one reference color and at least one verification color may be completely or partially the same. For example, one of at least one verification color may be the same as a specific one of at least one reference color. It is understandable that the verification color may also be determined based on at least one reference color, that is, the specific reference color may be used as the verification color. As shown in FIG4 , in the illumination sequence f, multiple reference colors “red, white…blue” and multiple verification colors “red…green” all contain red.

In some embodiments, at least one reference color and at least one verification color may have other relationships, which are not limited here. For example, at least one reference color and at least one verification color have the same or different color systems. For example, at least one reference color belongs to a warm color system (such as red, yellow, etc.), and at least one reference color belongs to a cool color system (such as gray, etc.).

In some embodiments, in the illumination sequence, the illumination corresponding to the at least one reference color can be arranged in front of or behind the illumination corresponding to the at least one verification color. As shown in FIG4 , in the illumination sequence e, the illuminations of multiple reference colors "red light, green light, blue light" are arranged in front of the illuminations of multiple verification colors "yellow light, purple light...cyan light". In the illumination sequence f, the illuminations of multiple reference colors "red light, white light...blue light" are arranged behind the illuminations of multiple verification colors "red light...green light". In some embodiments, the illumination corresponding to the at least one reference color can also be arranged alternately with the illumination corresponding to the at least one verification color, which is not limited here.

FIG5 is an exemplary flow chart of acquiring multiple target images according to some embodiments of this specification. In some embodiments, process 500 may be performed by an acquisition module. As shown in FIG5 , process 500 includes the following steps:

Step 510: Acquire multiple initial images.

As mentioned above, the multiple initial images are unprocessed images obtained from the terminal. In some embodiments, the multiple initial images may be images taken by an image acquisition device of the terminal, or may be images determined by the hijacked terminal based on images or videos uploaded by the hijacker. In some embodiments, the multiple initial images may include a first initial image and a second initial image.

As described above, the acquisition module may perform texture replacement on the multiple initial images to generate the multiple target images described in step 220. For example, the acquisition module may replace the texture of the multiple initial images with the texture in the specified image. The first initial image refers to the specified image in the multiple initial images, that is, the initial image that provides the texture for replacement. In some embodiments, the first initial image needs to contain the target object. For example, the acquisition module may obtain the first initial image containing the target object from the multiple initial images through image screening. Alternatively, the first initial image may be any one of the multiple initial images. For another example, the first initial image may be the one with the earliest shooting time among the multiple initial images. For another example, the first initial image may be the one with the simplest background among the multiple initial images. As an example only, the simplicity of the background is determined by the color types of the background. The fewer the color types of the background, the simpler the background. The simplicity of the background is also determined by the complexity of the lines of the background. The fewer the lines of the background, the simpler the background. In some embodiments, white light exists in the illumination sequence. The first initial image may be an initial image of the multiple target images whose acquisition time corresponds to the white light irradiation time.

The second initial image is an initial image of the replaced texture among the multiple initial images. In some embodiments, the second initial image may be any initial image except the first initial image. The second initial image may be one or more images.

In some embodiments, the terminal may collect the corresponding initial image according to the illumination time of the illumination in the illumination sequence. The acquisition module may obtain the multiple initial images from the terminal through the network. Alternatively, the hijacker may upload images or videos through the terminal device. The acquisition module may determine the multiple initial images based on the uploaded images or videos. For a detailed description of the terminal collecting the initial images, please refer to step 220, which will not be repeated here.

Step 520: Replace the texture of the second initial image with the texture of the first initial image to generate a processed second initial image.

In some embodiments, the acquisition module can realize texture replacement based on a color migration algorithm. Specifically, the acquisition module can migrate the color of the illumination when the second initial image is shot to the first initial image based on the color migration algorithm to generate a processed second initial image. The color migration algorithm is a method of migrating the color of an image to another image to generate a new image. The color migration algorithm includes but is not limited to the Reinhard algorithm, the Welsh algorithm, the fuzzy clustering algorithm, the adaptive migration algorithm, etc. In some embodiments, the color migration algorithm can extract the color features of the second initial image, and then migrate the color features of the second initial image to the first initial image to generate a processed second initial image. For more information about color features, see step 230 and its related description. For a detailed description of the color migration algorithm, see Figure 6 and its related description, which will not be repeated here.

It can be understood that the purpose of replacing the texture of the second initial image with the texture of the first initial image is to make the textures of all target images consistent, but the colors of all target images remain unchanged. Therefore, in some embodiments, the acquisition module can migrate the color features of the illumination when the second initial image is captured to the first initial image based on the color migration algorithm, so that the color features of the newly generated image are still the same as the second initial image, but the texture becomes the texture of the first initial image. For example, when there are N second initial images (N is an integer greater than or equal to 1), the color features of the illumination when each of the N second initial images is captured is migrated to the first initial image, and N newly generated images can be obtained. The color features of the N newly generated images respectively represent the colors of the illumination when the N second initial images are captured, but the textures of the N newly generated images are all the textures of the first initial image.

In some embodiments, the acquisition module may also use a texture feature migration algorithm to implement texture replacement. Specifically, the texture feature migration algorithm may extract texture features of the first initial image and texture features of the second initial image, and replace the texture features of the second initial image with the texture features of the first initial image to generate a processed second initial image.

In some embodiments, the method of extracting texture features may include but is not limited to a geometric method, a gray level co-occurrence matrix method, a model method, a signal processing method, and a machine learning model, etc. Among them, the machine learning model may include but is not limited to a deep neural network model, a recurrent neural network model, a custom model structure, etc., which are not limited here.

Step 530: Use the processed second initial image as one of the multiple target images.

It can be understood that the color of illumination when the processed second initial image is captured is the same as that of the second initial image, but the texture features come from the first initial image. If the illumination colors when the first initial image and the second initial image are captured are different, the first initial image and the processed second initial image can be two images with the same content but different colors. In some embodiments, the multiple initial images include one or more second initial images. For each second initial image, the acquisition module can replace the texture of the second initial image with the texture of the first initial image to generate the corresponding processed second initial image. Optionally, the acquisition module can also use the first initial image as one of the multiple target images. At this time, the multiple target images include the first initial image and one or more processed second initial images.

As mentioned above, since the distance and angle between the image acquisition device and the target object may change, the textures of multiple initial images may be different. Therefore, some embodiments of this specification use texture unification processing to make the textures in multiple target images the same, thereby reducing the impact of the texture in the target image on the illumination color recognition and better determining the authenticity of the multiple target images.

FIG. 6 is a schematic diagram of texture replacement according to some embodiments of the present specification.

As shown in FIG6 , among the m initial images, the first initial image is collected when irradiated with white light, and the other initial images are collected when irradiated with red light, orange light, cyan light…, and blue light, respectively. The acquisition module can select the first initial image 610-1 as the first initial image, and the initial images 610-2, 610-3…, and 610-m as the second initial images. In addition to the color, the texture of each second initial image is different from that of the first initial image. For example, the position of the target object in the second initial image 610-m is different from that in the first initial image 610-1. For another example, the shooting background of the target object in the second initial image 610-2, 610-3…, and 610-m is different from that in the first initial image 610-1. The texture differences of the initial images 610-1, 610-2, 610-3…, and 610-m may result in low accuracy of the image authenticity judgment results, increased data analysis, and the like.

To solve the above problem, the second initial image can be preprocessed using a color migration algorithm. As shown in FIG6 , the acquisition module extracts the color features of the m-1 second initial images 610-2, 610-3…610-m (i.e., the color features corresponding to red, orange, cyan,…blue, respectively). The acquisition module migrates the color features of the m-1 second initial images to the first initial image 610-1, and generates m-1 processed second initial images 620-2, 620-3…, 620-m. It can be understood that the processed second initial image incorporates the texture features of the first initial image and the color features of the second initial image, which is equivalent to the image obtained by replacing the texture of the second initial image with the texture of the first initial image.

In some embodiments, the first initial image and the second initial image are RGB images. In order to avoid the influence of the correlation between the color channels in the RGB color space, the acquisition module may first convert the first initial image and the second initial image from the RGB color space to the Lαβ color space. For example, the acquisition module may convert the target image (e.g., the first initial image or the second initial image) from the RGB color space to the Lαβ color space through a neural network. For another example, the acquisition module may convert the target image from the RGB color space to the LMS color space first, and then from the LMS color space to the Lαβ color space based on multiple transition matrices.

Further, the acquisition module can extract the color features of the converted second initial image and the converted first initial image in the Lαβ color space. In some embodiments, the acquisition module can calculate the average value μ _2j and the standard deviation value σ _2j of all pixels of the converted second initial image on each channel of Lαβ. Wherein, j represents the color channel sequence number in the Lαβ color space, 0≤j≤2. When j is equal to 0, 1, and 2, it represents the brightness channel L, the yellow-blue channel α, and the red-green channel β, respectively. The acquisition module can calculate the average value μ _1j and the standard deviation value σ _1j of all pixels of the converted first initial image on each channel of Lαβ.

Furthermore, the acquisition module may transfer the color features of the converted second initial image to the converted first initial image. In some embodiments, the acquisition module may determine the scaling factor λ _j =σ 2j /σ 1j of the corresponding channel based on the standard deviation value σ _1j of the converted first initial image and the standard deviation value σ _2j of the converted second initial image in each Lαβ channel. For each pixel point of the converted first initial image, the acquisition module may subtract the average value μ _1j of the channel from the value of the pixel point _in each Lαβ channel to obtain the updated value of the pixel point in each Lαβ channel. The acquisition module may then multiply the updated value of each pixel point in each Lαβ channel by the scaling factor _{λ j} of the channel, and add the average value μ _2j of the converted second initial image in the corresponding Lαβ channel to generate the processed second initial image.

In some embodiments, the acquisition module may further convert the processed second initial image from the Lαβ color space to the RGB color space.

Some embodiments of the present specification migrate the color features of the second initial image to the first initial image based on a color migration algorithm, which not only avoids the extraction of complex texture features, but also allows the processed second initial image to contain more detailed and accurate color feature information, thereby improving the efficiency and accuracy of determining the authenticity of the target image.

FIG7 is a flow chart of determining the authenticity of a target image based on color features according to some embodiments of this specification. In some embodiments, process 700 may be performed by a verification module. As shown in FIG7 , process 700 may include the following steps:

Step 710: extract color features of multiple target images.

For more details about the color characteristics, see step 230 and its related description.

Step 720: Determine the authenticity of the multiple target images based on the color features and the illumination sequence of the multiple target images.

In some embodiments, for each of the plurality of target images, the verification module may determine the color of the illumination when the target image was captured based on the color feature of the target image, and then determine the color corresponding to the target image based on the illumination sequence. Further, the verification module may determine the authenticity of the target image.

For example, when the verification color can be obtained based on at least one reference color fusion, the verification module can form a new color space (i.e., the reference color space in FIG. 9 ) based on the reference color features of at least one reference image. Further, the verification module can determine the color of the illumination when the verification image is photographed based on the new color space and the verification color features of the verification image. Furthermore, the verification module can determine the authenticity of the verification image in combination with the color corresponding to the verification image. For determining the authenticity of the target image based on the reference color space, please refer to FIG. 9 , FIG. 11 and related descriptions thereof, which will not be repeated here.

In some embodiments, the verification module can determine the color relationship between the multiple target images based on the color features of the multiple target images, and then determine the authenticity of the multiple target images based on the color relationship between the multiple target images and the color relationship between the multiple colors of the multiple illuminations in the illumination sequence. For determining the authenticity of the target image based on the color relationship, please refer to Figure 12 and its related description, which will not be repeated here.

In some embodiments, for each of the plurality of target images, the verification module may determine the degree of match between the color feature of the target image and the color feature of the corresponding illumination based on the color feature of the target image and the color feature of the corresponding illumination. Further, the verification module may determine the authenticity of the target image based on the degree of match. For example, if the degree of match between the color feature of the target image and the color feature of the corresponding illumination is greater than a preset threshold, the target image is authentic. The degree of match may be determined based on the similarity between the color feature of the target image and the color feature of the illumination. The similarity may be measured by Euclidean distance, Manhattan distance, etc. In some embodiments, the verification module may determine the degree of match between the first feature information and the second feature information based on the color feature of the first image sequence constructed by the plurality of target images (i.e., the first feature information in FIG. 14 ) and the color feature of the second image sequence constructed by the plurality of color template images (i.e., the second feature information in FIG. 14 ). Further, the verification module may determine the authenticity of the plurality of target images based on the degree of match. For more details on determining the authenticity of the plurality of target images based on the sequence, see FIG. 14 and its related description.

In some embodiments, the preset threshold set for image authenticity judgment in some embodiments of this specification may be related to the shooting stability. The shooting stability is the stability of the image acquisition device of the terminal when acquiring the target image. In some embodiments, the preset threshold is positively correlated with the shooting stability. It can be understood that the higher the shooting stability, the higher the quality of the acquired target image, and the more the color features extracted based on multiple target images can truly reflect the color of the light when being photographed, the larger the preset threshold. In some embodiments, the shooting stability can be measured based on the motion parameters of the terminal detected by the motion sensor of the terminal (for example, a vehicle-mounted terminal or a user terminal, etc.). For example, the motion speed, vibration frequency, etc. detected by the motion sensor. For example, the larger the motion parameter, or the larger the rate of change of the motion parameter, the lower the shooting stability. The motion sensor may be a sensor for detecting the driving condition of the vehicle, and the vehicle may be a vehicle used by the target user. The target user refers to the user to whom the target object belongs. For example, if the target user is a driver of an online car-hailing service, the motion sensor may be a motion sensor on the driver's side or on the vehicle terminal.

In some embodiments, the preset threshold value may also be related to the shooting distance and the shooting angle. The shooting distance is the distance between the image acquisition device and the target object when acquiring the target image. The shooting angle is the angle between the front of the target object and the terminal screen when the image acquisition device acquires the target image. In some embodiments, the shooting distance and the shooting angle are both negatively correlated with the preset threshold value. It can be understood that the shorter the shooting distance, the higher the quality of the acquired target image, and the more the color features extracted based on multiple target images can truly reflect the color of the illumination when being photographed, the larger the preset threshold value. The smaller the shooting angle, the higher the quality of the acquired target image, and similarly, the larger the preset threshold value. In some embodiments, the shooting distance and the shooting angle can be determined based on the target image by image recognition technology.

In some embodiments, the verification module may perform specific operations (e.g., averaging, standard deviation, etc.) on the shooting stability, shooting distance, and shooting angle of each target image, and determine a preset threshold based on the shooting stability, shooting distance, and shooting angle after the specific operations. For example, the verification module determines the corresponding sub-thresholds based on the shooting stability, shooting distance, and shooting angle after the specific operations, and then determines the preset threshold based on the sub-threshold corresponding to the shooting stability, the sub-threshold corresponding to the shooting distance, and the sub-threshold corresponding to the shooting angle. For example, the three sub-thresholds may be averaged, weighted averaged, etc.

FIG8 is a flow chart of determining the authenticity of a target image based on a color verification model according to some embodiments of this specification. In some embodiments, process 800 may be performed by a verification module. As shown in FIG8 , process 800 may include the following steps:

Step 810, obtaining a color verification model.

The color verification model is a model used to verify whether an image is authentic. The color verification model is a machine learning model with preset parameters. The preset parameters refer to the model parameters learned during the training process of the machine learning model. Taking a neural network as an example, the model parameters include weights and biases, etc. The preset parameters of the color verification model are determined during the training process. For example, the model acquisition module can train an initial color verification model based on multiple labeled training samples to obtain a color verification model.

In some embodiments, the color verification model can be stored in a storage device, and the verification module can obtain the color verification model from the storage device through a network. In some embodiments, the color verification model can be obtained through a training process. For the training process of the color verification model, please refer to Figures 10, 13 and 15 and their related descriptions.

Step 820 , based on the illumination sequence, the multiple target images are processed using a color verification model to determine the authenticity of the multiple target images.

In some embodiments, the color verification model may include a first verification model. The first verification model may include a first color feature extraction layer and a color classification layer. The first color feature extraction layer extracts the color features of the target image. The color classification layer determines the color corresponding to the target image based on the color features of the target image. For determining the authenticity of the target image based on the first verification model, see FIG. 10 and its related description.

In some embodiments, the color verification model may include a second verification model. The second verification model may include a second color feature extraction layer and a color relationship determination layer. The second color feature extraction layer extracts the color features of the target image. The color relationship determination layer determines the relationship between the colors corresponding to different target images (e.g., whether they are the same) based on the color features of the target image. For a detailed description of determining the authenticity of the target image based on the second verification model, see FIG. 13 and its related description.

In some embodiments, the color verification model may include a third verification model. The third verification model may include a first extraction layer, a second extraction layer, and a discrimination layer. The first extraction layer extracts color features of a sequence constructed by multiple target images. The second extraction layer extracts color features of a sequence constructed by multiple color template images. The discrimination layer determines the relationship between the two sequences based on the color features of the two sequences. For determining the authenticity of multiple target images based on the third verification model, see FIG. 14 and its related description.

FIG9 is an exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of the present specification. In some embodiments, flow chart 900 may be executed by a verification module. As shown in FIG9 , the process 900 may include the following steps:

In some embodiments, the multiple colors of illumination in the illumination sequence include at least one reference color and at least one verification color. Each of the at least one verification color can be determined based on at least a portion of the color of the at least one reference color. For example, each of the at least one verification color can be generated based on the fusion of one or more reference colors. The multiple images include at least one reference image and at least one verification image. Each of the at least one verification image corresponds to one of the at least one verification color. Each of the multiple reference images corresponds to one of the at least one reference color. As described in step 220, the correspondence between the target image and a specific color indicates that if the terminal is not hijacked (i.e., the target image is authentic), the target image should have the specific color.

Step 910: extracting a reference color feature of at least one reference image and a verification color feature of at least one verification image.

The reference color feature refers to the color feature of the reference image. The verification color feature refers to the color feature of the verification image. For color features and their extraction, please refer to the description of step 230.

In some embodiments, the verification module can extract color features of the image based on the first color feature extraction layer included in the first verification model. For details on extracting color features based on the first color feature extraction layer, please refer to FIG. 10 and its related description, which will not be repeated here.

Step 920 , for each of the at least one verification image, based on the verification color feature of the verification image and the reference color feature of the at least one reference image, determine the color of the illumination when the verification image was captured.

In some embodiments, the reference color features of at least one reference image can be used to construct a reference color space. The reference color space uses the at least one reference color as its color channel. Specifically, the reference color features corresponding to each reference image can be used as the reference value of the corresponding color channel in the reference color space.

In some embodiments, the color space (also referred to as the original color space) corresponding to the multiple target images may be the same as or different from the reference color space. For example, the multiple target images may correspond to an RGB color space, and the at least one reference color is red, blue, and green. Then, the original color space corresponding to the multiple target images and the reference color space constructed based on the reference color belong to the same color space. In this document, if the base colors or primary colors of two color spaces are the same, then the two color spaces may be regarded as the same color space.

As described above, the verification color can be obtained by fusion based on one or more reference colors. Therefore, the verification module can determine the color corresponding to the verification color feature based on the reference color feature and/or the reference color space constructed by it. In some embodiments, the verification module can map the verification color feature of the verification image based on the reference color space to determine the color of the illumination when the verification image is photographed. For example, the verification module can determine the parameters of the verification color feature on each color channel based on the relationship between the verification color feature and the reference value of each color channel in the reference color space, and then determine the color corresponding to the verification color feature based on the parameters, that is, the color of the illumination when the verification image is photographed.

For example, the verification module can extract the reference color features based on the reference images a, b, and c. As the reference values of color channel I, color channel II and color channel III respectively. Color channel I, color channel II and color channel III are three color channels of the reference color space. The verification module can extract the verification color feature based on the verification image d. And based on the verification color characteristics And the reference values of color channel I, color channel II and color channel III The relationship between Determine verification color characteristics Parameters δ ₁ , δ ₂ and δ ₃ on color channel I, color channel II and color channel III respectively. The verification module can determine the color corresponding to the verification color feature based on the parameters δ ₁ , δ ₂ and δ ₃ , that is, the color of the illumination when the verification image is taken. In some embodiments, the correspondence between the parameters and the color category can be preset or learned through a model.

In some embodiments, the reference color space may be the same as the color of the color channel of the original color space. For example, the original space color may be an RGB space, and the at least one reference color may be red, green, and blue. The verification module may construct a new RGB color space (i.e., reference color space) based on the reference color features of the three reference images corresponding to red, green, and blue, and determine the RGB value of the verification color feature of each verification image in the new RGB color space, thereby determining the color of illumination when the verification image is captured.

In some embodiments, the verification module can process the reference color features and the verification color features based on the color classification layer in the first verification model to determine the color of the illumination when the verification image was taken. For details, please refer to Figure 10 and its related description, which will not be repeated here.

Step 930 , determining the authenticity of the plurality of target images based on the illumination sequence and the color of the illumination when at least one verification image is captured.

In some embodiments, for each of at least one verification image, the verification module may determine the verification color corresponding to the verification image based on the illumination sequence. Further, the verification module may determine the authenticity of the verification image based on the verification color corresponding to the verification image. For example, the verification module determines the authenticity of the verification image based on a first judgment result of whether the verification color corresponding to the verification image is consistent with the color of the illumination when photographed. If the verification color corresponding to the verification image is the same as the color of the illumination when photographed, it indicates that the verification image is authentic, and if the verification color corresponding to the verification image is different from the color of the illumination when photographed, it indicates that the verification image is not authentic. For another example, the verification module determines the authenticity of the verification image based on whether the relationship between the verification colors corresponding to multiple verification images (for example, whether they are the same) is consistent with the relationship between the colors of the illumination when the multiple verification images are photographed.

In some embodiments, the verification module can determine whether the image acquisition device of the terminal is hijacked based on the authenticity of at least one verification image. For example, if the number of verification images with authenticity exceeds a first threshold, it indicates that the image acquisition device of the terminal is not hijacked. For another example, if the number of verification images without authenticity exceeds a second threshold (e.g., 1), it indicates that the image acquisition device of the terminal is hijacked.

In some embodiments, the verification module can combine the reference color space with other verification methods to determine the authenticity of multiple target images. In some embodiments, the verification module can determine the updated color features of each target image in the verification image and the reference image based on the reference color space. The updated color features refer to the features after the original color features are converted to the reference color space. Further, the verification module can replace the original color features based on the updated color features of each target image, and determine the authenticity of multiple target images in combination with other verification methods. For example, the verification module can determine the first color relationship between the multiple target images based on the updated color features of the multiple target images, and determine the authenticity of the multiple target images based on the first color relationship. For another example, the verification module determines the color features of the first image sequence constructed by the multiple target images based on the updated color features of the multiple target images, and determines the authenticity of the multiple target images based on the color features of the first image sequence. For related descriptions of other verification methods, please refer to other places in this specification, such as Figures 12 to 15 and their related descriptions.

Since both the reference image and the verification image are taken under the same external ambient light conditions, establishing a reference color space based on the reference image, and determining the color of the illumination when the verification image is taken based on the reference color space can make the determination result more accurate. Further, the authenticity of the target image is also determined more accurately. For example, when the illumination in the illumination sequence is weaker than the ambient light, the illumination irradiated to the target object may be difficult to detect. Or, when the ambient light is colored light, the illumination irradiated to the target object may be disturbed. When the terminal is not hijacked, the reference image and the verification image are taken under the same (or substantially the same) ambient light. The reference color space constructed based on the reference image incorporates the influence of ambient light, so compared with the original color space, the color of the illumination when the verification image is taken can be more accurately identified. In addition, the method disclosed in this article can avoid interference from the light-emitting element of the terminal. When the terminal is not hijacked, the reference image and the verification image are both taken under the illumination of the same light-emitting element. The reference color space can eliminate or weaken the influence of the light-emitting element and improve the accuracy of identifying the color of the illumination.

FIG. 10 is a schematic diagram of a first verification model according to some embodiments of the present specification.

In some embodiments, the verification module may determine the authenticity of multiple target images based on the first verification model and the illumination sequence. The first verification model may include a first color feature extraction layer and a color classification layer. The first color feature extraction layer may include a reference color feature extraction layer and a verification color feature extraction layer. As shown in FIG10 , the first verification model may include a reference color feature extraction layer 1030, a verification color feature extraction layer 1040, and a color classification layer 1070. The reference color feature extraction layer 1030 and the verification color feature extraction layer 1040 may be used to implement step 910. The color classification layer 1070 may be used to implement step 920. Further, the verification module determines the authenticity of the verification image based on the color and illumination sequence corresponding to the verification image.

The color feature extraction model (e.g., the first color feature extraction layer, the reference color feature extraction layer 1030, and the verification color feature extraction layer 1040, etc.) can extract the color features of the target image. In some embodiments, the type of the color feature extraction model can include a convolutional neural network model such as ResNet, DenseNet, MobileNet, ShuffleNet, or EfficientNet, or a recurrent neural network model such as a long short-term memory recurrent neural network. In some embodiments, the types of the reference color feature extraction layer 1030 and the verification color feature extraction layer 1040 can be the same or different.

The reference color feature extraction layer 1030 extracts reference color features 1050 of at least one reference image 1010. In some embodiments, the at least one reference image 1010 may include multiple reference images. The reference color feature 1050 may be a fusion of color features of the multiple reference images 1010. For example, the multiple reference images 1010 may be spliced and input into the reference color feature extraction layer 1030 after splicing, and the reference color feature extraction layer 1030 may output the reference color feature 1050. For example, the reference color feature 1050 is a feature vector spliced from the color feature vectors of the reference images 1010-1, 1010-2, and 1010-3.

The verification color feature extraction layer 1040 extracts the verification color feature 1060 of at least one verification image 1020. In some embodiments, the verification module may perform color judgment on each of the at least one verification image 1020. For example, as shown in FIG8 , the verification module may input at least one reference image 1010 into the reference color feature extraction layer 1030 and input the verification image 1020-2 into the verification color feature extraction layer 1040. The verification color feature extraction layer 1040 may output the verification color feature 1060 of the verification image 1020-2. The color classification layer 1070 may determine the color of the illumination when the verification image 1020-2 is photographed based on the reference color feature 1050 and the verification color feature 1060 of the verification image 1020-2.

In some embodiments, the verification module can simultaneously perform color judgment on multiple verification images 1020. For example, the verification module can input at least one reference image 1010 into the reference color feature extraction layer 1030, and input multiple verification images 1020 (including verification images 1020-1, 1020-2...1020-n) into the verification color feature extraction layer 1040. The verification color feature extraction layer 1040 can simultaneously output the verification color features 1060 of the multiple verification images 1020. The color classification layer 1070 can simultaneously determine the color of illumination when each verification image in the multiple verification images is captured.

For each of the at least one verification image, the color classification layer 1070 may determine the color of the illumination when the verification image was taken based on the reference color feature and the verification color feature of the verification image. For example, the color classification layer 1070 may determine a numerical value or probability based on the reference color feature and the verification color feature of the verification image, and then determine the color of the illumination when the verification image was taken based on the numerical value or probability. The numerical value or probability corresponding to the verification image may reflect the possibility that the color of the illumination when the verification image was taken belongs to each color. In some embodiments, the color classification layer 1070 may include but is not limited to a fully connected layer, a deep neural network, etc.

The first verification model is a machine learning model with preset parameters. It can be understood that the reference color feature extraction layer, the verification color feature extraction layer and the color classification layer included in the first verification model are machine learning models with preset parameters. The preset parameters of the first verification model can be determined in the model training process. For example, the acquisition module can train the initial first verification model based on the first training sample with a first label to obtain the preset parameters of the first verification model. The first training sample includes at least one sample reference image and at least one sample verification image of the sample target object, and the first label of the first training sample is the color of illumination when each sample verification image is taken. Among them, the color of illumination when at least one sample reference image is taken is the same as at least one reference color. For example, if the at least one reference color includes red, green, and blue, then the at least sample reference image includes three target images of the sample target object taken under red light, green light, and blue light.

In some embodiments, the acquisition module can input the first training sample into the initial first verification model, and update the parameters of the initial verification color feature extraction layer, the initial reference color feature extraction layer, and the initial color classification layer through training until the updated first verification model meets the first preset condition. The updated first verification model can be designated as the first verification model of preset parameters, in other words, the updated first verification model can be designated as the trained first verification model. The first preset condition can be that the loss function of the updated color feature model is less than a threshold, converges, or the number of training iterations reaches a threshold.

In some embodiments, the acquisition module can train the initial verification color feature extraction layer, the initial reference color feature extraction layer, and the initial color classification layer in the initial first verification model through an end-to-end training method. The end-to-end training method refers to inputting the training samples into the initial model, determining the loss value based on the output of the initial model, and updating the initial model based on the loss value. The initial model may include multiple sub-models or modules for performing different data processing operations, which will be regarded as a whole during training and updated simultaneously. For example, in the training of the initial first verification model, at least one sample reference image can be input into the initial reference color feature extraction layer, and at least one sample verification image can be input into the initial verification color feature extraction layer. A loss function is established based on the output result of the initial color classification layer and the first label, and the parameters of each initial model in the initial first verification model are updated simultaneously based on the loss function.

In some embodiments, the first verification model may be pre-trained by the processing device or a third party and then stored in a storage device, and the processing device may directly call the first verification model from the storage device.

In some embodiments of this specification, the authenticity of the verification image is determined by the first verification model, which can improve the efficiency of the authenticity verification of the target image. In addition, the use of the first verification model can improve the reliability of the authenticity verification of the target object, reduce or remove the influence of the performance difference of the terminal device, and further determine the authenticity of the target image. It can be understood that there are certain differences in the hardware of different terminals. For example, the same color color light emitted by the terminal screens of different manufacturers may have differences in parameters such as saturation and brightness, resulting in a large intra-class gap of the same color. The first training sample of the initial first verification model can be taken by terminals with different performances. The initial first verification model can be learned during the training process, so that the first verification model can consider the terminal performance difference when judging the color of the target object, and more accurately determine the color of the target image. Moreover, when the terminal is not hijacked, since the reference image and the verification image are taken under the same external ambient light conditions. In some embodiments, based on the reference color feature extraction layer in the first verification model, a reference color space is established, and when the authenticity of multiple target images is determined based on the reference color space, the influence of the external ambient light can be eliminated or weakened.

FIG11 is another exemplary flow chart of determining the authenticity of multiple target images according to some embodiments of this specification. In some embodiments, flow chart 1100 may be executed by a verification module. As shown in FIG11 , the flow chart 1100 includes the following steps:

Step 1110 , extracting verification color features of at least one verification image.

For a detailed description of extracting verification color features, please refer to step 910 and its related descriptions.

Step 1120: extracting reference color features of at least one reference image.

For a detailed description of extracting the reference color features, please refer to step 910 and its related descriptions.

Step 1130 : For each of the at least one verification image, based on the illumination sequence and the reference color feature, generate a target color feature of the verification color corresponding to the verification image.

The target color feature refers to the feature of the verification color corresponding to the verification image represented in the reference color space. In some embodiments, for each of the at least one verification image, the verification module may determine the verification color corresponding to the verification image based on the illumination sequence, and generate the target color feature of the verification image based on the verification color and the reference color feature. For example, the verification module may merge the color feature of the verification color with the reference color feature to obtain the target color feature.

Step 1140 , determining the authenticity of the plurality of target images based on the target color feature and the verification color feature corresponding to each of the at least one verification image.

In some embodiments, for each of the at least one verification image, the verification module may determine the authenticity of the verification image based on the similarity between the corresponding target color feature and the verification color feature. The similarity between the target color feature and the verification color feature may be calculated by vector similarity, for example, by Euclidean distance, Manhattan distance, etc. Exemplarily, when the similarity between the target color feature and the verification color feature is greater than a third threshold, the verification image is authentic, otherwise it is not authentic.

FIG12 is an exemplary flow chart of a method for determining the authenticity of multiple target images according to some embodiments of this specification. In some embodiments, process 1200 may be performed by a verification module. As shown in FIG12 , process 1200 includes the following steps:

As described above, the multiple colors corresponding to the multiple illuminations in the illumination sequence include at least one reference color and at least one verification color. In some embodiments, one or more of the at least one reference color is the same as one or more of the at least one verification color. The multiple target images include at least one reference image and at least one verification image, each of the at least one reference image corresponds to one of the at least one reference color, and each of the at least one verification image corresponds to one of the at least one verification color.

Step 1210 , extracting a reference color feature of each of at least one reference image and a verification color feature of each of at least one verification image.

For extracting the reference color features and verifying the color features, please refer to step 910 and its related description, which will not be repeated here.

In some embodiments, the verification module can extract the reference color feature and the verification color feature based on the second color feature extraction layer included in the second verification model. For details on extracting color features based on the second color feature extraction layer, please refer to FIG. 13 and its related description, which will not be repeated here.

Step 1220 : For each of the at least one reference image, determine a first color relationship between the reference image and each verification image based on a reference color feature of the reference image and a verification color feature of each verification image.

The first color relationship between the reference image and the verification image refers to the relationship between the color of the illumination when the reference image is captured and the color of the illumination when the verification image is captured. The first color relationship includes the same, different, or similar. In some embodiments, the first color relationship can be represented by a numerical value. For example, the same is represented by "1" and the different is represented by "0".

In some embodiments, at least one first color relationship determined based on at least one reference image and at least one verification image can be represented by a vector, and each element in the vector can represent a first color relationship between one of the at least one reference image and one of the at least one verification image. For example, if the first color relationship between one reference image and each of five verification images is the same, different, the same, the same, and different, respectively, then the first color relationship between one reference image and five verification images can be represented by a vector (1,0,1,1,0).

In some embodiments, at least one first color relationship determined based on at least one reference image and at least one verification image can also be represented by a verification code. The subcode at each position in the verification code can represent the first color relationship between one of the at least one reference image and one of the at least one verification image. For example, the first color relationship between the above-mentioned 1 reference image and 5 verification images can be represented by verification code 10110.

In some embodiments, the verification module may determine the first color relationship between the reference color feature of the reference image and the verification color feature of the verification image. For example, the verification module may determine the similarity between the reference color feature of the reference image and the verification color feature of the verification image, and determine at least one first color relationship based on the similarity and a threshold. For example, if the similarity is greater than the fourth threshold, it is judged to be the same, if it is less than the fifth threshold, it is judged to be different, or if it is greater than the sixth threshold and less than the fourth threshold, it is judged to be similar, etc. Among them, the fourth threshold may be greater than the fifth threshold and the sixth threshold, and the sixth threshold may be greater than the fifth threshold. In some embodiments, the similarity can be characterized by the distance between the reference color feature and the verification color feature. The distance may include, but is not limited to, Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, Mahalanobis distance, angle cosine distance, etc.

In some embodiments, the verification module may also obtain the first color relationship based on the color relationship determination layer included in the second verification model. For a detailed description of the color relationship determination layer, please refer to FIG. 13 and its related description, which will not be repeated here.

Step 1230: for each of the at least one reference color, determine a second color relationship between the reference color and each of the verification colors.

The second color relationship between the reference color and the verification color can indicate whether the two colors are the same, different, or similar. In some embodiments, the type and representation of the second color relationship can be similar to the first color relationship, which will not be described in detail here.

In some embodiments, the verification module can determine the second color relationship based on the reference color and the verification color category or color parameter. For example, if the reference color and the verification color have the same category or the numerical difference in the color parameter is less than a certain threshold, the two colors are judged to be the same, otherwise the two colors are judged to be different.

In some embodiments, the verification module may extract a first color feature of the color template image of the reference color and a second color feature of the color template image of the verification color. The verification module may further determine a second color relationship between the reference color and the verification color based on the first color feature and the second color feature. For example, the verification module may calculate the similarity between the first color feature and the second color feature to determine the second color relationship.

In some embodiments, at least one first color relationship and at least one second color relationship have a one-to-one correspondence. Specifically, the first color relationship between the reference image and the verification image corresponds to the second color relationship between the reference color corresponding to the reference image and the verification color corresponding to the verification image.

Step 1240: Determine authenticity of the plurality of target images based on at least one first color relationship and at least one second color relationship.

In some embodiments, the verification module may determine the authenticity of the plurality of target images based on part or all of at least one first color relationship and a corresponding second color relationship.

In some embodiments, the first color relationship and the second color relationship can be represented by a vector. In some embodiments, the verification module can select part or all of at least one first color relationship to construct a first vector, and construct a second vector based on the second color relationship corresponding to the selected first color relationship. Further, the verification module can determine the authenticity of multiple target images based on the similarity between the first vector and the second vector. For example, if the similarity is greater than a seventh threshold, the multiple target images are authentic. It can be understood that the arrangement order of the elements in the first vector and the second vector is determined based on the correspondence between the first color relationship and the second color relationship. For example, the element corresponding to a first color relationship in the first vector A is A _ij , and the element corresponding to the second color relationship corresponding to the first color relationship in the second vector B is B _ij .

In some embodiments, the first color relationship and the second color relationship can also be represented by a verification code. In some embodiments, the verification module can select part or all of at least one first color relationship to construct a corresponding first verification code, construct a corresponding second verification code based on the second color relationship corresponding to the selected first color relationship, and determine the authenticity of multiple target images. Similar to the first vector and the second vector, the position of the subcode in the first verification code and the second verification code is determined based on the corresponding relationship between the first color relationship and the second color relationship. For example, if the first verification code and the second verification code are different, the multiple target images are not authentic. For example, the first verification code is 10110 and the second verification code is 10111, then the multiple target images are not authentic. For another example, the verification module can determine the authenticity of multiple target images based on the number of identical subcodes in the first verification code and the second verification code. For example, if the number of identical subcodes is greater than the eighth threshold, the authenticity of the multiple target images is determined, and if the number of identical subcodes is less than the ninth threshold, it is determined that the multiple target images are not authentic. For example, the eighth threshold is 3, the ninth threshold is 1, the first verification code is 10110, the second verification code is 10111, and the first, second, third and fourth subcodes of the first verification code and the second verification code correspond to the same, then the multiple target images are determined to be authentic.

In some embodiments, the verification module may determine the color of the illumination when the verification image and the reference image were taken based on the reference color space, further determine the first color relationship, and then determine the authenticity of the multiple target images in combination with the corresponding second color relationship. In some embodiments, the verification module may determine the updated verification color features of the verification image and the updated reference color features of the reference image based on the reference color space. Further, the verification module determines the first color relationship based on the updated verification color features and the updated reference color features, and then determines the authenticity of the multiple target images in combination with the corresponding second color relationship.

As mentioned above, both the reference image and the verification image are taken under the same external ambient light conditions and illuminated by the same light-emitting element. Therefore, when determining the authenticity of multiple target images based on the relationship between the reference image and the verification image, the influence of the external ambient light and the light-emitting element can be eliminated or weakened, thereby improving the recognition accuracy of the lighting color.

FIG. 13 is a schematic diagram of a second verification model according to some embodiments of the present specification.

In some embodiments, the verification module can determine the authenticity of multiple target images based on the second verification model and the illumination sequence. As shown in FIG13, the second verification model can include a second color feature extraction layer 1330 and a color relationship determination layer 1360. The second color feature extraction layer 1330 can be used to implement step 1210, and the color relationship determination layer 1360 can be used to implement step 1220. Further, the verification module can determine the authenticity of multiple target images based on the first color relationship and the illumination sequence.

In some embodiments, at least one reference image and at least one verification image may constitute one or more image pairs. Each image pair includes at least one of the reference images and at least one of the verification images. The verification module may analyze the one or more image pairs separately to determine the first color relationship between the reference image and the verification image in the image pair. For example, as shown in FIG. 13 , the at least one reference image includes “1320-1…1320-y”, and the at least one verification image includes “1310-1…1310-x”. For illustrative purposes, the following is an example of an image pair consisting of the reference image 1320-y and the verification image 1310-1.

The second color feature extraction layer 1330 can extract the reference color feature 1350-y of the reference image 1320-y and the verification color feature 1340-1 of the verification image 1310-1. In some embodiments, the type of the second color feature extraction layer 1330 can include a convolutional neural network (CNN) model such as ResNet, ResNeXt, SE-Net, DenseNet, MobileNet, ShuffleNet, RegNet, EfficientNet or Inception, or a recurrent neural network model.

The input of the second color feature extraction layer 1330 may be an image pair (e.g., a reference image 1320-y and a verification image 1310-1). For example, the reference image 1320-y and the verification image 1310-1 may be spliced and input to the second color feature extraction layer 1330. The output of the second color feature extraction layer 1330 may be the color features of the image pair (e.g., a reference color feature 1350-y of the reference image 1320-y and a verification color feature 1340-1 of the verification image 1310-1). For example, the output of the second color feature extraction model 1330 may be the color features of the spliced verification color feature 1340-1 of the verification image 1310-1 and the reference color feature 1350-y of the reference image 1320-y.

The color relationship determination layer 1360 is used to determine the first color relationship of the image pair based on the color features of the image pair. For example, the verification module can input the reference color feature 1350-y of the reference image 1320-y and the verification color feature 1340-1 of the verification image 1310-1 into the color relationship determination layer 1360, and the color relationship determination layer 1360 outputs the first color relationship between the reference image 1320-y and the verification image 1310-1.

In some embodiments, the verification module may input a plurality of image pairs consisting of at least one reference image and at least one verification image into a second verification model. The second verification model may simultaneously output a first color relationship for each of the plurality of image pairs. In some embodiments, the verification module may input a certain pair of the plurality of image pairs into the second verification model. The second verification model may output the first color relationship for the image pair.

In some embodiments, the color relationship determination layer 1360 may be a classification model, including but not limited to a fully connected layer, a deep neural network, a decision tree, and the like.

The second verification model is a machine learning model with preset parameters. It can be understood that the second color feature extraction layer and the color relationship determination layer included in the second verification model are machine learning models with preset parameters. The preset parameters of the second verification model can be determined during the training process. For example, the acquisition module can train the initial second verification model based on the second training sample with a second label to obtain the second verification model. The second training sample includes one or more sample image pairs with a second label. Each sample image pair includes two target images of the sample target object taken under the same or different lighting. The second label of the second training sample can indicate whether the color of the illumination when the sample image pair was taken is the same.

In some embodiments, the acquisition module may input the second training sample into the initial second verification model, and update the parameters of the initial second color feature extraction layer and the initial color relationship determination layer through training until the updated second verification model meets the second preset condition. The updated second verification model may be designated as the second verification model of preset parameters, in other words, the updated second verification model may be designated as the trained second verification model. The second preset condition may be that the loss function of the updated second verification model is less than a threshold, converges, or the number of training iterations reaches a threshold.

In some embodiments, the acquisition module can train the initial second color feature extraction layer and the initial color relationship determination layer in the initial second verification model through an end-to-end training method. The end-to-end training method refers to inputting the training sample into the initial model, determining the loss value based on the output of the initial model, and updating the initial model based on the loss value. The initial model may include multiple sub-models or modules for performing different data processing operations, which will be regarded as a whole in training and updated simultaneously. For example, in the training of the initial second verification model, at least one sample reference image and at least one verification image can be input into the initial color feature extraction model, and a loss function is established based on the output result of the initial color relationship determination layer and the second label, and the parameters of each initial model in the initial second verification model are updated simultaneously based on the loss function.

In some embodiments, the second verification model may be pre-trained by the processing device or a third party and then stored in a storage device, and the processing device may directly call the second verification model from the storage device.

Based on the first color relationship and the second color relationship, the authenticity of multiple target images can be determined without identifying the type of illumination when the target image is photographed. It can be directly determined by comparing the color features to determine whether the type of illumination when the target image is photographed is consistent. This is equivalent to converting the color recognition task into a binary classification task for determining whether the colors are the same. In some embodiments, a second verification model can be used to determine the first color relationship. The color relationship determination layer of the second verification model can only include a small number of neurons (for example, two neurons) to determine whether the colors are the same. Compared with the color recognition network in the traditional method, the second verification model disclosed in this specification has a simpler structure. The computing resources (such as computing space) required for the target object analysis based on the second verification model are also relatively less, thereby improving the recognition efficiency of the illumination color. At the same time, the input of the model can be a target image corresponding to any color. Compared with other algorithms that require a limited number of input color types, the applicability of the embodiments of this specification is higher. Moreover, the use of the second verification model can improve the reliability of the target object authenticity verification, reduce or remove the impact of the performance differences of the terminal equipment, and further determine the authenticity of the target image. It is understandable that there are certain differences in the hardware of different terminals. For example, the same color light emitted by the terminal screens of different manufacturers may have differences in parameters such as saturation and brightness, resulting in a relatively large intra-class gap for the same color. The second training sample of the initial second verification model can be taken by terminals with different performance. Through learning during the training process, the initial second verification model can enable the second verification model to consider the terminal performance differences when judging the color of the target object and determine the color of the target image more accurately. In addition, when the terminal is not hijacked, the reference image and the verification image are taken under the same external ambient light conditions. Therefore, when the reference image and the verification image are processed based on the second verification model to determine the authenticity of multiple target images, the influence of the external ambient light can be eliminated or weakened.

FIG14 is another exemplary flow chart of a method for determining the authenticity of multiple target images according to some embodiments of this specification. In some embodiments, process 1400 may be performed by a verification module. As shown in FIG14 , process 1400 includes the following steps:

Step 1410: determine a first image sequence based on a plurality of target images.

The first image sequence is a collection of multiple target images arranged in a specific order. In some embodiments, the verification module can sort the multiple target images according to their respective shooting times to generate the first image sequence. For example, the multiple target images can be sorted from first to last according to their respective shooting times.

Step 1420: Determine a second image sequence based on the multiple color template images.

A color template image is a template image generated based on the color of the light in the light sequence. A color template image of a certain color refers to a pure color image that only contains that color. For example, a red color template image only contains red, no other colors other than red, and no texture.

In some embodiments, the verification module may generate the multiple color template images based on the illumination sequence. For example, the verification module may generate a color template image corresponding to the color of each illumination in the illumination sequence according to the color type and/or color parameter of the illumination. In some embodiments, the color template image of each color in the illumination sequence may be pre-stored in a storage device, and the verification module may obtain the color template image corresponding to the color of the illumination in the illumination sequence from the storage device through a network.

The second image sequence is a collection of multiple color template images arranged in order. In some embodiments, the verification module can sort the multiple color template images according to the illumination time of their corresponding illumination to generate the second image sequence. For example, the multiple color template images can be sorted from first to last according to the illumination time of their corresponding illumination. In some embodiments, the arrangement order of the multiple color template images in the second image sequence is consistent with the arrangement order of the multiple target images in the first image sequence. The illumination time of the illumination corresponding to the multiple color template images in the second image sequence corresponds to the shooting time of the multiple target images in the first image sequence. For example, if the multiple target images are arranged from first to last according to their shooting time, the multiple color template images are also arranged from first to last based on the illumination time of their corresponding illumination.

Step 1430: extract first feature information of the first image sequence.

The first feature information may include color features of multiple target images in the first image sequence. For more details about extracting color features, see step 230 and its related description. In some embodiments, the verification module may extract the first feature information of the first image sequence based on the first extraction layer in the third verification model. For extracting the first feature information based on the first extraction layer, see Figure 15 and its related description.

Step 1440: extract second feature information of the second image sequence.

The second feature information may include color features of a plurality of color template images in the second image sequence. For more details on extracting the color features, see step 230 and its related description. In some embodiments, the verification module may extract the second feature information based on a second extraction layer in the second color verification model. For more details on extracting the second color features based on the second extraction layer, see FIG. 15 and its related description.

Step 1450: Determine the authenticity of the multiple target images based on the first feature information and the second feature information.

In some embodiments, the verification module can determine a second judgment result of whether the color sequence of illumination when multiple target images in the first image sequence are captured is consistent with the color sequence of multiple template color images in the second image sequence based on the matching degree of the first feature information and the second feature information. For example, the verification module can use the similarity of the first feature information and the second feature information as the matching degree, and then determine the second judgment result based on the relationship between the similarity of the first feature information and the second feature information and the threshold. For example, if the similarity of the first feature information and the second feature information is greater than the tenth threshold, the second judgment result is consistent. If the similarity of the first feature information and the second feature information is less than the eleventh threshold, the second judgment result is inconsistent. Further, the verification module can determine the authenticity of the multiple target images based on the second judgment result. For example, if the second judgment result is consistent, the multiple target images are authentic.

In some embodiments, the verification module may determine the second judgment result based on the discrimination layer in the third color verification model. For more details on determining the second judgment result based on the discrimination layer, see FIG. 15 and its related description.

Some embodiments of the present specification generate a second image sequence based on an artificially constructed color template image, and determine the authenticity of multiple target images by comparing the second image sequence with the first image sequence (a sequence of multiple target images). Compared with directly identifying the color of the first image sequence, the method disclosed in the present specification can make the target image recognition task simpler. In some embodiments, a third verification model can be used to analyze the authenticity of the target image. Using the second image sequence can make the recognition task of the third verification model simpler and less difficult to learn, thereby making the recognition accuracy higher. In addition, the multiple target images in the first image sequence are all taken under the same external ambient light conditions and illuminated by the same light-emitting element. Therefore, when determining the authenticity of the multiple target images based on the relationship between the reference image and the verification image, the influence of the external ambient light and the light-emitting element can be eliminated or weakened, thereby improving the recognition accuracy of the illumination color.

FIG. 15 is a diagram showing an example structure of a third verification model according to some embodiments of the present specification.

In some embodiments, the verification module can determine the authenticity of multiple target images based on the third verification model and the illumination sequence. As shown in FIG15, the third color verification model can include a first extraction layer 1530, a second extraction layer 1540, and a discrimination layer 1570. For example, the verification module can use the third verification model to implement steps 1430-1450 to determine the second judgment result. Specifically, step 1430 is implemented based on the first extraction layer 1530, step 1440 is implemented based on the second extraction layer 1540, and step 1450 is implemented based on the discrimination layer 1570. Further, the verification module determines the authenticity of multiple target images based on the second judgment result and the illumination sequence.

In some embodiments, the input of the first extraction layer 1530 is the first image sequence 1510, and the output is the first feature information 1550. For example, the verification module may sequentially stitch together multiple target images in the first image sequence 1510 and input them into the first extraction layer 1530. The output first feature information 1550 may be a feature obtained by stitching together color features corresponding to the multiple target images in the first image sequence 1510. The input of the second extraction layer 1540 is the second image sequence 1520, and the output is the second feature information 1560. For example, the verification module may sequentially stitch together multiple color template images in the second image sequence 1520 and input them into the second extraction layer 1540. The output second feature information 1560 may be a feature obtained by stitching together color features corresponding to the multiple color template images in the second image sequence 1520.

In some embodiments, the types of the first extraction layer and the second extraction layer include, but are not limited to, convolutional neural network (CNN) models such as ResNet, ResNeXt, SE-Net, DenseNet, MobileNet, ShuffleNet, RegNet, EfficientNet, or Inception, or recurrent neural network models. The types of the first extraction layer and the second extraction layer may be the same or different.

In some embodiments, the input of the discriminant layer 1570 is the first feature information 1550 and the second feature information 1560, and the output is the second judgment result. In some embodiments, the discriminant layer can be a model for implementing classification, including but not limited to a fully connected layer, a deep neural network (DNN), etc.

The third verification model is a machine learning model with preset parameters. It can be understood that the first extraction layer, the second extraction layer and the discrimination layer included in the third verification model are machine learning models with preset parameters. The preset parameters of the third verification model can be determined in the model training process. For example, the acquisition module can train the initial third verification model based on the third training sample with a third label to obtain the third verification model. The third training sample includes a first sample image sequence and a second sample image sequence, the first sample image sequence is composed of a plurality of sample target images of a sample target object, and the second sample image sequence is composed of a plurality of sample color template images of a plurality of sample colors. The third label of the third training sample is whether the color sequence of the illumination when the plurality of sample target images of the first sample image sequence are photographed is consistent with the color sequence of the plurality of sample color templates in the second sample image sequence.

In some embodiments, the acquisition module can input the third training sample into the initial third verification model, and update the parameters of the initial first extraction layer, the initial second extraction layer, and the initial discrimination layer through training until the updated third verification model meets the first preset condition. The updated third verification model can be designated as the third verification model of preset parameters, in other words, the updated third verification model can be designated as the trained third verification model. The third preset condition can be that the loss function of the updated third verification model is less than a threshold, converges, or the number of training iterations reaches a threshold.

In some embodiments, the acquisition module can train the initial first extraction layer, the initial second extraction layer, and the initial discriminant layer in the initial third verification model through an end-to-end training method. The end-to-end training method refers to inputting the training samples into the initial model, determining the loss value based on the output of the initial model, and updating the initial model based on the loss value. The initial model may include multiple sub-models or modules for performing different data processing operations, which will be regarded as a whole during training and updated simultaneously. For example, in the training of the initial first verification model, the first sample image sequence can be input into the initial first extraction layer, the second sample image sequence can be input into the initial second extraction layer, and a loss function can be established based on the output result of the initial discriminant layer and the third label. Based on the loss function, the parameters of each initial model in the initial third verification model are updated simultaneously.

In some embodiments, all or part of the parameters of the first abstraction layer and the second abstraction layer may be shared.

In some embodiments of this specification, the authenticity of the target image is determined by the third verification model. It is not necessary to identify the type of illumination when the target image is taken, and the target recognition is performed directly by comparing whether the first image sequence containing the target image and the second image sequence containing the color template image are consistent. This is equivalent to converting the color recognition task into a binary classification task of judging whether the colors are the same. In some embodiments, the third verification model can be used to determine whether the first image sequence and the second image sequence are consistent. The discriminant layer of the second verification model can only include a small number of neurons (for example, two neurons) to judge whether the sequences are the same. Compared with the color recognition network in the traditional method, the second verification model disclosed in this specification has a simpler structure. The computing resources (such as computing space) required for the target object analysis based on the third verification model are also relatively less, thereby improving the recognition efficiency of the illumination color. At the same time, the input of the model can be a target image corresponding to any color. Compared with other algorithms that require a limited number of input color types, the embodiments of this specification are more applicable. Moreover, the use of the third verification model can improve the reliability of the target object authenticity verification, reduce or remove the impact of the performance differences of the terminal equipment, and further determine the authenticity of the target image. It is understandable that there are certain differences in the hardware of different terminals. For example, the same color light emitted by terminal screens of different manufacturers may have differences in parameters such as saturation and brightness, resulting in a relatively large intra-class gap for the same color. The third training sample of the initial third verification model can be taken by terminals with different performance. Through learning during the training process, the initial third verification model can make the trained third verification model consider the terminal performance differences when judging the color of the target object and determine the color of the target image more accurately. In addition, the multiple target images in the first image sequence are all taken under the same external ambient light conditions. Therefore, when the first image sequence is processed based on the third verification model to determine the authenticity of the multiple target images, the influence of the external ambient light can be eliminated or weakened.

In some embodiments, the verification module may determine the updated color features of multiple target images (including at least one verification target object image and at least one reference image) based on the reference color space, and generate updated first feature information of the first image sequence based on the updated color features of the multiple target images. Similarly, the verification module may generate updated second feature information of the second image sequence based on the updated color features of the multiple color template images. The verification module may further determine the authenticity of the multiple target images based on the updated first feature information (or first feature information) and the updated second feature information (or second feature information).

The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and/or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.

Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and/or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and/or use of terms in this specification shall prevail.

Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims (10)

1. A method of target identification, the method comprising:

acquiring a plurality of initial images, wherein the shooting time of the plurality of initial images has a corresponding relation with the irradiation time of a plurality of illuminations in an illumination sequence irradiated to a target object, the plurality of illuminations have a plurality of colors, and the plurality of initial images comprise a first initial image and at least one second initial image;

migrating color features of the second initial image onto the first initial image for each of at least one second initial image to cause texture of the first initial image to replace texture of the second initial image to generate a processed second initial image; and

Determining the authenticity of a plurality of target images based on the illumination sequence and color characteristics of the plurality of target images, the plurality of target images including the first initial image and the at least one processed second initial image.

2. The method of claim 1, the replacing the texture of the second initial image with the texture of the first initial image to generate a processed second initial image comprising:

and based on a color migration algorithm, migrating the color of illumination when the second initial image is shot to the first initial image, and generating a processed second initial image.

3. The method of claim 2, the color migration algorithm comprising one of a Reinhard algorithm, a Welsh algorithm, a fuzzy clustering algorithm, and an adaptive migration algorithm.

4. The method of claim 1, the plurality of colors comprising white, the first initial image captured at a time corresponding to a light illumination time of white in the light sequence.

5. The method of claim 1, the determining the authenticity of the plurality of target images based on the illumination sequence and color characteristics of the plurality of target images comprising:

determining the illumination color when the plurality of target images are shot; and

Determining the authenticity of the plurality of target images based on the illumination sequence and the colors of illumination when the plurality of target images are captured.

6. The method of claim 5, the determining a color of illumination when the plurality of target images are captured comprising:

For each of the plurality of target images, processing the target image based on a color verification model, determining the illumination color when the target image is shot, wherein the color verification model is a machine learning model with preset parameters.

7. The method of claim 1, the determining the authenticity of the plurality of target images based on the illumination sequence and color characteristics of the plurality of target images comprising:

Determining a first image sequence based on the plurality of target images;

determining a second image sequence based on a plurality of color template images, the plurality of color template images being generated based on the illumination sequence;

An authenticity of the plurality of target images is determined based on the first image sequence and the second image sequence.

8. A target recognition system, the system comprising:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a plurality of initial images, the shooting time of the plurality of initial images has a corresponding relation with the irradiation time of a plurality of illuminations in an illumination sequence irradiated to a target object, the plurality of illuminations have a plurality of colors, and the plurality of initial images comprise a first initial image and at least one second initial image;

a preprocessing module, configured to migrate, for each of at least one second initial image, a color feature of the second initial image onto the first initial image, so that a texture of the first initial image replaces a texture of the second initial image, so as to generate a processed second initial image; and

And the verification module is used for determining the authenticity of the plurality of target images based on the illumination sequence and the color characteristics of the plurality of target images, wherein the plurality of target images comprise the first initial image and the at least one processed second initial image.

9. An object discriminating apparatus, comprising at least one processor and at least one memory;

the at least one memory is configured to store computer instructions;

The at least one processor is configured to execute at least some of the computer instructions to implement the method of any one of claims 1 to 7.

10. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the method of any one of claims 1 to 7.