WO2020001648A1

WO2020001648A1 - Image processing method and apparatus and terminal device

Info

Publication number: WO2020001648A1
Application number: PCT/CN2019/093846
Authority: WO
Inventors: 别晓辉; 刘森玉
Original assignee: 华为技术有限公司
Priority date: 2018-06-29
Filing date: 2019-06-28
Publication date: 2020-01-02
Also published as: CN110727808A

Abstract

Disclosed are an image processing method and apparatus and a terminal device. The method comprises: processing an image analysis factor of an image in an image library by means of a first image analysis algorithm to obtain a first preferred value of the image, the image analysis factor comprising one or more of a geometric factor or a visual factor; processing historical behavior data of the image by means of a second image analysis algorithm to obtain a second preferred value of the image; processing the first preferred value and the second preferred value to obtain the priority of the image; optimizing the image library according to the priority of the images in the image library. An image with which a user satisfies can be screened out by adopting the embodiments of the present application, and optimization efficiency of the image library is efficiently improved.

Description

Image processing method, device and terminal equipment

Technical field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, device, and terminal device.

Background technique

Users can use the camera to capture beautiful moments of life anytime, anywhere, record them as beautiful memories, and also easily share and share with others. In the use of the camera, the expansion of the library is a very common problem. For example, in the process of using the camera, the user obtains satisfactory pictures by repeatedly framing and shooting multiple times, resulting in a large number of redundant or low-quality pictures in the gallery. In addition, the camera's continuous shooting, snapshot, or high-Dynamic Range (HDR) compositing functions have exacerbated the gallery expansion. Expansion of the gallery brings great inconvenience to the user's later browsing and cleaning. The user needs to browse a large number of redundant duplicate or low-quality pictures while browsing the pictures, reducing the subjective willingness of the user to browse the gallery. Therefore, how to optimize the gallery so as to filter out the pictures that the user is satisfied with is a problem that needs to be solved urgently.

Summary of the invention

The embodiments of the present application provide an image processing method and device, which optimize the gallery based on image analysis factors and historical behavior data of the user's operation of the image, which can screen out images that the user is satisfied with, and effectively improve the optimization efficiency of the gallery.

In a first aspect, an embodiment of the present application provides an image processing method. A terminal device processes an image analysis factor of an image in a gallery through a first image analysis algorithm to obtain a first preferred value of the image. The image analysis factor includes a geometric factor or One or more of the visual factors; the historical behavior data of the image is processed by the second image analysis algorithm to obtain the second preferred value of the image; the first preferred value and the second preferred value are processed to obtain the priority of the image Level; optimize the gallery according to the priority of each image in the gallery.

In this implementation manner, the terminal device processes the image analysis factors of the images in the gallery through the first image analysis algorithm to obtain the first preferred value of the image, which can ensure the high quality of the image; the terminal device uses the second image analysis algorithm to The historical behavior data of the image is processed to obtain the second preferred value of the image, which can ensure that the user has a greater tendency for the image. The terminal device optimizes the gallery based on the image analysis factors and the historical behavior data of the user's operation of the image, and can be filtered Produce user-satisfied images and effectively improve the efficiency of gallery optimization.

In a possible implementation manner, the historical behavior data includes one or more of the following: the number of operations performed on the image; the length of time the image is displayed on the display screen; and the number of times the image is displayed on the display screen.

In this implementation manner, the user operates the image more times, or the image is displayed on the display screen for a longer time, or the image is displayed more times on the display screen, and the terminal device may determine the user ’s The tendency is greater, that is, the probability that the image is a user-satisfied image is greater.

In a possible implementation manner, the number of operations performed on the image by the terminal device may specifically include: the number of times the image is shared, the number of times the image has been favorited, the number of times the image has been edited, or the acquisition time One or more of the number of images after the acquisition time of the image and the image background is the same as the image background of the image.

In a possible implementation manner, the specific method for the terminal device to optimize the gallery according to the priority of each image in the gallery may be: image clustering of each image according to attribute information of each image in the gallery to obtain at least For an image group, the attribute information includes one or more of face information, acquisition time, acquisition location, or image background; the highest priority image in the same image group is used as the summary image of the image group.

In this implementation, the terminal device performs image clustering on each image according to the attribute information of each image, and uses the highest priority image in the same image group as the summary image of the image group, which can optimize image classification and effectively improve the optimization of the gallery effectiveness.

In a possible implementation manner, after the terminal device uses the image with the highest priority in the same image group as the summary image of the image group, it can also display all the images contained in the image group when an image expansion instruction about the image group is detected. image.

In a possible implementation manner, after the terminal device optimizes the gallery according to the priority of each image in the gallery, the terminal device can also obtain the identity information of the receiving terminal; according to the identity information, obtain the optimized image contained in the gallery The degree of association with the receiving terminal; according to the degree of association, determine the shared image shared to the receiving terminal among the images contained in the optimized gallery, and send the shared image to the receiving terminal.

In this implementation, the terminal device obtains the degree of association between the image contained in the optimized gallery and the receiving terminal according to the identity information, and determines the sharing to receive among the images contained in the optimized gallery according to the degree of association. The shared image of the terminal, and sending the shared image to the receiving terminal can realize intelligent and personalized group shared images, and improve the efficiency of image sharing.

In a possible implementation manner, the specific manner in which the terminal device sends the shared image to the receiving terminal may be: sending index information of the shared image to the receiving terminal, where the index information includes an image identifier of the shared image; and receiving a thumbnail image from the receiving terminal Acquire the request; respond to the thumbnail acquisition request, send a thumbnail corresponding to the shared image to the receiving terminal; receive the image acquisition request about the target thumbnail from the receiving terminal; respond to the image acquisition request, send the target image corresponding to the target thumbnail to the receiving terminal.

In a possible implementation manner, before the terminal device acquires the identity information of the receiving terminal, it may also establish a WIFI point-to-point communication connection with the receiving terminal.

In a second aspect, an embodiment of the present application provides a computer storage medium that stores a program, and when the program is executed, causes a terminal device to execute any one of the image processing methods described in the first aspect.

In a third aspect, an embodiment of the present application provides a terminal device that has a function of implementing the behavior of the terminal device in the image processing method example described in the first aspect. The functions may be implemented by hardware, and may also be implemented by hardware executing corresponding software. The hardware or software includes one or more units or modules corresponding to the functions described above.

In a fourth aspect, an embodiment of the present application provides a terminal device. The terminal device may include a processor and a memory, and the processor is configured to support the terminal device to perform a corresponding function of the terminal device in the image processing method according to the first aspect. The memory is used for coupling with the processor, and it stores program instructions and data necessary for the terminal device.

In a fifth aspect, an embodiment of the present application provides a computer program product containing instructions, which when executed on a computer, causes the computer to execute the method described in the first aspect.

According to a sixth aspect, an embodiment of the present application provides a chip system. The chip system includes a processor for a terminal device to implement the functions involved in the foregoing aspect, for example, generating or processing data involved in the foregoing method and / or information.

In a design solution, the chip system further includes a memory, and the memory is configured to store program instructions and data necessary for the terminal device. The chip system may be composed of chips, and may also include chips and other discrete devices.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a schematic flowchart of an image processing method disclosed in an embodiment of the present application;

2 is a schematic flowchart of another image processing method disclosed in an embodiment of the present application;

3A is a schematic diagram of an image disclosed in an embodiment of the present application;

3B is a schematic diagram of another image disclosed in an embodiment of the present application;

4 is a schematic flowchart of another image processing method disclosed in an embodiment of the present application;

5A is a schematic diagram of a WIFI point-to-point communication connection disclosed in an embodiment of the present application;

FIG. 5B is a schematic diagram of a correlation degree disclosed in an embodiment of the present application; FIG.

5C is a schematic diagram of a shared image disclosed in an embodiment of the present application;

6 is a schematic structural diagram of an image processing apparatus disclosed by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a terminal device disclosed in an embodiment of the present application.

detailed description

The following describes the embodiments of the present application with reference to the drawings in the embodiments of the present application.

The image processing method in the embodiment of the present application may be applied to a terminal device. The terminal device includes, but is not limited to, a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or an individual user. Digital assistants, etc.

The terminal device's gallery can be located in the memory, and the gallery can store one or more images. Images can include pictures or videos. Users often view repeatedly or shoot multiple times during the video recording process. The terminal device stores a large number of redundant or low-quality images, which leads to the expansion of the gallery. The expansion of the gallery brings great inconvenience to the user's later browsing and cleaning. At the same time, when browsing pictures, you need to browse a lot of redundant or low-quality pictures, which wastes user time and reduces the subjective willingness of users to browse the gallery. Based on the above technical problems, an image processing method according to an embodiment of the present application is proposed. The terminal device processes the image analysis factors of the images in the gallery through a first image analysis algorithm to obtain a first preferred value of the image. The historical behavior data of the image is processed to obtain the second preferred value of the image, the first preferred value and the second preferred value are processed to obtain the priority of the image, and the gallery is optimized based on the priority of each image in the gallery. The embodiment of the present application optimizes a gallery based on image analysis factors and historical behavior data of user-operated images, and can filter out images that are satisfactory to users, effectively improving the optimization efficiency of the gallery.

The image analysis factors in the embodiments of the present application may include one or more of geometric factors or visual factors. The geometric factors can be obtained based on the composition rules of photographic aesthetics, and the geometric factors can be the rule of threes, the rule of diagonals, or the balance. Visual factors can include contrast, exposure, or color distribution. The embodiments of the present application can screen out high-quality images based on image analysis factors.

The historical behavior data of the image in the embodiment of the present application may include one or more of the following: the number of operations performed on the image, the length of time the image is displayed on the display screen, and the number of times the image is displayed on the display screen. The embodiment of the present application can predict the user's propensity based on the historical behavior data of the image, that is, predict the user's satisfaction with the image.

The method for optimizing the gallery in the embodiment of the present application may be: sort the images according to the priority of the images from high to low, and display the sorted images; or lower priority images among the images with higher similarity Delete images; or push higher priority images, etc.

Please refer to FIG. 1. FIG. 1 is an image processing method provided by an embodiment of the present application. The method is used to optimize the images in a gallery. The method includes, but is not limited to, the following steps:

Step S101: The terminal device processes the image analysis factors of the images in the gallery through a first image analysis algorithm to obtain a first preferred value of the image.

The terminal device can acquire the image in the gallery, and process the image analysis factor of the image through the first image analysis algorithm to obtain the first preferred value of the image. The first image analysis algorithm may be one or more of a geometric factor analysis algorithm or a visual factor analysis algorithm, and the image analysis factor may include one or more of a geometric factor or a visual factor. The image in the gallery can be obtained by the terminal device through the sensor component, communication component or the Internet and stored in the gallery. For example, the sensor component can be a camera or a scanner. The terminal device can take a picture of the target object through the camera to obtain an image, or The terminal device can scan the target object through the scanner to obtain an image. For example, the communication component is Bluetooth, infrared, Near Field Communication (NFC) or WIFI. The terminal device can establish a communication connection with the sending terminal through the communication component. Receive images from the sending terminal through the communication connection established with the sending terminal.

The terminal device can acquire all the images in the gallery. For each image, the terminal device can perform steps S101-S104 in the embodiment of the present application. Alternatively, the terminal device may acquire one or more images in the gallery every preset time interval. For each image, the terminal device may perform steps S101-S104 in the embodiment of the present application. Alternatively, the terminal device may acquire one or more images that have not been optimized in the gallery every preset time interval. For each image that is not optimized, the terminal device may perform steps S101-S104 in the embodiment of the present application.

In a possible implementation manner, if the first image analysis algorithm is a geometric factor analysis algorithm, the terminal device can process the geometric factors of the images in the gallery through the geometric factor analysis algorithm to obtain the geometric factor value of the image, and The geometric factor value is taken as the first preferred value. For example, the geometric factor analysis algorithm can be expressed by the following formula:

Among them, E _a represents the geometric factor value of the image, ω _RT represents the geometric factor is the weight of the trisection method, E _RT represents the geometric factor three-point method, ω _DA represents the geometric factor is the diagonal rule weight, and E _DA represents The diagonal of the geometric factor is regular, ω _VB means the geometric factor is the balance weight, and E _VB means the geometric factor is the balance.

In a possible implementation manner, if the first image analysis algorithm is a visual factor analysis algorithm, the terminal device may process the visual factor of the image in the gallery through the visual factor analysis algorithm to obtain the visual factor value of the image, and The visual factor value is taken as the first preferred value.

In a possible implementation manner, if the first image analysis algorithm includes a geometric factor analysis algorithm and a visual factor analysis algorithm, the terminal device may process the geometric factors of the images in the gallery through the geometric factor analysis algorithm to obtain the geometric factor value. The terminal device may also use the visual factor analysis algorithm to process the visual factors of the images in the gallery to obtain the visual factor values, and perform a weighted sum of the geometric factor values and the visual factor values to obtain the first preferred value of the image. Exemplarily, if the value of the geometric factor is E _a , the value of the visual factor is E _c , the weight of the geometric factor value is α, and the weight of the visual factor value is β, the terminal device can pass α * E _a + β * E _c , A first preferred value of the image is obtained.

Step S102: The terminal device processes the historical behavior data of the image by using a second image analysis algorithm to obtain a second preferred value of the image.

The terminal device may also process the historical behavior data of the image by using a second image analysis algorithm to obtain a second preferred value of the image. The embodiment of this application does not limit the sequence of execution of steps S101 and S102. For example, the terminal device may process the image analysis factors of the images in the gallery by using the first image analysis algorithm, and obtain the first preferred value of the image. Two image analysis algorithms process the historical behavior data of the image to obtain the second preferred value of the image. For another example, the terminal device may process the historical behavior data of the image by using the second image analysis algorithm to obtain the second preferred value of the image, and then process the image analysis factors of the image in the gallery through the first image analysis algorithm to obtain the image The first preferred value. For another example, the terminal device can simultaneously process the image analysis factors of the images in the gallery through the first image analysis algorithm to obtain the first preferred value of the image, and process the historical behavior data of the image through the second image analysis algorithm to obtain the image. The second preferred value.

The historical behavior data may include one or more of the following: the number of operations performed on the image; the length of time the image is displayed on the display screen; and the number of times the image is displayed on the display screen.

The number of operations performed on the image may specifically include: the number of times the image is shared, the number of times the image has been bookmarked, the number of edits performed on the image, or the acquisition time is after the image acquisition time and the image background One or more of the same number of images as the image background of the image. The number of edits to edit an image can refer to the number of edits to enlarge, reduce, adjust parameters, or mark the image in response to user operations. The parameter adjustment can be parameters such as the size, background color, shadow, or hue of the image Adjustment.

In a possible implementation manner, if the historical behavior data of the image is the number of operations performed on the image, the terminal device may determine that the more the number of operations performed by the user on the image indicates that the user's tendency toward the image is greater. The terminal device can obtain the operation value of the image by using the following formula, and use the operation value as the second preferred value of the image:

E ₀ = f (ZX ₀ )

Among them, E ₀ represents the operation value of the image, Z> 0, and X ₀ represents the number of operations performed on the image. Exemplarily, f (ZX ₀ ) may be a proportional function.

In a possible implementation manner, if the historical behavior data of the image is the display duration of the image on the display screen, the terminal device may determine that the longer the display duration of the image on the display screen is, the greater the tendency of the user to the image. . The display duration of the image on the display screen can refer to the sum of the display duration of the image when it is displayed on the display screen multiple times in the preset time period, or the sum of the display duration of the image when it is displayed on the display screen multiple times in the preset time period. Maximum display duration. The terminal device can obtain the display time value of the image by using the following formula, and uses the display time value as the second preferred value of the image:

E ₁ = f (t ₁ X ₁ )

Among them, E ₁ represents the display time value of the image, t ₁ > 0, and X ₁ represents the display time of the image on the display screen. Exemplarily, f (t ₁ X ₁ ) may be a proportional function.

In a possible implementation manner, if the historical behavior data of the image is the number of times the image is displayed on the display screen, the terminal device may determine that the more the image is displayed on the display screen, the greater the user ’s tendency towards the image . The number of times the image is displayed on the display screen may refer to the total number of times that the image is displayed multiple times on the display screen within a preset period of time. The terminal device can obtain the number of times the image is displayed by using the following formula, and use the value of the number of times of display as the second preferred value of the image:

E ₂ = f (t ₂ X ₂ )

Among them, E ₂ represents the number of times the image is displayed, t ₂ > 0, and X ₂ represents the number of times the image is displayed on the display screen. Exemplarily, f (t ₂ X ₂ ) may be a proportional function.

In a possible implementation manner, if the historical behavior data is that the user selects the image as a summary image, the terminal device may determine that the selection value of the image is 1, and use the selection value as the second preferred value of the image.

If the historical behavior data of the image includes the number of operations performed on the image, the length of time the image is displayed on the display screen, and two or more of the number of times the image is displayed on the display screen, the terminal device may use a second image analysis algorithm Each historical behavior data of the image is processed to obtain a processing result, and the processing result is weighted and summed to obtain a second preferred value of the image. For example, if the historical behavior data includes the number of times the image is shared, the number of edits to edit the image, and the length of time the image is displayed on the display screen, the terminal device uses the second image analysis algorithm corresponding to each historical behavior data to The historical behavior data is processed to obtain the number of times of sharing, the number of times of editing, and the value of display duration. The terminal device can multiply the value of the number of times of sharing and the weight corresponding to the number of times of sharing to obtain a first product, and the number of times of editing and the value Multiply the weights corresponding to the number of times to obtain a second product. Multiply the weight of the display duration by the weight of the display duration to obtain a third product. Add the first product, the second product, and the third product to obtain the product. The second preferred value of the image.

Step S103: The terminal device processes the first preferred value and the second preferred value to obtain the priority of the image.

After the terminal device obtains the first preferred value and the second preferred value of the image, it can perform weighted summation on the first preferred value and the second preferred value to obtain the priority of the image. For example, the terminal device may multiply the weight corresponding to the first preference value by the first preference value to obtain a fourth product, multiply the weight value corresponding to the second preference value by the second preference value to obtain a fifth product, and The fourth product and the fifth product are added to obtain the priority of the image.

Step S104: The terminal device optimizes the gallery according to the priority of each image in the gallery.

In a possible implementation manner, the terminal device may acquire an image with a similarity greater than a preset ratio threshold, and delete an image with a lower priority from the acquired image. In the embodiment of the present application, the user has taken multiple repeated images by repeatedly framing or continuous shooting. The terminal device processes the image analysis factors of the images in the gallery by using the first image analysis algorithm, and the images are processed by the second image analysis algorithm. The historical behavior data can be processed to determine that the image with a higher priority has a higher image quality and the user has a higher tendency for the image, and the terminal device can compare the image with a higher priority in an image with a similarity greater than a preset ratio threshold. The image is stored in the memory of the terminal device, and the lower-priority image among the images whose similarity is greater than the preset ratio threshold is deleted, which effectively improves the storage space of the memory and effectively improves the library optimization efficiency.

In a possible implementation manner, the terminal device may sort the images in the gallery according to the priority of the images from large to small, and then display the sorted images. Through the embodiments of the present application, the terminal device can display a higher priority image in front so that the image that the user preferentially browses is an image required by the user, which effectively improves the optimization efficiency of the gallery.

In the method described in FIG. 1, the terminal device processes the image analysis factors of the images in the gallery through a first image analysis algorithm to obtain a first preferred value of the image, and processes the historical behavior data of the image through a second image analysis algorithm. To obtain the second preferred value of the image, to process the first preferred value and the second preferred value, to obtain the priority of the image, and to optimize the gallery according to the priority of each image in the gallery, and to filter out the images that the user is satisfied with , Effectively improve the efficiency of library optimization.

Based on the image processing method shown in FIG. 1, please refer to FIG. 2. FIG. 2 is an image processing method provided by an embodiment of the present application. The method is used for clustering images in an optimized gallery. The method includes: But not limited to the following steps:

Step S201: The terminal device performs image clustering on each image according to attribute information of each image in the gallery to obtain at least one image group.

The terminal device may perform image clustering on each image according to attribute information of each image in the gallery to obtain at least one image group. Alternatively, the terminal device may perform image clustering on each image according to the attribute information of each image in the optimized gallery to obtain at least one image group. The attribute information may include one or more of face information, acquisition time, acquisition location, or image background.

In a possible implementation manner, the terminal device can obtain all the images in the gallery and use a preset image clustering algorithm to process the clustering factors of each image to achieve scene clustering. The preset image clustering algorithm may include a kmeams algorithm (a clustering algorithm based on partitioning), a Convolutional Neural Network (CNN) algorithm, or a Deep Residual Network (ResNet) algorithm. The clustering factors may include static features of the image and sensor data, static features may include color histograms or structural features, and the sensor data may include location information obtained by the terminal device acquired by the Global Positioning System (GPS) or The terminal equipment collected by the time synchronization system acquires the time information of the image and so on. Taking the schematic diagram of the image shown in FIG. 3A as an example, the terminal device may belong to the same image group for images with similar color histograms, and belong to the same image group for images collected at the same location, and set the acquisition time to The images in the preset time period belong to the same image group.

In a possible implementation manner, the terminal device may use a hierarchical clustering algorithm to cluster the color histogram to implement scene clustering.

In a possible implementation manner, the terminal device may modify the clustering result to improve the accuracy of the clustering.

In a possible implementation manner, the terminal device may also perform face recognition on each image in the gallery, and store images containing the same face in the same image group.

Step S202: The terminal device acquires priorities of each image in the same image group.

After the terminal device obtains at least one image group, it can obtain the priority of each image in the same image group. For the manner of obtaining the priority of the image, refer to the image processing method described in FIG. 1, which is not described in the embodiment of this application.

Step S203: The terminal device uses the image with the highest priority in the same image group as the digest image of the image group.

For the same image group, the terminal device can create a hierarchical directory, and use the image with the highest priority in the image group as the summary image of the image group, that is, the image with the highest priority as the first-level directory, and other images in the image group as two Level directory. For example, when a terminal device obtains multiple image groups through scene clustering, a scene hierarchical directory may be established, that is, for the images of the same scene, the image with the highest priority is used as the first-level directory among the images containing the scene. For another example, when the terminal device obtains multiple image groups through character clustering, a hierarchical classification of people can be established, that is, for images of the same person, the image with the highest priority is used as the first-level directory among the images containing the person. According to the embodiment of the present application, only the highest priority image in each image group can be displayed in the first-level catalog of the gallery, which prevents the user from searching for all the images contained in the gallery, reduces the image search delay, and effectively improves Gallery optimization efficiency.

In a possible implementation manner, when the terminal device detects the image expansion instruction about the image group, it can display all the images included in the image group. For example, the terminal device can display a virtual button with an image expansion function in the summary image. When the terminal device detects that the user clicks the virtual button, it can generate an image expansion instruction about the image group to which the summary image belongs, and respond to the image expansion instruction. Display all images contained in this image group. For another example, when the terminal device detects a user operation on the digest image, it can detect whether the operation submitted by the user matches the operation corresponding to the image expansion instruction. If the operation submitted by the user matches the operation corresponding to the image expansion instruction, the terminal device can generate the operation Operate the corresponding image expansion instruction, and display all images included in the image group in response to the image expansion instruction. The operation corresponding to the image expansion instruction may be set by a user, or may be set by default when the terminal device is shipped from the factory. The operation corresponding to the image expansion instruction may be a long-press, double-click, or multi-touch on the summary image, or a sliding track obtained by sliding on the summary image. The operation corresponding to the image expansion instruction can also be the sliding track obtained by the user sliding on the display screen. The sliding track corresponding to different image groups is different. When the terminal device detects the sliding track submitted by the user, the user can send the sliding track submitted by the user. Compare with the slide trajectory corresponding to each image expansion instruction, generate an image expansion instruction corresponding to the slide trajectory that matches the slide trajectory submitted by the user, and respond to the image expansion instruction to display all the images contained in the image group corresponding to the slide trajectory. image.

Taking the schematic diagram of the image shown in FIG. 3B as an example, a virtual button is displayed on the upper left of the summary image, and the number displayed in the virtual button may indicate the number of images included in the image group to which the summary image belongs. When the terminal device detects that the user clicks the virtual button, it can generate an image expansion instruction about the image group to which the summary image belongs, and display all images included in the image group in response to the image expansion instruction.

In the method described in FIG. 2, the terminal device performs image clustering on each image according to the attribute information of each image to obtain at least one image group, obtains the priority of each image in the same image group, and prioritizes the same image group. The highest image is used as the summary image of the image group, which can optimize the image classification and effectively improve the library optimization efficiency.

Based on the image processing method shown in FIG. 2, please refer to FIG. 4. FIG. 4 is an image processing method provided by an embodiment of the present application. The method is used for intelligently sharing an optimized processed image. The method includes, but is not limited to The following steps:

Step S401: The terminal device acquires the identity information of the receiving terminal.

The terminal device can obtain the identity information of the receiving terminal. The identity information can be used to identify the relationship between the receiving terminal and the terminal device, such as a family member, a colleague, or a friend. The family member may further include a spouse, a parent, or a sibling. The colleague may further include General manager, supervisor, minister or subordinate, etc. Friends may further include same-sex friends or heterosexual friends, etc. The identity information may also include avatar information of the user who uses the receiving terminal.

In a possible implementation manner, the terminal device may establish a WIFI point-to-point communication connection with the receiving terminal, and obtain the identity information of the receiving terminal through the WIFI point-to-point communication connection with the receiving terminal. For example, a terminal device can use WIFI Direct networking to establish a multi-terminal home domain networking to achieve point-to-point interconnection and image transmission through point-to-point interconnection, which can increase the transmission speed and distance of the image. Among them, the WIFI Direct standard means that devices in a wireless network can be connected to each other without a wireless router. The topology of the WIFI Direct network can be one-to-one or one-to-many. Taking the schematic diagram of the WIFI point-to-point communication connection shown in FIG. 5A as an example, the terminal device can find the receiving terminal through the "device discovery function" of WIFIDirect, and the terminal device can actively initiate an invitation to the receiving terminal to establish a communication connection, and the terminal device and the receiving terminal After the communication connection is established, the terminal device and the receiving terminal automatically establish a group. The terminal device can negotiate with the receiving terminal to determine whether the device used to manage the group is a terminal device or a receiving terminal. The device used to manage the group can change other devices. The device joins the group and sets the group as a temporary group or a permanent group. If the group is a temporary group, the device used to manage the group will delete the group after a preset period of time. If the group is a permanent group, the device for managing the group deletes the group when it receives a delete instruction for the group.

In addition, the embodiments of the present application include, but are not limited to, establishing a WIFI point-to-point communication connection with the receiving terminal, and can also establish Bluetooth, infrared, Near Field Communication (NFC) or WIFI routing networking with the receiving terminal. Connect to communicate with the receiving terminal, for example, to obtain the identity information of the receiving terminal.

Step S402: The terminal device obtains, according to the identity information, the degree of association between the image included in the optimized gallery and the receiving terminal.

The terminal device can obtain the correlation between the image contained in the optimized gallery and the receiving terminal according to the identity information, and the correlation can be greater than or equal to 0 and less than or equal to 1. For example, after the terminal device performs character clustering on the images in the gallery through the image processing method shown in FIG. 2, the user can personally mark the persons included in the image. For example, the image includes three persons. One person is the spouse of the second person in the image, the third person in the image is the child of the second person in the image, and the identity information of the receiving terminal is a colleague. Then the terminal device can determine the relationship between the various characters in the image as Family relationship, the relationship between the receiving terminal and the terminal device is a colleague relationship, then the correlation between the image and the receiving terminal is 0.1. As another example, after the terminal device performs character clustering on the images in the gallery through the image processing method shown in FIG. 2, the terminal device can compare the face in the image with the avatar information included in the identity information. When the face indicated by the avatar information, the terminal device may determine that the correlation between the image and the receiving terminal is 1. For another example, after the terminal device performs scene clustering on the images in the gallery through the image processing method shown in FIG. 2, if the image is a landscape photo, regardless of whether the receiving terminal and the terminal device are a family relationship, a colleague relationship, or a friend relationship, The correlation between the image and the receiving terminal is 0.8. For another example, after the terminal device performs scene clustering on the images in the gallery by using the image processing method shown in FIG. 2, if the scene of the image is an office place, and the relationship between the receiving terminal and the terminal device is a colleague relationship, the terminal device may determine The correlation between the image and the receiving terminal is 0.8.

Taking the schematic diagram of the degree of association shown in FIG. 5B as an example, if the image is a landscape image, the terminal device can determine that the degree of association between the image and all receiving terminals is 0.8; if the image is a person image, the person contained in the image is When the child is a child and the relationship between the terminal device and the receiving terminal is a family relationship, the terminal device can determine that the correlation between the image and the receiving terminal is 1; if the image is a person image, the person contained in the image and the receiving terminal When the relationship is a spouse relationship, the terminal device can determine that the correlation between the image and the receiving terminal is 0.8; if the image is a person image, the relationship between the person contained in the image and the receiving terminal is father-son, mother-child, father-daughter, or In the mother-daughter relationship, the terminal device can determine that the correlation between the image and the receiving terminal is 0.6; if the image is a person image and there is no relationship between the person contained in the image and the receiving terminal, the terminal device can determine that the image is related to the receiving terminal The correlation between the receiving terminals is 0.2; if the image is neither a person image nor a landscape image, the terminal device To determine the correlation between the image and the receiving terminal is 0.1.

Step S403: The terminal device determines a shared image shared to the receiving terminal among the images included in the optimized gallery according to the degree of association.

In a possible implementation manner, a user may set a correlation degree of images shared to different receiving terminals. For example, the user may set the degree of relevance of the image shared to the first terminal device to be greater than the first degree of relevance threshold, and the degree of relevance of the image shared to the second terminal device to be greater than the second degree of relevance threshold. If the terminal device determines that the receiving terminal is the first terminal device according to the identity information of the receiving terminal, the terminal device may determine an image in the gallery whose correlation degree is greater than the first correlation degree threshold value as a shared image.

In a possible implementation manner, the terminal device may determine an image in the gallery whose correlation degree is greater than a preset correlation degree threshold value as a shared image, and the preset correlation degree threshold value may be a correlation degree threshold value preset by the terminal device, for example, 0.5 or 0.7. Wait.

Step S404: The terminal device sends the shared image to the receiving terminal.

After the terminal device determines the shared image, it can send the shared image to the receiving terminal through a communication connection established in advance with the receiving terminal.

In a possible implementation manner, the terminal device may send index information of the shared image to the receiving terminal. The index information includes an image identifier of the shared image. The terminal device may receive a thumbnail image acquisition request from the receiving terminal, and respond to the thumbnail image acquisition request. Send a thumbnail corresponding to the shared image to the receiving terminal, receive an image acquisition request for the target thumbnail from the receiving terminal, and send a target image corresponding to the target thumbnail to the receiving terminal in response to the image acquisition request. The index information may also include storage paths of index files or thumbnail storage information of shared images.

Taking the schematic diagram of image sharing shown in FIG. 5C as an example, after the terminal device XX determines the shared image, it can generate an index file (for example, a file shared to YY) about the receiving terminal YY, and store the index information of the shared image to the receiving terminal In the index file of YY, the terminal device may send the index information in the index file of the receiving terminal YY to the receiving terminal YY. The receiving terminal YY may establish a shared file (for example, a shared file from XX) about the terminal device in advance, and store index information from the terminal device into the shared file. When the receiving terminal detects a user operation on one or more image identifiers, it may send a thumbnail image acquisition request to the terminal device, and the thumbnail image acquisition request carries the image identifier. The terminal device may respond to the thumbnail image acquisition request and send a thumbnail image corresponding to the shared image indicated by the image identification to the receiving terminal. The receiving terminal may display a thumbnail corresponding to the shared image on a display screen of the receiving terminal. When the receiving terminal detects a user operation on the target thumbnail, it may send an image acquisition request to the terminal device. The image acquisition request may carry the target thumbnail corresponding to the target thumbnail. The image ID of the shared image. The terminal device may respond to the image acquisition request and send a target image corresponding to the target thumbnail to the receiving terminal. When the receiving terminal detects a storage operation of the target image by the user, the target image may be stored in the memory.

In a possible implementation manner, the index file generated by the terminal device may be in a hidden state, that is, the index file is displayed only when a display request of the index file by the user is detected.

In a possible implementation manner, the terminal device may periodically detect whether a new image is stored in the gallery, and the terminal device obtains a correlation between the new image and the receiving terminal. If it is determined that the new image is shared according to the correlation Image, the terminal device may store index information of the new image in an index file.

In a possible implementation manner, if the terminal device deletes an image corresponding to an image identifier in the index file, the terminal device may delete index information about the image in the index file.

In the method described in FIG. 4, the terminal device obtains the identity information of the receiving terminal, and according to the identity information, obtains the correlation between the image contained in the optimized gallery and the receiving terminal, and according to the correlation, the optimized processing Among the images contained in the gallery, the shared image shared to the receiving terminal is determined, and the shared image is sent to the receiving terminal, which can realize intelligent and personalized group shared images and improve the efficiency of image sharing.

The method in the embodiment of the present application has been described in detail above, and the terminal device in the embodiment of the present application is provided below.

Please refer to FIG. 6. FIG. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application, which is used to implement the functions of the terminal device in the embodiments of FIG. 1, FIG. 2, and FIG. 4. , Software, or a combination of hardware and software to implement the solution of this application. Those skilled in the art should understand that the functional blocks described in FIG. 6 may be combined or separated into several sub-blocks to implement the solution of the present application. Therefore, the content described above in this application may support any possible combination or separation or further definition of the functional modules described below.

As shown in FIG. 6, the image processing apparatus may include a processing module 601 and a sending module 602, wherein the detailed description of each module is as follows.

A processing module 601 is configured to process an image analysis factor of an image in the gallery through a first image analysis algorithm to obtain a first preferred value of the image, where the image analysis factor includes one or more of a geometric factor or a visual factor Species

The processing module 601 is further configured to process the historical behavior data of the image by using a second image analysis algorithm to obtain a second preferred value of the image;

The processing module 601 is further configured to process the first preferred value and the second preferred value to obtain a priority of the image;

The processing module 601 is further configured to perform optimization processing on the gallery according to a priority of each image in the gallery.

In a possible implementation manner, the historical behavior data includes at least one of the following:

The number of operations performed on the image;

A display duration of the image on a display screen;

The number of times the image is displayed on the display screen.

In a possible implementation manner, the number of operations performed on the image includes:

The number of times that the image was shared, the number of times the image was favorited, the number of edits that the image was edited, or the acquisition time was after the acquisition time of the image and the image background and the image of the image One or more of the number of images with the same background.

In a possible implementation manner, the processing module 601 optimizes the gallery according to the priority of each image in the gallery, and is specifically configured to:

According to the attribute information of each image in the gallery, image clustering is performed on each image to obtain at least one image group. The attribute information includes one or more of face information, acquisition time, acquisition location, or image background. Species

The image with the highest priority in the same image group is used as the summary image of the image group.

In a possible implementation manner, the processing module 601 is further configured to use an image with the highest priority in the same image group as a summary image of the image group, and when an image expansion instruction about the image group is detected To display all the images contained in the image group.

In a possible implementation manner, the processing module 601 is further configured to obtain the identity information of the receiving terminal after optimizing the gallery according to the priority of each image in the gallery, and according to the identity information To obtain a degree of association between the image included in the optimized gallery and the receiving terminal, and according to the degree of association, determine a share shared to the receiving terminal among the images included in the optimized gallery image;

The device further includes:

The sending module 602 is configured to send the shared image to the receiving terminal.

In a possible implementation manner, the sending module 602 is specifically configured to:

Sending index information of the shared image to the receiving terminal, where the index information includes an image identifier of the shared image;

Receiving a thumbnail acquisition request from the receiving terminal;

In response to the thumbnail image acquisition request, sending the thumbnail image corresponding to the shared image to the receiving terminal;

Receiving an image acquisition request about a target thumbnail from the receiving terminal;

In response to the image acquisition request, sending a target image corresponding to the target thumbnail to the receiving terminal.

In a possible implementation manner, the processing module 601 is further configured to establish a WIFI point-to-point communication connection with the receiving terminal before acquiring the identity information of the receiving terminal.

It should be noted that the implementation of each module may also correspond to the corresponding description of the embodiments shown in FIG. 1, FIG. 2, and FIG. 4.

It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be another division manner. Each functional module in the embodiment of the present application may be integrated into one processing module, or each module may exist separately physically, or two or more modules may be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

Please refer to FIG. 7, which is a schematic structural diagram of a terminal device disclosed in an embodiment of the present application. As shown in FIG. 7, the terminal device 700 may include a baseband chip 110, a memory 115 (one or more computer-readable storage media), a radio frequency (RF) module 116, and a peripheral system 117. These components can communicate on one or more communication buses 114.

The peripheral system 117 is mainly used to implement the interaction function between the terminal device 700 and a user / external environment, and mainly includes input and output devices of the terminal device 700. In specific implementation, the peripheral system 117 may include at least one of a touch screen controller 118, a camera controller 119, an audio controller 120, and a sensor management module 121. Each controller may be coupled to a corresponding peripheral device (such as a touch screen 123, a camera 124, an audio circuit 125, and a sensor 126). In some embodiments, the touch screen 123 may be a touch screen configured with a self-capacitive floating touch panel, or a touch screen configured with an infrared type floating touch panel. In some embodiments, the camera 124 may be a 3D camera. It should be noted that the peripheral system 117 may further include other I / O peripherals.

The baseband chip 110 may integrate: one or more processors 111, a clock module 112, and a power management module 113. The clock module 112 integrated in the baseband chip 110 is mainly used to generate a clock required for data transmission and timing control for the processor 111. The power management module 113 integrated in the baseband chip 110 is mainly used to provide a stable, high-precision voltage for the processor 111, the radio frequency module 116, and peripheral systems.

A radio frequency (RF) module 116 is used to receive and send radio frequency signals, and mainly integrates a receiver and a transmitter of the terminal device 700. A radio frequency (RF) module 116 communicates with a communication network and other communication devices through radio frequency signals. In a specific implementation, the radio frequency (RF) module 116 may include, but is not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chip, a SIM card, and Storage media, etc. In some embodiments, the radio frequency (RF) module 116 may be implemented on a separate chip.

The memory 115 is coupled to the processor 111 and is configured to store various software programs and / or multiple sets of instructions. In specific implementation, the memory 115 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 115 may store an operating system (hereinafter referred to as a system), such as an embedded operating system such as ANDROID, IOS, WINDOWS, or LINUX. The memory 115 may also store a network communication program, which may be used to communicate with one or more additional devices, one or more terminal device devices, and one or more network devices. The memory 115 may also store a user interface program. The user interface program can display the content of the application program realistically through a graphical operation interface, and receive user control operations on the application program through input controls such as menus, dialog boxes, and buttons. .

The memory 115 may also store one or more application programs. As shown in Figure 7, these applications may include: social applications (such as Facebook), image management applications (such as photo albums), map applications (such as Google Maps), browsers (such as Safari, Google Chrome), etc. .

It should be understood that the terminal device 700 is only an example provided in the embodiment of the present application, and the terminal device 700 may have more or fewer components than those shown, may combine two or more components, or may have Different configurations of components are implemented.

Specifically, the terminal device described in the embodiment of the present application may be used to implement part or all of the processes in the method embodiment described in conjunction with FIG. 1, FIG. 2, and FIG. 4.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are wholly or partially generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, a computer, a server, or a data center. Transmission by wire (for example, coaxial cable, optical fiber, digital subscriber line) or wireless (for example, infrared, wireless, microwave, etc.) to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes one or more available medium integration. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (Solid State Disk (SSD)), and the like.

Claims

An image processing method, wherein the method includes:

Processing the image analysis factors of the images in the gallery through a first image analysis algorithm to obtain a first preferred value of the image, where the image analysis factors include one or more of geometric factors or visual factors;

Processing the historical behavior data of the image through a second image analysis algorithm to obtain a second preferred value of the image;

Process the first preferred value and the second preferred value to obtain the priority of the image;

According to the priority of each image in the gallery, the gallery is optimized.
The method of claim 1, wherein the historical behavior data comprises one or more of the following:

The number of operations performed on the image;

A display duration of the image on a display screen;

The number of times the image is displayed on the display screen.
The method according to claim 2, wherein the number of operations performed on the image comprises:

The number of times that the image was shared, the number of times the image was favorited, the number of edits that the image was edited, or the acquisition time was after the acquisition time of the image and the image background and the image of the image One or more of the number of images with the same background.
The method according to claim 1, wherein the optimizing the gallery according to the priority of each image in the gallery comprises:

According to the attribute information of each image in the gallery, image clustering is performed on each image to obtain at least one image group. The attribute information includes one or more of face information, acquisition time, acquisition location, or image background. Species

The image with the highest priority in the same image group is used as the summary image of the image group.
The method according to claim 4, wherein after the image having the highest priority in the same image group is used as a summary image of the image group, further comprising:

When an image expansion instruction for the image group is detected, all images included in the image group are displayed.
The method according to claim 1, wherein after performing optimization processing on the gallery according to the priority of each image in the gallery, further comprising:

Obtain the identity information of the receiving terminal;

Obtaining, according to the identity information, a degree of correlation between the image included in the optimized gallery and the receiving terminal;

Determining a shared image shared to the receiving terminal among the images included in the optimized gallery according to the correlation degree;

Sending the shared image to the receiving terminal.
The method according to claim 6, wherein the sending the shared image to the receiving terminal comprises:

Sending index information of the shared image to the receiving terminal, where the index information includes an image identifier of the shared image;

Receiving a thumbnail acquisition request from the receiving terminal;

In response to the thumbnail image acquisition request, sending the thumbnail image corresponding to the shared image to the receiving terminal;

Receiving an image acquisition request about a target thumbnail from the receiving terminal;

In response to the image acquisition request, sending a target image corresponding to the target thumbnail to the receiving terminal.
The method according to claim 6, wherein before the obtaining the identity information of the receiving terminal, further comprising:

A WIFI point-to-point communication connection is established with the receiving terminal.
An image processing device, wherein the device includes:

A processing module, configured to process an image analysis factor of an image in the gallery by using a first image analysis algorithm to obtain a first preferred value of the image, where the image analysis factor includes one or more of a geometric factor or a visual factor ;

The processing module is further configured to process the historical behavior data of the image by using a second image analysis algorithm to obtain a second preferred value of the image;

The processing module is further configured to process the first preferred value and the second preferred value to obtain a priority of the image;

The processing module is further configured to optimize the gallery according to the priority of each image in the gallery.
The apparatus according to claim 9, wherein the historical behavior data includes at least one of the following:

The number of operations performed on the image;

A display duration of the image on a display screen;

The number of times the image is displayed on the display screen.
The apparatus according to claim 10, wherein the number of operations performed on the image comprises:

The number of times that the image was shared, the number of times the image was favorited, the number of edits that the image was edited, or the acquisition time was after the acquisition time of the image and the image background and the image of the image One or more of the number of images with the same background.
The apparatus according to claim 9, wherein the processing module optimizes the gallery according to a priority of each image in the gallery, and is specifically configured to:

According to the attribute information of each image in the gallery, image clustering is performed on each image to obtain at least one image group. The attribute information includes one or more of face information, acquisition time, acquisition location, or image background. Species

The image with the highest priority in the same image group is used as the summary image of the image group.
The apparatus according to claim 12, wherein:

The processing module is further configured to: after the image with the highest priority in the same image group is used as a summary image of the image group, and when an image expansion instruction about the image group is detected, display all information contained in the image group. image.
The apparatus according to claim 9, wherein:

The processing module is further configured to obtain the identity information of the receiving terminal after optimizing the gallery according to the priority of each image in the gallery, and to obtain the information contained in the optimized gallery according to the identity information. A degree of association between an image and the receiving terminal, and according to the degree of association, determining a shared image shared to the receiving terminal among the images included in the optimized gallery;

The device further includes:

A sending module, configured to send the shared image to the receiving terminal.
The apparatus according to claim 14, wherein the sending module is specifically configured to:

Sending index information of the shared image to the receiving terminal, where the index information includes an image identifier of the shared image;

Receiving a thumbnail acquisition request from the receiving terminal;

In response to the thumbnail image acquisition request, sending the thumbnail image corresponding to the shared image to the receiving terminal;

Receiving an image acquisition request about a target thumbnail from the receiving terminal;

In response to the image acquisition request, sending a target image corresponding to the target thumbnail to the receiving terminal.
The apparatus according to claim 14, wherein:

The processing module is further configured to establish a WIFI point-to-point communication connection with the receiving terminal before acquiring the identity information of the receiving terminal.
A terminal device, wherein the terminal device includes a processor and a memory, and the processor establishes communication with the memory;

The processor is configured to execute the method according to any one of claims 1-8.
A computer storage medium, wherein the computer storage medium stores a program, and when the program is executed, the method according to any one of claims 1-8.