CN117789380B

CN117789380B - Self-service settlement method, system, electronic equipment and medium for shopping checkout

Info

Publication number: CN117789380B
Application number: CN202311808878.3A
Authority: CN
Inventors: 萧炽全; 冯沥梦
Original assignee: Shenzhen Junshida Technology Co ltd
Current assignee: Shenzhen Junshida Technology Co ltd
Priority date: 2023-12-26
Filing date: 2023-12-26
Publication date: 2024-07-30
Anticipated expiration: 2043-12-26
Also published as: CN117789380A

Abstract

A self-service settlement method, a self-service settlement system, electronic equipment and a self-service settlement medium for shopping checkout relate to the technical field of electronic cashing. The method comprises the following steps: acquiring a first image and the weight of the commodity to be settled on a settlement table, and determining a commodity identification result of the commodity to be settled according to the first image; if the commodity identification result corresponds to at least two commodities, determining the placement position of the object to be settled on the checkout stand according to the first image, and determining target shooting parameters according to the placement position; acquiring a second image which is secondarily shot on the basis of the target shooting parameters and used for settling the commodity, and determining target commodities in the commodities corresponding to the commodity identification result according to the second image; and calling the commodity price of the target commodity, and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight. The method achieves the effects of automatically identifying the commodity to be settled and improving the accuracy of calculating the settlement amount of the commodity.

Description

Self-service settlement method, system, electronic equipment and medium for shopping checkout

Technical Field

The application relates to the technical field of electronic cashing, in particular to a self-service settlement method, a self-service settlement system, electronic equipment and a self-service settlement medium for shopping checkout.

Background

With the development of technology and the change of consumer shopping habits, the conventional checkout mode of cashing by cashiers is slow and inefficient. In recent years, self-checkout systems have come into use in supermarkets and convenience stores, and such systems can automatically identify each commodity, acquire weight information, and quickly calculate prices. The system directly performs unified identification and weighing on all commodities placed in a checkout area so as to realize quick and accurate checkout and improve shopping experience.

Currently, in the existing self-service settlement method, an integral image of a fixed area of a commodity to be settled is photographed, the commodity is determined according to the integral image of the fixed area, and weighing is performed, so that the settlement amount is calculated. However, in an actual use scene, consumers often stack commodities at different positions of a checkout stand, shoot an overall image of a fixed area of the commodity to be settled, and often have the situations that shooting angles are too deviated or exceed a shooting range, so that errors occur when commodity identification is performed through the shot image, and the calculated commodity settlement amount is inaccurate.

Disclosure of Invention

The application provides a self-service settlement method, a self-service settlement system, electronic equipment and a medium for shopping checkout, which have the effects of automatically identifying commodities to be settled and improving the accuracy of calculating the settlement amount of the commodities.

In a first aspect, the present application provides a self-checkout method for shopping checkout, comprising:

acquiring a first image and commodity weight of a commodity to be settled on a settlement table, and determining a commodity identification result of the commodity to be settled according to the first image;

If the commodity identification result corresponds to at least two commodities, determining the placement position of the object to be settled on the checkout stand according to the first image, and determining target shooting parameters according to the placement position;

Acquiring a second image which is secondarily shot on the commodity to be settled based on the target shooting parameters, and determining target commodities in the commodities corresponding to the commodity identification result according to the second image;

And calling the commodity price of the target commodity, and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight.

By adopting the technical scheme, in the checkout process, the whole commodity image is utilized for quick preliminary identification, and the possible commodity category is determined. When the identification result is not unique, the system can intelligently judge the placement position of the commodity, and determine relevant parameters required by customizing and optimizing the re-shooting of the target commodity according to the placement position. After the clear local image after the re-shooting is obtained, the system calculates the similarity between different features by analyzing and extracting visual features such as local contours of the commodity and finally locks the target commodity. After the target commodity is successfully identified, the system can automatically call the price information of the commodity and calculate the settlement amount by combining the weighing weight. The whole process realizes full automation from quick positioning to accurate identification to automatic settlement, greatly simplifies the settlement link, reduces manual operation, provides more intelligent payment experience for users, and improves the settlement efficiency and accuracy compared with the traditional settlement mode.

Optionally, determining the overall shape and the overall color of the commodity to be settled according to the overall image; the overall shape and the overall color of the commodity to be settled are used as combined characteristics and are compared with standard characteristics of the inventory commodity, and the matching degree of the combined characteristics and the standard characteristics is obtained; and determining at least one paired commodity in each inventory commodity according to the matching degree, and taking the paired commodity as the commodity identification result.

By adopting the technical scheme, on the basis of obtaining the whole commodity image, the image is further processed and analyzed, the whole shape and the whole color visual characteristics of the commodity are extracted, and the whole shape and the whole color visual characteristics are taken as combined characteristics. Compared with the direct image recognition, the combined features can more fully describe the whole visual information of the commodity, comprise double discrimination of shape outline and color pattern, and are beneficial to the accuracy of subsequent recognition. And then, the system carries out high-efficiency matching on the dynamic captured commodity combination characteristics and standard characteristics of a commodity library, and can effectively reduce the identification range and exclude a large number of irrelevant commodities by calculating the matching degree between the characteristics, thereby quickly determining the most probable paired commodities. The added image processing and combined feature extraction steps form a more targeted identification flow, so that the accuracy of commodity identification by a checkout system is improved.

Optionally, if the commodity identification result corresponds to one commodity, the commodity is used as the target commodity, and the settlement amount of the commodity to be settled is determined according to the commodity price and the commodity weight of the target commodity.

By adopting the technical scheme, when the unique target commodity can be determined through primary identification, the subsequent complex judgment flow is skipped, the identification result is directly utilized, the whole checkout process is simplified, and the efficiency is improved. The system can quickly inquire the price information of the commodity, and directly calculates the weight value obtained by weighing to obtain the settlement amount. Repeated feature extraction and comparison and equivalent operation are avoided, and the calculated amount is reduced. The whole step only involves price inquiry and simple arithmetic calculation, and automatic checkout is directly completed. The alternative scheme optimizes and simplifies the flow aiming at the condition of unique identification result, and realizes high-efficiency automatic settlement.

Optionally, acquiring a shooting position of a camera on the checkout stand and a commodity size of the commodity to be settled, and determining a relative angle between the camera and the commodity to be settled according to the shooting position and the placing position; determining a horizontal adjustment angle and a vertical adjustment angle of the camera according to the relative angle; determining the relative distance between the commodity to be settled and the camera according to the placement position and the camera shooting position; determining a shooting focal length according to the relative distance and the commodity size; and taking the horizontal adjustment angle, the vertical adjustment angle and the shooting focal length of the camera as the target shooting parameters.

By adopting the technical scheme, the camera position information is intelligently acquired, and relative angle parameters between the camera position information and the commodity placement position are calculated by combining commodity placement position analysis, so that the horizontal and vertical adjustment angles of the camera are determined, and the shooting angle is reasonably set. The system can acquire the commodity size data, calculate the relative distance between the commodity and the camera, and set a proper focal length parameter according to the relative distance. The system can obtain a group of customized parameters including shooting angles and focal distances, and can carry out the shooting of the key areas after accurate planning. According to the scheme, the influence of key factors such as position, size and the like on parameter optimization is considered, and clear and complete commodity images are obtained through comprehensive analysis of all the elements, so that the accuracy of subsequent identification is improved. The scheme provides a set of feasible intelligent parameter optimization scheme for commodity customization key shooting, is suitable for different scenes, and enhances the intelligent coordination capacity of a checkout system.

Optionally, extracting local contour features of the commodity to be settled according to the local image; acquiring outline features of the commodities corresponding to the commodity identification result; and determining the corresponding target commodity in each commodity according to the local contour features of the commodity to be settled and the contour features of each commodity.

By adopting the technical scheme, the outline features of the key areas of the commodities to be settled in the local images are extracted, and the standard outline features of each candidate commodity in the primary identification result are obtained. The system calculates the similarity between the local contour features of the commodity to be settled and the standard features of each candidate commodity, and determines the best matched target commodity. The local outline features can effectively represent the key boundary information of the commodity, and are visual features with rich information and easy extraction. By matching with the standard feature library, the method can realize effective recognition from the primary recognition range to the specific commodity. Compared with the whole feature, the local feature contains finer target boundary details, which is beneficial to improving the recognition accuracy.

Optionally, if the to-be-settled commodity is detected to include at least two different commodities, the weighing operation of the to-be-settled commodity is stopped, and error prompt information is generated on a display screen of the settlement table.

By adopting the technical scheme, the mixture of different commodities on the checkout stand is detected through image analysis, the system can quickly stop the subsequent weighing operation of the commodity batch, errors are avoided when the weight is acquired, prompting abnormal information is generated on the display screen interface of the checkout stand, and a user is informed of the failure of checkout at the time and needs to operate again. The processing means for stopping weighing in time and prompting the user can effectively prevent the user from mistaking that the batch of mixed commodities is normally finished for checkout. When unexpected abnormal conditions such as commodity mixing and the like occur, the scheme can realize rapid stopping and prompt notification of the flow through intelligent detection and friendly interaction, and the fault tolerance, user experience and safety and reliability of the checkout system are enhanced.

Optionally, if the target commodity is a piece counting commodity with a fixed unit price, determining the commodity number of the piece counting commodity according to the first image and the second image; and determining the commodity total price of the counted commodity according to the commodity number and the unit price of the counted commodity.

By adopting the technical scheme, when the target commodity is identified as belonging to the commodity counting commodity, the system can automatically utilize the commodity quantity information in the obtained first image and second image to accurately judge the total quantity of the commodity counting commodity. After counting the quantity, inquiring the unit price of the commodity, and directly multiplying the unit price with the quantity obtained by recognition according to the prestored unit price, so that the total price of the commodity in the batch is rapidly determined. The checkout mode avoids the trouble of accurately weighing pieces one by one, greatly improves the settlement efficiency of the counted goods, expands the applicable commodity category range of the system, and is not limited to weight commodities. The scheme realizes effective support of the checkout scene containing the checked-out commodity, provides a new technical realization way for the checkout system, and has more universality in a settlement mode.

In a second aspect of the application, a self-checkout system for shopping checkout is provided.

The image identification module is used for acquiring a first image and the weight of the commodity to be settled on the settlement table, and determining the commodity identification result of the commodity to be settled according to the first image;

The parameter adjusting module is used for determining the placement position of the object to be settled on the checkout stand according to the first image if the commodity identification result corresponds to at least two commodities, and determining target shooting parameters according to the placement position;

the commodity identification module is used for acquiring a second image which is shot for the commodity to be settled for the second time based on the target shooting parameters, and determining target commodities in the commodities corresponding to the commodity identification result according to the second image;

and the amount calculation module is used for calling the commodity price of the target commodity and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight.

In a third aspect of the application, an electronic device is provided.

A self-service settlement system for shopping checkout comprises a memory, a processor and a program stored in the memory and capable of running on the processor, wherein the program can realize a self-service settlement method for shopping checkout when loaded and executed by the processor.

In a fourth aspect of the application, a computer readable storage medium is provided.

A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to implement a self-checkout method of shopping checkout.

In summary, one or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:

1. The application determines the possible commodity category by utilizing the whole commodity image to carry out quick preliminary identification in the checkout process. When the identification result is not unique, the system can intelligently judge the placement position of the commodity, and determine relevant parameters required by customizing and optimizing the re-shooting of the target commodity according to the placement position. After the clear local image after the re-shooting is obtained, the system calculates the similarity between different features by analyzing and extracting visual features such as local contours of the commodity and finally locks the target commodity. After the target commodity is successfully identified, the system can automatically call the price information of the commodity and calculate the settlement amount by combining the weighing weight. The whole process realizes full automation from quick positioning to accurate identification to automatic settlement, greatly simplifies the settlement link, reduces manual operation, provides more intelligent payment experience for users, and improves the settlement efficiency and accuracy compared with the traditional settlement mode.

2. According to the method, the outline features of the key areas of the commodities to be settled in the local images are extracted, and the standard outline features of each candidate commodity in the primary identification result are obtained. The system calculates the similarity between the local contour features of the commodity to be settled and the standard features of each candidate commodity, and determines the best matched target commodity. The local outline features can effectively represent the key boundary information of the commodity, and are visual features with rich information and easy extraction. By matching with the standard feature library, the method can realize effective recognition from the primary recognition range to the specific commodity. Compared with the whole feature, the local feature contains finer target boundary details, which is beneficial to improving the recognition accuracy.

3. According to the application, when the target commodity is identified as the commodity counting commodity, the system can automatically utilize the commodity quantity information in the obtained first image and the second image to accurately judge the total quantity of the commodity counting commodity. After counting the quantity, inquiring the unit price of the commodity, and directly multiplying the unit price with the quantity obtained by recognition according to the prestored unit price, so that the total price of the commodity in the batch is rapidly determined. The checkout mode avoids the trouble of accurately weighing pieces one by one, and greatly improves the efficiency of counting commodity settlement.

Drawings

FIG. 1 is a schematic flow chart of a self-service settlement method for shopping checkout provided by an embodiment of the application;

FIG. 2 is a schematic diagram of a self-checkout system for shopping checkout according to an embodiment of the present application;

Fig. 3 is a schematic structural diagram of an electronic device according to the disclosure.

Reference numerals illustrate: 300. an electronic device; 301. a processor; 302. a communication bus; 303. a user interface; 304. a network interface; 305. a memory.

Detailed Description

In order that those skilled in the art will better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments.

In describing embodiments of the present application, words such as "for example" or "for example" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "such as" or "for example" in embodiments of the application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "or" for example "is intended to present related concepts in a concrete fashion.

In the description of embodiments of the application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating an indicated technical feature. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In order to facilitate understanding of the method and system provided by the embodiments of the present application, a description of the background of the embodiments of the present application is provided before the description of the embodiments of the present application.

The embodiment of the application discloses a self-service settlement method for shopping checkout, which is characterized in that an image of a commodity to be settled on a checkout stand is identified, when a plurality of matching objects appear in an identification result, shooting parameters are adjusted according to a placement position, a local image is obtained, then the target commodity is determined, and the settlement amount is calculated according to the commodity weight of the commodity to be settled. The method is mainly used for solving the problem that the settlement amount is calculated inaccurately due to errors in commodity identification.

Those skilled in the art will appreciate that the problems associated with the prior art are solved by the present application, and a detailed description of a technical solution according to an embodiment of the present application is provided below, wherein the detailed description is given with reference to the accompanying drawings.

Referring to fig. 1, a self-settlement method of shopping checkout includes S10 to S40, specifically including the steps of:

S10: acquiring a first image and the weight of the commodity to be settled on a settlement table, and determining the commodity identification result of the commodity to be settled according to the first image.

The first image refers to an image obtained by integrally shooting the commodity to be settled through the camera.

Specifically, when a consumer places a purchased commodity on a checkout stand, the system controls the camera to capture an overall image of the commodity to be settled as a first image. The first image is taken because the overall information of the commodity to be settled, such as the number, shape, color, size and other characteristics, can be quickly acquired through the overall image, which provides an effective reference for subsequent commodity identification. Meanwhile, the system can start the weighing scale to stably weigh the commodity to be settled, and weight information of the commodity is obtained. After the first image and weight information are obtained, the system inputs the first image into the merchandise identification model for processing. The commodity identification model extracts the overall color, shape and other characteristics of the first image, compares the overall color, shape and other characteristics with standard characteristics in a commodity library, calculates the matching degree, and determines a possible commodity identification result according to the matching degree. The first image is processed through the commodity identification model, so that the image identification can quickly acquire the relevant characteristics of the commodity, and the system is helped to accurately judge the type of the commodity to be settled. And determining the commodity type possibly corresponding to the commodity to be settled in the first image according to the identification result output by the commodity identification model. Compared with direct manual commodity identification, the automatic and efficient commodity classification can be realized through the identification model, human errors are avoided, and self-service checkout efficiency is improved.

On the basis of the above embodiment, the specific step of determining the commodity identification result further includes S11 to S14:

S11: and determining the overall shape and the overall color of the commodity to be settled according to the overall image.

For example, after the first image is obtained, the overall characteristic information of the commodity to be settled needs to be obtained according to the overall image so as to realize accurate identification of the commodity. The overall shape and the overall color characteristics are obtained, and the overall shape and the overall color characteristics can effectively distinguish different commodities and are not influenced by shooting angle and position variation. Specifically, an image processing algorithm is used on the first image to extract the outline of the commodity to be settled, and the overall shape of the commodity to be settled is obtained. The overall shape reflects the appearance characteristics of the commodity, such as length, width, height, curved surface characteristics and the like. A color histogram of the first image is calculated to represent the dominant color distribution, i.e., the overall color, of the item to be settled. After the overall shape and the overall color characteristics are obtained, the system compares the overall shape and the overall color characteristics with the characteristics of standard commodity pictures in a commodity library. And calculating the correlation degree of the overall shape outline and the color histogram to obtain the preliminary matching degree. According to the matching degree result, the commodity type possibly corresponding to the commodity to be settled in the first image can be determined. By extracting the whole shape and the whole color as the identification features, the whole information of the image can be effectively utilized for preliminary filtration, the range of possible commodities is reduced, and compared with the direct use of the detail features, the whole features can rapidly and effectively complete preliminary identification, provide references for subsequent accurate identification, reduce the calculated amount and improve the system efficiency.

S12: and comparing the overall shape and the overall color of the commodity to be settled with the standard characteristics of the inventory commodity to obtain the matching degree of the combined characteristics and the standard characteristics.

The matching degree refers to the similarity degree between the characteristics of the commodity to be settled and the standard characteristics of the inventory commodity. The degree of matching is represented by calculating the correlation between feature vectors, the more similar the feature vectors are, the higher the degree of matching is.

For example, after the overall shape and overall color characteristics of the commodity to be settled are obtained, in order to realize accurate commodity identification, the two types of characteristics need to be combined into a comprehensive characteristic description, and compared with standard characteristics in a commodity library, and a plurality of characteristics can be combined to provide more comprehensive and unique commodity description information, so that the system can find the best matched commodity. Specifically, the extracted overall shape and the overall color feature vector are spliced into a combined feature description. The combined feature contains both geometric information of the shape outline and visual information of the color histogram. And taking out standard features of all commodities in the commodity library one by one, including standard shapes and color features of commodity pictures, and sequentially calculating the correlation between the combined features and each standard feature vector to obtain a matching degree array, wherein the matching degree array reflects the similarity of the commodities to be settled and each inventory commodity on the combined features. According to the sorting of the matching degree values from large to small, the top N commodities with the highest matching degree are used as possible recognition result candidates, so that the recognition accuracy can be improved, because the combined features provide richer descriptions, and similar commodities have high matching degree. The commodity is identified by constructing the combination characteristics, namely, the information of the shape and the color is fully utilized, so that the system can more accurately judge the commodity to be settled, lay a foundation for the follow-up accurate identification, and improve the intelligent level of self-service checkout.

S13: and determining at least one paired commodity in each inventory commodity according to the matching degree, and taking the paired commodity as a commodity identification result.

Illustratively, a matching degree threshold is preset, and the matching degree threshold is obtained according to historical commodity matching data. Screening all inventory commodities with the matching degree larger than the threshold value, wherein the filtered commodities have a certain probability of being commodities to be settled. The system selects the top-ranked products, i.e. the products with the highest matching degree, as possible recognition result candidates according to the matching degree ordering of the products. The first few high-match products were taken because the products with the highest match were often the actual products to be settled, while the next highest match also represented more similar products. The system returns the possible identification result commodities to finish the primary identification. Through the matching degree-based identification process, the whole shape and color characteristics can be effectively utilized, the commodity range is rapidly reduced, and the most probable commodity results are output. Compared with direct integral image recognition, the accuracy is greatly improved. Returning multiple options may avoid preference errors as opposed to taking a single highest item. The step utilizes the matching degree to realize accurate and reliable preliminary automatic commodity identification and provides reference for subsequent identification.

S14: if the commodity identification result corresponds to one commodity, the commodity is taken as a target commodity, and the settlement amount of the commodity to be settled is determined according to the commodity price and the commodity weight of the target commodity.

Illustratively, it is determined whether the number of recognition result items is equal to 1. If equal to 1, the single commodity is directly taken out as the target commodity. The system retrieves the unit price information of the target commodity from the commodity database, and simultaneously retrieves the obtained actual weight of the commodity, and calculates the settlement amount of the commodity. Through the process, when the identification result is unique, the system can skip the steps of candidate sorting, re-identification and the like, directly and accurately settle accounts, simplify the flow and improve the efficiency. Meanwhile, the function of overall feature recognition is exerted to the greatest extent, and quick automatic self-service checkout is realized.

In an alternative embodiment of the present application, there is also a process for detecting the type of goods to be settled on the checkout stand, specifically including: in determining the goods to be settled through image recognition, the system needs to determine whether the goods placed on the checkout stand are the same type of goods. If different kinds of groceries are detected to be mixed together, settlement cannot be completed correctly. At this time, the operation needs to be stopped in time and a prompt is given, so that errors are avoided. Specifically, after the local features of the commodity are obtained, information such as color, texture and the like of each region is extracted and compared with standard features. If there is a significant difference in the characteristics of the different regions, the system will determine that different items are involved. When detecting that two or more than two kinds of goods exist in the sundries to be settled, the system can send a control instruction to the weighing module, immediately stop the weighing operation of the current sundries, and avoid unreasonable pricing. And driving the checkout screen to prompt text information. By stopping weighing and giving prompts in time, mixed pricing errors of different commodities can be avoided, checkout accidents are reduced, clear error correction guidance is provided for users, and intellectualization and friendliness of checkout are improved. The step realizes the mixed detection and corresponding control of commodity categories, so that the intelligent checkout machine can automatically monitor and avoid the occurrence of identification errors, and the confusion of commodities to be settled is prevented.

S20: if the commodity identification result corresponds to at least two commodities, the placement position of the object to be settled on the checkout stand is determined according to the first image, and the target shooting parameters are determined according to the placement position.

Specifically, when the primary identification result outputs a plurality of possible commodities, it means that the determination cannot be performed by means of the overall characteristics of the first image, and more specific identification information needs to be acquired to accurately identify the commodities. The reason for determining the placement position and redetermining the shooting parameters is that by analyzing the specific positions of the commodities, targeted image shooting can be carried out, and a clearer identified picture can be obtained. Specifically, a specific bounding box of each item to be settled on the checkout stand is determined in the first image using image segmentation and object detection algorithms. The size and center coordinates of each commodity, i.e., the placement position, can be obtained. The system recalculates an optimal shooting parameter combination capable of clearly capturing the visual angle and the characteristics of each commodity as a target parameter according to the camera parameters, the light conditions and the position information of each commodity. After the target parameters are obtained, the parameters such as the position, focal length, aperture and the like of the camera are customized and adjusted according to each commodity, and local key shooting is carried out. Compared with the whole shooting, the customized shooting combined with the position information can greatly improve the definition and the recognition degree of the local commodity image. The method fully utilizes the position information provided by the first image, and achieves targeted re-shooting of commodity images by intelligently determining target shooting parameters, thereby providing important support for improving the recognition accuracy subsequently.

On the basis of the above embodiment, the specific steps of determining the target imaging parameter further include S21 to S23:

S21: the method comprises the steps of acquiring a shooting position of a camera on a checkout stand and the commodity size of the commodity to be settled, and determining the relative angle of the camera and the commodity to be settled according to the shooting position and the placing position.

Illustratively, the position coordinates, attitude angle, and parameters such as focal length of the camera mounted on the checkout stand are recorded in advance. These pieces of information constitute imaging position parameters. The system already judges the size and the dimension of each commodity in the first image, combines the camera coordinates and the commodity coordinates, and calculates and solves the included angle between the connecting line of the camera and the center of each commodity and the horizontal direction, namely the pitch angle by utilizing the geometric relation. And further, the camera needs to adjust the horizontal and vertical angles of shooting aiming at the commodity, so that the optimization of shooting visual angles is ensured. Through the step, the system can fully utilize the known conditions, and the relative angle parameter between the camera and the commodity is obtained by rapid calculation only according to the spatial position relation of the camera and the commodity, thereby playing an important role in the follow-up intelligent determination of the optimal shooting frame according to the position and the size of the commodity. The implementation of the step improves the automation level of the system and also enables the re-shooting to be more accurate and efficient.

S22: and determining the horizontal adjustment angle and the vertical adjustment angle of the camera according to the relative angles.

For example, after obtaining the relative angle of the camera and each commodity, the angle to be adjusted by the camera needs to be calculated according to the angle parameter, so as to ensure the optimization of the shooting angle. The adjustment angles in the horizontal and vertical directions are determined, respectively, because the angles in both directions affect the final photographing effect. Specifically, according to the pitch angle of the connecting line of the camera and the commodity center, which is calculated previously, the depression angle or the elevation angle is judged, and the angle value is calculated. On the premise of not changing the relative positions of the camera and the commodity, the horizontal angle and the vertical angle which enable the camera to rotate relative to the reference position are determined according to the angle value. For example, if the depression angle is 30 degrees, the camera needs to be rotated downward by 30 degrees. After the angle values in the horizontal direction and the vertical direction are obtained, the system can control the cradle head of the camera to dynamically rotate at corresponding angles, so that the angle of a shooting visual angle coincides with the connecting line of the commodity center, and the intelligent adjustment of the angle of the camera according to the commodity position is realized. Through the step, the commodity can be aligned to carry out local shooting only through the rotation angle under the condition of not changing other parameters. The flexible view finding based on the dynamic angle adjustment of the position information can greatly improve the image recognition effect and enable the follow-up re-shooting to be more accurate.

S23: and determining the relative distance between the commodity to be settled and the camera according to the placement position and the camera shooting position.

For example, in order to take a local and accurate image of each commodity, besides determining the angle, the distance between the commodity and the camera needs to be determined, so as to adjust the focal length parameter of the camera. The system calculates the exact relative distance from the positions of the two. Specifically, the system has acquired the placement coordinates of each commodity and the mounting coordinates of the camera. Based on the two position information, the system can solve the relative distance between the camera and each commodity center by using a triangle geometric relationship and a formula for calculating the linear distance between the two points. After the distance parameters of the commodities and the cameras are obtained, the focal length of the cameras can be dynamically adjusted accordingly in the follow-up process, and clear focusing of images is achieved.

S24: determining a shooting focal length according to the relative distance and the commodity size; and taking the horizontal adjustment angle, the vertical adjustment angle and the shooting focal length of the camera as target shooting parameters.

For example, after obtaining the distance parameter between the commodity and the camera, the system needs to calculate and determine a suitable camera focal length according to the distance parameter to optimize the shooting definition. And forming a final target shooting parameter combination according to the obtained angle parameter and the focal length parameter, and providing the optimal configuration for the re-shooting. Specifically, a reasonable focal length range is estimated according to the distance between the camera and the commodity size. And adjusting the focal length in the range for trial shooting, and selecting a focal length value for obtaining a clear image. For example, a short focal length is used for small close range items and a long focal length is used for large far range items. After determining the focal length, the system combines the adjusted horizontal angle, vertical angle, and calculated focal length and saves as a target parameter combination for the article. The process is repeated for all commodities to be identified, after target parameters of all the commodities are obtained, the system can dynamically adjust the camera according to the parameters when the system shoots again, and therefore local optimization shooting of each commodity is achieved. The multi-parameter combination of the precise focal length and the precise angle can obtain clear commodity images with high identification degree, and greatly improve the accuracy of subsequent identification.

S30: and acquiring a second image which is secondarily shot on the commodity to be settled based on the target shooting parameters, and determining target commodities in the commodities corresponding to the commodity identification result according to the second image.

The second image is an image obtained by locally shooting the commodity to be settled through the camera after the parameters of the camera are adjusted. The second image has smaller view range and only contains a local area of one commodity, and the resolution of the second image is higher, so that the detail characteristics of the single commodity are highlighted.

Specifically, after determining the target imaging parameters for each commodity, it is necessary to re-capture based on these optimized parameters to acquire a clearer image for precise identification. The secondary photographing is performed because the image quality and recognition effect can be greatly improved by customizing the parameters. Specifically, the position, angle and focal length of the camera are sequentially adjusted to match the target parameter combinations of each commodity. And after the parameters are adjusted in place, controlling the camera to re-shoot the commodity, and obtaining a clear image containing the main visual angle and the characteristics of the commodity as a second image. After obtaining the second image of all the products, the system extracts, for each product, the local features of the image, such as contours, textures, etc. And comparing the standard features of the possible identification results of the commodity with the standard features of the possible identification results of the commodity, and calculating the matching degree. And selecting the commodity with the highest matching degree with the second image as a final recognition result, namely the target commodity by the system. Through the identification flow based on the re-shooting, clear local images depicting commodity details can be obtained, so that visual features are accurately extracted, and accurate identification is realized. The local features of the second image may more accurately determine the specific merchandise than the global features of the first image. The step realizes accurate and efficient identification based on optimization parameter duplicate shooting.

On the basis of the above embodiment, the specific step of determining the target commodity further includes S31 to S32:

S31: and extracting the local contour features of the commodity to be settled according to the local image.

Illustratively, after a clear partial image is acquired after re-shooting, commodity features need to be further extracted based on the image to achieve accurate identification. Wherein the local profile feature is one of the important image features. The local outline features are extracted because the local outline features can effectively describe the local outline structure of the commodity, and the system is convenient to judge specific commodities. Specifically, the second commodity partial image is preprocessed, and the method comprises the steps of filtering, enhancing and the like. The edge detection operator, such as a canny operator, is used for detecting the edges of the commodity, and the overall outline of the commodity can be extracted according to the edge information. Contour features, such as inflection points of a logo contour, are extracted based on the contour to digitally describe the contour shape. These local profile features retain both shape information and have some dimensional, rotational invariance. After the local outline features of the digital expression are obtained, the system can compare the local outline features with the standard outlines of all possible commodities to judge the optimal matching degree, so as to determine the identification result. The method effectively acquires shape distinguishing information by extracting the local contour features, so that the commodity identification and distinguishing are more accurate, and the purpose of accurate checkout is achieved.

S32: acquiring outline features of each commodity corresponding to a commodity identification result; and determining corresponding target commodities in the commodities according to the local contour features of the to-be-settled objects and the contour features of the commodities.

For example, after extracting the local profile features of the commodity to be settled, the specific commodity needs to be accurately compared and determined. The system obtains the outline characteristics of the possible commodity result set and determines the target commodity with the best matching through comparison. Specifically, the standard contour features of each candidate commodity in the primary identification result are called from the commodity feature database. These standard profile features constitute a library of profile features for the candidate commodity set. And calculating the similarity or the correlation between the local contour features of the commodity to be settled and the contour features of each candidate commodity one by one. The similarity may be calculated by calculating a distance, such as a Euclidean distance, between the local outline vector of the item to be settled and each candidate item outline vector, the smaller the distance, the higher the similarity. After calculating the similarity of the outline features of each commodity in the candidate set and the commodity to be settled, the system can select the commodity which is the most matched with the local outline of the commodity to be settled, namely the commodity with the highest similarity, as the identification result, namely the target commodity.

S40: and calling the commodity price of the target commodity, and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight.

Specifically, after successfully identifying the target commodity of the commodity to be settled, the final settlement amount is required to be calculated according to the price information and the weight of the commodity. Specifically, the system searches the unit price information of the commodity, including the unit price, whether there is a member discount, etc., from the commodity database after confirming the target commodity, which constitutes commodity price data of the target commodity. And reading the weight value of the weighed target commodity, directly multiplying the unit price of the target commodity by the weight, and calculating the total price of the commodity, namely the final settlement amount of the commodity to be settled. Through the operation, the system can automatically complete calculation and determination of the amount of the commodity to be settled. After the final settlement amount is obtained, the system can control the interactive screen to display the result, and the amount is broadcasted through the voice module, so that the whole self-service settlement process is completed. The method realizes the effective combination of commodity identification and price information, so that the system can automatically calculate the settlement amount, greatly reduces the operation complexity and is convenient for shoppers.

On the basis of the above embodiment, there is also a process of detecting commodity pricing patterns, and the specific steps further include S41 to S42:

s41: and if the target commodity is a counting commodity with fixed unit price, determining the commodity number of the counting commodity according to the first image and the second image.

For example, when the identified target commodity belongs to a commodity whose price is calculated by piece, it is necessary to determine the number thereof in addition to the unit price. The first image and the second image are utilized because the overall positioning and the partial recognition are required to comprehensively judge the quantity. Specifically, the system may first determine the number of items on the first item's overall image using the target detection algorithm, which may yield an approximate number estimate. The boundary of the single commodity is determined in the second partial image by using a more accurate segmentation and identification algorithm, and the statistical quantity is further counted, so that the deviation of the judgment of the first image can be corrected. The system integrates the first image detection quantity and the second image identification quantity to be used as the final counted commodity quantity. By combining the global positioning and the local recognition of the image, the system can automatically analyze and judge the number of the counting commodities on the checkout stand. This avoids errors caused by manual statistics, greatly improving the checkout efficiency and accuracy.

S42: and determining the commodity total price of the counted commodity according to the commodity number and the unit price of the counted commodity.

For example, after successfully identifying the number of the counted goods, the total price of the goods is calculated to complete the settlement. The total price is calculated based on the quantity and the unit price because the price of the piece goods is calculated based on the number of pieces. Specifically, the system will first find the unit price data of the target piece goods from the goods database. The system will take out the number of products identified in the previous step. The unit price is multiplied by the quantity to obtain the total price of the commodity. Through the operation, the automatic calculation of the total price of the counted commodity is realized, and after the total price of the commodity is obtained, the system can be directly used as the settlement amount to finish the self-service checkout flow. The settlement capability according to the number of pieces is built in the system, manual operation is avoided, the intelligent synergistic effect is enhanced, better shopping experience is brought to the user, and meanwhile the problem of calculation errors of settlement amount caused by incorrect placement of commodities to be settled by a shopper is also avoided.

In another alternative embodiment of the present application, there is also a process of checking out a statement, specifically including: clear options and instructions are provided on the user interface of the self-service checkout stand to instruct the customer how to conduct the order checkout. An explicit "order checkout" or "pay separately" button is provided, which the customer can select before starting the scan or at any time during the checkout process. The customer may divide the items to be settled into different groups, each of which will be individually checked out, and the system adds them to different "shopping baskets" or "bills" depending on the items to be settled that the customer selects. When the grouping of the goods to be settled is completed, the customer can respectively settle the goods to be settled for each group. The system should process each separate bill one by one and the customer selects a different payment method for each bill. The customer may choose a different payment means, such as cash, credit card, debit card, mobile payment, or gift card, for each separate bill. After each bill payment is completed, confirmation of completion of the transaction is provided and a corresponding receipt is printed. If a problem is encountered during the order checkout process, a one-touch help button should be provided so that the customer can request assistance from the store clerk. The payment information of the customer is ensured to be kept safe in the process of bill settlement, and the information is prevented from being leaked or incorrectly processed.

In yet another alternative embodiment of the present application, there is also detection of suspicious checkout activity, the specific process comprising: when a customer puts the commodity to be settled on a self-service checkout stand, firstly, the image of the commodity to be settled is identified through a camera, and identification information of the commodity is obtained. And searching the known accurate weight of the commodity in the commodity information base through the identified commodity information. If the commodity is not recorded with weight in the commodity library, the system records the measured weight and the measured times thereof into the commodity information library. The number of measurements of the commodity increases each time the commodity is weighed. When the number of measurements reaches a preset number, the system will consider the current measured weight as the exact weight of the commodity and update it in the commodity information base. For a good of known exact weight, the system will compare the weighed metered weight to the exact weight recorded in the good information base. If the measured weight does not match the exact weight, the system will mark the merchandise on the transaction list for suspicious checkout, prompting the staff for subsequent inspection.

Referring to fig. 2, a self-service settlement system for shopping checkout according to an embodiment of the present application includes: the system comprises an image recognition module, a parameter adjustment module, a commodity recognition module and an amount calculation module, wherein:

The image identification module is used for acquiring a first image and the weight of the commodity to be settled on the settlement table and determining the commodity identification result of the commodity to be settled according to the first image;

The commodity identification module is used for acquiring a second image which is shot for the second time for the commodity to be settled based on the target shooting parameters, and determining target commodities in the commodities corresponding to the commodity identification result according to the second image;

On the basis of the embodiment, the image recognition module is further used for determining the overall shape and the overall color of the commodity to be settled according to the overall image; the overall shape and the overall color of the commodity to be settled are used as combined characteristics and are compared with standard characteristics of the inventory commodity to obtain the matching degree of the combined characteristics and the standard characteristics; and determining at least one paired commodity in each inventory commodity according to the matching degree, and taking the paired commodity as a commodity identification result.

On the basis of the embodiment, the image recognition module further includes determining a settlement amount of the commodity to be settled according to the commodity price and the commodity weight of the target commodity by taking the commodity as the target commodity if the commodity recognition result corresponds to one commodity.

On the basis of the embodiment, the parameter adjusting module is further used for acquiring the shooting position of the camera on the checkout stand and the commodity size of the commodity to be settled, and determining the relative angle between the camera and the commodity to be settled according to the shooting position and the placing position; determining a horizontal adjustment angle and a vertical adjustment angle of the camera according to the relative angles; determining the relative distance between the commodity to be settled and the camera according to the placement position and the camera shooting position; determining a shooting focal length according to the relative distance and the commodity size; and taking the horizontal adjustment angle, the vertical adjustment angle and the shooting focal length of the camera as target shooting parameters.

On the basis of the embodiment, the commodity identification module is further used for extracting local contour features of the commodity to be settled according to the local image; acquiring outline features of each commodity corresponding to a commodity identification result; and determining corresponding target commodities in the commodities according to the local contour features of the to-be-settled objects and the contour features of the commodities.

On the basis of the embodiment, the image recognition module further comprises stopping weighing operation of the commodity to be settled and generating error prompt information on a display screen of the checkout stand if the commodity to be settled is detected to comprise at least two different commodities.

On the basis of the embodiment, the money calculating module is further configured to determine the number of the items of the item of sale according to the first image and the second image if the target item of sale is a monovalent fixed item of sale; and determining the commodity total price of the counted commodity according to the commodity number and the unit price of the counted commodity.

It should be noted that: in the device provided in the above embodiment, when implementing the functions thereof, only the division of the above functional modules is used as an example, in practical application, the above functional allocation may be implemented by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the embodiments of the apparatus and the method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the embodiments of the method are detailed in the method embodiments, which are not repeated herein.

The application also discloses electronic equipment. Referring to fig. 3, fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, at least one communication bus 302.

Wherein the communication bus 302 is used to enable connected communication between these components.

The user interface 303 may include a Display screen (Display) interface and a Camera (Camera) interface, and the optional user interface 303 may further include a standard wired interface and a standard wireless interface.

The network interface 304 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Wherein the processor 301 may include one or more processing cores. The processor 301 utilizes various interfaces and lines to connect various portions of the overall server, perform various functions of the server and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and invoking data stored in the memory 305. Alternatively, the processor 301 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 301 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface diagram, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 301 and may be implemented by a single chip.

The Memory 305 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 305 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 305 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc.; the storage data area may store data or the like involved in the above respective method embodiments. Memory 305 may also optionally be at least one storage device located remotely from the aforementioned processor 301. Referring to fig. 3, an operating system, a network communication module, a user interface module, and an application program of a self-settlement method for shopping check-out may be included in the memory 305 as a computer storage medium.

In the electronic device 300 shown in fig. 3, the user interface 303 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 301 may be configured to invoke an application program in the memory 305 that stores a self-checkout method of shopping checkout, which when executed by the one or more processors 301, causes the electronic device 300 to perform the method as in one or more of the embodiments described above. It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all of the preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. And the aforementioned memory includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a magnetic disk or an optical disk.

The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure.

This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.

Claims

1. A self-checkout method for shopping checkout, comprising:

the commodity price of the target commodity is called, and the settlement amount of the commodity to be settled is determined according to the commodity price and the commodity weight;

the first image is an overall image of the commodity to be settled, and the determining the commodity identification result of the commodity to be settled according to the first image includes:

determining the overall shape and the overall color of the commodity to be settled according to the overall image;

The overall shape and the overall color of the commodity to be settled are used as combined characteristics and are compared with standard characteristics of the inventory commodity, and the matching degree of the combined characteristics and the standard characteristics is obtained;

determining at least one paired commodity in each inventory commodity according to the matching degree, and taking the paired commodity as the commodity identification result;

The second image is a partial image of the commodity to be settled, and the determining, according to the second image, a target commodity in the commodities corresponding to the commodity identification result includes:

Extracting local contour features of the commodity to be settled according to the local image;

acquiring outline features of the commodities corresponding to the commodity identification result;

and determining the corresponding target commodity in each commodity according to the local contour features of the commodity to be settled and the contour features of each commodity.

2. The self-checkout method for shopping checkout of claim 1, wherein after determining the commodity identification result of the object to be checkout based on the first image, further comprising:

and if the commodity identification result corresponds to one commodity, taking the commodity as the target commodity, and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight of the target commodity.

3. The self-settlement method for shopping check out according to claim 1, wherein the determining the target photographing parameter according to the placement position comprises:

acquiring a shooting position of a camera on the checkout stand and the commodity size of the commodity to be settled, and determining the relative angle of the camera and the commodity to be settled according to the shooting position and the placing position;

determining a horizontal adjustment angle and a vertical adjustment angle of the camera according to the relative angle;

determining the relative distance between the commodity to be settled and the camera according to the placement position and the camera shooting position;

Determining a shooting focal length according to the relative distance and the commodity size;

and taking the horizontal adjustment angle, the vertical adjustment angle and the shooting focal length of the camera as the target shooting parameters.

4. The self-checkout method of shopping checkout of claim 1, wherein the acquiring the first image and the weight of the commodity to be checkout on the checkout stand, before determining the commodity identification result of the commodity to be checkout according to the first image, further comprises:

if the commodity to be settled is detected to comprise at least two different commodities, stopping weighing operation of the commodity to be settled, and generating error prompt information on a display screen of the settlement table.

5. The self-checkout method for shopping checkout of claim 1, wherein after retrieving the commodity price of the target commodity and determining the checkout amount of the commodity to be checkout based on the commodity price and the commodity weight, further comprising:

If the target commodity is a counting commodity with fixed unit price, determining the commodity number of the counting commodity according to the first image and the second image;

and determining the commodity total price of the counted commodity according to the commodity number and the unit price of the counted commodity.

6. A self-checkout system for shopping checkout, the system comprising:

the amount calculation module is used for calling the commodity price of the target commodity and determining the settlement amount of the commodity to be settled according to the commodity price and the commodity weight;

7. An electronic device comprising a processor, a memory, a user interface, and a network interface, the memory for storing instructions, the user interface and the network interface for communicating to other devices, the processor for executing the instructions stored in the memory to cause the electronic device to perform a self-checkout method of shopping checkout as claimed in any one of claims 1 to 5.

8. A computer readable storage medium storing instructions which, when executed, perform the self-checkout method steps of shopping checkout of any one of claims 1 to 5.