JP6398331B2 - Information processing apparatus, control method, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, a control method, and a program.

  In general, a product for sale is provided with a symbol for obtaining information related to the product (hereinafter referred to as a product information symbol). Here, the symbol is, for example, a code such as a bar code or a character string. Further, a symbol for obtaining information related to the service (hereinafter referred to as a service information symbol) is attached to a product that is a target of service such as discount.

  In a POS (Point of Service) terminal that performs product settlement, a service is applied to a product by reading the product information symbol and the service information symbol. For example, the price of a product is specified by reading a product information symbol. Further, the discount rate is specified by reading the service information symbol. Then, the discount service is applied to the product by calculating the selling price of the product from the price of the product and the discount rate.

JP 2010-26745 A

  POS terminals need to correctly associate product information symbols with service information symbols. This is because if the association between the product information symbol and the service information symbol is incorrect, the service may not be applied to the service target product, or the service may be applied to a non-service target product.

  For example, the POS terminal device disclosed in Patent Literature 1 includes a scanner that scans product information symbols and a camera that captures discount stickers. Here, the camera is installed in the vicinity of the scanner so that the discount sticker is included in the imaging range of the camera when scanning the product information symbol. And this camera is comprised so that it may image in conjunction with the scan of the merchandise information symbol by a scanner.

  However, in this POS terminal device, the operator must use the POS terminal while grasping the positional relationship between the symbol and the reading device. Specifically, when a product is read by a POS terminal, the product information symbol must be readable by the scanner and the discount sticker can be read by the imaging device. There is. As a result, the convenience of POS terminals is reduced.

  The present invention has been made in view of the above problems. An object of the present invention is to provide a technique for easily associating a product information symbol attached to one product with a service information symbol.

The information processing apparatus provided by the present invention, a product area specifying means for specifying a product region representing the items from one image acquired, and product information symbol detection means for detecting product information symbol from the identified item area, the If the specified product area, and the service information symbol detection means for detecting service information symbol is a symbol for obtaining information related to the service, both before SL product information symbols and the service information symbol is detected, the Corresponding means for associating product information corresponding to the product information symbol and service information corresponding to the service information symbol.

The control method provided by the present invention is executed by a computer. The control method includes the product area specifying step of specifying a product region representing the items from one image acquired, and product information symbol detection step of detecting a product information symbols from the product region above identified, from the product region above identified , if the service information symbol detection step of detecting the service information symbol is a symbol for obtaining information related to the service, both before SL product information symbols and the service information symbol is detected, corresponding to the product information symbols An association step for associating the product information to be associated with the service information corresponding to the service information symbol.

  The program provided by the present invention causes a computer to have the functions of each functional component of the information processing apparatus provided by the present invention.

  According to the present invention, a technique for easily associating a product information symbol attached to one product with a service information symbol is provided.

1 is a block diagram illustrating an information processing apparatus according to a first embodiment. It is a figure which illustrates the goods picture in which one goods is imaged. It is a figure which shows notionally the hardware structural example of the information processing apparatus in 1st embodiment. 3 is a flowchart illustrating a flow of processing executed by the information processing apparatus according to the first embodiment. It is a figure which illustrates a mode that a part of barcode is detected. It is a figure which illustrates merchandise information in a table format. It is a figure which illustrates a mode that a part of discount seal is detected. It is a figure which illustrates service information in a table format. It is a figure which illustrates matching information in a table format. It is a figure which illustrates the goods picture in which three goods are imaged. 6 is a flowchart illustrating a flow of processing performed by the information processing apparatus according to the second embodiment. It is a figure which illustrates a mode that one goods was imaged from a different angle. 10 is a flowchart illustrating a flow of processing executed by the information processing apparatus according to the third embodiment. It is a figure which shows the structural example of a POS register | resistor apparatus. It is a figure which shows the structural example of a POS system. FIG. 10 is a block diagram illustrating an information processing apparatus according to a fourth embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[Embodiment 1]
FIG. 1 is a block diagram illustrating an information processing apparatus 2000 according to the first embodiment. In FIG. 1, arrows indicate the flow of information. Further, in FIG. 1, each block represents a functional unit configuration, not a hardware unit configuration.

  The information processing apparatus 2000 handles products with product information symbols. The product information symbol is a symbol for obtaining information about the product. Here, the symbol is a barcode, a two-dimensional code (such as a QR code (registered trademark)), a character string symbol, or the like. The product information symbol is a barcode or the like in which information specifying product information (product information ID or the like) is encoded, or a character string symbol representing information specifying product information.

  In addition, some products are targeted for services such as discounts. And the service information symbol is attached | subjected to the goods used as the object of service. The service information symbol is a symbol for obtaining information related to the service, and a bar code or the like in which information specifying the service information (service information ID or the like) is encoded, or a character string symbol representing information specifying the service information Etc.

  The information processing apparatus 2000 includes an image acquisition unit 2020, a product region specifying unit 2040, a product information symbol detection unit 2060, a service information symbol detection unit 2080, and an association unit 2100. The image acquisition unit 2020 acquires a product image that is an image obtained by capturing a product. The product region specifying unit 2040 specifies a product region that is a region representing a product from the product image. Here, the “region representing the product” means a region where the product is shown among the regions included in the product image. Further, the product area may be an area where the entire product is shown, or an area where a part of the product is shown. The product information symbol detection unit 2060 detects a product information symbol from the product image. The service information symbol detection unit 2080 detects a service information symbol from the product image. The associating unit 2100 includes product information corresponding to the product information symbol and service information corresponding to the service information symbol for the product information symbol and the service information symbol included in the product area representing one product. Associate.

  The operation of the information processing apparatus 2000 of this embodiment will be conceptually described with reference to FIG. FIG. 2 is a diagram illustrating a product image 10 in which one product (product 20) is captured. A bar code 70 and a discount sticker 80 are attached to the product 20 shown in the product image 1. The barcode 70 is a form of a product information symbol, and the discount sticker 80 is a form of a service information symbol. The information processing apparatus 2000 associates the product information corresponding to the barcode 70 with the service information corresponding to the discount sticker 80 by performing processing as follows.

  The image acquisition unit 2020 acquires the product image 10. The product region specifying unit 2040 specifies the product region 90 representing the product 20 from the product image 10. The product information symbol detection unit 2060 detects the barcode 70 from the product image 10. The service information symbol detection unit 2080 detects the discount sticker 80 from the product image 10. Here, both the barcode 70 and the discount sticker 80 exist inside the product area 90. Therefore, the service information symbol detection unit 2080 associates the product information corresponding to the barcode 70 and the service information corresponding to the discount sticker 80. For example, the product information corresponding to the barcode 70 is the price of the product 20. Further, the service information corresponding to the discount sticker 80 is information that “the price of the product is reduced by 30%”.

  The display provided on the product for obtaining product information and service information may be any display and is not limited to the above-described symbols. The merchandise area specifying unit 2040 and the merchandise information symbol detecting unit 2060 are configured to detect the display provided on these merchandise.

<Hardware configuration>
FIG. 3 is a diagram illustrating a hardware configuration of the information processing apparatus according to the first embodiment. As illustrated in FIG. 3, the information processing apparatus 2000 includes a CPU (Central Processing Unit) 1, a memory 2, an input / output interface (I / F) 3, a communication unit 4, and the like. The CPU 1 is connected to other units via a communication line such as a bus. The memory 2 is a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, or the like. The communication unit 4 exchanges signals with other devices and devices. A portable recording medium or the like can be connected to the communication unit 4. The input / output I / F 3 is connected to the display unit 5, the input unit 6, the imaging unit 7, the sound collecting unit 8, and the like.

  The display unit 5 displays a screen corresponding to drawing data processed by a CPU 1 or a GPU (Graphics Processing Unit) (not shown) such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. Is a unit. The input unit 6 is a unit that receives an input of a user operation, and is realized as a hardware button unit, a touch sensor, or the like, for example. The display unit 5 and the input unit 6 can be integrated and realized as a touch panel. The imaging unit 7 is a camera formed from a lens, an imaging element, and the like, and captures still images and moving images. The sound collection unit 8 is a microphone unit.

  The hardware configuration of the information processing apparatus 2000 is not limited to the configuration shown in FIG. The information processing apparatus 2000 may further have a configuration other than the configuration shown in FIG. In addition, the information processing apparatus 2000 may not include the display unit 5, the input unit 6, the imaging unit 7, and the sound collection unit 8. In this case, the information processing apparatus 2000 communicates via the communication unit 4 to display on the display unit 5 included in another device, to acquire an image captured by the imaging unit 7, and to obtain audio obtained by the sound collection unit 8. Data acquisition can be used.

<Process flow>
FIG. 4 is a flowchart illustrating the flow of processing executed by the information processing apparatus 2000 according to the first embodiment. In step S102, the image acquisition unit 2020 acquires a product image in which the product is captured. In step S104, the product area specifying unit 2040 specifies a product area representing the product from the product image. In step S106, the product information symbol detection unit 2060 detects a product information symbol from the product image. In step S108, the service information symbol detection unit 2080 detects a service information symbol from the product image. In step S110, the association unit 2100 corresponds to the product information corresponding to the product information symbol and the service information symbol for the product information symbol and the service information symbol included in the product area representing one product. Corresponding service information. Here, the order in which each step is executed is not limited to the order shown in FIG. For example, steps S106 and S108 may be executed before step S104.

  According to the present embodiment, product information and service information can be associated with each other by obtaining a product image captured by an imaging device that captures product information symbols and service information symbols. Specifically, for the product information symbol and service information symbol included in the product area representing one product, product information corresponding to the product information symbol and service information corresponding to the service information symbol are It is associated. Thereby, as compared with the case where the product information symbol and the service information symbol are individually read, the operation of associating the product information with the service information is facilitated and the association is accurate.

  Hereinafter, the present embodiment will be described in further detail.

<Details of Image Acquisition Unit 2020>
There are various methods by which the image acquisition unit 2020 acquires product images. For example, the image acquisition unit 2020 acquires a product image from the imaging unit 7. The image acquisition unit 2020 may acquire a product image from a storage unit or a database that stores the product image captured by the imaging unit 7.

<Details of Product Area Identification Unit 2040>
The product area specifying unit 2040 specifies the product area by performing object recognition processing on the product image. Here, object recognition in which an object captured in an image is extracted while being distinguished from the background is a known technique. Therefore, the detailed description regarding object recognition is abbreviate | omitted.

  For example, the product region specifying unit 2040 acquires the feature amount of the product from the storage unit or the like, and specifies the product region representing the product from the product image using the feature amount. As described above, the product area may be an area where the entire product is shown, or an area where a part of the product is shown. Further, for example, the product region specifying unit 2040 may acquire a feature amount for each product package. For example, a common feature amount is acquired for different types of lunch boxes in the same package.

  Moreover, the product area specifying unit 2040 may specify the product area without using the feature amount of the product. For example, the product area specifying unit 2040 detects a shape (such as a rectangular parallelepiped, a cone, a quadrangle, or a circle) that is considered as a product from the product image, and sets a portion corresponding to the detected shape as a product area. In addition, for example, the product area specifying unit 2040 specifies an area (hereinafter referred to as a background area) representing a product other than the product (such as a work table on which the product is placed) from the product image, and each of the regions other than the background area is set as a product area As specified. In particular, if the color of the workbench is composed of only one color or the pattern of the workbench is composed of a simple pattern or the like, the background area and the product area can be easily distinguished.

  Note that the information processing apparatus 2000 can operate correctly even if an area representing an object other than a product is erroneously specified as a product area. This is because merchandise information symbols are not attached to items other than merchandise, and merchandise information and service information are not erroneously associated with items other than merchandise. However, since the amount of calculation performed by the associating unit 2100 increases, it is preferable that the image acquisition unit 2020 specifies only an area representing a product as a product area.

<Details of Product Information Symbol Detection Unit 2060>
The merchandise information symbol detection unit 2060 identifies a merchandise information symbol by performing object recognition processing on the merchandise image. For example, the product information symbol detection unit 2060 acquires the feature amount of the product information symbol from the storage unit or the like, and detects the product information symbol from the product image using the feature amount.

  The product information symbol detection unit 2060 may perform the product information symbol detection process by focusing on the product region specified by the product region specifying unit 2040 instead of the entire product image. In this case, since the calculation amount of the process performed by the product information symbol detection unit 2060 is reduced, the time required for the process performed by the product information symbol detection unit 2060 is shortened.

  Depending on the form of the product information symbol, it may be sufficient for the product information symbol detection unit 2060 to detect only a part of the product information symbol. For example, when the product information symbol is the barcode 70 shown in FIG. 2, the information encoded by the barcode 70 is decoded even from only a part of the barcode 70, such as the region 100 indicated by the solid line in FIG. can do.

<Details of product information>
FIG. 6 is a diagram illustrating product information in a table format. The table shown in FIG. 6 is referred to as a product information table 200. The product information table 200 has four columns of product information ID 202, product name 204, price 206, and feature value 208. The product information ID 202 indicates an ID for specifying product information. The product name 204 indicates the name of the product. The price 206 indicates the price of the product. A feature amount 208 indicates a feature amount used for detecting a product information symbol from a product image.

  For example, the record on the first line in FIG. 6 indicates that information related to a product “product name is XXX and price is 1000 yen” is managed as product information of product information ID 202 = I001. Furthermore, this record indicates that the feature quantity of the product information symbol corresponding to the product information of “product information ID 202 = I001” is FI001.

  The product information table 200 may indicate information other than product names and prices.

<Details of Service Information Symbol Detection Unit 2080>
The service information symbol detection unit 2080 specifies a service information symbol by performing object recognition processing on the product image. For example, the product information symbol detection unit 2060 acquires the feature amount of the service information symbol from the storage unit or the like, and detects the service information symbol from the product image using the feature amount.

  Similarly to the product information symbol detection unit 2060, the service information symbol detection unit 2080 performs service information symbol detection processing focusing on the product region specified by the product region specification unit 2040, not the entire product image. Also good.

  Depending on the form of the service information symbol, the service information symbol detection unit 2080 may be sufficient to detect only a part of the service information symbol. For example, when the service information symbol is the discount sticker 80 shown in FIG. 2, the information represented by the discount sticker 80 can be obtained even from only a part of the discount sticker 80, such as the area 110 shown by the solid line in FIG. Can do.

<Details of service information>
FIG. 8 is a diagram illustrating service information in a table format. The table shown in FIG. 8 is referred to as a service information table 300. The service information table 300 has three columns of service information ID 302, service content 304, and feature amount 306. Service information ID 302 indicates an ID for identifying service information. Service content 304 indicates the content of the service. A feature quantity 306 indicates a feature quantity used to detect a service information symbol from a product image.

  For example, the record on the first line in FIG. 8 indicates that information related to “30% discount service” is managed as service information of “service information ID 302 = S001”. Further, this record indicates that the feature quantity of the service information symbol corresponding to the service information of “service information ID 302 = S001” is FS001.

  The service information table 300 may indicate information other than service contents.

<Details of association unit 2100>
For example, the associating unit 2100 determines that the product information symbol and the service information symbol are included in the product area using the product information symbol coordinates and the service information symbol coordinates in the product image. Here, the coordinates of the product information symbol may be arbitrary coordinates in the product information symbol. For example, the coordinates of the product symbol are the coordinates of the top left corner of the product symbol and the coordinates of the center of the product information symbol. The same applies to the coordinates of the service information symbol.

<Association information>
The associating unit 2100 may generate information in which the product information symbol and the service information symbol are associated with each other. This information is referred to as association information. FIG. 9 is a diagram illustrating the association information in a table format. The correspondence information in the table format shown in FIG. The association information table 400 has two columns, product information ID 402 and service information ID 404. Each record in the association information table 400 associates product information specified by the product information ID 402 with service information specified by the service information ID 404.

[Embodiment 2]
The configuration of the information processing apparatus 2000 according to the second embodiment is illustrated in FIG. 1 similarly to the information processing apparatus 2000 according to the first embodiment.

  The information processing apparatus 2000 according to the second embodiment associates product information and service information for each of the plurality of products when a plurality of products are shown in one product image. Specifically, the product area specifying unit 2040 specifies a product area representing each product for a plurality of products included in one product image. Then, the associating unit 2100 associates the above-described product information and service information with respect to a product region including both the product information symbol and the service information symbol among the specified product regions.

  The operation of the information processing apparatus 2000 according to this embodiment will be conceptually described with reference to FIG. FIG. 10 is a diagram illustrating a product image 60 in which three products 20-1, 20-2, and 20-3 are captured. The product region specifying unit 2040 specifies a product region 90-1 representing the product 20-1, a product region 90-2 representing the product 20-2, and a product region 90-3 representing the product 20-3. In addition, the product information symbol detection unit 2060 detects the barcode 70-1, the barcode 70-2, and the barcode 70-3. Further, the service information symbol detector 2080 detects the discount sticker 80-1 and the discount sticker 80-3.

  The associating unit 2100 associates the product information corresponding to the barcode 70-1 with the service information corresponding to the discount sticker 80-1. The association unit 2100 associates the product information corresponding to the barcode 70-3 and the service information corresponding to the discount sticker 80-3. As described above, the product information and the service information are associated with the product 20-1 and the product 20-3, respectively. On the other hand, since the discount sticker is not included in the product area 90-2, the product 20-2 is not associated.

  Note that the process performed by the product region specifying unit 2040, the process performed by the product information symbol detection unit 2060, and the process performed by the service information symbol detection unit 2080 are the same as those described in the first embodiment.

<Process flow>
FIG. 11 is a flowchart illustrating the flow of processing performed by the information processing apparatus 2000 according to the second embodiment. The information processing apparatus 2000 according to the second embodiment performs the processes in and after step S202 illustrated in FIG. 11 after executing the processes in steps S102 to S108 illustrated in FIG. In order to simplify the drawing, steps S102 to S106 are omitted in FIG.

  Steps S202 to S212 are loop processing A executed for each detected product information symbol. In step S202, the associating unit 2100 determines whether there is a merchandise information symbol that is not yet the target of the loop process A. If the loop process A has already been executed for all the product information symbols, the process in FIG. 11 ends. On the other hand, if there is a merchandise information symbol that is not yet subject to loop processing A, the associating unit 2100 selects one of them. Here, the selected product information symbol is written as product information symbol i. Then, the process of FIG. 11 proceeds to step S204.

  Steps S204 to S208 are loop processing B executed for each detected service information symbol. In step S204, the associating unit 2100 determines whether there is a merchandise information symbol that is not yet the target of the loop process B. If the loop process B has already been executed for all the product information symbols, the process in FIG. 11 proceeds to step S212. On the other hand, when there is a service information symbol that is not yet the target of the loop process B, the associating unit 2100 selects one of them. Here, the selected service information symbol is denoted as service information symbol j. Then, the process of FIG. 11 proceeds to step S206.

  In step S206, the associating unit 2100 determines whether or not the product area that includes the product information symbol i and the product area that includes the service information symbol j represent one product. If these two product areas do not represent one product (step S206: NO), the process of FIG. 11 proceeds to step S208. Since step S208 is the end of loop processing B, the processing in FIG. 11 proceeds to step S204.

  On the other hand, when the product area including the product information symbol i and the product area including the service information symbol j represent one product in step S206 (step S206: YES), the process of FIG. Proceed to S210. In step S210, the associating unit 2100 associates product information corresponding to the product information symbol i with service information corresponding to the service information symbol j.

  Since step S212 is the end of loop processing A, the processing in FIG. 11 proceeds to step S202.

<Action and effect>
According to this embodiment, when a plurality of products are included in one image, product information and service information can be associated with each product. Therefore, it is possible to reduce the effort for associating the product information with the service information as compared with the method of scanning the products one by one.

[Embodiment 3]
The configuration of the information processing apparatus 2000 according to the third embodiment is illustrated in FIG. 1 similarly to the information processing apparatus 2000 according to the first embodiment.

  The information processing apparatus 2000 according to the present embodiment associates product information and service information for a product even when the product information symbol and the service information symbol are captured in different images. For example, as illustrated in FIG. 12, the information processing apparatus 2000 according to the present embodiment assumes a situation in which it is difficult to simultaneously capture the product information symbol and the service information symbol. FIG. 12 is a diagram illustrating a state in which one product is imaged from different angles. The product 20 in FIG. 12 has a discount seal 80 attached to the upper surface (surface 21) and a barcode 70 attached to the bottom surface (surface 23).

  FIG. 12A shows a state in which the product 20 is imaged from an angle looking down at the surface 21. In this case, although the discount sticker 80 attached to the surface 21 is shown, the barcode 70 attached to the surface 23 is not shown. On the other hand, FIG. 12B shows a state in which the product 20 is imaged from an angle looking down at the surface 23. In this case, the bar code 70 attached to the surface 23 is shown, but the discount sticker 80 attached to the surface 21 is not shown.

  As described above, when the barcode 70 and the discount sticker 80 are separately attached to the two surfaces that are difficult to be imaged simultaneously, such as the top surface and the bottom surface, the product information symbol and the service information symbol are displayed from one product image. It is difficult to detect both. In addition, even when the barcode 70 and the discount sticker 80 are attached on the same surface, it may be difficult to detect both the product information symbol and the service information symbol from one product image. In order to cope with such a situation, the information processing apparatus 2000 according to the present embodiment, when a product information symbol and a service information symbol are captured in different images for one product, product information and service for the product. Correlate with information.

  Specifically, the associating unit 2100 of the present embodiment is 1) a product information symbol is imaged in the first product area in the first product image, and 2) is different from the first product image. When the service information symbol is displayed in the second product area in the product image of 2 and 3) the first product area and the second product area represent one product, the product information symbol area The product information corresponding to the product information symbol represented by is associated with the service information corresponding to the service information symbol represented by the service information symbol area.

  Here, it can be determined, for example, that the product represented by the first product area and the product represented by the second product area are one product as follows. First, it is assumed that the imaging unit 7 that captures product images is configured to repeatedly capture images. For example, the imaging unit 7 sequentially captures a plurality of images constituting the moving image. Then, the associating unit 2100 recognizes one product captured across the images from a plurality of images captured repeatedly. There are various methods for recognizing a single product taken across images. For example, there is a method of tracking a product area detected in a certain image across a plurality of images. Further, for example, when product areas having the same or similar feature amounts are specified for a plurality of images, the product areas may be regarded as representing one product. Moreover, you may consider that the product area | region contained in the image imaged within the predetermined time represents the same goods. Furthermore, when a product image is captured using a plurality of imaging devices and a product region having the same or similar feature amount is detected in each product image, the product region is regarded as representing one product. May be. However, the method for recognizing one product captured across images is not limited to the above method. There are various known methods for recognizing an object captured over a plurality of images as one object, and the associating unit 2100 can use these various methods. Here, a detailed description of the known method is omitted.

  Further, the association unit 2100 may consider that the target product has been switched when an image in which the product is not shown appears. For example, it is assumed that images are captured in the order of an image 1 where no product is shown, an image 2 where the product is shown, an image 5 and an image 6 where no product is shown. In this case, the associating unit 2100 considers that the product area included in the image 2 to the image 5 represents one product. Here, the associating unit 2100 may consider that the product area included in the captured image represents one product when the time when the product is repeatedly imaged is short. For example, the associating unit 2100 considers that the product included in the captured image represents one product when the image of the product is repeatedly captured within 1 second. In this case, when an image showing a product is repeatedly captured for longer than 1 second, for example, the association unit 2100 uses a technique such as tracking described above.

  For example, in the case of the product 20 illustrated in FIG. 12, the imaging unit 7 captures a series of flows from the state illustrated in FIG. 12A to the state in which the product 20 is turned over to the state illustrated in FIG. To do. Then, the information processing apparatus 2000 can track the captured moving image to find that the product area 90-a and the product area 90-b in FIG. 12 are product areas representing one product. Then, the associating unit 2100 associates the discount sticker 80 in the product area 90-a with the barcode 70 in the product area 90-b.

<Process flow>
FIG. 13 is a flowchart illustrating the flow of processing executed by the information processing apparatus 2000 according to the third embodiment. In step S302, the information processing apparatus 2000 initializes a variable x to 0. Where x is the ID assigned to the product.

  In step S304, the image acquisition unit 2020 acquires a product image. In step S306, the product area specifying unit 2040 specifies the product area from the product image. In order to simplify the flowchart, the processing flow in FIG. 13 is based on the premise that there is one product area included in one product image. When a plurality of product areas are included in one product image, the information processing apparatus 2000 performs the processing after step S308 for each product area.

  In step S308, the product area specifying unit 2040 determines whether or not the product represented by the specified product area has already been detected. Here, “a product has already been detected” does not include a situation where “a different type of product has been detected”. If the product represented by the identified product area has not yet been detected (step S308: NO), the process of FIG. 13 proceeds to step S310. On the other hand, if the product represented by the specified product area has already been detected (step S308: YES), the process of FIG. 13 proceeds to step S318.

  In step S310, the product region specifying unit 2040 puts information specifying the detected product into I [x]. Here, I [x] is a data area for storing information on a product to which x is assigned as an ID. In step S312, the merchandise area specifying unit 2040 initializes I [x] .ItemInfSymbol to 0. I [x] .ItemInfSymbol is a data area for storing information for specifying the product information symbol included in the product area for the product specified by x. In step S314, the merchandise area specifying unit 2040 initializes I [x] .ServInfSymbol to 0. I [x] .ServInfSymbol is a data area for storing information for specifying a service information symbol included in the product area for the product specified by x. In step S316, the product region specifying unit 2040 adds 1 to the value of x.

  In step S318, the product information symbol detection unit 2060 performs product information symbol detection processing for the specified product region. In step S320, the product information symbol detection unit 2060 determines whether or not a product information symbol is detected in the product area. If a product information symbol is detected in the product area (step S320: YES), the process of FIG. 13 proceeds to step S322. On the other hand, when the product information symbol is not detected in the product area (step S320: NO), the process of FIG. 13 proceeds to step S324.

  In step S322, the product information symbol detection unit 2060 stores information for specifying the detected product information symbol in I [x] .ItemInfSymbol.

  In step S324, the service information symbol detection unit 2080 performs a service information symbol detection process for the specified product area. In step S326, the service information symbol detection unit 2080 determines whether or not a service information symbol is detected in the product area. If a service information symbol is detected in the product area (step S326: YES), the process of FIG. 13 proceeds to step S328. On the other hand, when the product information symbol is not detected in the product area (step S326: NO), the process of FIG. 13 proceeds to step S330.

  In step S328, the service information symbol detection unit 2080 stores information identifying the detected service information symbol in I [x] .ServInfSymbol.

  In step S330, the associating unit 2100 determines whether or not the condition “I [x] .ItemInfSymbol ≠ 0 and I [x] .ServInfSymbol ≠ 0” is satisfied. This condition means that “both product information symbols and service information symbols are detected”. If the above condition is satisfied, the process of FIG. 13 proceeds to step S332. On the other hand, if this condition is not satisfied, the process in FIG. 13 proceeds to step S304.

  In step S332, the associating unit 2100 obtains product information corresponding to the product information symbol specified by I [x] .ItemInfSymbol and service information corresponding to the service information symbol specified by I [x] .ServInfSymbol. Associate. Then, the process of FIG. 13 proceeds to step S304.

  In step S304, the next product image is acquired, and the above-described processing is repeated.

<Action and effect>
According to the present embodiment, even if the product information symbol and the service information symbol attached to one product cannot be imaged simultaneously, the product information corresponding to the product information symbol attached to the product and the service Service information corresponding to the information symbol can be associated. Therefore, the convenience of the information processing apparatus 2000 is improved.

  In addition, when the product information symbol and the service information symbol must be captured simultaneously, the positional relationship between the product information symbol and the service information symbol is limited. Therefore, when a clerk or the like attaches a product information symbol or a service information symbol, the clerk or the like must work while being aware of the positional relationship between the product information symbol and the service information symbol. On the other hand, according to the information processing apparatus 2000 of the present embodiment, the positional relationship between the product information symbol and the service information symbol is not limited as described above. For this reason, it is possible to reduce the labor of the work performed by the store clerk.

[Example]
A plurality of examples will be described below, and the above-described embodiments will be described in more detail. The present invention is not limited in any way by the following examples.

<Example 1>
FIG. 14 is a diagram illustrating a configuration example of a POS register device. As shown in FIG. 14, the first embodiment shows an example in which the information processing device 2000 and the product image presentation method described above are applied to a POS (Point Of Sale) register device 30. The POS register device 30 according to the first embodiment includes a checker device 31 and a cashier device 41. The checker device 31 registers the product as a payment target, and sends information on the payment target product to the cashier device 41. The cashier apparatus 41 performs a settlement process for a product to be settled. The payment target can be registered in both the checker device 31 and the cashier device 41.

  The checker device 31 and the cashier device 41 according to the first embodiment include a CPU 1, a memory 2, an input / output I / F 3, a communication unit 4, and the like, as in FIG. The above-described information processing apparatus 2000 can be applied to both the checker apparatus 31 and the cashier apparatus 41, and the above-described product image presentation method can be executed in either the checker apparatus 31 or the cashier apparatus 41.

  The display unit 5 described above corresponds to at least one of the display unit 32 a and the display unit 42 of the touch panel unit 32. The input unit 6 described above corresponds to at least one of the touch panel 32 b, the keyboard 33, and the keyboard 43 of the touch panel unit 32. The imaging unit 7 described above corresponds to the scanner device 34. The scanner device 34 captures a product image that reflects the appearance of the product held over the reading window 36. The above-described sound collecting unit 8 is attached to, for example, the member 35 in the checker device 31, the operator of the POS register device 30, and the like.

  According to the first embodiment, the operator of the POS register device 30 holds the product over the reading window 36 when registering the product as a liquidation target. As a result, a product image of the product is captured, and product information and service information are associated with the product. As a result, the merchandise is registered with the payment target in association with the service.

  As described in the second embodiment, when a plurality of products are copied onto one product image, for example, the operator holds the plurality of products over the reading window 36 at the same time. Further, by installing the scanner device 34 on the member 35 or the like, the scanner device 34 may image the work table 37 from above. In this case, for example, the operator places a plurality of products on the work table 37 and operates the scanner device 34. Further, the scanner device 34 may be configured as an object that can be held by an operator's hand. In this case, the operator can capture the product information symbol and service information symbol of each product by capturing images of a plurality of products placed on the work table 37 or the like from a desired angle.

  Further, as described in the third embodiment, when the product information symbol and the service information symbol attached to one product cannot be captured simultaneously, for example, the operator holds the product over the reading window 36 while changing the angle. Both the product information symbol and the service information symbol attached to the product are imaged. Further, as described above, the scanner device 34 may be configured to be held by an operator by hand. In this case, for example, the operator can image both the product information symbol and the service information symbol attached to the product by imaging one product while changing the position of the scanner device 34.

  In the first embodiment, as illustrated in FIG. 14, the POS register device 30 includes the checker device 31 and the cashier device 41, but the POS register device 30 may be configured by only the cashier device 41. In this case, the cashier device 41 only needs to have the scanner device 34.

<Example 2>
FIG. 15 is a diagram illustrating a configuration example of a POS system. As shown in FIG. 15, the second embodiment shows an example in which the information processing apparatus 2000 and the product image presentation method described above are applied to the POS system 50. The POS system 50 according to the second embodiment includes a server device 51 and a store side configuration 52, and the server device 51 and the store side configuration 52 are communicably connected via a communication network 53. The store-side configuration 52 includes a store terminal 54, a stand 55, and the like that can be connected to the communication network 53 via a communication relay device. The stand 55 is a device for connecting the store terminal 54 and other devices (such as a handy scanner, a cash drawer, a receipt printer, etc.) in a communicable manner.

  The store terminal 54 is a general-purpose computer such as a PC (Personal Computer), a notebook PC, a tablet-type terminal, a smartphone, and the like. A similar function is realized. The store terminal 54 includes a touch panel unit 56, an imaging unit 57, a microphone (not shown), and the like. The display unit 5 described above corresponds to the display unit of the touch panel unit 56. The input unit 6 described above corresponds to the touch panel of the touch panel unit 56. The imaging unit 7 described above corresponds to the imaging unit 57. The sound collecting unit 8 described above corresponds to a microphone built in the store terminal 54.

  The server device 51 is a general server computer, and is realized as a WEB server, an application server, or the like. The store terminal 54 and the server device 51 have a CPU 1, a memory 2, an input / output I / F 3, a communication unit 4, and the like as in FIG. 3.

  The above-described information processing apparatus 2000 can be realized only as the store terminal 54 or as a combination of the server device 51 and the store terminal 54. When a part of the information processing device 2000 is realized by the server device 51, for example, the association unit 2100 is realized by the server device 51, and the other processing units are realized by the store terminal 54.

  According to the second embodiment, the operator of the store terminal 54 can associate the product information related to the product with the service information by holding the product over the imaging unit 57 when registering the product as a liquidation target. As a result, the product can be registered in the liquidation target in association with the applied service. The operator of the store terminal 54 performs the same operation when not only at the time of liquidation, but when, for example, a customer asks "How much is the price of the product after applying the service". Thereby, the product information of the product and the service information are associated with each other, and as a result, the customer can be informed of information regarding the price of the product after application of the service.

<Example 3>
The information processing device 2000 and the product image presentation method described above are not applicable only to the POS register device 30 and the POS system 50, but can be applied to various devices. For example, the information processing apparatus 2000 and the control method described above may be applied to a general-purpose computer such as a PC or a smart device used by general users. In this case, when the user wants to grasp the price after the service is applied to the product taken by the user, after the service is applied to the product by photographing the product with the imaging unit of the computer. You can see the price.

[Embodiment 4]
FIG. 16 is a block diagram illustrating an information processing apparatus 2000 according to the fourth embodiment. In FIG. 16, arrows indicate the flow of information. Further, in FIG. 16, each block represents a functional unit configuration, not a hardware unit configuration.

  The information processing apparatus 2000 includes a product region specifying unit 2040, a product information symbol detection unit 2060, a service information symbol detection unit 2080, and an association unit 2100. The product area specifying unit 2040 specifies the product area from the product image. The product information symbol detection unit 2060 detects a product information symbol from the product image. The service information symbol detection unit 2080 detects a service information symbol from the product image. The associating unit 2100 includes product information corresponding to the product information symbol and service information corresponding to the service information symbol for the product information symbol and the service information symbol included in the product area representing one product. Associate.

  Here, the details of the processing executed by the product area specifying unit 2040, the product information symbol detecting unit 2060, the service information symbol detecting unit 2080, and the associating unit 2100 according to the fourth embodiment are as follows. This is the same as the product information symbol detection unit 2060, the service information symbol detection unit 2080, and the association unit 2100.

As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.
A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
1. Product area specifying means for specifying a product area representing a product from the acquired image;
Product information symbol detecting means for detecting a product information symbol from the image;
Service information symbol detection means for detecting a service information symbol from the image;
For the product information symbol and the service information symbol included in the product area representing one product, the product information corresponding to the product information symbol is associated with the service information corresponding to the service information symbol. Association means to perform;
An information processing apparatus.
2. The product area specifying means specifies a product area representing each product when a plurality of products are included in the image,
The association means performs the association on a product area that includes both the product information symbol and the service information symbol among the specified product areas. The information processing apparatus described in 1.
3. The information processing apparatus acquires a plurality of images,
The association means includes a first product area including a product information symbol area in the first image, and a second product including a service information symbol area in a second image different from the first image. When an area represents one product, product information corresponding to the product information symbol represented by the product information symbol area is associated with service information corresponding to the service information symbol represented by the service information symbol area. Or 2. The information processing apparatus described in 1.
4). A control method executed by a computer,
A product area specifying step for specifying a product area representing the product from the image;
A product information symbol detection step for detecting a product information symbol from the image;
A service information symbol detection step for detecting a service information symbol from the image;
For the product information symbol and the service information symbol included in the product area representing one product, the product information corresponding to the product information symbol is associated with the service information corresponding to the service information symbol. A matching step to be performed;
Control method.
5). The product region specifying step specifies a product region representing each product when a plurality of products are included in the image,
In the associating step, the associating is performed for a merchandise area including both the merchandise information symbol and the service information symbol among the identified merchandise areas. The control method described in 1.
6). An image acquisition step of acquiring a plurality of images;
The association step includes a first product area including a product information symbol area in the first image, and a second product including a service information symbol area in a second image different from the first image. 3. When an area represents one product, product information corresponding to the product information symbol represented by the product information symbol area is associated with service information corresponding to the service information symbol represented by the service information symbol area. Or 5. The control method described in 1.
7). On the computer,
A product area specifying function for specifying a product area representing a product from an image;
A product information symbol detection function for detecting a product information symbol from an image;
A service information symbol detection function for detecting a service information symbol from an image;
For the product information symbol and the service information symbol included in the product area representing one product, the product information corresponding to the product information symbol is associated with the service information corresponding to the service information symbol. The mapping function to perform,
A program to give
8). The product area specifying function specifies a product area representing each product when a plurality of products are included in the image,
The associating function performs the associating with respect to a product area including both the product information symbol and the service information symbol among the specified product areas. The program described in.
9. The computer has an image acquisition function for acquiring a plurality of images,
The association function includes a first product area including a product information symbol area in the first image, and a second product including a service information symbol area in a second image different from the first image. 6. When the area represents one product, product information corresponding to the product information symbol represented by the product information symbol area is associated with service information corresponding to the service information symbol represented by the service information symbol area. Or 8. The program described in.

1 CPU
2 Memory 3 Input / output I / F
4 Communication unit 5 Display unit 6 Input unit 7 Imaging unit 8 Sound collection unit 10 Product image 20 Product 21, 22, 23 Surface 30 POS register device 31 Checker device 32 Touch panel unit 32a Display unit 32b Touch panel 33 Keyboard 34 Scanner device 35 Member 36 Reading window 37 Work table 41 Cashier device 42 Display unit 43 Keyboard 50 POS system 51 Server device 52 Store side configuration 53 Communication network 54 Store terminal 55 Stand 56 Touch panel unit 57 Imaging unit 60 Product image 70 Bar code 80 Discount seal 90 Product region 100 110 area 200 product information table 202 product information ID
204 Product name 206 Price 208 Feature quantity 300 Service information table 302 Service information ID
304 Service content 306 Feature value 400 Correlation information table 402 Product information ID
404 Service information ID
2000 Information processing apparatus 2020 Image acquisition unit 2040 Product region specifying unit 2060 Product information symbol detection unit 2080 Service information symbol detection unit 2100 Association unit

Claims (9)

A product area specifying means for specifying a product area representing a product from one acquired image;
Product information symbol detection means for detecting a product information symbol from the specified product region ;
Service information symbol detection means for detecting a service information symbol, which is a symbol for obtaining information related to a service, from the specified product area ;
If both before SL product information symbols and the service information symbol is detected, a correlating means for performing a product information corresponding to the commodity information symbols, the correspondence between the service information corresponding to the service information symbols,
An information processing apparatus. The product area specifying means specifies a product area representing each product when a plurality of products are included in the image,
The information processing apparatus according to claim 1, wherein the association unit performs the association on a product area including both the product information symbol and the service information symbol among the specified product areas. The information processing apparatus acquires a plurality of images,
The product area specifying means specifies the product area from each of a plurality of images,
Said correlating means comprises a first product region including the product information symbols identified from the first image, the service information symbols identified from another second image from said first image If the second product area including the product information represents one product, product information corresponding to the product information symbol represented by the product information symbol area, service information corresponding to the service information symbol represented by the service information symbol area, and The information processing apparatus according to claim 1 . A control method executed by a computer,
A product region specifying step for specifying a product region representing a product from one acquired image;
A product information symbol detection step for detecting a product information symbol from the specified product region ;
A service information symbol detection step for detecting a service information symbol, which is a symbol for obtaining information related to a service, from the specified product area ;
If both before SL product information symbols and the service information symbol is detected, the correspondence performing the commodity information corresponding to the commodity information symbols, the correspondence between the service information corresponding to the service information symbols,
A control method. The product region specifying step specifies a product region representing each product when a plurality of products are included in the image,
5. The control method according to claim 4, wherein in the associating step, the associating is performed for a product area including both the product information symbol and the service information symbol in the specified product area. An image acquisition step of acquiring a plurality of images;
The product region specifying step specifies the product region from each of a plurality of images,
The association step includes a first product region including the product information symbols identified from the first image, the service information symbols identified from another second image from said first image If the second product area including the product information represents one product, product information corresponding to the product information symbol represented by the product information symbol area, service information corresponding to the service information symbol represented by the service information symbol area, and The control method according to claim 4, wherein On the computer,
A product area specifying function for specifying a product area representing a product from one acquired image;
A product information symbol detection function for detecting a product information symbol from the specified product region ;
A service information symbol detection function for detecting a service information symbol that is a symbol for obtaining information related to a service from the specified product area ;
If both before SL product information symbols and the service information symbol is detected, and the product information corresponding to the commodity information symbols, the correlation function for the association between the service information corresponding to the service information symbols,
A program to give The product area specifying function specifies a product area representing each product when a plurality of products are included in the image,
The program according to claim 7, wherein the association function performs the association for a product area including both the product information symbol and the service information symbol among the specified product areas. The computer has an image acquisition function for acquiring a plurality of images,
The product area specifying function specifies the product area from each of a plurality of images,
The mapping function is a first product region including the product information symbols identified from the first image, the service information symbols identified from another second image from said first image If the second product area including the product information represents one product, product information corresponding to the product information symbol represented by the product information symbol area, service information corresponding to the service information symbol represented by the service information symbol area, and The program according to claim 7, which is associated with.

Images