JP6773102B2 - Information processing equipment and information processing programs - Google Patents

Description

The present invention relates to an information processing device and an information processing program.

Patent Document 1 has an object of accurately grasping the number of customers who are interested in a product, and is stored in a non-volatile memory of a storage device in response to a predetermined operation (pressing operation) in the operation unit. The number of operations is incremented, and the price data stored in the non-volatile memory of the storage device is read out in response to a predetermined operation (pressing operation) on the operation unit, and the price display based on the read price data is displayed. It is disclosed that the display unit is displayed for a certain period of time.

An object of Patent Document 2 is to enable a customer who reaches for a product displayed on a shelf, a show space, or the like to be provided with information specific to the customer regarding the product, and an information terminal device. Acquires user identification information that identifies the user from a user who reaches for the corresponding product in a non-contact manner, and transmits the acquired user identification information to the server device together with the product identification information corresponding to the information terminal device. The server device stores the attribute information about the user in association with the user identification information of each user, and each time the user identification information and the product identification information are received from each information terminal device, the server device is stored in association with the user identification information. It is disclosed that the user attribute information is read out and the user attribute information is associated with the product group identification information for processing.

JP-A-2009-092794 JP-A-2007-114957

An object of the present invention is to provide an information processing device and an information processing program that determine a customer's behavior for viewing a product tag when counting the customer's behavior for each behavior. ..

The gist of the present invention for achieving such an object lies in the inventions of the following items.
The invention of claim 1 is when a tag is hung on a product and the distance between the tag and a predetermined position is less than or less than a predetermined distance, the product by the customer. The tag is provided with a determination means for determining that the action is for viewing the tag and a counting means for counting each action determined by the determination means, and the tag is equipped with two detection units for detecting the movement. The determination means is an action for the customer to see the tag of the product when the movement detected by one detection unit and the movement detected by the other detection unit are in opposite directions. It is an information processing apparatus characterized by determining that.

The invention of claim 2 is a customer when a tag is hung on a product and the distance between the tag and a predetermined position is less than or less than a predetermined distance. It functions as a determination means for determining that it is an action for viewing the tag of the product, and a counting means for counting each action determined by the determination means, and the tag has two detections for detecting movement. A unit is attached, and the determination means is for viewing the tag of the product by the customer when the movement detected by one detection unit and the movement detected by the other detection unit are in opposite directions. It is an information processing program that determines that it is an action .

According to the information processing device of claim 1, when the customer's behavior is counted for each behavior, it is possible to determine the behavior of the customer to see the tag of the product. Further, it can be determined that the action is for the customer to see the tag of the product based on the movement detected by the detection unit attached to the tag.

According to the information processing program of claim 2 , when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined. Further, it can be determined that the action is for the customer to see the tag of the product based on the movement detected by the detection unit attached to the tag.

It is a conceptual module configuration diagram about the configuration example of the first embodiment. It is explanatory drawing which shows the system configuration example in the case of realizing the 1st Embodiment. It is a flowchart which shows the processing example (1) by 1st Embodiment. It is explanatory drawing which shows the processing example (1) by 1st Embodiment. It is a flowchart which shows the processing example (2) by 1st Embodiment. It is explanatory drawing which shows the processing example (2) by 1st Embodiment. It is a flowchart which shows the processing example (3) by 1st Embodiment. It is explanatory drawing which shows the processing example (3) by 1st Embodiment. It is a conceptual module configuration diagram about the configuration example of the second embodiment. It is explanatory drawing which shows the system configuration example in the case of realizing the 2nd Embodiment. It is a flowchart which shows the processing example by 2nd Embodiment. It is explanatory drawing which shows the processing example (1) by 2nd Embodiment. It is explanatory drawing which shows the example of the tag used in the 2nd Embodiment. It is explanatory drawing which shows the processing example (2) by 2nd Embodiment. It is explanatory drawing which shows the data structure example of the operation | count table. It is a block diagram which shows the hardware configuration example of the computer which realizes this embodiment.

Hereinafter, examples of various suitable embodiments for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram for a configuration example of the first embodiment.
The module generally refers to parts such as software (computer program) and hardware that can be logically separated. Therefore, the module in this embodiment refers not only to the module in the computer program but also to the module in the hardware configuration. Therefore, in the present embodiment, a computer program for functioning as those modules (a program for causing the computer to execute each procedure, a program for causing the computer to function as each means, and each function for the computer). It also serves as an explanation of the program), system and method for realizing the above. However, for convenience of explanation, words equivalent to "remember" and "remember" are used, but these words are stored in a storage device or stored when the embodiment is a computer program. It means to control so that it is stored in the device. Further, the modules may have a one-to-one correspondence with the functions, but in the implementation, one module may be configured by one program, a plurality of modules may be configured by one program, and conversely, one The module may be composed of a plurality of programs. Further, the plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers by a computer in a distributed or parallel environment. It should be noted that one module may include another module. In addition, hereinafter, "connection" is used not only for physical connection but also for logical connection (data transfer, instruction, reference relationship between data, etc.). "Predetermined" means that it is determined before the target process, not only before the process according to the present embodiment starts, but also after the process according to the present embodiment starts. However, if it is before the target process, it is used with the intention that it is determined according to the situation / state at that time or according to the situation / state up to that point. When there are a plurality of "predetermined values", they may be different values, or two or more values (including all values, of course) may be the same. Further, the description having the meaning of "if A, do B" is used to mean "determine whether or not it is A, and if it is determined to be A, do B". However, this excludes cases where it is not necessary to determine whether or not it is A.
In addition, a system or device is configured by connecting a plurality of computers, hardware, devices, etc. by communication means such as a network (including a one-to-one correspondence communication connection), and one computer, hardware, etc. It also includes cases where it is realized by a device or the like. "Device" and "system" are used as synonymous terms. Of course, the "system" does not include anything that is nothing more than a social "mechanism" (social system) that is an artificial arrangement.
In addition, for each process by each module or when multiple processes are performed in the module, the target information is read from the storage device, and after the processes are performed, the process results are written to the storage device. is there. Therefore, the description of reading from the storage device before processing and writing to the storage device after processing may be omitted. The storage device here may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The information processing device 100 according to the first embodiment determines (identifies) the behavior of the customer, and as shown in the example of FIG. 1, the image receiving module 120, the image processing module 130, and the information processing module. It has 140 and an information recording module 150.
In order to formulate measures for sales promotion, it is necessary to judge the customer's behavior (behavior) with respect to the product in the store or the like and count the behavior. This embodiment is used for that purpose. In particular, the action for the tag attached to the product is determined. The tag is a tag on which the attributes of the product are described. The attributes of the product include, for example, price, manufacturer, raw material, and the like. Specifically, it corresponds to a price tag or the like.

The photographing module 110 is connected to the image receiving module 120 of the information processing apparatus 100. The photographing module 110 photographs a product and its surroundings (also referred to as a product area). Customers enter the product area to purchase the product, to confirm the product, and so on. Therefore, the image taken by the photographing module 110 may include a customer together with the product.
Specifically, the photographing module 110 is a 2D camera, a 3D camera, a thermo camera, an omnidirectional camera, or the like for capturing a still image or a moving image. Further, the photographing module 110 may be fixed, or may be capable of changing the angle, position, and the like. Further, the relationship between the product and the photographing module 110 may be 1 to 1, 1 to N, N to 1, or N to N. .. In the present embodiment, a one-to-one case will be described as an example. In the case of N to 1, it is sufficient to recognize the product and determine the behavior of the customer for each product as described later, and in the case of 1 to N, one product or the product from a plurality of images. The action for the tag may be determined, and the final determination result may be obtained from a plurality of determination results by using a majority vote or the like.

The image receiving module 120 is connected to the photographing module 110 and the image processing module 130. The image receiving module 120 receives an image taken by the photographing module 110 and passes the image to the image processing module 130.
The image processing module 130 is connected to the image receiving module 120 and the information processing module 140. The image processing module 130 has a customer feature extraction module 132 and a tag recognition module 134.
The customer feature extraction module 132 extracts the customer features from the image received by the image reception module 120. For example, the position of the product, the body of the customer, the position of the head of the customer, the position of the hand (including the arm), the direction of the line of sight, the position of the face, the eyes, and the like are extracted as features. For example, the product photographed by the photographing module 110 is a predetermined product, and the characteristics (color, shape, size, etc.) of the predetermined product are extracted in advance and stored as a dictionary. Allows the product to be recognized from within the image. In addition, human recognition, human part (hand, head, etc.) recognition, line-of-sight direction, etc. are recognized using existing technology. The position may be a position in the coordinate system in the image or a position relative to the product.
The tag recognition module 134 recognizes the product tag from the image received by the image reception module 120. For example, the tag photographed by the photographing module 110 is predetermined, and the characteristics (color, shape, size, etc.) of the predetermined tag are extracted in advance and stored as a dictionary. Allows the tag to be recognized from within the image. Further, the tag may have a different mark, information image (for example, two-dimensional code, etc.) or the like for each product or type of product, or may be coated with a paint that reacts to a specific wavelength. However, a device that emits a specific wavelength (sound) may be provided. The tag recognition module 134 recognizes these. In the case of a tag that reacts to a specific wavelength, the photographing module 110 may include a sensor that reacts to the specific wavelength.

The information processing module 140 is connected to the image processing module 130 and the information recording module 150. The information processing module 140 includes an action identification module 142, a line-of-sight detection module 144, and a counting module 146.
The action identification module 142 determines the action for viewing the tag of the product by the customer from the image taken by the customer.
The behavior identification module 142 sees the behavior of the customer entering the product area, the behavior of touching the product, the behavior of touching the tag, the behavior of taking out the tag, the behavior of returning the tag, and the behavior of the product as judgments. There are actions, actions that look at tags, etc.
For example, if the tag is attached at a position where it is necessary to touch the product in order to see the tag, the behavior identification module 142 detects the tag in the image after touching the product, if the customer Judges that it is an action to see the tag of the product. In this case, the tag is inside the product and is attached at a position where it is necessary to touch the product in order to see the tag. This determination process will be described later with reference to FIGS. 3 and 4.
Further, the shooting module 110 and the tag are located adjacent to each other, and the action specifying module 142 is an action for the customer to see the tag of the product when the customer's line of sight is toward the shooting module 110 in the image. Judge that there is. For the line-of-sight detection, the processing result of the line-of-sight detection module 144 is used. This determination process will be described later with reference to FIGS. 5 and 6.
In addition, the tag is hung at the bottom of the product, and when the action identification module 142 detects that the tag is touched in the image, it is determined that the action is for the customer to see the tag of the product. To do. This determination process will be described later with reference to FIGS. 7 and 8.

The line-of-sight detection module 144 detects the line-of-sight of the customer by using the processing result of the customer feature extraction module 132. For example, it detects a line of sight to the photographing module 110, a line of sight to something other than the photographing module 110, and the like.
The counting module 146 counts for each action determined by the action specifying module 142. That is, the number of times the action is performed is measured. The number of actions may be counted for each of the above-mentioned actions, or the number of predetermined actions may be counted.
The information recording module 150 is connected to the information processing module 140. The information recording module 150 stores the counting result by the counting module 146. In addition, the behavior of the customer may be associated with the characteristics of the customer and stored.

FIG. 2 is an explanatory diagram showing an example of a system configuration in the case of realizing the first embodiment.
The information processing device 100, the photographing module 110A, the photographing module 110B, the photographing module 110C, the photographing module 110D, and the product management device 210 are each connected via a communication line 290. The communication line 290 may be wireless, wired, or a combination thereof, and may be, for example, an intranet as a communication infrastructure, the Internet, or the like. A plurality of photographing modules 110 may be installed for each product in one store, or may be installed in a plurality of stores.
The product management device 210 obtains a processing result (information stored in the information recording module 150) by the information processing device 100 and presents the information or the like to the user. In addition, in cooperation with POS (Point Of Sale system) etc., the behavior of customers for purchased products and the behavior of customers who did not purchase but entered the product area should be presented so that they can be compared. You may.

FIG. 3 is a flowchart showing a processing example (1) according to the first embodiment.
In step S302, the photographing module 110 photographs the inside of the product area.
In step S304, the image receiving module 120 receives an image.
In step S306, the customer feature extraction module 132 extracts the customer features from the image.
In step S308, the tag recognition module 134 recognizes the tag from the image.
In step S310, the behavior identification module 142 identifies the behavior of the customer.
In step S312, the action identification module 142 determines the identified action. If the action is an action in which a customer enters the product area, the process proceeds to step S314, if the action is an action that touches a product or a tag, the action proceeds to step S316, and if the action is an action in which a tag is taken out, the process proceeds to step S318, and the tag is inserted. If the action is restored, the process proceeds to step S320.
In step S314, the counting module 146 counts the actions of the customer entering the product area.
In step S316, the counting module 146 counts the actions of touching the goods or tags.
In step S318, the counting module 146 counts the action of taking out the tag.
In step S320, the counting module 146 counts the action of returning the tag.
In step S322, the counting module 146 records the counting result in the information recording module 150.

FIG. 4 is an explanatory diagram showing a processing example (1) according to the first embodiment. In particular, a processing example in which the behavior identification module 142 in step S310 identifies the behavior of the customer will be described. Hereinafter, an apparel store as a store and a shirt as a product will be described as an example.
In this example, the photographing module 110 is attached to the ceiling of the store and photographs the product area. Therefore, the captured image is a photograph of the customer's head 430, hand 440, and shirt 410 taken from above, as in the example of the image column in FIG.
Further, the tag 420 is attached at a position where it is necessary to touch the shirt 410 in order to see the tag 420. For example, it may be attached to the inside of the shirt 410, or the tag 420 may be inserted inside out between the buttons of the shirt 410. In the example of the image column in FIG. 4, the tag 420 is also described, but in cases other than FIG. 4C, it is in the shirt 410 and is not photographed in the image.

In the example of FIG. 4, it is assumed that the vehicle advances in the direction of the arrow on the time axis 400.
The example of FIG. 4 (A) shows an example of detecting the behavior of the customer entering the product area. This means that the customer enters the product area when it is detected that the customer's head 430, etc. stays for a predetermined time or longer from the state where the customer (customer's head 430, hand 440, etc.) is not photographed. It may be judged as an action.
In the example of FIG. 4B, the case where the tag 420 of the shirt 410 is picked up is shown. By detecting that the customer's hand 440 enters the area inside the shirt 410 and stays for a predetermined time or longer, it is determined that the action is "touching the product".
In the example of FIG. 4C, the case where the tag 420 is taken out in order to read the tag 420 of the shirt 410 is shown. By detecting that the hand 440 (or the tag 420) is separated from the area of the shirt 410 by a predetermined distance and stays for a predetermined time or longer from the state of the example of FIG. 4 (B), the “tag is attached”. It is determined that the action is taken out. The "predetermined distance" here may be the length of the string 425 for attaching the tag 420 to the shirt 410. In addition, this "action to take out the tag" may be regarded as "action to see the tag".
In the example of FIG. 4D, the case where the tag 420 of the shirt 410 is finished and returned is shown. From the state of the example of FIG. 4C, by detecting that the customer's hand 440 has left the shirt 410, it is determined that the action is "the action of returning the tag."

FIG. 5 is a flowchart showing a processing example (2) according to the first embodiment.
In step S502, the photographing module 110 photographs the inside of the product area.
In step S504, the image receiving module 120 receives the image.
In step S506, the customer feature extraction module 132 extracts the customer features from the image.
In step S508, the action identification module 142 identifies the action.
In step S510, the action identification module 142 determines the identified action. If the action is an action in which a customer enters the product area, the action proceeds to step S512, if the action is an action of seeing the product, the action proceeds to step S514, and if the action is an action of touching the product, the action proceeds to step S516, and the tag is viewed. If it is an action, the process proceeds to step S518.
In step S512, the counting module 146 counts the actions of the customer entering the product area.
In step S514, the counting module 146 counts the actions of seeing the product.
In step S516, the counting module 146 counts the actions of touching the product.
In step S518, the counting module 146 counts the actions of seeing the tag.
In step S520, the counting module 146 records the counting result in the information recording module 150.

FIG. 6 is an explanatory diagram showing a processing example (2) according to the first embodiment. In particular, a processing example in which the behavior identification module 142 in step S508 identifies the behavior of the customer will be described.
In this example, the photographing module 110 (camera 690 in the figure) is installed at a position adjacent to the tag 620. Further, in order to take a picture of the shirt 610, it may be installed at a position away from the shirt 610. Specifically, in the example of FIG. 6, a tag 620 is attached to the lower side (floor) of the product, a camera 690 is installed next to the tag 620, and an obliquely upper front image is taken.

The example of FIG. 6A shows a case where the customer touches the shirt 610 with his left hand.
When it is detected that the customer 630 or the like stays for a predetermined time or longer from the state in which the customer 630 has not been photographed, it may be determined as an action that the customer has entered the product area.
Then, the line-of-sight detection module 144 detects that the line of sight is the line of sight to the shirt 610, and when the line of sight is longer than a predetermined time, it is determined that the action is "the behavior of seeing the product". The line-of-sight detection module 144 may detect that the line of sight to the shirt 610 is the line of sight to the shirt 610 by detecting that the position of the pupil 651 in the eyeball 650 is in the direction of the shirt 610. Further, by detecting that the position of the white eye 652 in the eyeball 650 is in the direction opposite to that of the shirt 610, it may be detected as the line of sight to the shirt 610.
Further, by detecting that the customer's left hand 640 enters the area inside the shirt 610 and stays for a predetermined time or longer, it is determined that the behavior is "touching the product".
The example of FIG. 6B shows a case where the customer is looking at the tag 620.
The line-of-sight detection module 144 detects that the line of sight is the line of sight to the camera 690, and if the line of sight is longer than a predetermined time, it is determined that the action is "the action of looking at the tag". Since the tag 620 and the camera 690 are adjacent to each other, the line of sight to the camera 690 may be the line of sight to the tag 620. The line-of-sight detection module 144 may detect that the line of sight to the camera 690 is the line of sight to the camera 690 by detecting that the position of the pupil 655 in the eyeball 654 is the center in the eyeball 654. Further, it is detected as a line of sight to the camera 690 when the positional relationship that the pupil 655 in the eyeball 654 is surrounded by the white eye 656, the area ratio of the pupil 655 in the eyeball 654 is equal to or more than a predetermined threshold value, and the like. You may try to do it. The left hand 640 may be in the position of the left hand 644 in a state of touching the shirt 610, or may be in the position of the left hand 642 in a state of being away from the shirt 610. In addition, this "action to see the tag" may be regarded as "action to see the tag".

FIG. 7 is a flowchart showing a processing example (3) according to the first embodiment.
In step S702, the photographing module 110 photographs the inside of the product area.
In step S704, the image receiving module 120 receives an image.
In step S706, the customer feature extraction module 132 extracts the customer features from the image.
In step S708, the tag recognition module 134 recognizes the tag from the image.
In step S710, the action identification module 142 identifies the action.
In step S712, the action identification module 142 determines the identified action. If the action is an action in which a customer enters the product area, the process proceeds to step S714, if the action is an action of touching a product, the process proceeds to step S716, and if the action is an action of touching a tag, the process proceeds to step S718.
In step S714, the counting module 146 counts the actions of the customer entering the product area.
In step S716, the counting module 146 counts the actions of touching the product.
In step S718, the counting module 146 counts the actions of touching the tag.
In step S720, the counting module 146 records the counting result in the information recording module 150.

FIG. 8 is an explanatory diagram showing a processing example (3) according to the first embodiment. In particular, a processing example in which the behavior identification module 142 in step S710 identifies the behavior of the customer will be described.
In this example, the photographing module 110 is attached so as to photograph the product area from the side. That is, it is a position where the shirt 810 and the customer 830 are photographed from the side.
Further, the tag 820 is attached in a state of being suspended from the lower part of the shirt 810. Therefore, in the image in which the customer has not been photographed, the shirt 810 and the tag 820 are not in the overlapping positions. That is, the shirt 810 is in the upper area, and the tag 820 is photographed in the area below it. Further, the tag 820 may be turned upside down in a suspended state. In this case, the customer is more likely to pick up the tag 820 in order to see the contents of the tag 820.

In the example of FIG. 8 (A), the case where the customer touches the shirt 810 with the hand 840 is shown.
When it is detected that the customer 830 or the like stays for a predetermined time or longer from the state in which the customer 830 has not been photographed, it may be determined as the behavior of the customer entering the product area.
Then, by detecting that the hand 840 enters the area in the shirt 810 and stays for a predetermined time or longer, it is determined that the action is "touching the product".
In the example of FIG. 8B, the case where the customer touches the tag 820 is shown.
By detecting that the hand 842 enters the area in the tag 820 and stays for a predetermined time or longer, it is determined that the action is "touching the tag". In addition, this "action to touch the tag" may be regarded as "action to see the tag".

FIG. 9 is a conceptual module configuration diagram for a configuration example of the second embodiment.
The information processing device 900 according to the second embodiment determines the behavior of the customer, and as shown in the example of FIG. 9, includes the detection information receiving module 920, the information processing module 940, and the information recording module 950. Have.
In this embodiment, a tag is hung on the product. A device for measuring the distance is attached to the tag. For example, an RFID (Radio Frequency Identifier) or the like that can measure the distance depending on the strength of the radio wave may be used. Then, a device that wirelessly communicates with the device attached to the tag is installed near the product. For example, when the product is clothes, the device may be embedded in a hanger or the like. In this case, the distance between the tag and the hanger when the tag is touched by the customer is different from the distance between the tag and the hanger when the tag is hung. The tag may be hung at the bottom of the item to increase the certainty of different distances when the tag is picked up.

The detection module 910 is connected to the detection information receiving module 920 of the information processing device 900. As described above, the detection module 910 detects the distance between the device attached to the tag and the predetermined position.
The detection information receiving module 920 is connected to the detection module 910 and the information processing module 940. The detection information reception module 920 receives the distance detected by the detection module 910 and passes it to the information processing module 940.
The information processing module 940 is connected to the detection information receiving module 920 and the information recording module 950. The information processing module 940 has an action identification module 942 and a counting module 946.
The action identification module 942 determines that the action is for the customer to see the tag of the product when the distance between the tag and the predetermined position is less than or less than the predetermined distance. To do. Here, as the "predetermined position", for example, there is a position of a device (for example, the above-mentioned hanger) equipped with a device attached to the tag and a corresponding device. Further, the "predetermined distance" is a distance between the tag and a predetermined position when the tag is suspended (that is, the customer is not touching the tag). When the customer touches (picks up) the tag, the tag is close to the predetermined position. Therefore, if the above conditions (when the distance between the tag and the predetermined position is less than or less than the predetermined distance) are satisfied, the customer's action to see the tag of the product. It is determined as.
The counting module 946 has the same function as the counting module 146.
The information recording module 950 is connected to the information processing module 940. The information recording module 950 has the same function as the information recording module 150.

FIG. 10 is an explanatory diagram showing an example of a system configuration in the case of realizing the second embodiment.
The information processing device 900, the detection module 910A, the detection module 910B, the detection module 910C, the detection module 910D, and the product management device 210 are each connected via a communication line 290. It is the same as the system configuration example shown in the example of FIG. 2, in which the photographing module 110 is changed to the detection module 910 and the information processing device 100 is changed to the information processing device 900.

FIG. 11 is a flowchart showing a processing example according to the second embodiment.
In step S1102, the detection module 910 detects the state of the sensor / radio wave transmission module 1225.
In step S1104, the detection information reception module 920 receives the status information.
In step S1106, the action identification module 942 identifies the action.
In step S1108, the action identification module 942 determines the identified action. If the action is an action of touching a product, the process proceeds to step S1110, and if the action is an action of touching a tag, the process proceeds to step S1112.
In step S1110, the counting module 946 counts the actions of touching the product.
In step S1112, the counting module 946 counts the actions of touching the tag.
In step S1114, the counting module 946 records the counting result in the information recording module 950.

FIG. 12 is an explanatory diagram showing a processing example (1) according to the second embodiment. In particular, a processing example in which the behavior identification module 942 in step S1106 identifies the behavior of the customer will be described.
As shown in the example of FIG. 12A, the sensor / radio wave transmission module 1225 is suspended from the lower part of the shirt 1210. A sensor / radio wave transmission module 1225 is attached to the tag 1220. A radio wave transmission / reception module 1250 is attached to the hanger 1255. The sensor / radio wave transmission module 1225 communicates with the radio wave transmission / reception module 1250, and the radio wave transmission / reception module 1250 measures the distance to the sensor / radio wave transmission module 1225. Then, the measured distance is transmitted to the detection information reception module 920. Further, an acceleration sensor or the like is attached to the sensor / radio wave transmission module 1225, detects the movement of the tag 1220 (sensor / radio wave transmission module 1225), transmits information indicating the movement to the radio wave transmission / reception module 1250, and transmits radio waves. The transmission / reception module 1250 transmits information indicating the movement to the detection information reception module 920. Specifically, it detects up / down / left / right movements and transmits information on the movements by radio waves.
If the customer 1230 touches the shirt 1210 with his hand 1240, the tag 1220 is likely to shake. Therefore, when movement is detected by the acceleration sensor or the like of the sensor / radio wave transmission module 1225, and the distance between the sensor / radio wave transmission module 1225 and the radio wave transmission / reception module 1250 is a predetermined distance (or movement). When the change in distance between the front and back is within a predetermined range), it is determined that the action is "touching the product".
As in the example shown in FIG. 12B, when the customer 1230 touches the tag 1220 with the hand 1242, the distance between the sensor / radio wave transmission module 1225 and the radio wave transmission / reception module 1250 is an example shown in FIG. 12A. It will be shorter than in the case of. Therefore, when the movement is detected by the acceleration sensor or the like of the sensor / radio wave transmission module 1225 and the distance between the sensor / radio wave transmission module 1225 and the radio wave transmission / reception module 1250 is less than or less than a predetermined distance. , Judged as "action of touching the tag". Further, as a condition for this determination, it may be added that the motion is detected by the acceleration sensor or the like of the sensor / radio wave transmission module 1225. In addition, this "action to touch the tag" may be regarded as "action to see the tag".

FIG. 13 is an explanatory diagram showing an example of the tag 1300 used in the second embodiment. The tag 1220 shown in the example of FIG. 12 is a general one in which one of the tags 1220 is suspended by a thread, but the tag 1300 shown in the example of FIG. 13 may be used.
Four threads 1390 (thread 1390A, thread 1390B, thread 1390C, thread 1390D) are attached to the four corners of the tag 1300 so as to keep the tag 1300 parallel to the floor. If the tag 1300 is to be kept parallel to the floor, the number of threads 1390 may not be four (for example, three, five, etc.), or another device (jig) may be used. It may be hung.
Two sensors / radio wave transmission modules 1310 (sensor / radio wave transmission module 1310A, sensor / radio wave transmission module 1310B) for detecting motion are attached to the tag 1300. It is desirable that the sensor / radio wave transmission module 1310 is one at the top and one at the bottom in the direction in which the characters (including figures) described on the tag 1300 can be read. That is, it is necessary to tilt the tag 1300 in order to read characters and the like, but at that time, one sensor / radio wave transmission module 1310 moves upward, and the other sensor / radio wave transmission module 1310 moves downward. Will be done. Further, it is desirable that the characters and the like in the tag 1300 are written on the back side (the side that cannot be seen in a normal state). This is because the tag 1300 must be picked up in order to read the characters and the like. Three or more sensors / radio wave transmission modules 1310 may be attached to one tag 1300.

FIG. 14 is an explanatory diagram showing a processing example (2) using the tag 1300 according to the second embodiment.
(1) At a certain apparel store, when a customer picks up a product 1410, two sensors / radio wave transmission module 1310A and a sensor / radio wave transmission module 1310B attached to a tag 1300 attached to the product 1410 transmit a signal. The radio wave receiving module 1450 receives the signal. The radio wave receiving module 1450 receives radio wave information transmitted from the sensors / radio wave transmitting modules 1310A and 1310B. It is installed for each product type. In addition, it has product information to which an acceleration sensor to which radio wave information is transmitted is attached.
The information of the product 1410 recorded in advance in the radio wave receiving module 1450 and the operation information of the sensor / radio wave transmitting module 1310A and the sensor / radio wave transmitting module 1310B are collectively transmitted to the information processing apparatus 1460. When the operation information of the sensor / radio wave transmission module 1310A and the sensor / radio wave transmission module 1310B are both in the "z-axis positive direction (upward direction)", the information processing device 1460 (action identification module) having a function as the information processing device 900. It is determined by 942) that the product has been picked up, and the information is recorded in the information recording module 1470 having a function as the information recording module 950. In this way, when the movements detected by both the sensor / radio wave transmission module 1310 are movements in the same direction, it is determined that the product has been picked up.

(2) After that, if the customer picks up the tag 1300 (or turns it over), the signals transmitted from the sensor / radio wave transmission module 1310A and the sensor / radio wave transmission module 1310B are transmitted as in the case of (1). The receiving module 1450 receives the information and transmits the information to the information processing device 1460 together with the information of the product 1410. The operation information of the sensor / radio wave transmission module 1310A and the sensor / radio wave transmission module 1310B are "z-axis positive direction (upward direction)" and "z-axis negative direction (downward direction)", respectively, and the information processing device 1460 (action identification). Module 942) determines that the action is "touching the tag" (or "action that flips the tag"), and is recorded in the information recording module 1470. In addition, this "action of touching the tag" (or "action of turning over the tag") may be regarded as "action for seeing the tag". That is, when the movement detected by one sensor / radio wave transmission module 1310 and the movement detected by the other sensor / radio wave transmission module 1310 are in opposite directions, it is determined that the action is for the customer to see the tag of the product. To do.

FIG. 15 is an explanatory diagram showing an example of a data structure of the operation / count table 1500. The operation / count table 1500 is a processing result recorded in the information recording module 150 and the information recording module 950 (information recording module 1470) in the first embodiment and the second embodiment.
The operation / count table 1500 has an operation column 1510 and a count column 1520. The action column 1510 stores the action. The count column 1520 stores the number of times the action was performed.
Further, as the processing result, the date and time when the action was performed (year, month, day, hour, minute, second, second or less, or a combination thereof) may be included.
Using these processing results, for example, "behavior in which a product is picked up" may be determined to be "interested in the product", and "action to see a tag" may be "action to see the product". It may be determined that "I am interested in the price", or if the action is detected multiple times within a predetermined time, it may be determined that the action is performed by one person (same customer). The clerk may be notified of these determination results in real time.

A hardware configuration example of the information processing apparatus (information processing apparatus 100, information processing apparatus 900) of the present embodiment will be described with reference to FIG. The configuration shown in FIG. 16 is configured by, for example, a personal computer (PC) or the like, and shows a hardware configuration example including a data reading unit 1617 such as a scanner and a data output unit 1618 such as a printer.

The CPU (Central Processing Unit) 1601 includes various modules described in the above-described embodiment, that is, an image reception module 120, an image processing module 130, a customer feature extraction module 132, a tag recognition module 134, an information processing module 140, and an action. Processing according to a computer program that describes the execution sequence of each module such as the specific module 142, the line-of-sight detection module 144, the counting module 146, the detection information receiving module 920, the information processing module 940, the action specifying module 942, and the counting module 946. It is a control unit to execute.

The ROM (Read Only Memory) 1602 stores programs, calculation parameters, and the like used by the CPU 1601. The RAM (Random Access Memory) 1603 stores a program used in the execution of the CPU 1601, parameters that are appropriately changed in the execution, and the like. These are connected to each other by a host bus 1604 composed of a CPU bus or the like.

The host bus 1604 is connected to an external bus 1606 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1605.

A pointing device 1609 such as a keyboard 1608 and a mouse is an input device operated by an operator. The display 1610 has a liquid crystal display device, a CRT (Cathode Ray Tube), or the like, and displays various information as text or image information.

The HDD (Hard Disk Drive) 1611 has a built-in hard disk, drives the hard disk, and records or reproduces programs and information executed by the CPU 1601. Images, detection information, counting results, etc. are stored on the hard disk. In addition, various computer programs such as various other data processing programs are stored.

The drive 1612 reads out the data or program recorded on the removable recording medium 1613 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and transfers the data or program to the interface 1607 and the external bus 1606. , Bridge 1605, and RAM 1603 connected via the host bus 1604. The removable recording medium 1613 can also be used as a data recording area similar to a hard disk.

The connection port 1614 is a port for connecting the external connection device 1615, and has a connection portion such as USB or IEEE 1394. The connection port 1614 is connected to the CPU 1601 and the like via the interface 1607, the external bus 1606, the bridge 1605, the host bus 1604, and the like. The communication unit 1616 is connected to the communication line and executes data communication processing with the outside. The data reading unit 1617 is, for example, a scanner, and executes a document reading process. The data output unit 1618 is, for example, a printer, and executes a document data output process.

The hardware configuration of the information processing apparatus shown in FIG. 16 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 16, and the module described in the present embodiment is executed. Any configuration may be possible. For example, some modules may be configured with dedicated hardware (for example, an Applied Special Integrated Circuit (ASIC)), and some modules are in an external system and are connected by a communication line. Further, a plurality of systems shown in FIG. 16 may be connected to each other by a communication line so as to cooperate with each other. Further, it may be incorporated in a copier, a fax machine, a scanner, a printer, a multifunction device (an image processing device having any two or more functions of a scanner, a printer, a copier, a fax machine, etc.) and the like.

Further, in the description of the above-described embodiment, in comparison with a predetermined value, "greater than or equal to", "less than or equal to", "greater than", and "less than (less than)" are inconsistent with the combination. May be "greater than", "less than (less than)", "greater than or equal to", and "less than or equal to", respectively, as long as

The described program may be stored in a recording medium and provided, or the program may be provided by a communication means. In that case, for example, the program described above may be regarded as an invention of "a computer-readable recording medium on which the program is recorded".
The "computer-readable recording medium on which a program is recorded" means a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum "DVD-R, DVD-RW, DVD-RAM, etc." and DVD + RW. Standards such as "DVD + R, DVD + RW, etc.", compact discs (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc., Blu-ray discs (CD-RW) Blu-ray® Disc), opto-magnetic disc (MO), flexible disc (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM® )), Flash memory, random access memory (RAM), SD (Secure Digital) memory card and the like.
Then, the program or a part thereof may be recorded on the recording medium and stored, distributed, or the like. Further, by communication, for example, a wired network used for a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, an intranet, an extranet, or wireless communication. It may be transmitted using a transmission medium such as a network or a combination thereof, or may be carried on a carrier.
Further, the program may be a part of another program or may be recorded on a recording medium together with a separate program. Further, the recording may be divided into a plurality of recording media. Further, it may be recorded in any mode as long as it can be restored, such as compression and encryption.

The above-described embodiment may be grasped as follows.
For example, there are the following issues.
An object of the present embodiment is to provide an information processing device and an information processing program that determine a customer's behavior for viewing a product tag when counting the customer's behavior for each behavior. It is supposed to be.
[A1] A determination means for determining the behavior of the customer to see the tag of the product from the image taken by the customer.
An information processing device including a counting means for counting each action determined by the determination means.

[A2] The tag is attached at a position where it is necessary to touch the product in order to see the tag.
The determination means is characterized in that, when the tag is detected in the image after touching the product, it is determined that the customer is acting to see the tag of the product [A1]. The information processing device described in.

[A3] The photographing means for capturing the image and the tag are located adjacent to each other.
When the customer's line of sight is directed toward the photographing means in the image, the determination means determines that the customer is acting to see the tag of the product [A1]. The information processing device described in.

[A4] The tag is hung from the bottom of the product.
The determination means according to [A1], wherein when the touching the tag is detected in the image, the determination means determines that the customer is acting to see the tag of the product. Information processing device.

[A5] When a tag is hung on a product and the distance between the tag and a predetermined position is less than or less than a predetermined distance, the customer can use the tag of the product. Judgment means to judge that it is an action to see,
An information processing device including a counting means for counting each action determined by the determination means.

[A6] Two or more detection units for detecting movement are attached to the tag.
When the movement detected by one detection unit and the movement detected by the other detection unit are in opposite directions, the determination means determines that the action is for the customer to see the tag of the product. The information processing apparatus according to [A5].

[A7] Computer
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
An information processing program for functioning as a counting means for counting each action determined by the determination means.

[A8] Computer
To see the tag of the product by the customer when the tag is hung on the product and the distance between the tag and the predetermined position is less than or less than the predetermined distance. Judgment means to judge that it is an action,
An information processing program for functioning as a counting means for counting each action determined by the determination means.

The above-mentioned invention has the following effects.
According to the information processing device of [A1], when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined.

According to the information processing device of [A2], when the tag is attached to a position where it is necessary to touch the product in order to see the tag, it is possible to determine the action for viewing the tag.

According to the information processing device of [A3], when the photographing means for capturing an image and the tag are located at adjacent positions, it is possible to determine the action for viewing the tag.

According to the information processing device of [A4], when the tag is hung at the bottom of the product, the action for viewing the tag can be determined.

According to the information processing device of [A5], when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined.

According to the information processing device of [A6], it can be determined that the action is for the customer to see the tag of the product based on the movement detected by the detection unit attached to the tag.

According to the information processing program of [A7], when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined.

According to the information processing program of [A8], when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined.

Further, the above-described embodiment may be grasped as follows.
[B1] Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It is provided with a counting means for counting each action determined by the determination means.
The tag is attached at a position where it is necessary to touch the product in order to see the tag.
When the tag is detected in the image after touching the product, the determination means determines that the customer is acting to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
An information processing device characterized by this.

[B2] Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It is provided with a counting means for counting each action determined by the determination means.
The photographing means for capturing the image and the tag are located adjacent to each other.
When the customer's line of sight is directed toward the photographing means in the image, the determination means determines that the action is for the customer to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
An information processing device characterized by this.

[B3] Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It is provided with a counting means for counting each action determined by the determination means.
The tag is hung from the bottom of the product.
When the determination means detects that the tag is touched in the image, it determines that the customer is acting to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
An information processing device characterized by this.

[B4] Computer
Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It functions as a counting means that counts each action determined by the determination means.
The tag is attached at a position where it is necessary to touch the product in order to see the tag.
When the tag is detected in the image after touching the product, the determination means determines that the customer is acting to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
Information processing program.

[B5] Computer
Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It functions as a counting means that counts each action determined by the determination means.
The photographing means for capturing the image and the tag are located adjacent to each other.
When the customer's line of sight is directed toward the photographing means in the image, the determination means determines that the action is for the customer to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
Information processing program.

[B6] Computer
Shooting means and
Judgment means for determining the behavior of the customer to see the tag of the product from the image taken by the customer,
It functions as a counting means that counts each action determined by the determination means.
The tag is hung from the bottom of the product.
When the determination means detects that the tag is touched in the image, it determines that the customer is acting to see the tag of the product.
The tag is coated with a paint that reacts to a specific wavelength, or the tag is provided with a device that emits a specific wavelength.
The photographing means includes a sensor that reacts to the specific wavelength.
The determination means recognizes the tag based on the specific wavelength.
Information processing program.

The above-mentioned invention has the following effects.
According to the information processing device of [B1], when the customer's behavior is counted for each behavior, it is possible to determine the behavior of the customer to see the tag of the product.

According to the information processing device of [B2], when the photographing means for capturing an image and the tag are located at adjacent positions, it is possible to determine the action for viewing the tag.

According to the information processing device of [B3], when the tag is hung at the bottom of the product, the action for viewing the tag can be determined.

According to the information processing program of [B4], when the customer's behavior is counted for each behavior, the customer's behavior for viewing the product tag can be determined.

According to the information processing program of [B5], when the photographing means for capturing an image and the tag are located adjacent to each other, the action for viewing the tag can be determined.

According to the information processing program of [B6], when the tag is hung at the bottom of the product, the action for viewing the tag can be determined.

100 ... Information processing device 110 ... Shooting module 120 ... Image reception module 130 ... Image processing module 132 ... Customer feature extraction module 134 ... Tag recognition module 140 ... Information processing module 142 ... Action identification module 144 ... Line of sight detection module 146 ... Counting module 150 ... Information recording module 210 ... Product management device 290 ... Communication line 900 ... Information processing device 910 ... Detection module 920 ... Detection information reception module 940 ... Information processing module 942 ... Action identification module 946 ... Counting module 950 ... Information recording module 1450 ... Radio waves Receive module 1460 ... Information processing device 1470 ... Information recording module

Claims (2)

To see the tag of the product by the customer when the tag is hung on the product and the distance between the tag and the predetermined position is less than or less than the predetermined distance. Judgment means to judge that it is an action,
It is provided with a counting means for counting each action determined by the determination means .
Two detectors for detecting movement are attached to the tag.
When the movement detected by one detection unit and the movement detected by the other detection unit are in opposite directions, the determination means determines that the action is for the customer to see the tag of the product. An information processing device characterized by. Computer,
To see the tag of the product by the customer when the tag is hung on the product and the distance between the tag and the predetermined position is less than or less than the predetermined distance. Judgment means to judge that it is an action,
It functions as a counting means that counts each action determined by the determination means .
Two detectors for detecting movement are attached to the tag.
Information for determining that the determination means is an action for the customer to see the tag of the product when the movement detected by one detection unit and the movement detected by the other detection unit are in opposite directions. Processing program.

Images