WO2021002126A1

WO2021002126A1 - Control device, optical system, control method, and program

Info

Publication number: WO2021002126A1
Application number: PCT/JP2020/021188
Authority: WO
Inventors: 薫江口; 達明里中; 正太島田
Original assignee: キヤノン株式会社
Priority date: 2019-07-02
Filing date: 2020-05-28
Publication date: 2021-01-07
Also published as: JP2021010136A

Abstract

[Problem] To provide a control device with which it is possible to place an order for an article relating to a subject being observed via an optical device. [Solution] A control device comprising an optical system, a display means, and a control means for controlling a display on the display means, wherein the control means controls the display on the display means so that a subject image obtained via the optical system and information relating to purchase of an article associated with the subject can be observed simultaneously by a user.

Description

Control devices, optical systems, control methods, and programs

The present invention relates to a control device for ordering an article using an optical device.

In recent years, in the field of mail-order sales, a system for ordering goods (goods) via a mail-order sales site by connecting to the Internet using a mobile terminal such as a PC or smartphone has become widespread. Patent Document 1 discloses a control device for purchasing a product related to a subject displayed on a television on the spot. Patent Document 2 discloses an electronic camera that displays advertisement information in a live view according to the position and orientation of the imaging unit.

JP-A-2002-335518 JP-A-2009-130688

However, with the control device disclosed in Patent Document 1, since the user needs to operate the mouse to purchase the product, the product cannot be purchased without disturbing the viewing or watching the event. The electronic camera disclosed in Patent Document 2 does not disclose a method for purchasing a product.

Therefore, an object of the present invention is to make it possible to order an article related to the subject being observed through the optical device.

The control device as one aspect of the present invention includes an optical system, a display means, and a control means for controlling the display of the display means. The control means controls the display of the display means so that the image of the subject obtained through the optical system and the information regarding the purchase of the article associated with the subject are simultaneously observed by the user.

An optical system as another aspect of the present invention includes the control device and a storage means for storing subject data and article data.

The control method as another aspect of the present invention includes an acquisition step of acquiring information regarding the purchase intention of the article associated with the subject observed by the user via the optical system, and the purchase of the article in the acquisition step. When the information regarding the intention is acquired, it has a creation step of creating ordering information of the article and an ordering process of transmitting the ordering information created in the creating step to the order receiving means.

The program as another aspect of the present invention causes the computer to execute the control method.

Other objects and features of the present invention will be described in the following examples.

According to the present invention, it is possible to order an article related to the subject being observed through the optical device.

It is a block diagram of the optical system in Example 1. FIG. It is explanatory drawing of the article ordering method in Example 1. FIG. It is a flowchart of the article ordering method in Example 1. It is explanatory drawing of the article ordering method in the modification of Example 1. FIG. It is a block diagram of the optical system in Example 2. It is explanatory drawing of the article ordering method in Example 2. It is a flowchart of the article ordering method in Example 2. It is explanatory drawing of the article ordering method in Example 2. It is explanatory drawing of the article ordering method in Example 2. It is a block diagram of the optical system in Example 3. It is a flowchart of the article ordering method in Example 3. It is explanatory drawing of the article ordering method in Example 3. It is explanatory drawing of the article ordering method in Example 3.

Hereinafter, examples of the present invention will be described in detail with reference to the drawings.

First, Example 1 of the present invention will be described. In this embodiment, it is assumed that a user uses an optical device in a stadium where a sports game is held. Then, the outline of this embodiment will be described by taking as an example a case where a user places an order for an article related to a subject observed by an optical device.

When the user observes a specific subject with an optical device, information on articles related to the subject is output on a small display. For example, items such as towels and key chains corresponding to each player. When the user indicates the purchase intention for the goods by the method described later, the order information regarding the goods transaction is immediately transmitted to the seller. Then, the ordered item is delivered to your seat.

Hereinafter, the configuration and processing flow of the optical system of this embodiment will be described in detail.

First, the configuration of the optical system in this embodiment will be described with reference to FIG. FIG. 1 is a block diagram of the optical system 10. The optical system 10 is roughly divided into a component on the user side and a component on the seller side.

First, the components on the user side will be explained. The user side is mainly composed of the optical device 100. The optical device 100 includes an optical unit 110 having an optical system 111 and an image pickup element 112. The optical unit 110 photoelectrically converts the image of the subject formed via the optical system 111 by the image sensor 112 and converts it into image information. The optical device 100 has a display unit (display means) 130 for displaying image information. The display unit 130 has a small display such as a liquid crystal monitor or an electronic viewfinder (EVF) as a display means. When the display means is an EVF, the user can immerse himself in the observation and can concentrate more on the subject observation, which is preferable.

The optical system 111 is preferably a telephoto optical system having an angle of view of 25 degrees or less (equivalent to 100 mm in 35 mm equivalent) at all angles of view (total diagonal angles of view). More preferably, the optical system 111 is a telephoto optical system having an angle of view of 10 degrees or less in all diagonal angles of view. This makes it possible to easily limit the observation target. On the contrary, when the optical system 111 is a wide-angle optical system, it is not preferable because an object existing in a wide range is projected and it becomes difficult to determine which subject's article information should be displayed.

The optical device 100 has a storage unit 180 capable of recording and holding personal information of a user. Here, "personal information" is information necessary for making an article available for purchase, such as name and credit card information. In addition, the location information of the user at the event venue is also included in the personal information. For example, if the user's seat is specified in advance, the seat number is also included in the personal information. Further, even when the seat is not decided, the personal information may include the user's position information acquired by a position information acquisition device such as a beacon installed at the venue or GPS (Global Positioning System). In that case, the user needs to carry a device corresponding to the position information acquisition device, and the optical device 100 may include the function. In addition, these personal information may be associated with the personal information entered at the time of purchasing the sports watching ticket.

Further, the optical device 100 includes a temporary storage unit (temporary storage means) 170 for temporarily holding article information (information regarding purchase of articles) described later, and a communication unit (communication unit) capable of communicating with an external server described later. Ordering means) 190. Further, the optical device 100 has a control unit (control means) 150 for displaying the article information obtained from the outside via the communication unit 190 on the display unit 130 at an arbitrary timing. Further, the optical device 100 has a communication unit 140 for determining the purchase intention of the user based on the operation of the user. The communication unit 140 is an acquisition means for acquiring information on the purchase intention. The communication unit 140 has a communication means (not shown) that can be operated by the user. The communication means is an operation member such as a button or a dial, or a device capable of transmitting the user's intention such as a touch panel or a line-of-sight detection device.

Further, the optical device 100 has a subject recognition unit (recognition means) 120 for recognizing or identifying a subject that the user is observing through the optical system 111. In this embodiment, the subject recognition unit 120 has an image processing unit 121 in order to recognize the subject. The image processing unit 121 has an image processing means for recognizing a subject by performing face recognition or object recognition on a captured image by image processing. Further, the optical device 100 has an ordering information creating unit (creating means) 160 that creates ordering information of an article based on personal information and article information described later according to the purchase intention of the user. The ordering information created by the ordering information creating unit 160 can be transmitted to the seller side via the communication unit 190.

Next, the components on the seller side will be explained. The seller side is mainly composed of a server and a terminal 230. The server has an article information holding unit (first storage means) 200 for holding article information (article data) regarding the article to be sold. The "article information" is the name of the article, the image of the article, the description for the article, the price information, and the like. Further, the server has a subject information holding unit (second storage means) 210 for holding subject information (subject data). The "subject information" is information necessary for outputting article information related to the subject to a small display when the user observes a specific subject with the optical device 100. When the subject is recognized by the face, the face information of the subject becomes the subject information. When recognizing by the uniform number (characters) attached to the uniform worn by the subject, the uniform number information (character information) becomes the subject information. When recognizing a subject with a specific marker, the information contained in the marker becomes the subject information.

The article information holding unit 200 and the subject information holding unit 210 in this embodiment are cloud servers existing on the Internet. Each of the data stored in the article information holding unit 200 and the data stored in the subject information holding unit 210 also includes information for associating the subject information with the article information. A specific example is shown in FIG.

2A and 2B are explanatory views of an article ordering method using the optical system 10, FIG. 2A shows the stored data of the subject information holding unit 210, and FIG. 2B shows the stored data of the article information holding unit 200. .. The "subject information" in FIG. 2A includes subject information (face information of subject A, information on the uniform number of subject B, etc.) necessary for recognizing each of subject A, subject B, and subject C. Is included. The "article information" in FIG. 2B includes information on the articles to be sold (names and prices of towels, key chains, etc.) as described above. The article information holding unit 200 and the subject information holding unit 210 each hold an "article information number" as information for associating the article information with the subject information. As a result, the same article information numbers possessed by the article information holding unit 200 and the subject information holding unit 210 are linked to each other. Therefore, when the subject observed by the optical device 100 is recognized as the subject A, the optical device 100 acquires the article information (information of the towel 1 and the towel 2 in this example) which is the article information number 001. .. Further, when a plurality of article information (towel 1 and towel 2) are included as in the article information number 001, one subject is given an "article information sub number" as shown in FIG. 2 (b). Can have multiple article information.

The server also has an order receiving unit (ordering means) 220 for controlling the order status of the seller. In this embodiment, the order receiving unit 220 is a cloud server existing on the Internet. In this embodiment, the article information holding unit 200, the subject information holding unit 210, and the ordering unit 220 have been described as independent cloud servers for the sake of simplicity, but the present invention is not limited thereto. For example, one server may have the functions of the article information holding unit 200, the subject information holding unit 210, and the order receiving unit 220. The seller also has a terminal 230 for actually receiving an order from the user. For example, the terminal 230 can receive order information from the order receiving unit 220 and receive an instruction as to what item should be moved or delivered to where. The above is a description of the configuration of the optical system in this embodiment.

Next, the operation (article ordering method) of the optical system 10 of this embodiment will be described with reference to FIG. FIG. 3 is a flowchart of an article ordering method. Each step of FIG. 3 is mainly executed by each part of the optical system 10 by the control unit 150 or based on a command of the control unit 150 or the like.

First, the following three things are done as advance preparations.
(1) The user records his / her personal information in the optical device 100 he / she uses. The content of personal information is as described above.
(2) The seller registers the article information of the article to be sold in the article information holding unit 200.
(3) The seller registers the subject information in the subject information holding unit 210.

After the above-mentioned advance preparation is completed, in step S101, the user connects the optical device 100 to the network in the stadium. The connection method may be wireless connection or wired connection. Subsequently, in step S102, the article information and the subject information are downloaded from the article information holding unit 200 and the subject information holding unit 210 to the optical device 100. When a plurality of article information and subject information exist, they may be downloaded all at once, or if it takes time, they may be downloaded separately. Further, when the time when the player included in the subject information participates is decided, the download may be executed at the timing of that time.

Subsequently, in step S103, the user observes a specific subject with the optical device 100. Subsequently, in step S104, the control unit 150 of the optical device 100 confirms whether or not the subject (observed subject) observed by the user is included in the subject information. If the subject information regarding the observation subject does not exist in the subject information, the process returns to step S103. On the other hand, if the subject information regarding the observed subject is included in the subject information, the process proceeds to step S105. In step S105, the optical device 100 recognizes the subject based on the subject information. Subsequently, in step S106, the control unit 150 searches for and identifies the relevant article information based on the subject recognition result and the information (article information number) for associating the subject information with the article information.

Subsequently, in step S107, the control unit 150 displays the article information specified in step S106 on the small display which is the display unit 130. The display method may be superimposed on the image being observed, or a band-shaped space may be prepared on the side or bottom of the screen for display. As described above, in this embodiment, the display unit 130 can simultaneously display information about the subject and the article. Here, the control unit 150 can determine whether or not to display the article information on the display unit 130 based on the position of the optical device 100. For example, when the position of the optical device 100 is outside the stadium, the control unit 150 determines that the article information is not displayed on the display unit 130. As a result, the article information cannot be displayed from a place other than the stadium, which is preferable because it is possible to prevent an illegal purchase operation. Further, preferably, when it is determined that the control unit 150 does not display the article information on the display unit 130 based on the position of the optical device 100, the following operation is preferably performed. Specifically, it is preferable that the control unit 150 automatically erases the subject information and the article information stored in the temporary storage unit 170. As a result, the temporary storage unit 170 can be efficiently used. In addition, it is preferable because it is possible to prevent a purchase operation from outside the stadium due to unauthorized access.

Subsequently, in step S108, the communication unit 140 determines whether or not the user intends to purchase based on the user operation result in the communication means. If it is determined that the user has a purchase intention, the process proceeds to step S109. In step S109, the ordering information creation unit 160 creates ordering information based on the goods information and personal information that the user has indicated the purchase intention. Subsequently, in step S110, the control unit 150 transmits the ordering information created in step S109 to the order receiving unit 220 on the seller side.

Subsequently, in step S111, the order receiving unit 220 receives the ordering information transmitted in step S110, and the salesperson at the venue transmits the necessary information among the ordering information to the terminal 230. However, since it is sufficient that the salesperson can access the ordering information in step S111, the means thereof is not limited. As an order notification, the fact that the order has been received may be notified to the terminal 230, and the salesperson who receives the notification may access the order information on the order unit 220 via the terminal 230. The ordering information may include information about where to deliver the goods. Immediate delivery is also possible if the ordering information includes user location information such as seating information.

Subsequently, in step S112, the salesperson who has accessed the ordering information actually takes action. Based on the ordering information, the salesperson confirms the inventory of the ordered goods, whether or not payment is made, and the delivery method. The ordering information includes the user's location information. Therefore, when the delivery to the user's seat is desired, the salesperson or the agent delivers to the user's seat. If it has been paid, it will be delivered on the spot and the process will end, and if it has not been paid, it will be accounted for on the spot. At that time, personal information may be collated as identity verification. Alternatively, a serial number or barcode unique to the optical device 100 may be attached to confirm that the optical device 100 is the one used at the time of ordering. In addition, instead of delivery, the user may visit the store at any time to receive the item, or use a locker dedicated to receiving the item. This is the step from ordering to delivery.

On the other hand, if the user has no intention of purchasing in step S108, the process proceeds to step S120. Here, "there is no purchase intention" means, for example, that the user operation indicating the purchase intention is not performed for a certain period of time in the communication means. Specifically, it corresponds to a state in which the purchase decision button is not pressed even though the article information is displayed. When there is a plurality of article information corresponding to the subject, the display is switched to other article information after a certain period of time has elapsed. In this case, the process returns to step S107 in order to switch the display content of the article information. On the other hand, if there is only one article information corresponding to the subject, the process proceeds to step S121. In step S121, the optical device 100 may hold the display of the current article information or erase the display after a certain period of time. Here, although the article information is switched at a certain time, the display content of the article information may be switched to the content of other article information by the user's operation.

As a modification of this embodiment, it can also be used in the following situations. For example, it can be used to order food and drink while watching sports at a stadium. In a situation where a salesperson sells food and drink such as beer at the stadium, the subject information and the article information are configured as shown in FIG. 4A and 4B are explanatory views of an article ordering method in the present modification, FIG. 4A shows the stored data of the subject information holding unit 210, and FIG. 4B shows the stored data of the article information holding unit 200.

Subject A to D in FIG. 4A are salespeople, the salesperson of subject A is beer, the salesperson of subject B is highball, and so on, and each salesperson sells different items. In this example, since each salesperson sells one item, the item information subnumber is unnecessary. The flow of ordering goods at this time is the same as in FIG. The flow will be explained briefly by interweaving concrete examples as follows.

When the user wants to order beer, the salesperson of subject A is observed with the optical device 100. The subject recognition unit 120 recognizes that the subject is the subject A by face recognition or the like. Subsequently, the optical device 100 accesses the subject information holding unit 210 and the article information holding unit 200 via the communication unit 190, searches based on the article information number, and causes the display unit 130 to display the article information (beer). .. This allows the user to be informed that the beer is on sale.

Next, if the user has a purchase intention, the user conveys the purchase intention to the optical device 100 by the communication means of the communication unit 140. As a result, the optical device 100 creates ordering information such as the position of the user (or the position of the optical device 100) and the number of beer orders in the ordering information creating unit 160, and orders the ordering information to the ordering unit 220 via the communication unit 190. To send. The salesperson carries the terminal 230, and the order notification is transmitted from the ordering unit 220 to the terminal 230. The salesperson confirms the order notification and delivers the beer to the user. Payment may be made at the time of delivery, or payment may be made in advance by a credit card or the like when the purchase intention is transmitted to the optical device 100 by means of communication. Further, preferably, when the salesperson is observed by the optical device 100, information such as an estimated time until delivery of the ordered item and the number of people waiting is also displayed. This is preferable because the user can order food and drink in consideration of the waiting time.

According to this embodiment, it is possible for the user to quickly order an item corresponding to the subject being observed through the optical device without disturbing the viewing or watching of the event.

Next, Example 2 of the present invention will be described. In this embodiment, it is assumed that the user uses the optical device 100a at a venue of a music event where a live performance of an artist is held.

First, the configuration of the optical system 10a in this embodiment will be described with reference to FIG. FIG. 1 is a block diagram of the optical system 10a. The optical device 100a of the present embodiment has the optical unit 110a, the subject recognition unit 120a, and the display unit 130a in place of the optical unit 110, the subject recognition unit 120, and the display unit 130. It is different from the optical device 100.

Specifically, the optical unit 110a does not have an image sensor 112, and is composed of only an optical system 111 capable of focusing (focus position adjustment) on a subject at an arbitrary distance. The display unit 130a has a finder for observing the optical image formed by the optical system 111. FIG. 6 is an explanatory diagram of an article ordering method and shows a display area of a finder. As shown in FIG. 6, the finder has an optical image display area 601 and an article information display area 602. The optical image display area 601 has, for example, a frosted glass plate (a so-called focus plate as in a single-lens reflex camera), and is an area for capturing an optical image formed by the optical system 111. The article information display area 602 has a liquid crystal display and an organic EL display, and is an area capable of displaying article information as described in the first embodiment.

The subject recognition unit (recognition means) 120a has a sensor unit 122 that identifies the subject that the user is observing via the optical system 111. The sensor unit 122 has measuring means for measuring the position, posture, time of the optical device 100a, and the distance from the optical device 100a to the place where the optical system 111 is in focus. Specifically, the sensor unit 122 includes a GPS, a gyro sensor, a clock, and focus position information acquisition means of the optical system 111. The position of the optical device 100a is measured by GPS, and the posture of the optical device 100a is measured by a gyro sensor. In addition, the focus position information of the optical system 111 is used to measure which area the user is paying attention to.

By combining the information, the subject recognition unit 120a calculates which area in the subject space the user is observing. Also, by incorporating the time information from the clock, the timing of observation can be grasped. Here, the focus position information acquisition means of the optical system 111 actually focuses at a distance from the optical device 100 based on the relative position information of the focusing lens of the optical system 111 of the optical unit 110, for example. It is a means to calculate whether or not the lenses are matched. Further, in addition to GPS, a beacon or the like to which BLE (Bluetooth Low Energy) is applied may be provided. As a result, the position of the optical device 100a (that is, the position of the user) can be detected even indoors with a certain degree of accuracy. With such a configuration, it is possible to know when and where the user is observing. However, although it is possible to know when and where the user is observing, it is not possible to directly recognize the subject as it is. Therefore, information on the time and position where the subject is scheduled to exist is added to the "subject information" possessed by the subject information holding unit 210, which is a component on the seller side. Then, when the user is observing the position matching the subject information with the optical device 100a at the matching time, the article information is displayed in the article information display area 602.

Next, the operation of the optical system 10a of this embodiment will be described. First, as in the first embodiment, the following three things are performed as advance preparations.
(1) The user records his / her personal information in the optical device 100a used by him / her. The content of personal information is as described above.
(2) The seller registers the article information of the article to be sold in the article information holding unit 200.
(3) The seller registers the subject information in the subject information holding unit 210.

At this time, the difference from the first embodiment is that the data stored in the subject information holding unit 210 includes the subject name, the position information of each subject, and the time information in the subject information. The subject name is a name for distinguishing a plurality of subjects, and can be arbitrarily given. The position information is, for example, GPS information, and represents latitude and longitude in degrees, minutes, and seconds.

9A and 9B are explanatory views of an article ordering method using the optical system 10a. FIG. 9A shows the stored data of the subject information holding unit 210, and FIG. 9B shows the stored data of the article information holding unit 200. Shown. In the example of the subject A in FIG. 9A, "12 ° 34'56.7" N "and" 12 ° 34'56.7 "E" correspond to GPS information, and the north latitude is 12 degrees 34 minutes 56.7. Seconds, 12 degrees 34 minutes 56.7 seconds east longitude (the last alphabet E, W, S, N indicates east, west, south, north). When a beacon is used instead of GPS for use in an indoor venue, the relative position information between the beacons and the relative position information between each beacon and the optical device 100a may be input.

FIG. 8 is an explanatory diagram of an article ordering method using the optical system 10a. For example, in the case of the positional relationship as shown in FIG. 8, the position of the subject A represents a circular range having a radius of 1 m at a position of beacon 001 to 4 m and a beacon 007 to 8 m. Further, the data stored in the subject information holding unit 210 includes time information for each subject. Taking the subject A in FIG. 9 (a) as an example, when the user observes the center of the main stage between "10:00 and 15 seconds" and "10:30 AM", it corresponds to the article information number of the subject A. The article information to be used is displayed on the display unit 130.

Next, with reference to FIG. 7, the operation (article ordering method) of the optical system 10a of this embodiment will be described. FIG. 7 is a flowchart of an article ordering method. Each step of FIG. 7 is mainly executed by each part of the optical system 10a by the control unit 150 or based on a command of the control unit 150 or the like.

First, in step S201, the user connects the optical device 100a to the network in the venue. The connection method may be wireless connection or wired connection. Subsequently, in step S202, the control unit 150 of the optical device 100a downloads the article information and the subject information from the article information holding unit 200 and the subject information holding unit 210. If there are a plurality of article information and subject information, they may be downloaded all at once, or if it takes time, they may be downloaded separately.

Subsequently, in step S203, the user observes a specific subject space (the specific subject may not actually exist) with the optical device 100a at a certain time. Subsequently, in step S204, the optical device 100a uses the sensor unit 122 of the subject recognition unit 120a to determine the position of the user, the direction / position the user is looking at, and the distance from the optical device 100 to the in-focus position. Measure. The position of the user is the latitude / longitude information indicated by the GPS when GPS is used. When a beacon is used, it can be measured by triangulation based on the radio field intensity between two or more beacons whose positions are known and the optical device 100a as in the prior art. Then, the sensor unit 122 measures the position of the optical device 100a, the posture of the optical device 100a (the direction in which the user is looking), and the distance from the optical device 100a to the in-focus position.

Subsequently, in step S205, the optical device 100a confirms whether or not the position and time measured in step S204 are included in the subject information. If the position and time are not included in the subject information, the process returns to step S204. On the other hand, if the position and time are included in the subject information, the process proceeds to step S206. In step S206, the optical device 100a recognizes the subject based on the subject information. Subsequently, in step S207, the optical device 100a searches for and identifies the relevant article information based on the subject recognition result and the information (article information number) for associating the subject information with the article information. Subsequently, in step S208, the article information specified in step S207 is displayed in the article information display area 602 of the display unit 130 shown in FIG. In this embodiment, since the subsequent steps S209 to S213, S220, and S221 are the same as the steps S108 to 112, S120, and S221 of the first embodiment, their description will be omitted.

Next, Example 3 of the present invention will be described. In this embodiment, it can be used at an event venue where a sports game or an artist's live performance is held, and it is assumed that it is used in the scenes described in Example 1 and Example 2.

First, the configuration of the optical system 10b in this embodiment will be described with reference to FIG. FIG. 10 is a block diagram of the optical system 10b. The optical system 10b of the present embodiment includes a communication device 300 represented by a smartphone between the optical device 100b and the order receiving unit 220, and an article information holding unit 200 that also serves as a subject information holding unit. In that respect, it differs from Example 1 and Example 2. The configuration of the optical system 10b of this embodiment is roughly divided into a component on the user side and a component on the seller side, as in the case of the first embodiment.

First, the components on the user side will be described. The user side is mainly composed of an optical device 100b and a communication device 300. The configuration of the optical device 100b is different from that of the optical device 100 of the first embodiment in the following three points.
(1) Since the ordering information creating unit 160 is provided in the communication device 300 described later, it is not described in the optical device 100b. Further, the ordering information creation unit 160 in the communication device 300 does not have to have as hardware, and may have as software (a part of the application).
(2) Since the personal information is stored in the storage unit of the communication device 300, it is not necessary to record the personal information in the storage unit 180 of the optical device 100b. The storage unit 180 may be used to store the captured image or video.
(3) The subject recognition unit 120 is moved to an external cloud server. For example, complicated subject recognition using deep learning can be performed without imposing a load on the main body of the optical device 100b. The image processing unit 121 of the optical device 100b plays a role of adjusting the brightness, contrast, and the like so that the user can easily see the captured image information on the display unit 130.

Next, the communication device 300 in this embodiment will be described. The communication device 300 is a portable communication device such as a smartphone. The communication device 300 has a communication unit (ordering means) 330 for communicating with an external device, and a storage unit 310 for storing personal information and the like necessary for ordering goods. Further, the communication device 300 has an intention confirmation unit 141 for confirming the user's intention to purchase goods. The intention confirmation unit 141 is an acquisition means for acquiring information regarding an intention to purchase an article. The intention confirmation unit 141 has a user-operable intention confirmation transmission means (not shown). The intention confirmation communication means is a device capable of transmitting the presence or absence of the user's intention confirmation, such as an operation member such as a button or a dial, or a touch panel. Further, the communication device 300 has an order information creation unit 160 that creates order information based on the result of confirmation of the user's intention to purchase goods. Further, the communication device 300 has a control unit 320 that controls them. The communication device 300 can be connected to the optical device 100b and the order receiving unit 220 described later by a wired or wireless network. The above are the components on the user side.

Next, the components on the seller side will be explained. The optical system 10 of the first embodiment has an article information holding unit 200 and a subject information holding unit 210, but since the article information holding unit 200 is configured to hold the subject information in this embodiment, it is an independent subject information holding unit. Does not need. Further, in this embodiment, since the article information and the subject information are managed collectively, the total amount of data held in one cloud server is large, but the information for associating the subject information and the article information is unnecessary. There is a merit.

FIG. 12 is an explanatory diagram of an article ordering method using the optical system 10b, and shows the stored data of the article information holding unit 200. Similar to the first embodiment, the “article information” includes the name of the article, the image of the article, the description for the article, the price information, and the like. Further, the "subject information" includes information necessary for outputting article information related to the subject to a small display when the user observes a specific subject with an optical device. When the subject is recognized by the "face", the face information of the subject becomes the subject information. When recognizing by the uniform number attached to the uniform worn by the subject, the uniform number information becomes the subject information. When recognizing a subject with a specific marker, the information contained in the marker becomes the subject information. Since other configurations such as the order receiving unit 220 and the terminal 230 are the same as those in the first embodiment, their description will be omitted.

Next, the operation of the optical system 10b of this embodiment will be described. First, the following two things are done as advance preparations.
(1) The user records his / her personal information in the communication device 300 he / she uses. Alternatively, if personal information is stored on an external Internet server, information for accessing it may be recorded. The content of the personal information is as described in the first embodiment.
(2) The seller registers the article information and the subject information of the article to be sold in the article information holding unit 200.

Next, with reference to FIG. 11, the operation (article ordering method) of the optical system 10b of this embodiment will be described. FIG. 11 is a flowchart of an article ordering method. Each step of FIG. 11 is mainly executed by the control unit 150 or the control unit 320, or by each unit of the optical system 10b based on a command of the control unit 150 or the control unit 320 or the like.

First, in step S301, the user connects the communication device 300 to the network in the venue. The connection method may be wireless connection or wired connection. Subsequently, in step S302, the user brings the optical device 100b and the communication device 300 into a communicable state. Also for this, the connection method may be wireless connection or wired connection. Note that steps S301 and S302 are not limited to this order, and the order may be reversed.

Subsequently, in step S303, the user observes a specific subject with the optical device 100b. Subsequently, in step S304, the optical device 100b transfers the captured image or the captured image to the subject recognition unit 120 via the communication unit 190 of the optical device 100b. In this embodiment, the subject recognition unit 120 is an external cloud server, and although it is described in FIG. 10 that it directly communicates with the communication device 300, it may directly communicate with the optical device 100b.

Subsequently, in step S305, the subject recognition unit 120 confirms whether or not the observed subject (observed subject) is included in the subject information. If the subject information about the observed subject does not exist in the subject information, the process returns to step S304. On the other hand, if the subject information includes subject information about the observed subject, the process proceeds to step S306. In step S306, the optical device 100b recognizes the subject based on the transferred captured image or captured image and the subject information. At this time, an existing general-purpose database may be used, or a database specialized for the subject of an arbitrary event or the like may be used.

Subsequently, in step S307, the optical device 100b searches for and identifies the relevant article information based on the subject recognition result. In this embodiment, since the subject information and the article information are stored on one server, the "article information number" used when communicating between the servers as described in the first and second embodiments is Not needed. Instead, as shown in FIG. 12, for example, the data in the list showing that "subject A" and "towel 1, towel 2" are linked is "information for linking the subject information and the article information". ". In this example, the list is shown in an easy-to-understand manner, but the display method is not limited to this as long as the subject information and the article information are linked. When there are a plurality of article information (towel 1 and towel 2) as in the example of the subject A, by assigning an "article information sub-number", a plurality of article information can be assigned to one subject as in the first and second embodiments. It is possible to have goods information. At this time, the optical device 100b may transmit the subject recognition result to the communication device 300 or the optical device 100 to further analyze the subject, or the subject recognition unit 120 directly accesses the article information holding unit 200 to provide article information. May be specified.

Subsequently, in step S308, the control unit 150 of the optical device 100b displays the article information specified in step S307 on the small display which is the display unit 130. The display method may be superimposed on the image being observed, or a band-shaped space may be prepared on the side or bottom of the screen for display. Subsequently, in step S309, the communication unit 140 determines whether or not the user has an intention to purchase, based on the user operation result in the communication means possessed by the communication unit 140. If it is determined that there is a purchase intention, the process proceeds to step S310. In step S310, the intention confirmation unit 141 of the communication device 300 confirms the purchase intention of the user based on the operation of the user in the communication device 300. For example, the user operates a specific button (not shown) in the optical device 100b while the specific article information is displayed on the display unit 130. At this time, as shown in FIG. 13, the communication device 300 (for example, a smartphone) is displayed so as to confirm the purchase intention of the user, and the purchase intention of the user is confirmed. FIG. 13 is an explanatory diagram of an article ordering method, and shows a display of the communication device 300 at the time of confirming the purchase intention. The button operation is an example, and a means other than the button operation may be used. Further, every time the article information is displayed on the display unit 130 of the optical device 100b, the display as shown in FIG. 13 is automatically displayed on the screen of the communication device 300 without allowing the user to perform an operation such as a button operation. You may check each time. In that case, the purchase intention determination in step 309 is always YES, and only the purchase intention confirmation in step S310 is performed.

If the intention confirmation unit 141 confirms that the user has a purchase intention in step S310, the process proceeds to step S311. In step S311, the ordering information creation unit 160 of the communication device 300 creates ordering information based on the goods information and personal information that the user has indicated the purchase intention. Subsequently, in step S312, the communication device 300 transmits the order information created in step S311 to the order receiving unit 220 on the seller side via the communication unit 330. Subsequently, in step S313, the order receiving unit 220 receives the ordering information transmitted in step S312, and the terminal 230 of the store at the venue or the salesperson carries the information required by the salesperson at the venue among the ordering information. Send to terminal 230. However, in step S313, the means is not limited as long as the salesperson can access the ordering information. The fact that the order has been received may be notified to the terminal 230 as an order notification, and the salesperson who receives the notification may access the order information on the order unit 220 via the terminal 230. The ordering information may include information about where to deliver the goods. If the position information of the optical device 100b is included in the ordering information, immediate delivery is also possible.

Subsequently, in step S314, the salesperson who has accessed the ordering information actually takes action. Since this is the same as that of the first embodiment, the description thereof will be omitted. If it is determined in step S309 that there is no purchase intention, or if the user's purchase intention cannot be confirmed in step 310, the process proceeds to step S320. Since steps S321 and S322 are the same as those in the first embodiment, the description thereof will be omitted.

According to this embodiment, regarding the control method for ordering an article using the optical device 100b, the user can quickly order the article corresponding to the subject being observed through the optical device 100b. ..

Further, as a modification of this embodiment, it is preferable that the article ordering method described in Examples 1 to 3 is effective only when the user is in a specific venue. As a result, it is possible to prevent unnecessary download of subject information and the like, so that it is possible to prevent an unexpected increase in the amount of communication. Further, as another modification, when the user is wondering whether to purchase a specific item, the communication unit 140 may be provided with a communication means so that the purchase operation can be performed later. For example, a "buy later" button is added in addition to the "buy" and "do not buy" buttons.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

In each embodiment, the control devices (optical devices 100, 100a, 100b) are an acquisition means (communication unit 140, intention confirmation unit 141), a creation means (order information creation unit 160), and an ordering means (communication unit 190). , 330). The acquisition means acquires information regarding the purchase intention of the article associated with the subject being observed by the user via the optical system 111 (or determines whether or not the user has the purchase intention). When the creating means acquires the information regarding the purchase intention of the article, the creating means creates the ordering information of the article. The ordering means transmits the ordering information to the ordering means (ordering unit 220).

Preferably, the control device has recognition means (subject recognition units 120, 120a) and specific means (control unit 150). The recognition means recognizes the subject being observed through the optical system. The identifying means identifies an article associated with the subject. Preferably, the specific means includes subject data (FIGS. 4 (a), 9 (a), 12) and article data (FIG. 4) stored in the storage means (article information holding unit 200, subject information holding unit 210). An article associated with the subject is identified based on (b), FIG. 9 (b), and FIG. 12). According to each embodiment, a control device, an optical system, a control method, which allows a user to order an article related to a subject being observed through an optical device without disturbing the viewing or watching of an event. And the program can be provided.

Although preferable examples of the present invention have been described above, the present invention is not limited to these examples, and various modifications and modifications can be made within the scope of the gist thereof.

Claims

It has an optical system, a display means, and a control means for controlling the display of the display means.
The control means controls the display of the display means so that the image of the subject obtained through the optical system and the information regarding the purchase of the article associated with the subject are simultaneously observed by the user. A control device characterized by.
An acquisition means for the user to acquire information regarding the purchase intention of the article, and
When the acquisition means acquires the information regarding the purchase intention of the article, the creating means for creating the ordering information of the article and the creating means.
The control device according to claim 1, further comprising an ordering means for transmitting the ordering information to the ordering means.
A recognition means for recognizing the subject and
The control device according to claim 1 or 2, further comprising a specific means for identifying the article associated with the subject.
The control device according to claim 3, wherein the specific means identifies the article associated with the subject based on subject data and article data stored in the storage means.
The recognition means has an image processing means for recognizing the subject by performing image processing on the image of the subject.
The control device according to claim 3 or 4, wherein the image processing means recognizes the subject by using at least one of a face, a character, and a marker.
The recognition means has a sensor unit for identifying the subject, and has a sensor unit.
The control device according to claim 3 or 4, wherein the sensor unit measures the posture of the control device.
The control device according to claim 6, wherein the sensor unit further measures at least one of the position and time of the control device or the distance to a place where the optical system is in focus. ..
The control device according to any one of claims 1 to 7, wherein the display means is an electronic viewfinder.
The control means according to any one of claims 1 to 8, wherein the control means determines whether or not to display the information regarding the purchase of the article on the display means based on the position of the control device. The control device described.
Further having a temporary storage means for temporarily storing information regarding the purchase of the article.
When the control means determines that the information regarding the purchase of the article is not displayed on the display means based on the position of the control device, the information regarding the purchase of the article stored in the temporary storage means is erased. The control device according to claim 9.
The control device according to any one of claims 1 to 10, wherein the angle of view of the optical system is 25 degrees or less in all angles of view.
The control device according to any one of claims 1 to 11.
An optical system comprising: a storage means for storing subject data and article data.
The storage means
The first storage means for storing the subject data and
The optical system according to claim 12, further comprising a second storage means for storing the article data.
The acquisition process of acquiring information about the purchase intention of the article associated with the subject that the user is observing through the optical system,
When the information regarding the purchase intention of the article is acquired in the acquisition step, the creation step of creating the ordering information of the article and the creation step.
A control method comprising an ordering process for transmitting the ordering information created in the producing process to an ordering means.
A program characterized by causing a computer to execute the control method according to claim 14.