JP5546394B2 - Imaging control system, control device, control method, and program - Google Patents

Description

  The present invention relates to an imaging control system that controls an imaging apparatus, a control apparatus and control method for the imaging apparatus, and a program.

  2. Description of the Related Art Conventionally, there is an imaging control system that includes an imaging device connected to a network and a client device that controls the imaging device via the network. In the imaging control system, the pan and tilt drive mechanism of the imaging apparatus can be controlled by an operation on the client apparatus. An imaging control system for the purpose of enabling a user to easily point an imaging device in the direction of a target when panning and tilting the imaging device using such an imaging control system is known. (For example, Patent Document 1). The imaging control system described in Patent Document 1 stores a table in which the relationship between pan angle and name display characters such as “east”, “west”, “south”, and “north” is stored in a memory in the imaging device. The imaging device is directed at the pan angle corresponding to the input name display character, or the name display character corresponding to the direction in which the imaging device is directed is displayed on the monitor.

JP 2004-266876 A

  Further, when performing pan / tilt driving of the imaging device, as a method for helping the user to operate the imaging direction of the imaging device in a target direction, it is conceivable to display the pan / tilt driving direction superimposed on the captured image. . However, when the pan and tilt drive directions are displayed in a superimposed manner, there is a problem that the user may not be able to recognize the drive direction in a specific imaging direction.

  Below, the problem which arises in a drive direction display is demonstrated. A description will be given of how the driving direction is displayed on the display image and how the driving direction is changed with the operation of the tilt mechanism with reference to FIG.

  In FIG. 9A, the axis 901 is the imaging direction of the imaging device 110, the display image 910 is an image displayed on the display device, the imaging center 911 is the center of the captured image, and the pan rotation center 912 is the pan rotation center on the display image. Indicates the position. In FIG. 9A, a pan direction guide 913 indicates a guide indicating the pan driving direction by the pan driving mechanism, and a tilt direction guide 914 indicates a direction guide indicating the tilt driving direction by the tilt driving mechanism. The pan direction guide 913 and the tilt direction guide 914 are expressed as straight lines or curves passing through the imaging center 911. These direction guides dynamically change in shape in accordance with changes in the imaging direction, that is, operations of the pan, tilt, and rotation mechanisms.

  A pan operation icon 915 for operating the pan driving mechanism of the imaging apparatus and a tilt operation for operating the tilt driving mechanism of the imaging apparatus are provided at both ends of the pan direction guide 913 and the tilt direction guide 914, respectively. An icon 916 is displayed. The user can drive the imaging device in the pan direction by performing an operation such as clicking on the pan operation icon 915 with the pointer. Similarly, the user can operate the tilt operation icon 916 to drive the imaging apparatus in the tilt direction.

  When the shaft 901 is nearly perpendicular to the pan rotation axis 203 as shown in FIG. 9A, the pan direction guide 913 approaches a straight line and the curvature becomes small. A display image when the tilt mechanism is operated from the state of the shaft 901 and the imaging direction is brought close to the pan rotation axis will be described with reference to FIG. As shown in FIG. 9B, as the imaging direction approaches the pan rotation axis 203, the distance between the imaging center 911 and the pan rotation center 912 is closer on the display image 920, and the curvature of the pan direction guide 913 is growing. As described above, when the imaging direction is in the vicinity of the pan rotation axis 203, the pan direction guide 913 is a circle or arc having a small radius, and it is difficult for the user to recognize the pan driving direction. Further, if the tilt operation is further continued from the state of the image pickup apparatus, the pan direction guide converges to a point and cannot be displayed when the image pickup direction reaches the pan rotation axis. Accordingly, the pan operation icon 915 is not displayed on the display image. Therefore, when the imaging direction overlaps the pan rotation axis, the user cannot operate the pan driving mechanism of the imaging apparatus using the pan operation icon 915.

  As described above, since the pan direction guide displayed so as to pass through the imaging center in a specific state becomes circular or cannot be recognized, there is a problem that the pan operation icons overlap or are not displayed. . In view of the above, an object of the present invention is to solve the above-described problem and to display an icon for performing a pan operation even when the shaft 901 approaches the pan rotation shaft 203.

  In order to achieve the above object, a control device according to the present invention includes an image pickup unit that picks up an image on the image pickup element with an intersection between the optical axis and the image pickup element as an image pickup center, and a pan drive unit that drives the image pickup unit to pan. And a tilt driving means for operating the pan driving means, and a tilt for operating the tilt driving means. An operation means; and a display control means for displaying an icon for inputting a command to the pan operation means so as to be superimposed on a position indicating a pan driving direction in a captured image picked up by the image pickup means. Superimposes the icon so that the distance from the pan rotation center to the icon is longer than the distance from the pan rotation center to the imaging center in the captured image. Wherein the display Te.

  With the above configuration, the present invention can display an icon for performing a pan operation even when the shaft 901 approaches the pan rotation shaft 203.

The block diagram for demonstrating the imaging control system of this invention. Explanatory drawing of an imaging device having a pan, tilt, and rotation mechanism. The figure for demonstrating the display image in Example 1 of this invention. FIG. 6 is a diagram for explaining a display image at the time of rotation driving in the first embodiment. The flowchart for demonstrating operation | movement of the control apparatus concerning this invention. The flowchart for demonstrating operation | movement of the imaging device concerning this invention. The figure for demonstrating the virtual spherical surface centering on an imaging device. The figure for demonstrating the display image in Example 2 of this invention. Explanatory drawing of a prior art example.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Example 1
FIG. 1 shows a block diagram of an imaging control system in the first embodiment of the present invention. In FIG. 1, the imaging device 110 performs imaging. The client device 130 performs setting of the imaging device 110, display of a captured image, and recording. A network 190 connects the imaging device 110 and the client device 130. The network 190 includes a plurality of routers, switches, cables, and the like that satisfy a communication standard such as Ethernet (registered trademark). In the present invention, any communication standard, scale, and configuration may be used as long as communication between each server and client can be performed. For example, the network 190 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

  First, the configuration of the imaging device 110 will be described. The imaging unit 111 includes an imaging element and an optical system that connects an image of a subject on the imaging element, and performs imaging on the imaging element with an intersection between the optical axis of the optical system and the imaging element as an imaging center. The image sensor is an image sensor such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charged Coupled Device).

  The pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 drive the imaging unit 111 based on an instruction from the CPU 115 described later. The pan driving unit 112 pans the imaging unit 111. The tilt driving unit 113 drives the imaging unit 111 to tilt. The rotation driving unit 114 rotates a lens unit 207 (described later) constituting the imaging unit 111 around a rotation rotation shaft 208 (described later).

  The video processing unit 116 processes the video signal captured by the imaging unit 111. The video processing performed in the video processing unit 116 is conversion of a digital video signal of an image signal captured by the imaging unit 111, encoding of the digital video signal, and the like. For example, YUV can be used as the format of the digital video signal. Also, as a method for encoding a digital video signal, for example, MPEG4, H.264, and the like. A system based on a standard such as H.264, MJPEG or JPEG can be used. The video signal subjected to the video processing by the video processing unit 116 is temporarily stored in the memory 117 and is output from the network 190 to the client device 130 via the communication control unit 118 under the control of the CPU 115 described later.

  The communication control unit 118 performs packetization processing on the encoded digital video signal. Further, the communication control unit 118 performs a packet multiplexing process according to a predetermined format in order to output a digital video signal to the network 190 according to the control of the CPU 115. As the predetermined format, for example, a format such as HTTP (Hypertext Transfer Protocol), RTP (Real-time Transport Protocol), or the like can be used. The communication control unit 118 transmits the digital video signal imaged by the imaging unit 111 to the network 190.

  The CPU 115 reads and executes a program stored in the memory 117, thereby controlling the operation of each component included in the imaging device 110. Further, the CPU 115 controls the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 in accordance with a command received from the client device 130 via the network 190. Further, the CPU 115 transmits the pan, tilt, and rotation positions of the imaging unit 111 to the client device 130 via the communication control unit 118 based on the driving results of the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114. Further, the CPU 115 acquires the angle of view of the image capturing unit 111, the size of the captured image captured by the image capturing unit 111, and the like from the image capturing unit 111 and transmits the acquired image to the client device 130.

  An example of the imaging device 110 is shown in FIGS. 2 (a) and 2 (b). FIG. 2A is a side view of the imaging device 110. In FIG. 2A, the pan rotating part is composed of a bottom case 201 and a turntable 202, and the turntable 202 rotates around a pan rotating shaft 203. The pan driving unit 112 is configured by a stepping motor or the like, and pans the imaging unit 111 by rotating the turntable 202.

  FIG. 2B is a front view of the imaging device 110. The tilt rotation unit is composed of a lens column 204 and a lens case 206, and has a structure in which the lens case 206 rotates about a tilt rotation axis 205. The tilt driving unit 113 is configured by a stepping motor or the like, and tilts the imaging unit 111 by rotating the lens case 206. The rotation part of the rotation is constituted by a lens case 206 and a lens unit 207, and the lens unit 207 rotates around a rotation rotation axis 208. The rotation drive unit 114 is configured by a stepping motor or the like, and rotates the imaging unit 111 by rotating the lens unit 207.

  Next, the client device 130 will be described with reference to FIG. Under the control of the CPU 137, the client device 130 receives the captured image transmitted from the imaging device 110 via the communication control unit 131, temporarily stores it in the memory 132, and sends it to the display control unit 133.

  The display control unit 133 superimposes a pan direction guide indicating the pan driving direction by the pan driving unit 112 on the captured image captured by the imaging unit 111. Further, the display control unit 133 superimposes a tilt direction guide indicating a tilt driving direction by the tilt driving unit 113 on the captured image captured by the imaging unit 111. Then, the display control unit 133 causes the display device 170 to display a pan direction guide indicating the pan driving direction by the pan driving unit 112 and a tilt direction guide indicating the tilt driving direction by the tilt driving unit 113 on the captured image. The display mode will be described later with reference to FIG. Further, the display control unit 133 causes an icon for inputting a command to the pan operation unit 138 to be superimposed and displayed at a position indicating the pan driving direction in the captured image captured by the imaging unit 111. Here, on the line representing the pan driving direction means, for example, on a pan main guide 313 described later with reference to FIG. 3 or on a pan auxiliary guide 315 described later. Here, the bread main guide 313 or the bread auxiliary guide 315 does not need to be actually superimposed on the captured image, and an icon is superimposed at a position where the bread main guide 313 or the bread auxiliary guide 315 should be superimposed. It only has to be done.

  The UI control unit 136 receives an instruction from the input device 150 described later, and transmits a command for controlling the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 to the imaging device 110 via the communication control unit 131 by the CPU 137. To do. The UI control unit 136 includes a pan operation unit 138 for operating the pan drive unit 112, a tilt operation unit 139 for operating the tilt drive unit 113, and a rotation operation unit 140 for operating the rotation drive unit 114. . The UI control unit 136 also generates a pan operation icon (hereinafter referred to as a pan operation button) and a tilt operation icon (hereinafter referred to as a tilt operation button) for inputting commands to the pan operation unit 138 and the tilt operation unit 139. It has the generating part 141 which performs. The CPU 137 reads and executes the icon display program stored in the memory 132 and controls the generation unit 141 to generate an icon. Furthermore, the UI control unit 136 includes a pointer unit 142 that instructs the display control unit 133 to display a pointer on the display device 170 in accordance with an instruction from an input device 150 such as a mouse described later.

  The pan operation unit 138 transmits a command for controlling the pan driving unit 112 to the imaging device 110 when detecting that the pan operation button displayed on the display device 170 is clicked with the pointer over the pan operation button. In addition, the tilt operation unit 139 transmits a command for controlling the tilt drive unit 113 to the imaging device 110 when it is detected that the pointer is placed on the tilt operation button displayed on the display device 170 and clicked. A user performs an operation of clicking on each operation button with a pointer over the input button 150, which will be described later. The operation of each operation button is not limited to using a pointer, and each operation button may be operated by a user touching the touch panel. The operation method of each operation button is not limited to the above method, and any operation may be used as long as an instruction is input to the pan operation unit 138 via the operation button. The pan operation button and the tilt operation button are displayed in a superimposed manner at positions indicating the pan driving direction and the tilt driving direction in the captured image, respectively. By doing so, the user can more intuitively perform the pan driving operation and the tilt driving operation.

  The input device 150 is a device for inputting a command for the user to operate the imaging device 110. The input device 150 outputs pan, tilt, and rotation drive commands input by the user to the UI control unit 136. The input device 150 is, for example, a keyboard or a mouse. The user operates the imaging device 110 by operating an operation button or the like displayed on the display device 170 described later using a keyboard or a mouse. Alternatively, the input device and the display device may be integrally configured, and a command to the imaging device 110 may be input using a touch panel or the like.

  The display device 170 displays a captured image captured by the imaging device 110 and a direction guide indicating a driving direction such as panning and tilting by display control of the display control unit 133. Further, the display device 170 displays an icon such as an operation button or a pointer in response to an instruction from the display control unit 133. Although the client device 130 and the display device 170 are shown as independent devices in FIG. 1, the client device 130 and the display device 170 may be configured as a single unit. For example, when a PC (Personal Computer) is used as the client device 130, the PC can also function as the display device 170 by displaying captured images, icons such as direction guides and operation buttons on the PC display. it can.

  Next, an example of a display image displayed on the display device 170 by the display control of the display control unit 133 is shown in FIG. FIG. 3A is a diagram when the imaging direction is nearly perpendicular to the pan rotation axis 203. In FIG. 3A, a display image 310 is a display image displayed on the display device 170 as a result of the display control unit 133 performing display control on the captured image received by the client device 130 from the imaging device 110. The imaging center 311 is an intersection between the optical axis of the optical system that connects the subject image on the imaging device and the imaging device of the imaging unit 111.

  The pan rotation center 312 indicates the position of the pan rotation center in the plane of the display image 310. The pan main guide 313 is a pan direction guide that indicates a pan movement direction so as to overlap a movement path along which the imaging center 311 moves by pan driving. A method for setting the pan rotation center and a display method for the pan main guide 313 will be described later with reference to FIG. The tilt direction guide 314 is a tilt direction guide that indicates a tilt movement direction so as to overlap a movement path along which the imaging center 311 moves by tilt driving. The pan auxiliary guide 315 is a pan direction guide that does not overlap the moving path along which the imaging center 311 moves. The pan auxiliary guide 315 is displayed at an appropriate interval outside the pan main guide 313 with respect to the pan rotation center 312 and indicates the pan driving direction to the user in the same manner as the pan main guide 313. In this manner, the display control unit 133 superimposes the pan direction guide that does not overlap the moving path along which the imaging center 311 moves by pan driving on the captured image. A display method of the bread assist guide 315 will be described later.

  Further, the display control unit 133 displays a pan operation button 317 and a tilt operation button 318 on the display image 310. The pan operation button 317 is an icon used for the user to drive the imaging unit 111 in the pan direction. In the state of FIG. 3A, the pan operation buttons 317 are displayed at both ends of the pan main guide 313. The tilt operation button 318 is an icon used by the user to drive the imaging unit 111 in the tilt direction. In the state of FIG. 3A, the tilt operation buttons 318 are displayed at both ends of the tilt direction guide 314. Of the two pan operation buttons 317, the upper pan operation button 317 is an icon for instructing pan driving in the upward direction, and the lower pan operation button 317 commands pan driving in the downward direction. It is an icon for. Of the two tilt operation buttons 318, the left tilt operation button 318 is an icon for instructing pan driving in the left direction, and the right pan operation button 317 is instructing pan driving in the right direction. It is an icon to do. By displaying the pan operation button and the tilt operation button on the pan direction guide and the tilt direction guide, respectively, the user can more intuitively recognize the drive direction and perform the pan drive operation and the tilt drive operation.

  Here, a display method of the pan main guide 313 and a setting method of the pan rotation center 312 will be described with reference to FIG. In FIG. 3E, a pan rotation axis 203 is a rotation axis for the imaging unit 111 to perform pan rotation, and an imaging apparatus position 300 is a position of the imaging apparatus. In FIG. 3E, a captured image 302 is a captured image virtually arranged so as to be orthogonal to the imaging direction of the imaging unit 111, and a vector 301 is a vector indicating the direction of the center of the captured image 302. The size of the vector 301 is set according to the angle of view of the captured image 302 and the size of the captured image 302.

The display control unit 133 performs display control of the pan main guide 313 and setting of the pan rotation center 312. First, the display control unit 133 obtains the size of the vector 301 from the angle of view acquired from the imaging device 110 and the size of the captured image 302. The size of the vector 301 is obtained by, for example, (Equation 1). However, the size of the vector 301 is L, the length of the diagonal line of the captured image 302 is 2d, and the diagonal view angle of the captured image 302 is 2α.
(Formula 1) L = d / tan α
Further, the display control unit 133 obtains an angle 303 formed by the pan rotation axis 203 and the vector 301 from the tilt angle acquired from the imaging device 110. Then, the display control unit 133 derives the length of the pan rotation radius 304 from the previously obtained size L of the vector 301 and the angle 303. The length of the pan rotation radius can be obtained by (Equation 2). However, the length of the pan rotation radius 304 is r, and the angle 303 is β.
(Formula 2) r = L · sin β
The bread rotation radius 304 thus obtained is used as a radius, and the circumference of a circle orthogonal to the pan rotation axis 203 is obtained as the pan rotation trajectory 305. Then, the pan main guide 313 is displayed by projecting and depicting the pan rotation trajectory 305 viewed from the imaging device position 300 from the imaging device position 300 to the virtual captured image 302.

  Further, the intersection of the plane including the captured image 302 and the pan rotation axis 203 is set as the pan rotation center 312. However, in the present embodiment, when the angle 303 is 90 degrees, the pan rotation center 312 is on a line that passes through the imaging center of the captured image 302 in the plane of the captured image 302 and is parallel to the pan rotation axis 203, and above the drawing. It shall be set as existing. The setting method of the pan rotation center 312 when the angle 303 is 90 degrees is not limited to this. When the angle 303 is 90 degrees, the pan rotation center 312 may not be set on the plane of the captured image 302.

  The display method of the pan main guide 313 and the setting method of the pan rotation center 312 are not limited to this. For example, a point moved from the imaging center 311 of the display image 310 by a distance equal to the pan rotation radius 304 in the tilt drive direction and upward in FIG. Then, an arc of a circle drawn with the pan rotation radius 304 as a radius from the pan rotation center 312 may be displayed as the pan main guide 313. Further, the bread main guide 313 may not be represented by an arc. For example, the pan driving direction may be indicated by displaying an arrow as shown in FIG. For example, the arrow can be drawn so as to circumscribe a curve drawn by projecting the pan rotation trajectory 305. Alternatively, the arrow can be displayed so that both ends of the arrow are inscribed in a curve drawn by projecting the pan rotation trajectory 305. These are examples for indicating the pan driving direction, and the display method is not limited to the above.

  Next, a display method of the bread assist guide 315 will be described. The bread auxiliary guide 315 can be displayed so that the distance from the bread main guide 313 is constant. Alternatively, the pan driving direction may be indicated by displaying an inscribed or circumscribing arrow on a curve displayed so that the distance from the pan main guide 313 is constant in the same manner as the pan main guide 313. The pan assist guide 315 is displayed such that the distance from the pan rotation center 312 to the pan assist guide 315 is longer than the distance from the pan rotation center 312 to the imaging center 311 in FIG. In FIG. 3A, the bread main guide 313 is shown as a solid line and the bread auxiliary guide 315 is shown as a broken line, but the bread main guide 313 may be shown as a broken line, or the bread auxiliary guide 315 may be shown as a solid line. The display method of the pan auxiliary guide 315 is not limited to the above-described method, and any method may be used as long as it indicates the pan driving direction so as not to overlap the moving path along which the imaging center 311 moves by pan driving.

  3A, the imaging apparatus 111 is instructed to be tilted from the input device 150 and is tilted in a direction toward the pan rotation center position (upward in the figure). The display image 320 when approaching will be described with reference to FIG. When the imaging direction approaches the pan rotation axis 203, the imaging center 311 and the pan rotation center 312 approach, so the radius of the pan main guide 313 becomes small and the driving direction becomes difficult to recognize. However, since the pan auxiliary guide 315 is displayed outside the pan main guide 313, the pan driving direction can be indicated by an arc having a larger radius, so that the user can easily recognize the pan driving direction.

  When the imaging direction approaches the pan rotation axis 203, the pan operation button 317 displayed at one end of the pan main guide 313 is changed to the pan operation button 317 displayed at the other end of the pan main guide 313. It will be displayed overlapping. However, by switching the display position of the pan operation button 317 and displaying the pan operation button 317 at both ends of the pan assist guide 315, one pan operation button 317 is displayed without overlapping the other pan operation button 317. be able to. That is, the display control unit 133 displays the pan operation button 317 on the pan main guide 313 when the angle formed by the pan rotation axis 203 and the optical axis of the imaging unit 111 is larger than a predetermined threshold. Then, when the angle formed by the pan rotation axis 203 and the optical axis of the imaging unit 111 is equal to or smaller than a predetermined threshold, the display control unit 133 switches the position where the pan operation button 317 is displayed. A method for setting the threshold will be described later. In this way, the display control unit 133 superimposes the pan operation button 317 at a position where the distance from the pan rotation center 312 to the pan operation button 317 is longer than the distance from the pan rotation center 312 to the image capture center 311 in the captured image. Display.

  Here, an example of a threshold value determination method will be described. For example, an angle 303 at which the distance between the pan rotation center 312 and the imaging center 311 is a quarter of the length of the tilt direction guide 314 displayed in the display image 310 can be used as the threshold value. That is, when the display image 310 includes all circles centered on the pan rotation center 312 and having a radius of the distance between the imaging center 311 and the pan rotation center 312, the display position of the pan operation button 317 is switched. can do.

In FIG. 3E, when the intersection of the plane of the captured image 302 and the pan rotation axis 203 is the pan rotation center 312, the distance x between the image pickup center 311 and the pan rotation center 312 is expressed as (Equation 3) below. Is done. Here, the size of the vector 301 is L, and the angle 303 is β.
(Formula 3) x = L · tan β
Accordingly, when the threshold value is set for the angle 303, the angle β is set such that the value of L · tan β is shorter than a quarter of the length of the tilt direction guide 314 displayed in the display image 310. It can be a threshold. The threshold setting method is not limited to the above method, and may be set by another method.

  The image pickup unit 111 is driven to tilt from the state of FIG. 3B toward the pan rotation center position, and the display image 330 when the image pickup center 311 and the pan rotation axis 203 coincide with each other is shown in FIG. It explains using. In this state, the pan rotation center position derived from the state value of the imaging device 110 and the imaging center 311 coincide with each other, so that the pan main guide 313 converges to one point and cannot be displayed. However, since the pan auxiliary guide 315 set to be displayed outside the pan main guide 313 continues to be displayed without converging, the user can recognize the pan driving direction. Further, when the imaging center 311 and the pan rotation axis coincide with each other, the pan operation buttons 317 displayed at both ends of the pan main guide 313 are not displayed on the display image 330. However, displaying the pan operation buttons 317 at both ends of the pan assist guide 315 can prevent the pan operation buttons 317 from being displayed with the tilt movement of the imaging unit 111.

  Further, like the display image 340 shown in FIG. 3D, the display control unit 133 can display the second bread auxiliary guide 316 inside the bread main guide 313. This second pan auxiliary guide 316 shows in advance how the pan main guide 313 changes when it is tilt-driven in the direction toward the pan rotation center 312 (upward in the figure). Can be shown. Therefore, even if the pan main guide converges and is not displayed as described above, the moving direction of the captured image can be appropriately conveyed to the user without causing confusion to the user. In this way, the imaging control system or control device according to the present embodiment can allow the user to more easily recognize the driving direction of the imaging device.

  FIG. 4 shows a display image 410 when the imaging unit 111 is rotated from the state of FIG. When the imaging unit 111 is rotated, the display image 340 in FIG. 3D rotates around the imaging center 311. The display control unit 133 rotates the directions of the pan main guide 313, the tilt direction guide 314, the pan auxiliary guide 315, and the pan auxiliary guide 316 around the imaging center 311 with the rotation of the display image 340 and superimposes it on the captured image. In this way, the display control unit 133 superimposes the pan direction guide and the tilt direction guide on the captured image in accordance with the drive of the rotation driving unit 114.

  As a result, the client device 130 can indicate to the user the pan and tilt drive directions when the imaging unit 111 is rotated. Further, the display control unit 133 rotates the display positions of the pan operation button 317 and the tilt operation button 318 around the imaging center 311 with the rotation of the display image 340 and superimposes it on the captured image. Accordingly, the client device 130 can display the pan operation button 317 and the tilt operation button 318 in the pan and tilt directions when the imaging unit 111 is driven to rotate. Even when the pan rotation center position and the imaging center 311 coincide with each other and the pan main guide 313 converges to one point and cannot be displayed, the pan assist guide 315 can indicate the pan driving direction to the user. In addition, even when the pan rotation center position and the imaging center 311 coincide with each other and the pan main guide 313 converges to one point and cannot be displayed, the pan operation button 317 is displayed on the pan auxiliary guide 315 to display the pan operation button 317. An operation button 317 can be shown to the user.

  Furthermore, when the tilt is driven in the direction toward the pan rotation center 312 by the pan auxiliary guide 316, it can be shown to the user in advance how the pan main guide 313 changes. When the rotation driving is performed, the pan driving direction and the tilt driving direction are inclined as shown in FIG. 4, so that it is difficult for the user to grasp the pan driving direction and the tilt driving direction. However, according to the present embodiment, the pan driving direction and the tilt driving direction can be shown to the user even when the optical axis of the imaging apparatus approaches the pan rotation axis.

  Next, the operation of the client device 130 according to the present embodiment will be described with reference to the flowchart of FIG. The processing flow of FIG. 5 shows a program for causing the client apparatus 130 to execute the procedure shown in FIG. The CPU 137 built in the client device 130 is a computer, and executes a program read from the memory 132 built in the client device 130.

  First, the client apparatus 130 enters a PTR (pan / tilt / rotation) setting mode (S100). The PTR setting mode refers to a mode that enables operation setting of the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 of the imaging device 110 from the client device 130. Specifically, in the PTR setting mode, the display control unit 133 that has received a command from the CPU 137 causes the display device 170 to display images for pan, tilt, and rotation operation settings, a pan operation button 317, and a tilt operation button 318. Etc. are displayed. Then, the client device 130 issues a command instruction to the imaging device 110 in accordance with an instruction input by the user operating the operation button from the input device 150.

  Next, as described with reference to FIGS. 3 and 4, the client apparatus 130 adjusts to the current pan, tilt, and rotation angles according to the pan main guide 313, the tilt direction guide 314, the pan auxiliary guide 315, and The pan auxiliary guide 316 is displayed (S110). The client device 130 may perform the process of step S100 and the process of step S110 simultaneously. That is, the client device 130 may cause the display device 170 to simultaneously display the pan operation button 317, the tilt operation button 318, the pan main guide 313, the tilt direction guide 314, the pan auxiliary guide 315, and the pan auxiliary guide 316. .

  Next, the client device 130 determines whether a driving instruction for the imaging device 110 is input from the input device 150 (S120). When an instruction to drive the imaging device 110 is input from the input device 150 to the client device 130 using operation buttons such as a pan operation button 317 and a tilt operation button 318 (Yes in S120), the client device 130 A drive command corresponding to the input operation is sent to the imaging apparatus 110 (S130). Thereafter, the process returns to S110.

  Subsequently, the operation of the imaging apparatus 110 in the present embodiment will be described with reference to the flowchart of FIG. The processing flow of FIG. 6 shows a program for causing the imaging apparatus 110 to execute the procedure shown in FIG. The CPU 115 built in the imaging apparatus 110 is a computer, and executes a program read from the memory 117 built in the imaging apparatus 110.

  The imaging device 110 first initializes the imaging device 110 (S200). In step S200, the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 are initialized, and the imaging apparatus and the like are initialized. By the initialization, the origin positions of the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 are confirmed, and the current imaging position is detected. Then, the current position of the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114 and the angle of view of the captured image are transmitted as statuses to the client device 130 (S210). When the initialization is completed, the imaging device 110 transmits a captured image to the client device 130 (S220). Then, the imaging device 110 waits for receiving a command from the client device 130 (S230).

  When a command is input from the client device 130, the imaging device 110 performs any one of pan driving, tilt driving, rotation driving, or a plurality of driving according to the command (S240). Pan driving, tilt driving, and rotation driving are performed by the pan driving unit 112, the tilt driving unit 113, and the rotation driving unit 114, respectively. The imaging device 110 transmits the current position after the driving instructed from the client device 130 and the angle of view of the captured image to the client device 130 as a status (S250). Thereafter, it waits for command reception again (S230).

  In addition to the method of always displaying the bread auxiliary guide 315 and the bread auxiliary guide 316 displayed outside or inside the bread main guide 313 described above, only when the imaging area of the imaging unit 111 is within a specific range. You may make it display.

  For example, when the angle 303 formed by the pan rotation axis 203 for the pan driving unit 112 to drive the image capturing unit 111 and the vector 301 of the image capturing unit 111 is equal to or smaller than the threshold, the display control unit 133 performs the pan assist guide 315. And the pan assist guide 316 can be superimposed on the captured image. Here, it is assumed that the direction of the vector 301 coincides with the direction of the optical axis of the imaging unit 111. Alternatively, the pan assist guide 315 and the pan assist guide 316 can be displayed when the distance between the imaging center 311 and the pan rotation center 312 shown in FIG.

  As described above, the distance between the imaging center 311 and the pan rotation center 312 is determined based on the length of the pan rotation radius 304 in FIG. Further, the length of the pan rotation radius 304 changes according to the angle of view of the imaging unit 111 and the size of the captured image 302. Accordingly, the display control unit 133 changes the threshold value of the angle 303 according to the angle of view of the imaging unit 111 and the size of the captured image, so that the distance between the imaging center 311 and the pan rotation center 312 is equal to or less than a predetermined value. The bread auxiliary guide 315 and the bread auxiliary guide 316 can be displayed. Other methods may be adopted as a method for determining whether the imaging region of the imaging unit 111 is within a specific range, and is not limited to these methods.

  In this way, when it is highly likely that the pan main guide 313 converges to the pan rotation center 312 when the imaging unit 111 is tilt driven, the pan auxiliary guide 316 displays the pan main guide 313 when the tilt driving is continued. The user can be informed that it will disappear. In addition, even after the pan main guide 313 is not displayed, the pan driving guide 315 can inform the user of the pan driving direction. Furthermore, when the imaging area of the imaging unit 111 is not within a specific range, the pan assist guide 315 and the pan assist guide 316 are not displayed, so that it is possible to prioritize the visibility of the captured image. In addition to using the threshold value, the user may be allowed to switch between display and non-display. The display method of the pan and tilt movement direction of the present invention is not limited to the method described above.

  According to the configuration described above, the display control unit 133 superimposes the pan direction guide that does not overlap the moving path along which the imaging center moves by pan driving on the captured image. Therefore, even when the imaging center and the pan rotation center overlap and the pan main guide 313 is no longer displayed, the user can recognize the moving direction. Also, by displaying the pan assist guide 316 on the inner side (pan rotation center side) of the imaging center moving path, the user recognizes how the pan main guide changes before the imaging center and the pan rotation center overlap. be able to. In this way, the present embodiment can appropriately convey the moving direction of the imaging region to the user. Therefore, the user can more easily recognize the driving direction of the imaging apparatus 110 than when only the pan main guide is displayed on the display image.

  Furthermore, according to the above-described configuration, the display control unit 133 causes the pan operation button 317 to be superimposed and displayed on the pan direction guide that does not overlap the moving path along which the imaging center moves by pan driving. Thus, the imaging control system according to the present embodiment can appropriately display the pan operation icon even when the optical axis of the imaging device approaches the pan rotation axis. Accordingly, even when the imaging center and the pan rotation center overlap and the pan main guide is not displayed, the user can drive the imaging unit 111 to pan using the pan operation button 317.

  In the present embodiment, the case where the display control unit 133 switches the position where the pan operation button 317 is displayed when the angle 303 formed by the pan rotation axis 203 and the optical axis of the imaging unit 111 is equal to or smaller than a predetermined threshold has been described. The display position of the operation button 317 may not be switched. That is, regardless of the value of the angle 303, the pan operation button 317 may be displayed on the pan assist guide 315 in FIG.

  In this embodiment, the case where the bread main guide 313, the bread auxiliary guide 315, and the bread auxiliary guide 316 are displayed has been described. However, the bread main guide 313, the bread auxiliary guide 315, and the bread auxiliary guide 316 are not displayed. Only the operation button 317 may be displayed. When only the pan operation button 317 is displayed, an arrow is displayed on the pan operation button 317 to indicate the pan drive direction so that the user can easily recognize the pan drive direction, or the pan operation button 317 is triangular. For example, the vertex may indicate the pan driving direction. In this way, the user can intuitively recognize the pan driving direction in the display image and perform the pan driving operation.

(Example 2)
A second embodiment of the present invention will be described. In the present embodiment, the pan operation button is displayed at a predetermined position that is line-symmetric with respect to a straight line that passes through the imaging center and indicates the tilt direction in the display image.

  Since the configuration of the imaging system according to the second embodiment is realized by the same configuration as the imaging control system described with reference to FIG. 1 in the first embodiment, description of each configuration is omitted. About each structure, it attaches | subjects and uses the same number as the number used in FIG.

  In the second embodiment, a display image displayed on the display device 170 by the display control of the display control unit 133 will be described with reference to FIG. FIG. 7A is a diagram when the imaging direction is nearly perpendicular to the pan rotation axis 203. The imaging center 711 is an intersection between the optical axis of the optical system that connects the image of the subject on the imaging device and the imaging device of the imaging unit 111. The pan rotation center 712 indicates the position of the pan rotation center on the plane of the display image 710. The method for setting the pan rotation center 712 is the same as the method described with reference to FIG.

  The pan main guide 713 is a pan direction guide indicating the pan direction in the display image 710. The pan main guide 713 is represented as a circle, arc, or straight line that passes through the pan operation button 715 with the pan rotation center 712 as the center. In this embodiment, the pan main guide 713 does not pass through the imaging center 711 and is superimposed on the captured image so that the distance from the pan rotation center 712 to the pan main guide 713 is longer than the distance from the pan rotation center to the imaging center 711. Displayed. The tilt main guide 714 is a tilt main guide that indicates the tilt movement direction so as to overlap the movement path along which the imaging center 711 moves by tilt driving.

  Further, the display control unit 133 causes the pan operation button 717 and the tilt operation button 718 to be superimposed on the display image 710 for display. The pan operation button 717 is an icon for inputting a command to the pan operation unit 138. In the state of FIG. 7A, the pan operation button 717 is superimposed on a predetermined position of the display image 710. The tilt operation button 718 is an icon for inputting a command to the tilt operation unit 139. In the state of FIG. 3A, the tilt operation buttons 718 are displayed on both ends of the tilt main guide 714.

  Here, the display position of the pan operation button 717 in the display image 710 will be described. The two pan operation buttons 717 shown in FIG. 7A are displayed symmetrically with respect to the tilt main guide 714 so that the distances from the pan rotation center 712 are equal. By being displayed in this way, the two pan operation buttons 717 are located at the same distance from any point on the straight line of the tilt main guide 714. Therefore, the two pan operation buttons 717 can be displayed on a circle centered on the pan rotation center 712 existing on the straight line of the tilt main guide 714. Further, the distance between the pan operation button 717 and the pan rotation center 712 is set to be longer than the distance from the pan rotation center 712 to the imaging center 711.

  In the state of FIG. 7A, tilt drive is instructed from the input device, for example, the imaging direction is tilted in a direction toward the pan rotation center 712 (upward in the figure), and the imaging direction is set to the pan rotation axis. A display image when approaching will be described with reference to FIG. When the imaging direction approaches the pan rotation axis, the imaging center 721 and the pan rotation center position 722 approach each other. On the other hand, since the display position of the pan operation button 717 does not change even when the tilt is driven, the pan main guide 713 maintains an appropriate radius and does not lose visibility. FIG. 7C illustrates a display image when the tilt driving is further performed in the direction from the state of FIG. 7B toward the pan rotation center position and the imaging direction coincides with the pan rotation axis. In this state, the pan rotation center 712 derived from the state value of the network camera coincides with the imaging center 711. Even in this state, since the position of the pan operation button 717 does not change, the pan main guide 713 does not become smaller than a certain radius, and the operator can confirm the drive direction and can be operated by the operation button.

FIG. 8 is a view for explaining an example of a display image when the rotation mechanism is operated. FIG. 8A is a diagram of a display image 810 when the rotation mechanism is operated in a state where the imaging direction is close to perpendicular to the pan rotation axis 203. The pan operation button 717 and the tilt operation button 718 are controlled to rotate and display in accordance with the driving of the rotation mechanism. Accordingly, the pan main guide 713 and the tilt main guide 714 also rotate. FIG. 8B shows a display image 820 when the tilt drive is performed in the direction toward the pan rotation center 712 in the state of FIG. 8A and the imaging direction coincides with the pan rotation axis. At this time, since the display position of the pan operation button 717 does not change even when the tilt is operated, the pan main guide 713 maintains a constant radius, and each mechanism can be operated by the operation buttons without losing visibility. . In this way, the imaging control system according to the present embodiment can appropriately display the pan operation icon even when the optical axis of the imaging apparatus approaches the pan rotation axis.
Since the operation of the imaging control system according to the present embodiment is the same as that described with reference to FIGS. 5 and 6, the description thereof is omitted.

  The imaging control system according to the second embodiment displays the distance between the pan operation button 717 and the pan rotation center 712 longer than the distance from the pan rotation center 712 to the imaging center 711 while the image pickup apparatus is panning and tilting. A pan operation button 717 is displayed at a predetermined position of the image. In the imaging control system according to the second embodiment, the pan main guide 713 is displayed so as to be superimposed on the captured image so as to always pass the pan operation button 717. Accordingly, even when the optical axis of the imaging device 110 approaches the pan rotation axis 203, an icon for performing a pan operation can be displayed.

(Other examples)
In the first and second embodiments, the case where the imaging apparatus 110 has a rotation mechanism has been described. On the other hand, when there is no rotation mechanism, it is possible to perform virtual rotation by rotating the image on the client device, but the present invention can be implemented with such a configuration. Even in such a case, the pan driving direction and the tilt driving direction can be shown to the user even when the optical axis of the imaging apparatus approaches the pan rotation axis, as in the case where the imaging unit 111 is rotated. .

  Further, in the first and second embodiments, the case where the tilt main guide is displayed in addition to the pan main guide and the pan auxiliary guide has been described. However, only the pan auxiliary guide may be displayed. Even when only the pan assist guide is displayed, the moving direction of the imaging region can be appropriately conveyed to the user without the pan direction guide disappearing due to the tilt drive of the imaging unit 111.

  Further, the client device 130 described in the first embodiment and the second embodiment may be integrated with the imaging device 110. That is, the present invention can be implemented even when the client device 130 superimposes the pan main guide 313 and the pan auxiliary guide 315 on the captured image captured by the image capturing device 110 and then transmits the display image to the display device 170 via the network 190. It is.

DESCRIPTION OF SYMBOLS 110 Image pick-up device 111 Image pick-up part 112 Pan drive part 113 Tilt drive part 114 Rotation drive part 130 Client apparatus 133 Display control part 138 Pan operation part 139 Tilt operation part 140 Rotation operation part 311 Imaging center 313 Pan main guide 315 Pan auxiliary guide 721 Parallel line 722 Meridian

Claims (11)

Imaging means for imaging on the imaging element with the intersection of the optical axis and the imaging element as the imaging center;
Pan driving means for pan driving the imaging means;
A control device for controlling an image pickup apparatus having a tilt drive means for tilt driving the image pickup means;
Pan operating means for operating the pan driving means;
A tilt operating means for operating the tilt driving means;
Display control means for superimposing and displaying an icon for inputting a command to the pan operation means at a position indicating a pan driving direction in a captured image captured by the imaging means;
The display control unit displays the icon in a superimposed manner so that a distance from a pan rotation center to the icon is longer than a distance from the pan rotation center to the imaging center in the captured image. apparatus.   When the angle formed by the pan rotation axis and the optical axis for pan driving the image pickup means by the pan drive means is equal to or less than a predetermined threshold value, the display control means moves from the pan rotation center to the icon in the image pickup apparatus. The control device according to claim 1, wherein the icon is superimposed and displayed so that a distance is longer than a distance from a pan rotation center to the imaging center.   The display control unit is configured to perform the panning driving in the captured image by pan driving when an angle formed by a pan rotation axis for the pan driving unit to pan-drive the imaging unit and the optical axis is larger than a predetermined threshold. The control apparatus according to claim 2, wherein the icon is displayed in a superimposed manner on a moving route along which the character moves.   The control device according to claim 2, wherein the display control unit changes the threshold value according to an angle of view of the imaging unit. A pan direction guide showing a pan driving direction without overlapping a moving path along which the imaging center moves by pan driving by the pan driving means, and a distance from the pan rotation center to the pan direction guide is from the pan rotation center to the imaging Comprising superimposing means for superimposing the pan direction guide far from the distance to the center on the captured image;
The control device according to claim 1, wherein the display control unit superimposes and displays the icon on the pan direction guide superimposed by the superimposing unit.   When an angle formed by a pan rotation axis for the pan driving unit to pan drive the imaging unit and the optical axis is equal to or less than a predetermined threshold, the panning drive by the pan driving unit moves to a moving path along which the imaging center moves. The control device according to claim 5, wherein the icon is displayed superimposed on a pan direction guide that does not overlap. The display control unit superimposes on the captured image a pan main guide indicating a moving path along which the imaging center moves by pan driving in the captured image,
When the angle formed by the pan rotation axis for driving the image pickup means by the pan driving means and the optical axis is larger than a predetermined threshold, the icon is displayed superimposed on the pan main guide. The control device according to claim 6.   The display control means displays the plurality of icons superimposed on the captured image at a predetermined position that is line symmetric with respect to a straight line indicating the tilt direction through the imaging center, and the pan direction passing through the icons The control device according to claim 5, wherein a guide is displayed by being superimposed on the captured image. An imaging control system having an imaging device and a control means for controlling the imaging device,
The imaging device
Imaging means for imaging on the imaging element with the intersection of the optical axis and the imaging element as the imaging center;
Pan driving means for pan driving the imaging means;
Tilt drive means for tilt driving the imaging means,
The control means includes
Pan operating means for operating the pan driving means;
A tilt operating means for operating the tilt driving means;
Display control means for superimposing and displaying an icon for inputting a command to the pan operation means at a position indicating a pan driving direction in a captured image captured by the imaging means;
The display control means causes the icon to be superimposed and displayed so that a distance from a pan rotation center to the icon is longer than a distance from the pan rotation center to the imaging center in the captured image. Control system. Imaging means for imaging on the imaging element with the intersection of the optical axis and the imaging element as the imaging center;
Pan driving means for pan driving the imaging means;
A display control method for controlling display of a captured image captured by an imaging device having a tilt driving unit that tilt-drives the imaging unit,
A generating step for generating an icon for inputting a command to the pan operating means for operating the pan driving means;
Display that causes the display control means to superimpose and display the icon at a position indicating the panning direction so that a distance from the pan rotation center to the icon is longer than a distance from the pan rotation center to the imaging center in the captured image. A display control method comprising: a control step. Imaging means for imaging on the imaging element with the intersection of the optical axis and the imaging element as the imaging center;
Pan driving means for pan driving the imaging means;
A computer for controlling an image pickup apparatus having a tilt drive means for driving the image pickup means to tilt;
A generation procedure for generating an icon for inputting a command to the pan operation means for operating the pan driving means;
Display in which the icon is superimposed and displayed at a position indicating the pan direction so that the distance from the pan rotation center to the icon is farther than the distance from the pan rotation center to the imaging center in the captured image captured by the imaging device A program characterized by causing a control procedure to be executed.

Images