JP2018160761A - Environmental image display system and environment image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress occurrence of complication for a user and display an environment image around an unmanned mobile object.SOLUTION: An imaging device 200 captures an entire-celestial-sphere image as a whole while having an overlapping region by using a plurality of photographing optical systems having the same width of field of view and different photographing directions and transmits photographing result data corresponding to each of the plurality of photographing optical systems to an image processing device 300. In addition, a head mounted display device 400 detects the posture change amount from the previous detection time point, and sends the detection result to the image processing device 300. The image processing device 300 performs stitching processing on a plurality of pieces of photographed result data to generate entire-celestial-sphere image data and then extract an image of a display area determined corresponding to the posture of the head mounted display device 400 from the entire-celestial-sphere image to generate display image data. After that, the head mounted display device 400 displays an image corresponding to the display image data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an environmental image display system and an environmental image display method, and more particularly to an environmental image display system and an environmental image display method for displaying an environmental image in an area where human entry is difficult.

  In recent years, in order to grasp the situation of areas where human bodies are dangerous and people cannot enter, a technology has been proposed that uses a camera mounted on a wirelessly maneuverable unmanned moving body to capture an environmental image of the area (Refer to Patent Document 1: hereinafter referred to as “conventional example”). In this conventional technique, an environment image is captured by a camera mounted on an unmanned moving body, and the captured result is transmitted to a display device used by a user by wireless communication. In the display device, an environmental image corresponding to the received photographing result is displayed.

  In the conventional technique, there are three cameras: a bird's-eye view camera directed downward for photographing the surroundings of the unmanned moving body, a front camera for photographing the front side, and a rear camera for photographing the rear side. Is mounted on an unmanned moving body.

JP 2013-112028 A

  In the conventional example, as described above, the photographing results of the three cameras, the overhead camera, the front camera, and the rear camera, are displayed. However, in the conventional example, there is no disclosure about how to display the imaging results of the three cameras.

  For example, it is conceivable that each of the photographing results of three cameras is displayed as it is on three display devices prepared corresponding to each of the three cameras. In such a case, when an image around the point of interest spans the display images on two or more display devices, the image around the point is placed on the same display device for the convenience of observation. In order to display, it is necessary to change the posture of the unmanned moving body by wireless control. Such an operation makes the user feel complicated.

  For this reason, a new technique that can display an environmental image around an unmanned moving body while suppressing the occurrence of complexity for the user is desired. Meeting this requirement is one of the problems to be solved by the present invention.

  An environment image display system according to the present invention is disposed on a wirelessly maneuverable unmanned moving body, photographs a peripheral image of the unmanned moving body, and transmits photographing result data by wireless communication; and processes the photographing result data An image processing device that generates display image data; and a head-mounted display device that is connected to the image processing device and displays an image according to the display image data. The imaging device has a wide imaging field of view. A omnidirectional camera that uses a plurality of imaging optical systems having the same imaging direction and different imaging directions and has an overlapping region and captures an image covering the entire celestial sphere as a whole; A wireless transmission unit that transmits imaging result data corresponding to each to the image processing device, wherein the image processing device includes a plurality of imaging result data transmitted from the imaging device. A radio reception unit for receiving; a omnidirectional image data generation unit for generating omnidirectional image data by performing a stitching process on the plurality of received imaging result data; and corresponding to an attitude of the head mounted display device A display image data generation unit that extracts the image of the display area determined in this way from the omnidirectional image and generates the display image data, and the head mounted display device displays an image corresponding to the display image data. A display unit for displaying; a posture change amount detection unit that detects a posture change amount of the head mounted display device from a previous detection time point and sends a detection result to the image processing device; and the display image data generation unit includes: Initial display image data in which the optical axis direction of the specific imaging optical system selected from the plurality of imaging optical systems is the initial line-of-sight direction of the user of the head mounted display device Generating a environmental image display system, characterized in that.

  In this environment image display system, a peripheral image of the unmanned moving body is photographed by the omnidirectional camera of the photographing device disposed on the unmanned moving body that can be wirelessly operated. This omnidirectional camera uses a plurality of photographic optical systems having the same field of view and different photographic directions, and shoots an image covering the omnidirectional sphere as a whole while having overlapping areas. And the radio transmission part of an imaging device transmits the imaging result data corresponding to each of the some imaging optical system which is an imaging result with the omnidirectional camera of an imaging device to an image processing apparatus.

  In the image processing apparatus, the omnidirectional image data generation unit receives a plurality of imaging result data transmitted from the imaging apparatus via the wireless reception unit. Subsequently, the omnidirectional image data generation unit performs a stitching process on the plurality of received imaging result data to generate omnidirectional image data.

  The omnidirectional image corresponding to the omnidirectional image data generated in this way includes a joint portion by stitching processing. At such joints, some discontinuity of the image occurs.

  Further, the posture change amount detection unit of the head mounted display device detects the posture change amount of the head mounted display device from the previous detection time point, and sends the detection result to the image processing device. Subsequently, the display image data generation unit of the image processing apparatus extracts an image of a display area determined in accordance with the attitude of the head mounted display device sent from the head mounted display device from the omnidirectional image, and displays the display image data. Generate. Here, using the optical axis direction of the specific imaging optical system selected from the plurality of imaging optical systems as the initial line-of-sight direction of the user of the head-mounted display device, the display image data generation unit changes the attitude of the head-mounted display device. Based on the reception result of the quantity, the position of the display image in the omnidirectional image is specified, and display image data is generated. Then, the display image data generation unit sends the generated display image data to the head mounted display device.

  In addition, when the display image corresponding to the extracted display image data includes a joint by stitching processing, the display image includes discontinuity. When such discontinuity occurs in the central portion of the display image, the user feels the most discomfort. As the discontinuity portion is further away from the central portion of the display image, the sense of discomfort for the user becomes smaller. In addition, when the joint by a stitching process is not contained in a display image, the discomfort for a user by generation | occurrence | production of the discontinuity in a display image does not generate | occur | produce.

  In the head mounted display device, the display unit receives display image data sent from the image processing device. Then, the display unit displays an image according to the display image data.

  Among the images displayed in this way, there is no image discontinuity in the central portion of the image corresponding to the initial line-of-sight direction (hereinafter referred to as “initial display image”), so at least the initial display image is displayed. It becomes an image in which the occurrence of discomfort for the user due to the occurrence of discontinuity in the image is suppressed. Then, by changing the posture of the head and the posture of the head mounted display device, the user can display an image in an arbitrary direction around the unmanned moving body and observe the surroundings of the moving body.

  Therefore, according to the environment image display system of the present invention, it is possible to display an environment image around the unmanned moving body while suppressing the troublesomeness for the user and the occurrence of discomfort for the user due to the discontinuity of the image. can do.

  In the environmental image display system of the present invention, any one of the image processing device and the head mounted display device has an initial visual direction input that sets the initial visual direction to a direction according to the current head mounted display device posture. An input unit to be performed can be further provided. In this case, using the input unit, the user who is wearing the head mounted display device can set a direction in which a desired posture change such as a direction facing the front is easy to change as the initial line-of-sight direction. For this reason, the convenience for the user can be improved.

  In the environment image display system of the present invention, the omnidirectional image data generation unit may perform a blending process on the overlapping area during the stitching process. In this case, even when the stitching processing part is included in the display image, it is possible to reduce the user's uncomfortable feeling when viewing the display image.

  In the environment image display system of the present invention, the area corresponding to each of the photographing optical systems in the omnidirectional image has a size that can include the display area of the image corresponding to the initial display image data. It can be. In this case, the initial display image does not include a stitching processing portion that may cause image discontinuity. For this reason, it is possible to effectively suppress the occurrence of discomfort for the user due to the occurrence of discontinuity in the image.

  In the environment image display system of the present invention, the number of the plurality of photographing optical systems may be “2”. In this case, since the stitching processing portion can be minimized, it is possible to minimize the uncomfortable feeling for the user due to the occurrence of the discontinuity of the image.

  An environmental image display method according to the present invention includes an imaging device that is arranged on an unmanned mobile body that can be wirelessly controlled, captures a peripheral image of the unmanned mobile body, and transmits the imaging result data by wireless communication; and processes the imaging result data An environmental image display used in an environmental image display system comprising: an image processing device that generates display image data; and a head-mounted display device that is connected to the image processing device and displays an image according to the display image data. In the method, the imaging device uses a plurality of imaging optical systems having the same imaging field of view and different imaging directions, and has an overlapping region and an image covering the entire celestial sphere as a whole. A photographing step of photographing and transmitting photographing result data corresponding to each of the plurality of photographing optical systems to the image processing device; and the image processing device comprising the photographing step A celestial sphere image data generating step for generating omnidirectional image data by performing a stitching process on the plurality of photographing result data transmitted in the step; and the image processing device is in an attitude of the head mounted display device. A display image data generation step of extracting an image of a display area determined correspondingly from the omnidirectional image and generating the display image data; and the head mounted display device displays an image corresponding to the display image data A display step; and a posture change amount detection step in which the head mounted display device detects a posture change amount of the head mounted display device from a previous detection time point and sends a detection result to the image processing device. In the image data generation step, the optical axis direction of the specific imaging optical system selected from the plurality of imaging optical systems is determined by the user of the head mounted display device. Generating an initial display image data to the period gaze direction is an environmental image display wherein the.

  In this environmental image display method, in the imaging process, the imaging apparatus uses a plurality of imaging optical systems having the same imaging field of view and different imaging directions, and has an overlapping area, but as a whole. An image covering the celestial sphere is photographed, and photographing result data corresponding to each of the plurality of photographing optical systems is transmitted to the image processing apparatus. In parallel with the photographing process, in the posture change amount detection step, the head mount display device detects the posture change amount of the head mount display device from the previous detection time point, and sends the detection result to the image processing device.

  When receiving a plurality of imaging result data transmitted from the imaging device, in the omnidirectional image data generation step, the image processing device performs a stitching process on the plurality of imaging result data to obtain the omnidirectional image data. Generate. Subsequently, in the display image data generation step, the image processing device extracts a display area image determined according to the attitude of the head mounted display device from the omnidirectional image to generate display image data. In the display step, the head mounted display device displays an image corresponding to the display image data.

  Therefore, according to the environment image display method of the present invention, the environment image around the unmanned mobile object is displayed while suppressing the troublesomeness for the user and the uncomfortable feeling for the user due to the discontinuity of the image. can do.

  As described above, according to the environmental image display system and the environmental image display method of the present invention, it is possible to unmanned while suppressing the troublesomeness for the user and the uncomfortable feeling for the user due to the discontinuity of the image. There is an effect that an environment image around the moving body can be displayed.

It is a figure which shows the structure of the environmental image display system which concerns on one Embodiment of this invention. It is a block diagram for demonstrating the structure of the imaging device of FIG. It is a figure for demonstrating the picked-up image by the imaging device of FIG. It is a block diagram for demonstrating the structure of the image processing apparatus of FIG. FIG. 5 is a diagram for explaining a relationship between an omnidirectional image corresponding to the omnidirectional image data generated by the omnidirectional image data generation unit of FIG. 4 and an image captured by the imaging apparatus of FIG. 2. It is a block diagram for demonstrating the structure of the head mounted display apparatus of FIG. 3 is a flowchart for explaining processing executed by the image processing apparatus of FIG. 1. It is a figure for demonstrating the example of a relationship between an omnidirectional image and a display image.

  Hereinafter, an embodiment of the present invention will be described with reference to FIGS. In the following description and drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

[Constitution]
As shown in FIG. 1, an environmental image display system 100 according to an embodiment includes an imaging device 200, an image processing device 300, and a head mounted display device 400.

  Here, the imaging device 200 is disposed in the unmanned moving body MV. The unmanned moving body MV is remotely operated by a remote control device RM. In the present embodiment, the unmanned mobile body MV is a vehicle that travels on the ground.

<Configuration of photographing apparatus 200>
As shown in FIG. 2, the photographing apparatus 200 includes an omnidirectional camera 210 and a wireless communication unit 220.

  The omnidirectional camera 210 has two photographing optical systems OP1 and OP2. The widths of the photographing fields corresponding to the photographing optical systems OP1 and OP2 are the same. Here, the optical axis direction of the photographing optical system OP1 is the first direction that is the forward direction of the unmanned moving body MV. The optical axis direction of the photographing optical system OP2 is the reverse direction of the first direction, that is, the second direction that is the rear direction of the unmanned moving body MV.

  Then, the omnidirectional camera 210 uses the imaging optical systems OP1 and OP2, and captures an image covering the entire celestial sphere as a whole while having an overlapping region. Then, the omnidirectional camera 210 sends the photographing result data corresponding to each of the photographing optical systems OP1 and OP2 to the wireless communication unit 220.

  Note that FIG. 3A shows an imaging range of an image corresponding to the imaging optical system OP1. FIG. 3B shows a shooting range of an image corresponding to the shooting optical system OP2.

  In FIG. 3A, the horizontal direction is the width direction of the unmanned mobile body MV, and the vertical direction is the height direction of the unmanned mobile body MV. The first direction is a (0 °, 0 °) direction.

  In FIG. 3B, the horizontal direction is the width direction of the unmanned mobile body MV, and the vertical direction is the height direction of the unmanned mobile body MV. The second direction is a (0 °, 0 °) direction.

  The wireless communication unit 220 receives two pieces of imaging result data sent from the omnidirectional camera 210. Then, the wireless communication unit 220 transmits the two shooting result data to the image processing apparatus 300 by wireless communication.

<Configuration of Image Processing Device 300>
As shown in FIG. 4, the image processing apparatus 300 includes a first wireless communication unit 310 and an omnidirectional image data generation unit 320. Further, the image processing apparatus 300 includes a display image data generation unit 330 and a second wireless communication unit 340.

  The first wireless communication unit 310 performs wireless communication with the imaging device 200. The first wireless communication unit 310 receives two pieces of shooting result data transmitted from the shooting apparatus 200. Then, the first wireless communication unit 310 sends the two shooting result data to the omnidirectional image data generation unit 320.

  The omnidirectional image data generation unit 320 receives the two imaging result data sent from the first wireless communication unit 310. Subsequently, the omnidirectional image data generation unit 320 performs stitching processing on the two imaging result data to generate omnidirectional image data.

  FIG. 5 shows an omnidirectional image FSI corresponding to the omnidirectional image data, an image (1) corresponding to the photographing optical system OP1, and images (2-1) to (2-) corresponding to the photographing optical system OP2. The relationship with 8) is shown. In FIG. 5, the stitching processing area is shown with small black circles scattered.

  In the present embodiment, the omnidirectional image data generation unit 320 performs a blending process on an overlapping area between an image corresponding to the photographing optical system OP1 and an image corresponding to the photographing optical system OP2 during the stitching process. ing. For this reason, the discontinuity of the image in the vicinity of the joint in the omnidirectional image is reduced as compared with the case where the blend process is not performed.

  In the present embodiment, during the blending process, the omnidirectional image data generation unit 320 increases the blend ratio of the image region corresponding to the imaging optical system to which each pixel belongs as the distance from the joint increases. It has become.

  The display image data generation unit 330 receives the omnidirectional image data sent from the omnidirectional image data generation unit 320. Further, the display image data generation unit 330 uses the second wireless communication unit 340 to receive the posture change amount of the head mounted display device 400 from the previous detection time point (hereinafter simply referred to as “posture change”) received from the head mounted display device 400. And an initial gaze direction setting command. Subsequently, the display image data generation unit 330 sets the display area in the omnidirectional image based on the initial line-of-sight direction set according to the latest initial line-of-sight direction setting command and the attitude change amount of the head mounted display device 400. decide.

  Next, the display image data generation unit 330 extracts the image of the determined display area from the omnidirectional image corresponding to the latest omnidirectional image data, and generates display image data. Then, the display image data generation unit 330 sends the generated display image data to the second wireless communication unit 340.

  The second wireless communication unit 340 performs wireless communication with the head mounted display device 400. The second wireless communication unit 340 receives the display image data sent from the display image data generation unit 330. Then, the second wireless communication unit 340 transmits the display image data to the head mounted display device 400.

  Further, the second wireless communication unit 340 receives the posture change amount of the head mounted display device 400 transmitted from the head mounted display device 400. Then, the second wireless communication unit 340 sends the posture change amount to the display image data generation unit 330.

  Further, the second wireless communication unit 340 receives the initial line-of-sight direction setting command transmitted from the head mounted display device 400. Then, the second wireless communication unit 340 sends the initial line-of-sight direction setting command to the display image data generation unit 330.

<Configuration of Head Mount Display Device 400>
As shown in FIG. 6, the head mounted display device 400 includes a wireless communication unit 410 and a display unit 420. The head mounted display device 400 includes an attitude change amount detection unit 430 and an input unit 440.

  The wireless communication unit 410 performs wireless communication with the image processing apparatus 300. The wireless communication unit 410 receives display image data transmitted from the image processing apparatus 300. Then, the wireless communication unit 410 sends the display image data to the display unit 420.

  In addition, the wireless communication unit 410 receives the posture change amount of the head mounted display device 400 sent from the posture change amount detection unit 430. Then, the wireless communication unit 410 sends the posture change amount to the image processing apparatus 300.

  In addition, the wireless communication unit 410 receives an initial line-of-sight direction setting command sent from the input unit 440. Then, the wireless communication unit 410 sends the initial line-of-sight direction setting command to the image processing device 300.

  The display unit 420 includes a display device such as a liquid crystal panel. The display unit 420 receives display image data sent from the wireless communication unit 410. Then, the display unit 420 displays an image corresponding to the display image data.

  The posture change amount detection unit 430 includes, for example, a three-dimensional gyro sensor. The posture change amount detection unit 430 calculates the posture change amount from the previous detection time point by detecting the three-dimensional angular velocity at each time point. Then, posture change amount detection unit 430 sends the posture change amount as a detection result to wireless communication unit 410.

  The input unit 440 includes a push button. Such a push button is a button for a momentary operation that is turned “ON” by a push operation and turned “OFF” when the push operation is completed. The input unit 440 sends an initial line-of-sight direction setting command to the wireless communication unit 410 each time a pressing operation is performed on the pressing button.

[Operation]
Next, the operation of the environment image display system 100 configured as described above will be described mainly focusing on the processing executed by the image processing apparatus 300.

  Note that the photographing apparatus 200 has started operation, and the photographing apparatus 200 sequentially transmits photographing result data to the image processing apparatus 300. In addition, it is assumed that the head mounted display device 400 has also started operation, and the head mounted display device 400 sequentially transmits the detected posture change amount to the image processing device 300. Further, the pressing operation of the pressing button of the input unit 440 in the head mounted display device 400 has been pressed once in the past, and an initial line-of-sight direction setting command has been transmitted from the head mounted display device 400 to the image processing device 300. And

  In the processing of the image processing apparatus 300 executed under the above-described operating environment, as shown in FIG. 7, first, in step S11, the omnidirectional image data generation unit 320 causes the first wireless communication unit 310 to operate. Then, it is determined whether or not new imaging result data sent from the imaging apparatus 200 has been received. If the result of the determination in step S11 is negative (step S11: N), the process proceeds to step S14 described later.

  If the result of the determination in step S11 is affirmative (step S11: Y), the process proceeds to step S12. In this step S12, the omnidirectional image data generation unit 320 causes the new photographing result data (two new photographing results of photographing result data corresponding to the photographing optical system OP1 and photographing result data corresponding to the photographing optical system OP2). Omnidirectional image data is generated based on (data). When generating the omnidirectional image data, the omnidirectional image data generation unit 320 performs the above-described stitching process to generate omnidirectional image data. Then, the omnidirectional image data generation unit 320 sends the generated omnidirectional image data to the display image data generation unit 330.

  Next, in step S13, the display image data generation unit 330 is based on the accumulation of reception results of posture change amounts from the reception time of the latest initial gaze direction setting command at that time via the second wireless communication unit 340. The display area image is extracted from the omnidirectional image corresponding to the latest omnidirectional image data to generate display image data. Then, the display image data generation unit 330 sends the generated display image data to the head mounted display device 400 via the second wireless communication unit 340. As a result, an image corresponding to the display image data is displayed on the display unit 420 of the head mounted display device 400.

  Next, in step S <b> 14, the display image data generation unit 330 determines whether a new initial line-of-sight direction setting command sent from the imaging apparatus 200 has been received via the second wireless communication unit 340. If the result of the determination in step S14 is negative (step S14: N), the process returns to step S11.

  If the result of the determination in step S14 is affirmative (step S14: Y), the process proceeds to step S15. In step S15, the display image data generation unit 330 generates new initial display image data. When generating the new initial display image data, the display image data generation unit 330 uses the (0 °, 0 °) position in the latest omnidirectional image as the center position of the new initial display image, and displays the image in the display area. Extraction is performed to generate new initial display image data. Then, the display image data generation unit 330 sends the generated initial display image data to the head mounted display device 400 via the second wireless communication unit 340. As a result, an image corresponding to the new initial display image data is displayed on the display unit 420 of the head mounted display device 400.

  When the process of step S15 ends, the process returns to step S11. Thereafter, the processes of steps S11 to S15 are repeated.

  Note that FIG. 8A shows the position of the display image region DSR in the omnidirectional image FSI immediately after the initial line-of-sight direction setting is performed. FIG. 8B shows an example of the position of the display image region DSR in accordance with the posture change of the head mounted display device 400 after the initial line-of-sight direction is set.

  As described above, in the environmental image display system 100 of the present embodiment, a peripheral image of the unmanned moving body MV is photographed by the omnidirectional camera 210 of the photographing apparatus 200 arranged on the unmanned moving body MV that can be wirelessly operated. . The omnidirectional camera 210 has the same field of view and uses two photographic optical systems OP1 and OP2 whose photographic directions are mutually different, and has an overlapping area, and displays an image covering the entire celestial sphere as a whole. Take a picture. Then, the radio transmission unit 220 of the imaging apparatus 200 transmits imaging result data corresponding to each of the two imaging optical systems OP1 and OP2 that are imaging results of the omnidirectional camera 210 to the image processing apparatus 300.

  In the image processing device 300, the omnidirectional image data generation unit 320 receives the two imaging result data transmitted from the imaging device 200 via the first wireless reception unit 310. Subsequently, the omnidirectional image data generation unit 320 performs stitching processing on the two received photographing result data to generate omnidirectional image data.

  In addition, the posture change amount detection unit 430 of the head mounted display device 400 detects the posture change amount of the head mounted display device 400 from the previous detection, and sends the detection result to the image processing device 300. Subsequently, the display image data generation unit 330 of the image processing device 300 extracts the image of the display area determined in accordance with the posture of the head mounted display device 400 sent from the head mounted display device 400 from the omnidirectional image, Display image data is generated. Here, with the optical axis direction of the specific imaging optical system OP1 selected from the two imaging optical systems OP1 and OP2 as the initial line-of-sight direction of the user of the head-mounted display device 400, the display image data generation unit 330 Based on the attitude of the mount display device 400, the position of the display image in the omnidirectional image is specified, and display image data is generated. Then, the display image data generation unit 330 sends the generated display image data to the head mounted display device 400.

  In the head mounted display device 400, the display unit 420 receives display image data sent from the image processing device 300 via the wireless communication unit 410. Then, the display unit 420 displays an image according to the display image data.

  Therefore, according to the environment image display system 100 of the present embodiment, the environment around the unmanned mobile MV is suppressed while suppressing the troublesomeness for the user and the uncomfortable feeling for the user due to the discontinuity of the image. An image can be displayed.

  Further, in the environmental image display system 100 according to the present embodiment, an input unit that performs an initial line-of-sight direction input in which the head-mounted display device 400 sets the initial line-of-sight direction to a direction according to the current posture of the head-mounted display device 400 440 is provided. For this reason, using the input unit 440, a direction in which the user wearing the head mounted display device 400 can easily change a desired posture, such as a direction facing the front, can be set as the initial line-of-sight direction. For this reason, the convenience for the user can be improved.

  Further, in the environmental image display system 100 of the present embodiment, the omnidirectional image data generation unit 320 performs a blending process on the overlapping region during the stitching process. For this reason, since the joint processing is performed on the joint portion of the image, even if the joint portion is included in the display image, it is possible to reduce the uncomfortable feeling of the user who viewed the display image. .

  In the environment image display system 100 of the present embodiment, the area corresponding to each of the imaging optical systems OP1 and OP2 in the omnidirectional image can include an image display area corresponding to the initial display image data. Have come to have. For this reason, the initial display image does not include a stitching processing portion that may cause image discontinuity. As a result, it is possible to effectively suppress the occurrence of discomfort for the user due to the occurrence of image discontinuity.

  In the environment image display system of the present invention, the number of photographic optical systems in the omnidirectional camera 210 is “2”. For this reason, since the stitching processing portion can be minimized, it is possible to minimize the occurrence of discomfort for the user due to the occurrence of image discontinuity.

[Modification of Embodiment]
The present invention is not limited to the above embodiment, and various modifications can be made.

  For example, in the above embodiment, the unmanned moving body is an unmanned vehicle traveling on the ground, but the unmanned moving body may be an unmanned airplane such as a so-called drone.

  In the above embodiment, the number of photographing optical systems used in the photographing apparatus is “2”, but may be “3” or more.

  In addition, the blend method employed in the stitching process in the above embodiment is an example, and another blend method may be employed.

  In the above embodiment, the image processing device and the head mounted display device are wirelessly connected. However, the image processing device and the head mounted display device may be connected by wired connection.

  In the above embodiment, the head-mounted display device includes the input unit that performs the initial line-of-sight direction input for setting the initial line-of-sight direction to a direction corresponding to the current head-mounted display device posture. The image processing apparatus may include the unit.

  As described above, the present invention can be applied to the environmental image display system and the environmental image display method for displaying the environmental image around the unmanned mobile body.

  DESCRIPTION OF SYMBOLS 100 ... Environmental image display system, 200 ... Imaging device, 210 ... Spherical camera, 220 ... Wireless communication part (wireless transmission part), 300 ... Image processing apparatus, 310 ... 1st wireless communication part (wireless reception part), 320 ... spherical image data generation unit, 330 ... display image data generation unit, 340 ... second wireless communication unit, 400 ... head mounted display device, 410 ... wireless communication unit, 420 ... display unit, 430 ... posture change amount detection unit 440 ... input unit, MV ... moving body, RM ... remote control device, OP1, OP2 ... imaging optical system, FSI ... global image, DSR ... display image area.

Claims (6)

An imaging device that is disposed on a wirelessly maneuverable unmanned moving body, photographs a peripheral image of the unmanned moving body, and transmits photographing result data by wireless communication;
An image processing device that processes the photographing result data to generate display image data;
A head mounted display device connected to the image processing device and displaying an image according to the display image data,
The imaging device
An omnidirectional camera that uses a plurality of imaging optical systems having the same imaging field of view and different imaging directions, and that captures an image covering the entire celestial sphere while having an overlapping region;
A wireless transmission unit that transmits imaging result data corresponding to each of the plurality of imaging optical systems to the image processing device;
The image processing apparatus includes:
A wireless receiver for receiving a plurality of photographing result data transmitted from the photographing device;
An omnidirectional image data generation unit that performs a stitching process on the plurality of received imaging result data to generate omnidirectional image data;
A display image data generating unit that extracts an image of a display area determined in accordance with the posture of the head mounted display device from an omnidirectional image and generates the display image data;
The head mounted display device
A display unit for displaying an image corresponding to the display image data;
A posture change amount detection unit that detects a posture change amount of the head mounted display device from the previous detection time point and sends a detection result to the image processing device;
The display image data generation unit generates initial display image data in which an optical axis direction of a specific imaging optical system selected from the plurality of imaging optical systems is an initial line-of-sight direction of a user of the head mounted display device To
An environmental image display system characterized by that. Either of the image processing device and the head mounted display device,
An input unit for performing an initial line-of-sight input for setting the initial line-of-sight direction to a direction corresponding to the current head-mounted display device posture;
The environmental image display system according to claim 1, further comprising:   The environmental image display system according to claim 1, wherein the omnidirectional image data generation unit performs a blending process on the overlapping area in the stitching process.   The area corresponding to each of the imaging optical systems in the omnidirectional image has a size capable of including an image display area corresponding to the initial display image data. The environmental image display system according to any one of the above.   The environmental image display system according to claim 1, wherein the number of the plurality of photographing optical systems is “2”. An imaging device disposed on a wirelessly maneuverable unmanned moving body, photographing a peripheral image of the unmanned moving body, and transmitting photographing result data by wireless communication; and processing the photographing result data to generate display image data An environmental image display method used in an environmental image display system, comprising: an image processing device; and a head mounted display device connected to the image processing device and displaying an image according to the display image data,
The imaging apparatus uses a plurality of imaging optical systems having the same imaging field of view and different imaging directions, and captures an image covering the entire celestial sphere while having an overlapping region, A photographing step of transmitting photographing result data corresponding to each of the photographing optical systems to the image processing apparatus;
An omnidirectional image data generation step in which the image processing device performs a stitching process on a plurality of imaging result data transmitted in the imaging step to generate omnidirectional image data;
A display image data generating step in which the image processing device extracts an image of a display area determined in accordance with the attitude of the head mounted display device from an omnidirectional image and generates the display image data;
A display step in which the head-mounted display device displays an image corresponding to the display image data;
The head-mounted display device detects a posture change amount of the head-mounted display device from a previous detection time point, and sends a detection result to the image processing device; a posture change amount detection step;
In the display image data generation step, initial display image data is generated in which the optical axis direction of the specific imaging optical system selected from the plurality of imaging optical systems is the initial line-of-sight direction of the user of the head mounted display device To
An environmental image display method characterized by the above.

Images