JP6613686B2 - Movie display device, movie display method, and program - Google Patents

Description

  The present invention relates to a moving image display device, a moving image display method, and a program.

  For the purpose of monitoring by a surveillance camera, sports broadcasting, and the like, a technique has been devised that automatically tracks a predetermined image from a moving image obtained by imaging at a wide angle and displays the predetermined image in an enlarged manner.

  For example, in Patent Document 1, an image moving in a moving image (hereinafter referred to as a moving image) is displayed so that it can be specified by the user, and the moving image specified by the user is combined with the moving image and displayed. A video display device is disclosed. Patent Document 2 discloses a technique for detecting the position of an object from each frame of a moving image and determining a region of interest in each frame of the moving image based on the detection result of the object.

  By the way, when a fish-eye camera or an omnidirectional camera is used for wide-angle imaging, an image with a very wide field of view can be obtained by a single imaging. Since the image obtained by the above includes distortion, it is necessary to track the predetermined image after correcting the distortion by partially cutting out the predetermined image included in the image. Therefore, Patent Documents 3, 4 and 5 devise a method for tracking a predetermined image suitable for an image obtained by imaging with a fisheye camera or an omnidirectional camera.

  However, when a predetermined image is cut out and displayed from a moving image obtained by imaging with a fisheye camera or an omnidirectional camera, it is difficult to determine which portion of the moving image is displayed. Therefore, there is a method of displaying a moving image together with a predetermined image so that it is easy to understand which portion of the moving image is cut out, but it is obtained by capturing an image with an angle of view of 180 degrees or more. It is difficult to display a moving image without distortion on a plane, and visibility of a region corresponding to a predetermined image in the moving image is deteriorated.

  In Patent Document 3, a predetermined image in a moving image is panned by panning a cutout position of the predetermined image from the moving image or zooming that gradually changes the magnification of the moving image along the moving path of the predetermined image. A method is described in which the position of the region corresponding to the image can be easily grasped. However, in this method, the region corresponding to the predetermined image cannot be directly recognized from the moving image, and when the predetermined image moves while the moving image is displayed, the region corresponds to the predetermined image in the moving image. The area is unknown.

  The present invention has been made in view of the above, and when a predetermined image included in a moving image is cut out and displayed, a moving image display device and a moving image display that make it easy to grasp a region corresponding to the predetermined image in the moving image It is an object to provide a method and a program.

In order to solve the above-described problems and achieve the object, the present invention provides a cutout unit that cuts out a first image that is a part of a frame and includes a predetermined image from a frame that forms a moving image, A generation unit that generates a second image in which the distortion of the predetermined image is less than or equal to a predetermined value, and a display control unit that displays the display mode of the region corresponding to the first image in the second image differently from other regions The predetermined image moves in the moving image based on a detection unit that detects the position of the predetermined image in the frame and a detection history of the position of the predetermined image from the frame on which the second image has been displayed. A predicting unit that predicts a first range, wherein the generating unit generates the second image in which the distortion of the first range is equal to or less than the predetermined value .

  According to the present invention, when a predetermined image included in a moving image is cut out and displayed, it is possible to easily grasp a region corresponding to the predetermined image in the moving image.

FIG. 1A is a block diagram illustrating an example of a hardware configuration of the moving image display apparatus according to the first embodiment. FIG. 1B is a block diagram illustrating an example of a functional configuration of the moving image browsing apparatus according to the first embodiment. FIG. 2 is a flowchart illustrating an exemplary flow of a moving image display process in the moving image display apparatus according to the first embodiment. FIG. 3 is a diagram illustrating a frame display example in the moving image display apparatus according to the first embodiment. FIG. 4 is a diagram for explaining a partial image generation process in the moving image display apparatus according to the first embodiment. FIG. 5 is a diagram for explaining panorama development in the moving image display apparatus according to the first embodiment. FIG. 6 is a diagram for explaining panorama development in the moving image display apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of a display mode of the target object region in the moving image display apparatus according to the first embodiment. FIG. 8 is a diagram illustrating an example of a display mode of the target object region in the moving image display apparatus according to the first embodiment. FIG. 9 is a diagram illustrating a display example of a partial image and an entire image in the moving image display apparatus according to the first embodiment. FIG. 10 is a diagram illustrating a display example of a partial image and an entire image in the moving image display apparatus according to the first embodiment. FIG. 11 is a block diagram illustrating an example of a functional configuration of the moving image browsing apparatus according to the second embodiment. FIG. 12 is a flowchart illustrating an exemplary flow of a moving image display process in the moving image display apparatus according to the second embodiment. FIG. 13 is a diagram for explaining an example of a motion prediction range prediction process in the moving image display apparatus according to the second embodiment. FIG. 14 is a block diagram illustrating an example of the configuration of the motion browsing apparatus according to the third embodiment. FIG. 15 is a flowchart illustrating an exemplary flow of a moving image display process performed by the moving image display apparatus according to the third embodiment. FIG. 16 is a block diagram illustrating an example of a configuration of a moving image browsing apparatus according to the fourth embodiment. FIG. 17 is a flowchart illustrating an example of a moving image reading process performed by the moving image display apparatus according to the fourth embodiment. FIG. 18 is a flowchart illustrating an exemplary flow of a moving image display process performed by the moving image display apparatus according to the fourth embodiment.

  Exemplary embodiments of a moving image display device, a moving image display method, and a program will be described below in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1A is a block diagram illustrating an example of a hardware configuration of the moving image display apparatus according to the first embodiment. As shown in FIG. 1A, the moving image display device according to the present embodiment includes a moving image browsing device 10, a storage device 20, an input / output device 30, and an image display device 40 (an example of a display unit). Yes. The moving image browsing device 10 controls display of various images on the image display device 40. The storage device 20 stores various information such as moving images to be displayed on the image display device 40. The input / output device 30 is an operation unit for a user of the moving image display device to input various operations. The image display device 40 is a display unit that is controlled by the moving image browsing device 10 and displays a moving image or the like.

  FIG. 1B is a block diagram illustrating an example of a functional configuration of the moving image browsing apparatus according to the first embodiment. In the present embodiment, as shown in FIG. 1B, the moving image browsing apparatus 10 includes a frame reading unit 101, an attention object determination unit 102, an attention object tracking unit 103, a partial image generation unit 104, and an entire image generation unit. 105, a target object region emphasis unit 106, and an image display unit 107.

  Next, a moving image display process in the moving image display apparatus according to the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart illustrating an exemplary flow of a moving image display process in the moving image display apparatus according to the first embodiment.

  First, the frame reading unit 101 reads a moving image to be displayed frame by frame from the storage device 20 according to the operation information input from the input / output device 30 (step S201). Here, the moving image to be displayed is an example of a moving image that is obtained by imaging of an imaging device that can capture images with an angle of view exceeding 180 degrees, such as a fish-eye camera or an omnidirectional camera, and includes a plurality of frames. is there. That is, the moving image to be displayed is a moving image obtained by imaging at a wide angle. In the present embodiment, the moving image to be displayed is a moving image instructed to be displayed by the user via the input / output device 30 among the moving images stored in the storage device 20.

  First, the frame reading unit 101 sequentially reads from the first frame of the moving image to be displayed or the frame instructed by the user via the input / output device 30. In the second and subsequent frame reading, the frame reading unit 101 reads the next frame after the previously read frame. In the present embodiment, the frame reading unit 101 reads frames constituting a moving image from the storage device 20, but the present invention is not limited to this. For example, the frame reading unit 101 has a wide angle such as a fisheye camera or an omnidirectional camera. You may acquire a flame | frame directly from the imaging device which can image.

  Next, the target object determination unit 102 determines a target object (an example of a predetermined image) that is an image designated by the user via the input / output device 30 in the read frame (step S202). FIG. 3 is a diagram illustrating a frame display example in the moving image display apparatus according to the first embodiment. In the present embodiment, the image display unit 107 displays the frame read by the frame reading unit 101 on the image display device 40 as shown in FIG.

  Then, as shown in FIG. 3, the object-of-interest determining unit 102 is a region designated by the user via the input / output device 30 (for example, a rectangular region R) in the frame displayed on the image display device 40. The image inside is determined as the target object AT. The object-of-interest determination unit 102 does not need to determine the object of interest every time a frame is read by the frame reading unit 101. If the user does not specify an area in the frame via the input / output device 30, the object-of-interest determination unit 102 The image in the area specified last by is continued to be the target object.

  That is, the object-of-interest determination unit 102 basically only needs to determine the object of interest once when a frame is first read from a moving image to be displayed. Accordingly, the target object determination unit 102 designates a new area by the user via the input / output device 30 when the target object tracking unit 103 (to be described later) fails to track the target object or during the display of the moving image. In this case, the target object is determined again.

  The target object tracking unit 103 (an example of a detection unit) detects the position of the target object in the last read frame (hereinafter referred to as the current frame) every time the frame is read by the frame reading unit 101. The object of interest is tracked (step S203). In the present embodiment, the target object tracking unit 103 detects the position of the target object in the current frame using a general object position detection method described in Non-Patent Document 1. However, the target object tracking unit 103 does not detect the position of the target object in the current frame when the target object determination unit 102 does not determine the target object.

  The partial image generation unit 104 (an example of a cutout unit) cuts out an image including a target object at a position that is a part of the current frame and detected by the target object tracking unit 103 from the current frame, and the cutout image Is generated as a partial image (an example of a first image) (step S204). FIG. 4 is a diagram for explaining a partial image generation process in the moving image display apparatus according to the first embodiment. In the present embodiment, the partial image generation unit 104 acquires a current frame unit sphere CB in which the current frame is represented by a unit sphere B as shown in FIG. Specifically, when the current frame is a frame obtained by imaging with a fisheye camera or an omnidirectional camera, the partial image generation unit 104 specifies the shooting direction of the current frame based on the projection method, and The current frame unit sphere CB is obtained by plotting the current frame on the unit sphere from the shooting direction (see Patent Document 3).

  Then, the partial image generation unit 104 uses an image that is a part of the current frame unit sphere CB and includes the target object AT as a center (projection center O) and is projected on a two-dimensional plane (projection plane S). Generate as a partial image.

  The entire image generation unit 105 (an example of the generation unit) generates an entire image (an example of a second image) in which the distortion of the object of interest in the current frame is set to a predetermined value or less (step S205). In the present embodiment, the entire image generation unit 105 generates an entire image obtained by panoramic development of the current frame. For example, the whole image generation unit 105 uses the equirectangular projection to panoramicly develop the current frame so that the distortion of the object of interest in the current frame is a predetermined value or less. Thereby, even when the current frame is panorama developed using equirectangular projection, it is possible to easily grasp the region corresponding to the object of interest in the entire image. 5 and 6 are diagrams for explaining panoramic development in the moving image display apparatus according to the first embodiment. Specifically, as shown in FIG. 5, the entire image generation unit 105 represents the shooting direction D of the current frame unit sphere CB as longitude θ and latitude ψ, and represents the current frame unit sphere CB as longitude θ and latitude. The whole image G (so-called equirectangular image) expressed in the two-dimensional orthogonal coordinate system of ψ is converted.

  In the equirectangular projection, as shown in FIG. 6, as the latitude ψ of the position of the object of interest AT in the current frame unit sphere CB increases, the object of interest in the entire image G is stretched in the horizontal direction. On the other hand, in the equirectangular projection, as shown in FIG. 6, when the latitude ψ of the position of the object of interest in the current frame unit sphere CB is 0 degree, the distortion of the object of interest in the entire image G is the smallest. Therefore, the entire image generation unit 105 sets the latitude ψ of the position of the target object AT included in the current frame unit sphere CB to 0 degrees. Next, the entire image generation unit 105 generates an entire image G obtained by panoramic development of the current frame unit sphere CB in which the position of the object of interest AT is set to latitude ψ: 0 °. Thereby, the whole image generation unit 105 can obtain the whole image G in which the distortion of the object of interest in the current frame is small.

  In the present embodiment, the whole image generation unit 105 panoramicly expands the current frame using equirectangular projection, but the present invention is not limited to this. For example, other cylindrical projection, conic projection, azimuth projection, etc. The current frame may be panorama developed using.

  Returning to FIG. 2, the target object region emphasizing unit 106 highlights the display mode of the region corresponding to the partial image (hereinafter referred to as the target object region) in the whole image generated by the whole image generating unit 105 ( Step S206). 7 and 8 are diagrams illustrating an example of a display mode of the target object region in the moving image display apparatus according to the first embodiment. In the present embodiment, as shown in FIG. 7, the target object region emphasizing unit 106 displays a rectangular frame 700 surrounding the target object AT included in the whole image G displayed on the image display device 40, as the whole image G. The display mode of the target object area is highlighted by being included in. In the present embodiment, the target object area emphasizing unit 106 highlights the display mode of the target object area included in the entire image, but the display mode of the target object area is changed to an area other than the target object area of the total image. However, the display mode is not limited to this. For example, the target object region emphasizing unit 106 may include a boundary line 800 between the target object region r corresponding to the partial image g in the whole image G and other regions in the whole image G as shown in FIG.

  The image display unit 107 displays at least the entire image among the partial image generated by the partial image generation unit 104 and the entire image generated by the entire image generation unit 105 on the image display device 40 (step S207). This makes it easy to grasp the region corresponding to the partial image in the entire image when the object of interest included in the moving image obtained by capturing at a wide angle with a fish-eye camera or an omnidirectional camera is displayed. be able to.

  In the present embodiment, the image display unit 107 displays the partial image and the entire image in different areas of the display screen of the image display device 40. FIGS. 9 and 10 are diagrams illustrating display examples of the partial image and the entire image in the moving image display apparatus according to the first embodiment. For example, the image display unit 107 may display the partial image g and the entire image G side by side on the display screen of the image display device 40 as shown in FIG. Alternatively, the image display unit 107 may superimpose and display the entire image G on a part of the partial image g on the display screen of the image display device 40 as shown in FIG. In the present embodiment, the target object region emphasizing unit 106 and the image display unit 107 function as an example of a display control unit that changes the display mode of the target object region in the entire image from other regions.

  Thereafter, the frame reading unit 101 determines whether or not the current frame is the last frame constituting the moving image to be displayed (step S208). When it is determined that the current frame is not the last frame (step S208: No), the frame reading unit 101 returns to step S201 and reads the next frame of the current frame. On the other hand, when it is determined that the current frame is the last frame (step S208: Yes), the frame reading unit 101 ends reading of the frame.

  As described above, according to the image display device according to the first embodiment, the display mode of the area corresponding to the partial image in the entire image can be different from that of other areas. When an object of interest included in a moving image obtained by imaging with a camera or the like at a wide angle is cut out and displayed, an area corresponding to a partial image in the entire image can be easily grasped.

(Second Embodiment)
In the present embodiment, based on the detection history of the position of the object of interest from the frame in which the entire image has been displayed, a range in which the object of interest moves in the entire image (hereinafter referred to as an operation prediction range) is predicted, and the operation prediction is performed. This is an example of generating an entire image with a range distortion set to a predetermined value or less. In the following description, description of the same parts as in the first embodiment will be omitted.

  FIG. 11 is a block diagram illustrating an example of a functional configuration of the moving image browsing apparatus according to the second embodiment. As illustrated in FIG. 11, the moving image browsing apparatus 1100 according to the present embodiment includes an entire image generation unit 1102 to which an operation range prediction unit 1101 is added and whose operation is different from that of the first embodiment. .

  Next, a moving image display process performed by the moving image display apparatus according to the present embodiment will be described with reference to FIG. FIG. 12 is a flowchart illustrating an exemplary flow of a moving image display process in the moving image display apparatus according to the second embodiment.

  In the present embodiment, the motion range prediction unit 1101 is based on the detection history of the position of the target object by the target object tracking unit 103 (that is, the detection history of the position of the target object from the frame in which the entire image has been displayed). A motion prediction range (an example of a first range) that is a range in which the object of interest moves within the moving image is predicted (step 1201). FIG. 13 is a diagram for explaining an example of a motion prediction range prediction process in the moving image display apparatus according to the second embodiment. Specifically, the motion range prediction unit 1101 assumes that the object of interest moves linearly. Next, the motion range prediction unit 1101 is detected in the current frame F, starting from the position P-1 of the target object detected in the frame F-1 that is one frame before the current frame F, as shown in FIG. A motion prediction line l that is a half line passing through the position P of the object of interest is set as a motion prediction range.

式（１）において、｛ａ_ｉ｝^Ｍ _ｉ＝１は、軌道モデルのパラメータである。そして、動作範囲予測部１１０１は、式（１）で表される軌道モデルに従って、カレントフレーム以降のフレームにおける注目物体の位置を動作予測範囲として求める。 Alternatively, the motion range prediction unit 1101 obtains a parametric target object trajectory (hereinafter referred to as a trajectory model) according to the target object position distribution in a frame (past frame) before the current frame. Further, the motion range prediction unit 1101 may identify the parameters of the trajectory model using the least square method. For example, the motion range prediction unit 1101 obtains a trajectory model represented by the Mth order polynomial shown in the following equation (1), where (x (t), y (t)) is the position of the object of interest at time t.

In equation (1), {a _i } ^M _{i = 1} is a parameter of the trajectory model. Then, the motion range prediction unit 1101 obtains the position of the object of interest in the frames subsequent to the current frame as the motion prediction range according to the trajectory model represented by Expression (1).

Alternatively, the motion range prediction unit 1101 may obtain the motion prediction range from the position of the object of interest in the frame read last using the state space model. For example, when the state space model is used for tracking the target object, the motion range prediction unit 1101 obtains the motion prediction range based on the state space model using the parameter (position of the target object) at that time. For example, the motion range prediction unit 1101 obtains a motion prediction range by repeatedly solving the recurrence formula using an M-order polynomial state space model expressed by the following formula (2).

The motion prediction range prediction methods described above are all methods for obtaining a straight line or a curve as the motion prediction range, but a region in which the obtained straight line or curve has a fluctuation range may be used as the motion prediction range. Specifically, the motion range prediction unit 1101 obtains the motion prediction range as a probability distribution p (x) with respect to the position (x (t)) of the target object. For example, the motion range prediction unit 1101 obtains the probability distribution p (x) of the position of the object of interest using the following equation (3) in which a random term ε is added to the state space model of the above equation (2).

Further, even when the motion range prediction unit 1101 obtains the motion prediction range as the trajectory f (x) = 0, according to the following equation (4), the motion range prediction range is given a probability density function. It can be expressed as In Expression (4), L is the length of the trajectory f (x).

Alternatively, the motion range prediction unit 1101 can also represent the motion prediction range by a probability density function in which the trajectory f (x) has an appropriate width σ, as shown in the following equation (5).

  Returning to FIG. 12, the entire image generation unit 1102 generates an entire image in which the distortion of the motion prediction range in the current frame is set to a predetermined value or less (step S1202). For example, if the equirectangular projection is used for panorama development of the current frame and the motion prediction range obtained by the motion range prediction unit 1101 is linear as shown in FIG. The current frame is panoramicly developed so that the motion prediction range in the distance cylindrical image is located at latitude ψ: 0 °.

式（６）において、Ｓは注目物体の位置ｘの定義域であり、ｗ（ｘ）は注目物体の位置ｘにおける全体画像の歪みを反映した重み関数である。重み関数ｗ（ｘ）は、正距円筒図法において、下記の式（７）により表される。

By the way, in equirectangular projection, an image in the current frame is multiplied by 1 / c ψ in the horizontal direction at latitude ψ. Therefore, the entire image generation unit 1102 uses the value obtained by adding 1 / c ψ to the motion prediction range (definite integral value) as an evaluation function, and even if the current frame is panorama expanded so that the evaluation function is minimized. good. For example, when the motion prediction range is represented by the probability density function p (x), the entire image generation unit 1102 expands the current frame in a panoramic manner so that the evaluation function J shown in the following equation (6) is minimized. To do.

In Expression (6), S is a domain of the position x of the target object, and w (x) is a weighting function that reflects the distortion of the entire image at the position x of the target object. The weight function w (x) is expressed by the following equation (7) in equirectangular projection.

That is, the weighting function w (x) represents the distance that the position multiplied by 1 / c ψ in the horizontal direction is away from the position without distortion in the current frame. Here, the conversion from the position x of the object of interest in the current frame to ψ (and θ) is expressed by the following equation (8) using the rotation R and the translation T.

  Therefore, the entire image generation unit 1102 solves the optimization problem for obtaining the rotation R and translation T that minimize Equations (6) and (7). Since this optimization problem is a nonlinear optimization problem, it is solved by a nonlinear programming method such as Newton's method or conjugate gradient method.

  As described above, the weighting function w (x) only needs to represent the degree of distortion at the position x of the target object in the entire image. Therefore, even when the current frame is panorama developed using a method other than the equirectangular projection, the current frame is panorama developed using the degree of distortion of the object of interest in the entire image by another method.

  An arbitrary coordinate system can be used as the coordinate system of the position x of the object of interest. For example, a coordinate system on the current frame, a coordinate system on a partial image, or a coordinate system on the entire image may be used.

  As described above, according to the image display apparatus according to the second embodiment, when a target object included in a moving image obtained by imaging at a wide angle is cut out and displayed, the future movement of the target object is considered. Thus, since the motion prediction range in the entire image is highlighted, it is possible to make it easier to grasp the position of the region corresponding to the partial image in the entire image.

(Third embodiment)
The present embodiment is an example in which the horizontal direction of the current frame is detected, and the current frame is panorama developed while maintaining the horizontal direction of the current frame. In the following description, portions different from the first and second embodiments will be described.

  FIG. 14 is a block diagram illustrating an example of the configuration of the motion browsing apparatus according to the third embodiment. As illustrated in FIG. 14, the moving image browsing apparatus 1400 according to the present embodiment includes an entire image generation unit 1401 that operates differently from the second embodiment. In the present embodiment, the imaging device that captured the moving image stored in the storage device 20 detects the acceleration when the moving image is captured by the acceleration sensor mounted on the imaging device, and the acceleration detection result Is added as metadata of a moving image. Then, the entire image generation unit 1401 detects the gravitational direction in the frame constituting the moving image and the horizontal direction orthogonal to the gravitational direction based on the detection result of the acceleration added to the moving image to be displayed.

  Next, moving image display processing by the moving image display apparatus will be described with reference to FIG. FIG. 15 is a flowchart illustrating an exemplary flow of a moving image display process performed by the moving image display apparatus according to the third embodiment.

  In the present embodiment, the entire image generation unit 1401 generates an entire image in which the current frame is panorama developed so that the distortion of the motion prediction range is a predetermined value or less while maintaining the horizontal direction of the current frame ( Step S1501). Empirically, the object of interest included in the current frame often moves in the horizontal direction, and the visibility of the object of interest is better when the current frame is not inclined in the horizontal direction. Therefore, the entire image generation unit 1401 expands the current frame in a panorama manner under the constraint that the horizontal direction of the entire image matches the horizontal direction of the image obtained by actual imaging (that is, the current frame). Note that the entire image generation unit 1401 performs panoramic development of the current frame in the same manner as in the first and second embodiments.

  As described above, according to the image display device according to the third embodiment, the current frame is panorama developed while maintaining the horizontal direction of the current frame. Therefore, when displaying the entire image, the entire image is displayed. The inclination of the target object area in the area is reduced, and the target object area can be more easily grasped.

(Fourth embodiment)
In this embodiment, positions of target objects in all frames constituting a moving image are detected in advance, and a target object position group (an example of a position group) that is a set of positions of target objects in all frames This is an example in which the motion range of the object of interest is determined based on the above and an entire image is generated in which the motion range distortion in the current frame is set to a predetermined value or less. In the following description, description of the same parts as those in the first, second, and third embodiments is omitted.

  FIG. 16 is a block diagram illustrating an example of a configuration of a moving image browsing apparatus according to the fourth embodiment. As shown in FIG. 16, the moving image browsing apparatus 1600 of the moving image display apparatus according to the fourth embodiment is different in operation from the above-described embodiment, and includes a frame reading unit 1601, a target object determining unit 1602, and a target object tracking. A unit 1603, a partial image generation unit 1604, an entire image generation unit 1605, an attention object region enhancement unit 1606, and an image display unit 1607.

  Next, moving image display processing by the moving image display apparatus will be described with reference to FIGS. 17 and 18. FIG. 17 is a flowchart illustrating an example of a moving image reading process performed by the moving image display apparatus according to the fourth embodiment. FIG. 18 is a flowchart illustrating an exemplary flow of a moving image display process performed by the moving image display apparatus according to the fourth embodiment.

  In the present embodiment, the frame reading unit 1601 reads a moving image to be displayed frame by frame from the storage device 20 regardless of the input of operation information from the input / output device 30 (step S1701). Also in this embodiment, the frame reading unit 1601 sequentially reads from the first frame of the moving image to be displayed or the frame instructed by the user via the input / output device 30.

  The target object determination unit 1602 determines an image designated by the user via the input / output device 30 as the target object in the read frame (step S1702). Every time a frame is read by the frame reading unit 1601 (step S1703), the target object tracking unit 1603 tracks the target object by detecting the position of the target object in the frame (step S1704). The target object tracking unit 1603 stores the detected position of the target object in the storage device 20 in association with the frame in which the position is detected.

  Then, when the position of the target object is detected by the target object tracking section 1603, the frame reading unit 1601 determines whether or not the last read frame is the last frame constituting the moving image to be displayed (step). S1705). If it is determined that the last read frame is not the last frame constituting the moving image to be displayed (step S1705: No), the frame reading unit 1601 reads the next frame after the last read frame (step S1705). S1703). On the other hand, when it is determined that the last read frame is the last frame constituting the moving image to be displayed (step S1705: Yes), the frame reading unit 1601 ends the frame reading process. That is, the object-of-interest tracking unit 1603 detects in advance the position of the object of interest in all frames constituting the moving image.

  Thereafter, when display of a moving image is instructed via the input / output device 30, the frame reading unit 1601 reads frames constituting the moving image (display target moving image) instructed to display from the storage device 20. (Step S1801). Next, the partial image generation unit 1604 selects a target object that is a part of the current frame from the current frame read by the frame reading unit 1601 and exists at a position stored in the storage device 20 in association with the current frame. Cut out the containing image. Then, the partial image generation unit 1604 generates a partial image based on the clipped image (step S1802). The partial image generation method is the same as that in the first embodiment.

  The overall image generation unit 1605 is based on a target object position group that is a set of target object positions stored in the storage device 20 in association with a plurality of frames constituting a moving image to be displayed. The motion range of the object is determined, and an entire image in which the current frame is panoramic developed so that the distortion of the motion range is a predetermined value or less is generated (step S1803). As a result, the entire image in which the distortion of the current frame is reduced is generated in consideration of the range in which the partial image moves, so that the target object region in the entire image can be more easily grasped.

  In the present embodiment, the entire image generation unit 1605 includes a parametric such as a normal distribution, a uniform distribution, or a mixture distribution thereof in the probability distribution p of the position of the object of interest in each of a plurality of frames constituting the moving image to be displayed. A model is assumed, and its parameter is identified as the motion range of the object of interest. Alternatively, the entire image generation unit 1605 may use an experience distribution using the position group of the target object as it is in each of a plurality of frames constituting the moving image to be displayed as the target object motion range.

  The attention object area emphasizing unit 1606 highlights the display mode of the attention object included in the entire image generated by the entire image generation unit 1605 as in the first embodiment (step S1804). As in the first embodiment, the image display unit 1607 displays at least the entire image among the partial image generated by the partial image generation unit 1604 and the entire image generated by the entire image generation unit 1605, as an image display device. 40 (step S1805). Thereafter, the frame reading unit 1601 determines whether or not the current frame is the last frame constituting the moving image to be displayed (step S1806).

  If it is determined that the current frame is not the last frame (step S1806: NO), the frame reading unit 1601 returns to step S1801 and reads the next frame of the current frame. On the other hand, if it is determined that the current frame is the last frame (step S1806: YES), the frame reading unit 1601 finishes reading the frame.

  As described above, according to the image display device according to the fourth embodiment, the entire image in which the distortion of the current frame is reduced is generated in consideration of the range in which the partial image moves. It is possible to make it easier to grasp the area corresponding to.

  Note that the program executed by the moving image browsing apparatuses 10, 1100, 1400, and 1600 of the present embodiment is provided by being incorporated in advance in a ROM (Read Only Memory) or the like. The programs executed by the moving image browsing apparatuses 10, 1100, 1400, and 1600 of the present embodiment are files in an installable format or an executable format, such as CD-ROM, flexible disk (FD), CD-R, DVD ( The recording medium may be recorded on a computer-readable recording medium such as a digital versatile disk).

  Furthermore, the program executed by the moving image browsing apparatus 10, 1100, 1400, 1600 of the present embodiment is stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. You may do it. Further, the program executed by the moving image browsing apparatuses 10, 1100, 1400, and 1600 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  The programs executed by the moving image browsing apparatuses 10, 1100, 1400, and 1600 according to the present embodiment are the above-described units (frame reading units 101 and 1601, target object determination units 102 and 1602, target object tracking units 103 and 1603, parts The module configuration includes the image generation units 104 and 1604, the entire image generation units 105, 1102, 1401, and 1605, the target object region enhancement units 106 and 1606, the image display units 107 and 1607, and the motion range prediction unit 1101). As actual hardware, a CPU (Central Processing Unit) reads out a program from the ROM and executes it, so that the above-described units are loaded on the main storage device, and the frame reading units 101 and 1601, the object-of-interest determining units 102 and 1602, Attention object tracking units 103 and 1603, partial image generation units 104 and 1 Reference numeral 604, an entire image generation unit 105, 1102, 1401, 1605, an attention object region enhancement unit 106, 1606, an image display unit 107, 1607, and an operation range prediction unit 1101 are generated on the main storage device.

10, 1100, 1400, 1600 Movie browsing device 20 Storage device 30 Input / output device 40 Image display device 101, 1601 Frame reading unit 102, 1602 Target object determination unit 103, 1603 Target object tracking unit 104, 1604 Partial image generation unit 105, 1102, 1401, 1605 Whole image generation unit 106, 1606 Object-of-interest region enhancement unit 107, 1607 Image display unit 1101 Motion range prediction unit

JP 2008-203756 A JP 2014-085845 A Japanese Patent No. 5233926 Japanese Patent No. 5733088 JP 2012-244479 A

Y. Wu, J. Lim, M.H. Yang, “Object Tracking Benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Pre-Prints, 2015.

Claims (7)

A cutout unit that cuts out a first image that is a part of the moving image and includes a predetermined image from the frame;
A generating unit that generates a second image in which distortion of the predetermined image in the frame is set to be equal to or lower than a predetermined value;
A display control unit that displays a display mode of an area corresponding to the first image in the second image different from other areas;
A detection unit for detecting a position of the predetermined image in the frame;
A prediction unit that predicts a first range in which the predetermined image moves in the moving image based on a detection history of the position of the predetermined image from the frame in which the second image has been displayed;
With
The said production | generation part is a moving image display apparatus which produces | generates the said 2nd image which made distortion of the said 1st range below the said predetermined value . A cutout unit that cuts out a first image that is a part of the moving image and includes a predetermined image from the frame;
A generating unit that generates a second image in which distortion of the predetermined image in the frame is set to be equal to or lower than a predetermined value;
A display control unit that displays a display mode of an area corresponding to the first image in the second image different from other areas;
A detection unit that detects in advance the position of the predetermined image in all the frames constituting the moving image;
With
The generation unit determines an operation range of the predetermined image based on a position group that is a set of positions of the predetermined image in all the frames, and the distortion of the operation range is set to the predetermined value or less. Generate two images,
Movie display device. The generating unit uses a equirectangular, moving picture display apparatus according to claim 1 or 2, the frame and panoramic to generate the second image. A frame constituting a moving image, comprising the steps of: to cut out the first image including a part at and and a predetermined image in the frame,
Generating a second image in which the distortion of the predetermined image in the frame is less than or equal to a predetermined value;
A step of Ru is displayed the display mode of the area corresponding to the first image in the second image by varying the other regions,
Detecting the position of the predetermined image in the frame;
Predicting a first range in which the predetermined image moves in the moving image based on a detection history of the position of the predetermined image from the frame on which the second image has been displayed;
Including
The step of generating the second image generates the second image in which the distortion of the first range is equal to or less than the predetermined value.
Video display method. Cutting out a first image that is a part of the moving image and includes a predetermined image from the frame;
Generating a second image in which the distortion of the predetermined image in the frame is less than or equal to a predetermined value;
Displaying a display mode of an area corresponding to the first image in the second image different from other areas;
Detecting in advance the position of the predetermined image in all the frames constituting the moving image;
Including
The step of generating the second image determines an operation range of the predetermined image based on a position group that is a set of positions of the predetermined image in all the frames, and distortion of the operation range is less than or equal to the predetermined value. Generating the second image,
Video display method. Computer
A cutout unit that cuts out a first image that is a part of the moving image and includes a predetermined image from the frame;
A generating unit that generates a second image in which distortion of the predetermined image in the frame is set to be equal to or lower than a predetermined value;
A display control unit that displays a display mode of an area corresponding to the first image in the second image different from other areas;
A detection unit for detecting a position of the predetermined image in the frame;
A prediction unit that predicts a first range in which the predetermined image moves in the moving image based on a detection history of the position of the predetermined image from the frame in which the second image has been displayed;
Function as
The said production | generation part is a program which produces | generates the said 2nd image which made distortion of the said 1st range the said predetermined or less . Computer
A cutout unit that cuts out a first image that is a part of the moving image and includes a predetermined image from the frame;
A generating unit that generates a second image in which distortion of the predetermined image in the frame is set to be equal to or lower than a predetermined value;
A display control unit that displays a display mode of an area corresponding to the first image in the second image different from other areas;
A detection unit that detects in advance the position of the predetermined image in all the frames constituting the moving image;
Function as
The generation unit determines an operation range of the predetermined image based on a position group that is a set of positions of the predetermined image in all the frames, and the distortion of the operation range is set to the predetermined value or less. Generate two images,
program.

Images