JP2012063547A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for performing correction processing for motion picture sickness prevention on a partial area possibly causing motion picture sickness in a display screen.SOLUTION: The image processing method which includes: specifying an attention area of a user in a display screen (S101); specifying a candidate object which is comparatively larger than the attention area and in which the dispersion value of a pixel value is equal to or larger than a threshold value (S102 to S105); specifying a correction target object in which cycles of movement between frame images are within a specific range and the amount of the movement is equal to or larger than the threshold value, out of the candidate objects (S106 and S107). The correction target object is rearranged on an update target frame image in which the correction target object is positioned in the attention area, and displayed on the display screen.

Description

  The present invention relates to an image display technique in a display device, particularly a display device having a large display surface.

  Conventionally, thin and large-screen displays typified by liquid crystal displays and plasma displays have become widespread. However, while being able to view beautiful images with a high sense of realism, there are concerns about adverse effects on the living body such as video sickness and light-sensitive seizures. For example, there are many image shakes caused by camera shake in videos taken by home video cameras by ordinary people. It is thought that motion sickness is likely to be induced by such camera shake, rotation, and rapid zooming. In addition, it is considered that video sickness is more likely to be induced as the video to be viewed is larger, as in the case of viewing on a large screen display. For example, the following method has been devised as a method for preventing video sickness or photosensitive seizures on the playback / display device side.

  (1) When it is detected that the content screen is shaken, a method of reducing the screen and displaying it has been devised (Patent Document 1).

  (2) A method has been devised in which a distance between a display screen and a user is detected, and when the detected distance is short, the screen is reduced and displayed (Patent Document 2).

  (3) Based on the size of the display area, viewing distance, and luminance characteristics, a method has been devised for detecting a dangerous section that may cause a photosensitive attack (Patent Document 3).

JP 2008-299241 A JP 2006-235307 A JP 2007-194741 A

  However, in the above conventional example, there is a case where video sickness cannot be prevented by a display having a large display surface. For example, in a large-screen display environment such as a wall display, when close to the screen and gazing at only a part of the area, video sickness prevention correction cannot be performed. Even if the entire content screen is not shaken, if the portion being watched is shaken, there is a possibility of causing motion sickness, and this cannot be corrected by the conventional technology.

  The present invention has been made in view of the above problems, and provides a technique for performing correction processing for the purpose of preventing image sickness on a partial region that may cause image sickness in a display screen. .

  In order to achieve the object of the present invention, for example, an image processing apparatus of the present invention comprises the following arrangement. That is, an image processing apparatus that displays a moving image including a frame image in which one or more objects are arranged, the means for specifying a user's gaze area on a display screen for displaying the moving image, and the moving image Means for identifying an object having an area in which the ratio of the gaze area to the area of the gaze region is equal to or greater than a threshold and having a variance of pixel values equal to or greater than the threshold as candidate objects, Among these, a specifying means for specifying a candidate object whose movement cycle between frame images is within a specified range and the amount of movement is equal to or greater than a threshold as a correction target object, and the correction target object when displayed on the display screen Is a frame image that is located in the gaze area as an update target frame image, and the frame image on the update target frame image Display control means for updating the update target frame image by rearranging the main target object and displaying the updated update target frame image on the display screen, wherein the display control means includes the respective update target frame images. The position of the correction target object on the respective update target frame images is determined so that the period of the movement of the correction target object during the period is equal to or smaller than the threshold value or the amount of the movement is smaller than the threshold value. The correction target object is rearranged at the determined arrangement position.

  ADVANTAGE OF THE INVENTION According to this invention, the correction process aiming at prevention of image sickness can be performed with respect to the partial area | region which may cause image sickness within a display screen.

FIG. 2 is a block diagram illustrating an example of a functional configuration of the image processing apparatus 1. The block diagram which shows the function structural example of the image processing apparatus 1 at the time of applying to a television apparatus. The figure which shows an example of a display screen. The figure which shows an example of a display screen. The figure which shows an example of a display screen. The figure which shows an example of a display screen. The flowchart of (Process 1). The flowchart of (Process 2).

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
In the present embodiment, on the display screen, a candidate area that may cause video sickness when the user gazes is extracted, and if this candidate area is included in the display area that the user views, Image sickness prevention correction is performed on the candidate area. Although various types of motion sickness prevention correction can be considered, in the present embodiment, a process of slowing down the movement of the image object is performed.

<Example of motion sickness prevention correction operation>
First, with reference to FIGS. 3 to 6, the motion sickness prevention correction process for the region that the user is gazing at will be described. 3 to 6 each show an example of a display screen (frame image 50). In the present embodiment, among the objects included in the area that the user is gazing at, the above-mentioned image is obtained by dulling the movement of an object that occupies a relatively large proportion in this area and has a periodic movement. Realize sickness prevention correction processing.

  FIG. 3 shows an example of the frame image 50. The frame image 50 is an image of a certain frame in a moving image composed of a plurality of frames, and includes a person object 51, a house object 52, and a chimney object 53. The person object 51 and the chimney object 53 are periodically shaken. . That is, the display position and orientation of the person object 51 and the chimney object 53 are changed over a plurality of frames.

  FIG. 4 shows an area 54 that the user can watch as the user approaches the frame image 50 of FIG. Since the user is too close to the frame image 50, the entire frame image 50 cannot be seen, and only a partial region (region 54) of the frame image 50 can be seen. In this case, since the object moving periodically is not shown in the region 54 that can be observed by the user, even if the user observes the region 54, the user does not get sick. However, in this embodiment, the motion sickness prevention correction process is not performed in such a case.

  FIG. 5 shows an area 54 that the user can watch as the user approaches the frame image 50 of FIG. Since the user is too close to the frame image 50, the entire frame image 50 cannot be seen, and only a partial region (region 54) of the frame image 50 can be seen. In this case, a chimney object 53 moving periodically is shown in an area 54 that can be observed by the user. However, since the area of the chimney object 53 is relatively small with respect to the area of the region 54, even if the user observes the region 54, the user does not get sick. However, in this embodiment, the motion sickness prevention correction process is not performed in such a case.

  FIG. 6 shows an area 54 that can be watched by the user as he / she approaches the frame image 50 of FIG. 3. Since the user is too close to the frame image 50, the entire frame image 50 cannot be seen, and only a partial region (region 54) of the frame image 50 can be seen. In this case, a periodically moving human object 51 is shown in the region 54 that can be observed by the user. Since the area of the person object 51 is relatively large with respect to the area of the region 54, if the user observes the region 54, there is a possibility of causing motion sickness. In this embodiment, in such a case, the motion sickness prevention correction process is performed on the person object 51. In FIG. 6, a human object 51 that has been subjected to the video sickness prevention correction process is shown, and the periodic movement is slow.

  As described above, in the present embodiment, for an object having a relatively large area and periodically moving in the region observed by the user on the display screen, correction processing is performed to slow down the movement. .

<Image processing apparatus according to this embodiment>
A functional configuration example of the image processing apparatus 1 according to the present embodiment will be described using the block unit of FIG. The viewing area detection unit 2 detects an area (gaze area) that the user observes on the display screen of the display unit 7. Of course, when the user observes the display screen of the display unit 7 from a position sufficiently away from the display screen of the display unit 7, the user observes the entire display screen of the display unit 7. In addition, when viewing close to the viewing angle of the user, the viewing angle range is observed by the user.

  The content input unit 3 inputs a moving image including a frame image including one or more objects, and sends each frame image to the dangerous area extraction unit 4 and the video sickness correction unit 6.

  The dangerous area extraction unit 4 extracts an area that may cause video sickness from the frame image input from the content input unit 3 as a candidate area. Details of the candidate area extraction processing will be described later.

  The area match determination unit 5 determines whether or not a candidate area is included in the gaze area. When the candidate region is included in the gaze region, the image sickness correction unit 6 performs image motion sickness prevention correction processing on the candidate region. In the present embodiment, a process for slowing the motion in the candidate area is performed using a filter in the time axis direction. Then, the motion sickness correction unit 6 sends the frame image subjected to the motion sickness prevention correction process to the display unit 7 at the subsequent stage. Of course, the video sickness correction unit 6 sends the frame image input from the content input unit 3 to the subsequent display unit 7 as long as the candidate region is not included in the gaze region. The display unit 7 displays the frame image received from the video sickness correction unit 6.

  Next, a functional configuration example of the image processing apparatus 1 when applied to a television apparatus will be described with reference to the block diagram of FIG. The camera 21 includes a plurality of camera devices capable of capturing moving images, and is installed above the image processing device 1. Each camera device is installed to observe a user who observes the display screen of the display panel 31, and the captured image of each frame captured by each camera device is sent to the subsequent image recognition unit 22.

  The image recognition unit 22 recognizes a user who is observing the display panel 31 by performing image recognition on the captured image transmitted from each camera device. In this image recognition, for example, it is determined whether or not this object is a human body by detecting the contour or feature point of the object in the captured image and matching it with a database created in advance. Similarly, matching is performed to determine the position of the person and the orientation of the face. Various methods can be considered for such a series of recognition techniques performed by the image recognition unit 22, and any of them may be used.

  The viewing area calculation unit 23 calculates the user's field of view from the user's position and face orientation. Then, the viewing area calculation unit 23 obtains an area observed by the user on the display panel 31 as a gaze area from the calculated visual field.

  The tuner 24 receives and demodulates a broadcast wave supplied from an external antenna (not shown), and extracts TS data. The decoder 25 separates compressed video, compressed audio, and control data from the TS data. The decoder 25 obtains moving image data and audio data by decoding compressed video and compressed audio. This moving image is composed of a frame image including one or more objects.

  A video I / F (interface) 26 inputs content from outside. For example, it is configured with interfaces of various standards such as HDMI and DVI, and inputs various video contents. This video content includes a moving image including a frame image including one or more objects.

  The selector 27 selects either the decoder 25 or the video I / F 26, and sends the data output from the selected one to the subsequent dangerous area extraction unit 4 and the image quality improvement processing unit 28. The operation of the dangerous area extraction unit 4 is as described above, and an area that may cause video sickness is extracted as a candidate area from the frame image input from the selector 27.

  The image quality enhancement processing unit 28 performs image processing such as enlargement / reduction processing and color correction processing on the frame image received from the selector 27. The OSD synthesis unit 29 superimposes various operation information such as channel number and volume setting on the frame image subjected to the image processing by the image quality improvement processing unit 28. Note that the image quality improvement processing unit 28 and the OSD composition unit 29 are provided because the image processing apparatus 1 is a television apparatus, and are not essential to achieve the object of the present embodiment. That is, a frame image may be input directly from the selector 27 to the video sickness correction unit 6.

  The display panel control unit 30 generates a drive signal for displaying a display screen on the display panel 31. The display panel control unit 30 generates a synchronization signal that matches the drive timing of the display panel 31, and outputs a frame image in accordance with the synchronization signal. The display panel control unit 30 also performs signal voltage level conversion and auxiliary signal creation. The display panel 31 is a display device that performs display by an arbitrary display method such as liquid crystal, plasma, projection, CRT, organic EL, or SED.

  The overall control unit 32 includes a CPU, peripheral circuits, a memory, and the like, and performs overall operation control of the image processing apparatus 1 including control for motion sickness prevention correction. Although omitted in FIG. 2, the image processing apparatus 1 includes an audio processing unit for reproducing audio, an amplifier, a speaker, a power source, and the like.

<About video sickness prevention processing>
The motion sickness prevention correction process includes the following two processes.

(Processing 1) It is determined whether or not an object that may cause video sickness is included in the frame image sent from the selector 27. Processing to be candidates for prevention correction processing (Processing 2) When an object that is a candidate for video sickness prevention correction processing in (Processing 1) is included in the user's gaze area, video is displayed for this object. Process for Performing Sickness Prevention Correction Process First, (Process 1) will be described with reference to FIG. 7 showing a flowchart of the process. In step S <b> 101, the image recognition unit 22 recognizes the position of the user and the orientation of the face from the group of frame images obtained from the camera 21. Then, the viewing area calculation unit 23 calculates the user's field of view from the position recognized by the image recognition unit 22 and the orientation of the face. From the calculated field of view, the viewing area (viewing area) Area). The viewing area calculation unit 23 obtains the area (size) of the gaze area.

  In step S <b> 102, the dangerous area extraction unit 4 detects an object (image object) from the frame image input from the selector 27. In step S103, the dangerous area extraction unit 4 determines whether or not the ratio of the area of the detected object to the area of the gaze area is equal to or greater than a specified value. If the result of this determination is that it is greater than or equal to the specified value, the process proceeds to step S104, and if it is smaller than the specified value, the process returns to step S102 and the next object is detected.

  In step S104, the dangerous area extraction unit 4 obtains a variance value of pixel values of each pixel constituting the object detected in step S102 (an object having a ratio equal to or larger than a specified value with respect to the area of the gaze area). In step S105, the dangerous area extraction unit 4 determines whether or not the obtained variance value is equal to or greater than a specified value. That is, in step S104 and step S105, it is determined whether or not the texture undulation of the object is equal to or greater than a specified value.

  If the result of determination in step S105 is greater than or equal to the specified value, the process proceeds to step S106, and if it is smaller than the specified value, the process returns to step S102 to detect the next object.

  In step S106, the dangerous area extraction unit 4 obtains a global motion vector (GMV) of the object detected in step S102 (an object having an area with a ratio equal to or larger than a specified value with respect to the area of the gaze area). Since the method of obtaining the high-frequency motion vector is well known, the description thereof is omitted.

  In step S107, the dangerous area extraction unit 4 determines whether or not the cycle of GMV movement (object movement) between frame images is within a specified range and the amount (magnitude) of this movement is equal to or greater than a threshold value. . That is, it is determined whether or not the object detected in step S102 may cause video sickness.

  As a result of the determination in step S107, if the GMV motion cycle is within the specified range and the amount of this motion is greater than or equal to the threshold value, the process proceeds to step S108; otherwise, the process returns to step S102. Perform next object detection.

  In step S108, the object detected in step S102 (an object having an area with a ratio equal to or larger than a specified value with respect to the area of the gaze area) is set as a correction candidate. For example, if the object's movement cycle is 0.5 to 2 Hz and this object is shaking with a size of 3% or more with respect to the area of the gaze area, this object is determined to be an object that may cause motion sickness And correction candidates.

  That is, it is determined that an object having a larger proportion in the field of view is more likely to induce video sickness. In addition, it is determined that an object having a rough texture with few undulations is less likely to induce video sickness. If there is a stationary texture on the object, it is determined that the possibility of inducing video sickness is low. For example, in the case of an image in which a bird in a bird cage is moving, it is determined that the possibility of video sickness is low because the bird cage is stationary.

  If all the objects are detected from the frame image received from the selector 27, the present process is terminated (actually, the process proceeds to step S152 in FIG. 8). On the other hand, if an object that has not yet been detected remains, the process returns to step S102 via step S109 to detect the next object.

  Next, (Process 2) will be described with reference to FIG. 8 showing a flowchart of the process. As described above, the process according to the flowchart of FIG. 8 is performed on the frame image after the process according to the flowchart of FIG. 7 is performed on the frame image for one frame.

  In step S152, the area match determination unit 5 determines that the object that is the correction candidate in step S108 is within the gaze area obtained by the viewing area calculation unit 23 in the frame image received from the selector 27 via the dangerous area extraction unit 4. Determine whether it is included. As a result of this determination, if it is included, the process proceeds to step S153. If it is not included, the process proceeds to step S154.

  In step S153, the motion sickness correction unit 6 sets a correction candidate object included in the gaze area in the frame image from the OSD composition unit 29 as a correction target object, and performs video motion prevention correction on the correction target object. Process. In this process, first, the correction target object is cut out from the frame image. Then, the arrangement position of the cut out correction target object on each frame image so that the period of the movement of the correction target object between the frame images is equal to or less than the threshold value or the amount of the movement is smaller than the threshold value. To decide. Then, the extracted correction candidate object is arranged at the determined arrangement position. In this way, the possibility of causing video sickness is reduced by removing the fluctuation of the period of the specific area. Then, the video sickness correction unit 6 sends out the frame image in which the arrangement position of the correction target object is corrected to the display panel control unit 30 at the subsequent stage.

  When the process proceeds from step S152 to step S154, the video sickness correction unit 6 sends the frame image from the OSD synthesis unit 29 to the display panel control unit 30 as it is.

  In step S154, the display panel control unit 30 performs operation control for displaying the frame image received from the video sickness correction unit 6 on the display panel 31. Note that the process according to the flowchart of FIG. 7 and the process according to the flowchart of FIG. 8 are performed for each frame.

  As described above, according to the present embodiment, the motion sickness prevention correction process can be performed even when only a partial area of the image is viewed close to the display screen of the large screen. Can increase the sex.

<Modification>
In the embodiment described above, the process for specifying the correction candidate object is performed for each frame. However, in order to reduce the processing load, the process may be performed for each scene change. In addition, as the technique for detecting the gaze region, the image recognition technique is used in the above embodiment. However, other techniques may be used to detect the orientation and position of the user's face. For example, the user's line-of-sight direction may be detected as the direction of the user's face. Any known algorithm can be used as a method of detecting the line of sight. The user's position may be detected by a separately provided distance sensor.

  In the above-described embodiment, the process of slowing down the motion of the object is performed as the video sickness prevention correction. However, any video sickness prevention correction technique may be used. For example, you may perform the process which blunts the edge of an object.

[Second Embodiment]
A moving image captured by a video camera or the like as well as a television image may be applied to the moving image. Further, instead of the television image, a moving image file that manages the background image and the object independently, such as MPEG-4, may be used. In this case, one frame image is generated by arranging the object at the arrangement position determined in the arrangement position determination process (step S153) on the background image, and the generated frame image is displayed.

  The first and second embodiments are examples of the following processing, and embodiments other than the first and second embodiments may be considered as long as the following processing is included. In a technique for displaying a moving image including a frame image in which one or more objects are arranged, first, a user's gaze area on a display screen for displaying a moving image is specified. Then, among the objects appearing in the moving image, an object having an area where the ratio of the gaze area to the area of the gaze area is equal to or greater than the threshold and whose pixel value variance is equal to or greater than the threshold is specified as a candidate object. Then, among the identified candidate objects, candidate objects whose motion period between frame images is within a specified range and whose motion amount is equal to or greater than a threshold are identified as correction target objects.

  Then, a frame image in which the correction target object is located in the gaze area when displayed on the display screen is used as the update target frame image, and the correction target object is rearranged on the update target frame image to update the update target frame image. To do. Then, the updated frame image to be updated is displayed on the display screen (display control).

  In the rearrangement of correction target objects in this display control, the following processing is performed. That is, the correction target object on each update target frame image is set so that the period of movement of the correction target object between the respective update target frame images is equal to or smaller than the threshold value, or the amount of motion is smaller than the threshold value. Determine the placement position. Then, the correction target object is rearranged at the determined arrangement position.

  Note that some of the units shown in FIGS. 1 and 2 may be implemented by a computer program. In this case, this computer program is executed by a controller such as the overall control unit 32 to realize the function of the corresponding functional unit.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (4)

An image processing apparatus for displaying a moving image composed of frame images in which one or more objects are arranged,
Means for identifying a user's gaze area on a display screen for displaying the moving image;
Means for identifying, as candidate objects, objects that have an area in which the ratio of the gaze area to the area of the gaze area is equal to or greater than a threshold value among the objects that appear in the moving image, and whose pixel value variance value is equal to or greater than the threshold value;
Among the candidate objects, a specifying unit that specifies, as a correction target object, a candidate object in which a period of movement between frame images is within a specified range and the amount of movement is equal to or greater than a threshold value;
A frame image in which the correction target object is located in the gaze area when displayed on the display screen is set as an update target frame image, and the correction target object is rearranged on the update target frame image to thereby update the frame image to be updated. And display control means for displaying the updated frame image to be updated on the display screen,
The display control means includes
The correction target on the respective update target frame images so that the period of movement of the correction target object between the respective update target frame images is equal to or less than a threshold value, or the amount of the movement is smaller than the threshold value. An image processing apparatus characterized by determining an arrangement position of an object and rearranging the correction target object at the determined arrangement position.   The specifying unit obtains a global motion vector (Global Motion Vector) of the candidate object for each frame image, and the period of motion between the frame images is within a specified range from the high-frequency motion vector obtained for each frame image. The image processing apparatus according to claim 1, wherein the candidate object whose amount of movement is equal to or greater than a threshold is specified as a correction target object. An image processing method performed by an image processing apparatus that displays a moving image composed of frame images in which one or more objects are arranged,
A step of specifying a gaze area of the image processing device to identify a user's gaze area on a display screen for displaying the moving image;
The means for identifying the object of the image processing apparatus has an area in which the ratio of the gaze area to the area of the gaze area is greater than or equal to a threshold among the objects appearing in the moving image, and the variance of the pixel values is greater than or equal to the threshold Identifying the object as a candidate object,
A specifying step of specifying, as a correction target object, a candidate object in which the specifying means of the image processing apparatus has a period of movement between frame images within a specified range and the amount of movement is equal to or greater than a threshold value among the candidate objects;
When the display control means of the image processing apparatus displays the image on the display screen, the frame image in which the correction target object is located in the gaze area is set as an update target frame image, and the correction target object is displayed on the update target frame image. A display control step of updating the frame image to be updated by rearranging the image, and displaying the updated frame image to be updated on the display screen,
In the display control step,
The correction target on the respective update target frame images so that the period of movement of the correction target object between the respective update target frame images is equal to or less than a threshold value, or the amount of the movement is smaller than the threshold value. An image processing method, comprising: determining an arrangement position of an object; and rearranging the correction target object at the determined arrangement position.   A computer program for causing a computer to function as each unit included in the image processing apparatus according to claim 1.

Images