JP2011066577A - Image processing program, display system, image processor, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing program for efficiently transmitting image information, a display system, an image processor and an image processing method. <P>SOLUTION: A detecting means 1a detects a first kind of an area 11 and a second kind of an area 12, having variations larger than those of the first kind of an area 11 included in each image according to the change among images 10, 10a, 10b, ..., which are input in a time-series manner. A transmitting means 1b time-series transmits image information of the first kind of an area 11 and image information of the second kind of an area 12, by making the number of times of transmitting the image information of the first kind of an area 11 in predetermined time width smaller than the number of times of transmitting the image information of the second kind of an area 12 in the predetermined time width. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing program for transmitting image information to a display device, a display system, an image processing device, and an image processing method.

  Conventionally, a display system in which a computer and a display device (a monitor or a projector) are connected via a network has been used. In this display system, a computer transmits image data to a display device. The display device displays the image data received from the computer on a monitor screen or a screen.

  Here, as a network for connecting the computer and the display device, a wireless IP (Internet Protocol) network, a telephone line network, or the like can be considered. In such a transmission path, there is a limit to the amount of data that can be transmitted / received per unit time (hereinafter referred to as transmission capacity). For this reason, when the transmission capacity is compressed, there may be a delay before the image data generated on the computer side is reproduced on the display device. In this case, for example, a delay or a frame drop occurs in the image displayed by the display device.

  On the other hand, it is conceivable that the amount of data is reduced (compressed) by encoding image data generated on the computer side, and the compressed image data is sent to the display device. In this way, it is possible to reduce a delay that may occur due to communication even if the transmission capacity is small.

  However, when image data is compressed, there is a problem that the image quality deteriorates and it is difficult to see the image displayed on the display device. For this reason, when generating image data for moving images, an area input by an operator or an area displaying a person is acquired, and the compression ratio of this area is lower than that of other areas to improve image quality relatively. Thus, a method for making this region easy to see is known (see, for example, Patent Documents 1 and 2). In addition, a method is known in which the number of frames in a high image quality area is larger than that in a low image quality area to further improve the image quality in the high image quality area (see, for example, Patent Document 3).

JP 2003-309835 A JP 2008-005349 A Japanese Patent Laid-Open No. 09-130767

  By the way, an image can include a region that changes greatly during a predetermined time and a region that hardly changes. In this case, there is a problem that it is inefficient from the viewpoint of using the transmission capacity of the network to transmit the image data of the region that hardly changes at the same frequency as the region that changes greatly.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing program, a display system, an image processing apparatus, and an image processing method that efficiently transmit image information.

  In order to solve the above problems, an image processing program is provided. A computer that executes the image processing program functions as a detection unit and a transmission unit. The detection means includes a first type region included in each image and a second type region having a larger amount of change than the first type region according to a change between the plurality of images input in time series. And detect. The transmission means sets the number of transmissions of the image information of the first type area in the predetermined time width as a number less than the number of transmissions of the image information of the second type area in the predetermined time width. The image information of the area and the image information of the second type area are sequentially transmitted.

  Moreover, in order to solve the said subject, a display system is provided. This display system includes an image processing device and a display device. The image processing apparatus includes a detection unit and a transmission unit. The detection means includes a first type region included in each image and a second type region having a larger amount of change than the first type region according to a change between the plurality of images input in time series. And detect. The transmission means sets the number of transmissions of the image information of the first type area in the predetermined time width as a number less than the number of transmissions of the image information of the second type area in the predetermined time width. The image information of the area and the image information of the second type area are sequentially transmitted. The display device includes receiving means and display means. The receiving means sequentially receives the image information of the first type area and the image information of the second type area transmitted by the transmitting means. The display means, when the receiving means receives the image information of the first type area and the image information of the second type area, the image information of the first type area and the image information of the second type And when the receiving means receives the image information of the second type area and does not receive the image information of the first type area, the image information of the first type area previously received and the current reception The image information of the second type area is displayed.

  In order to solve the above problems, an image processing apparatus having the same function as a computer that executes the image processing program is provided. In order to solve the above problem, an image processing method is provided that performs the same processing as a computer that executes the image processing program.

  According to the image processing program, display system, image processing apparatus, and image processing method, image information can be transmitted efficiently.

It is a figure which shows the display system of 1st Embodiment. It is a figure which shows the whole structure of a display system. It is a figure which shows the hardware constitutions of a computer. It is a figure which shows the hardware constitutions of a display apparatus. It is a figure which shows the function structure of each apparatus of 2nd Embodiment. It is a figure which shows the specific example of a mesh structure. It is a figure which shows the example of a data structure of a data output control table. It is a figure which shows the example of a data structure of the mesh management table of 2nd Embodiment. 10 is a flowchart illustrating image data transmission processing according to the second embodiment. It is a flowchart which shows the area | region determination processing of 2nd Embodiment. It is a flowchart which shows the transmission area specific process of 2nd Embodiment. It is a flowchart which shows the encoding process of 2nd Embodiment. It is a flowchart which shows the reception process of the image data of 2nd Embodiment. It is a flowchart which shows the setting change process of the environment of 2nd Embodiment. It is a figure which shows the 1st specific example of the transmission method of the image data of 2nd Embodiment. It is a figure which shows the 2nd specific example of the transmission method of the image data of each area | region of 2nd Embodiment. It is a figure which shows the function structure of each apparatus of 3rd Embodiment. It is a figure which shows the example of a data structure of the mesh management table of 3rd Embodiment. It is a flowchart which shows the area | region determination processing of 3rd Embodiment. It is a flowchart which shows the encoding process of 3rd Embodiment. It is a flowchart which shows the encoding process of 3rd Embodiment. It is a figure which shows each area | region of the image of 3rd Embodiment. It is a figure which shows the function structure of each apparatus of 4th Embodiment. It is a figure which shows the example of a data structure of the mesh management table of 4th Embodiment. It is a flowchart which shows the transmission area specific process of 4th Embodiment. It is a figure which shows the specific example of the transmission method of the image data of 4th Embodiment. It is a figure which shows the function structure of each apparatus of 5th Embodiment. It is a figure which shows the example of a data structure of the continuation frame number threshold value setting table of 5th Embodiment. It is a figure which shows the example of a data structure of the mesh management table of 5th Embodiment. It is a flowchart which shows the transmission area specific process of 5th Embodiment. It is a figure which shows the specific example of the transmission method of the secondary high quality image of 5th Embodiment.

Hereinafter, embodiments will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a display system according to a first embodiment. This display system has a computer 1 and a display device 2. The computer 1 has detection means 1a and transmission means 1b.

  The detection means 1a detects the first type region 11 and the first type region 11 included in each image according to changes between the images 10, 10a, 10b,. The second type region 12 having a larger change amount is detected. The amount of change can be detected by, for example, the degree of change in the pixel value of each pixel. For example, when the difference between the pixel value of the target pixel of the image 10a and the pixel value of the corresponding pixel in the image 10 that is the immediately preceding image is greater than or equal to a predetermined value, the target pixel is included in the second type region 12. Can be considered. Note that the amount of change may be detected by comparing with a set of a plurality of pixels.

  The transmission means 1b sets the number of transmissions of the image information of the first type area 11 in the predetermined time width as the number of times smaller than the number of transmissions of the image information in the second type area 12 in the predetermined time width. The image information of the type 11 area 11 and the image information of the second type area 12 are sequentially transmitted to the display device 2.

  For example, the transmission unit 1b transmits the image information 12a, 12b, 12c, 12d, and 12e for the second type area 12 and the image for the first type area 11 during a predetermined time width. The information 11a and 11b are transmitted twice.

  The display device 2 displays the image information of the first type region 11 and the image information of the second type region 12 received from the transmission unit 1b. The display device 2 is, for example, a monitor or a projector.

  For example, when receiving the image information 11a of the first type region 11 and the image information 12a of the second type region, the display device 2 displays an image obtained by combining the image information 11a and 12a. Next, when receiving the image information 12b, the display device 2 updates the displayed image information 12a with the image information 12b. Thereafter, the same applies to the case where the display device 2 sequentially receives the image information 12c and 12d. When the display device 2 receives the image information 11b and 12e, the display device 2 updates the displayed image information 11a with the image information 11b, and updates the displayed image information 12d with the image information 12e.

  According to the computer 1, the first type region 11 included in each image and the first type region 11 included in each image according to the change between the plurality of images 10, 10 a, 10 b,. The second type region 12 having a larger change amount than the first type region 11 is detected. The transmission means 1b sets the number of transmissions of the image information of the first type region 11 in the predetermined time width to be smaller than the number of transmissions of the image information of the second type region 12 in the predetermined time width. The image information of the type 11 area 11 and the image information of the second type area 12 are sequentially transmitted to the display device 2.

  Thereby, it becomes possible to transmit image information efficiently. Specifically, the number of transmissions of the image information of the first type region 11 having a smaller change amount than that of the second type region 12 is smaller than that of the image information of the second type region 12. For this reason, it is possible to prevent the transmission capacity from being used unnecessarily in order to transmit the image information of the first type region 11. Further, the image quality of the image of the first type region 11 and the image of the second type region 12 can be improved by using the transmission capacity corresponding to the transmission capacity. In addition, the transmission capacity can be used for other data communications.

In the following second to fifth embodiments, application examples of the display system according to the first embodiment will be described.
[Second Embodiment]
FIG. 2 is a diagram illustrating the overall configuration of the display system. The display system according to the second embodiment includes a computer 100 and a display device 200. The computer 100 and the display device 200 are connected via the network 10. The network 10 is a wireless local area network (LAN) that connects the computer 100 and the display device 200 wirelessly.

  The computer 100 transmits image data to be displayed on the display device 200 to the display device 200 via the network 10. As the data format of the image data transmitted from the computer 100 to the display device 200, a lossy compression method such as Motion JPEG (Motion Joint Photographic Experts Groups) is used. By using the irreversible compression method, the amount of data can be easily reduced as compared to the lossless compression method, and the processing time required for the encoding / decoding process can be shortened. For this reason, the irreversible compression method is suitable when image data is transmitted via the network 10 and the same content is desired to be viewed simultaneously on a plurality of screens.

The display device 200 projects image data received from the computer 100 onto the screen 20. The display device 200 is a projector, for example.
The computer 100 has a function of transmitting image data to the display device 200. This function is realized by the computer 100 executing an image processing program. Such an image processing program can be recorded on a computer-readable recording medium 30 and distributed.

  Further, the image processing program may be stored from the recording medium 30 in a server device (not shown) connected to the network 10. In this case, the computer 100 can download the image processing program from the server device via the network 10.

  As the recording medium 30, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory can be used. Examples of the magnetic recording device include an HDD (Hard Disk Drive), a flexible disk (FD), and a magnetic tape. Examples of the optical disc include a CD (Compact Disc), a CD-R (Recordable) / RW (ReWritable), a DVD (Digital Versatile Disc), and a DVD-R / RW / RAM (Random Access Memory). Magneto-optical recording media include MO (Magneto-Optical disk).

  The display device 200 also has a function of receiving the image data transmitted by the computer 100 and projecting it on the screen 20. The display device 200 can download a program for this purpose from the server device via the network 10.

  The computer 100 and the display device 200 may be directly connected without using the network 10. In this case, it is conceivable that the computer 100 and the display device 200 are connected by a wireless USB (Wireless Universal Serial Bus).

  FIG. 3 is a diagram illustrating a hardware configuration of the computer. The computer 100 includes a central processing unit (CPU) 101, a random access memory (RAM) 102, an HDD 103, a display processing unit 104, a display 105, an input processing unit 106, a keyboard 107, a mouse 108, and a communication processing unit 109.

The CPU 101 controls the entire computer 100.
The RAM 102 temporarily stores at least part of an OS (Operating System) program and application software (hereinafter referred to as an application) program to be executed by the CPU 101. The RAM 102 stores various data necessary for processing by the CPU 101.

  The HDD 103 stores OS programs and application programs. The HDD 103 stores various data necessary for processing by the CPU 101. Instead of the HDD 103, for example, another nonvolatile storage device such as an SSD (Solid State Drive) can be used.

The display processing unit 104 is connected to the display 105. The display processing unit 104 displays an image on the screen of the display 105 in accordance with a command from the CPU 101.
The input processing unit 106 is connected to a keyboard 107 and a mouse 108. The input processing unit 106 transmits signals sent from the keyboard 107 and the mouse 108 to the CPU 101.

  The communication processing unit 109 is connected to the network 10 and transmits / receives image data to / from the display device 200. As the content of the image data, for example, an image displayed on the display 105 by the display processing unit 104 can be considered. In this way, for example, the user can make a presentation by operating the computer 100 and displaying the operation screen on the screen 20 via the display device 200.

  FIG. 4 is a diagram illustrating a hardware configuration of the display device. The display device 200 includes a CPU 201, a RAM 202, an NVRAM (Non Volatile RAM) 203, a display processing unit 204, a projection unit 205, an input processing unit 206, an operation unit 207, and a communication processing unit 208.

The CPU 201 controls the entire display device 200.
The RAM 202 temporarily stores at least part of an OS program and application programs to be executed by the CPU 201. The RAM 202 stores various data necessary for processing by the CPU 201.

The NVRAM 203 stores OS programs and application programs. The NVRAM 203 stores various data necessary for processing by the CPU 201.
The display processing unit 204 is connected to the projection unit 205. The display processing unit 204 causes the projection unit 205 to execute image projection on the screen in accordance with a command from the CPU 201.

The input processing unit 206 is connected to the operation unit 207. The input processing unit 206 transmits a signal sent from the operation unit 207 to the CPU 201.
The communication processing unit 208 is connected to the network 10 and transmits / receives data to / from the computer 100.

  FIG. 5 is a diagram illustrating a functional configuration of each device according to the second embodiment. The computer 100 includes a control information storage unit 110, a mesh storage unit 120, a mesh management information storage unit 130, a mesh generation unit 140, a change detection unit 145, an area determination unit 150, a transmission area specifying unit 160, an encoding unit 170, and a transmission unit. 180, a receiving unit 185, a communication band monitoring unit 190, and a frame rate setting receiving unit 195. These functions are realized by the CPU 101 executing the image processing program. However, some or all of these functions may be realized by dedicated hardware.

  The control information storage unit 110 stores data output control information for specifying a compression rate (or image quality) when encoding image data. The data output control information includes information such as a transmission capacity of a transmission path between the computer 100 and the display device 200 and a frame rate (the number of drawing screens per unit time) when the display device 200 reproduces an image. .

  The mesh storage unit 120 is a buffer for storing a mesh obtained by dividing the image into square regions with a predetermined number of pixels. The mesh storage unit 120 temporarily stores a mesh generated for image data input in time series.

The mesh management information storage unit 130 stores mesh management information including the position of the mesh in the image, the classification of moving images and still images, and the like.
The mesh generation unit 140 sequentially receives image data. The mesh generation unit 140 generates a mesh by dividing the image data, and stores the mesh in the mesh storage unit 120.

  The change detection unit 145 compares the mesh at the same position in the image stored in the mesh storage unit 120 with the mesh immediately before the corresponding position, and detects the change of the corresponding mesh from the previous mesh. Here, the change detection unit 145 can detect a change in the mesh based on whether there is a difference in pixel values (such as parameters indicating color and luminance) of pixels included in both meshes. Specifically, the change detection unit 145 detects that there is a change when there is a pixel having a difference in pixel value, and detects that there is no change when there is no difference in pixel value in any pixel. In addition, for example, the change detection unit 145 detects that there is a change when there is a pixel having a difference greater than or equal to a predetermined value in the pixel value, and detects that there is no change when any pixel does not have a difference greater than or equal to the predetermined value. It is also possible to do.

  The region determination unit 150 identifies a region (a set of a plurality of meshes) detected by the change detection unit 145 as having a change as a moving image region. Further, the region determination unit 150 identifies the region detected by the change detection unit 145 as no change as a still image region. The area determination unit 150 generates mesh management information based on the moving image and still image classification of each mesh specified in this way, and stores the mesh management information in the mesh management information storage unit 130.

  The transmission area specifying unit 160 specifies an area (transmission area) to be transmitted to the display device 200 based on the mesh management information stored in the mesh management information storage unit 130. The transmission area specifying unit 160 specifies the entire moving image area as the transmission area.

  Further, the transmission area specifying unit 160 specifies a part of the still image area as the transmission area. That is, the transmission area specifying unit 160 further divides the still image area, and sequentially specifies the divided areas as transmission areas at a plurality of timings. In this way, it is possible to allocate a larger amount of transmission data per unit area of the still image region than the amount of transmission data per unit area of the moving image region. That is, it is possible to improve the image quality of the still image region over the image quality of the moving image region by controlling the compression rate accompanying the subsequent encoding.

  Furthermore, the transmission area specifying unit 160 sets the frequency of specifying the still image area as the transmission area smaller than the frequency of specifying the moving picture area as the transmission area. For example, after transmitting the image data of the image quality of the still image area, the transmission area specifying unit 160 does not specify the area as a transmission area until the area becomes a moving image area or a predetermined time elapses. That is, by thinning out the still image area with respect to the input rate of the image data, the transmission rate (or the number of transmissions) of the still image area can be made smaller than the transmission rate of the moving image area.

The transmission area specifying unit 160 outputs information on the moving image area or the still image area specified as the transmission area to the encoding unit 170.
The encoding unit 170 sequentially encodes the meshes of the transmission area specified by the transmission area specifying unit 160 to reduce the data amount (compression). The encoding unit 170 includes a compression rate determination unit 171, a moving image region encoding unit 172, and a still image region encoding unit 173.

  The compression rate determination unit 171 determines the encoded data amount of a mesh (moving image mesh) group included in the moving image region or a mesh (still image mesh) group included in the still image region. The greater the amount of data after encoding, the lower the compression rate and the smaller the deterioration of the image, and the higher the image quality. Also, the smaller the encoded data amount, the higher the compression rate and the greater the degradation of the image, so that the image quality becomes lower.

  The compression rate determination unit 171 sets the still image mesh compression rate lower than the compression rate of the moving image mesh, and can display a higher quality image than the moving image region for the still image region. Details of the compression rate determination processing by the compression rate determination unit 171 will be described later.

  The moving image region encoding unit 172 encodes the moving image mesh in the transmission region so as to be close to the encoded data amount of the moving image region determined by the compression rate determining unit 171. For example, when encoding in the Motion JPEG format, the moving image region encoding unit 172 performs encoding by sequentially performing DCT (Discrete Cosine Transform) conversion, quantization processing, and Huffman encoding processing on the moving image mesh. In this case, the moving image region encoding unit 172 performs encoding using, for example, tables (quantization table or Huffman code table) having a maximum data amount that does not exceed the encoded data amount of the moving image region. Alternatively, encoding is performed using tables that are closest to the encoded data amount.

The moving image region encoding unit 172 outputs the compressed moving image data generated by encoding to the transmission unit 180.
The still image region encoding unit 173 encodes the still image mesh in the transmission region so as to be close to the encoded data amount of the still image region determined by the compression rate determination unit 171. It should be noted that the motion JPEG format encoding process may be applied to the still image area in the same manner as the moving image area encoding unit 172.

The still image region encoding unit 173 outputs the still image compressed data generated by encoding to the transmission unit 180.
Note that the encoding unit 170 includes information on reproducing an image on the display device 200 in each compressed data. For example, when encoding in the Motion JPEG format, the encoding unit 170 is information indicating the quantization table used for quantization, the Huffman code table used for Huffman encoding, and the position (such as coordinates on the screen) of each mesh. Is included in each compressed data.

The transmission unit 180 transmits compressed data including moving image compressed data or still image compressed data acquired from the encoding unit 170 to the display device 200.
The receiving unit 185 receives setting information of a frame rate at which the display device 200 performs image reproduction from the display device 200. The receiving unit 185 reflects the content of the frame rate setting information in the data output control information stored in the control information storage unit 110.

  The communication band monitoring unit 190 acquires the transmission capacity of the transmission path between the computer 100 and the display device 200 from the transmission unit 180 and the reception unit 185. The communication band monitoring unit 190 reflects information on the transmission capacity in the current communication environment in the data output control information stored in the control information storage unit 110.

  The frame rate setting accepting unit 195 accepts a frame rate setting change when the display device 200 reproduces an image. The frame rate setting change is input, for example, by the user operating the keyboard 107 or the mouse 108 according to the setting change interface displayed on the display 105. The frame rate setting reception unit 195 notifies the display device 200 of the change in the received frame rate setting, and reflects the changed content in the data output control information stored in the control information storage unit 110.

The display device 200 includes a reception unit 210, a decoding unit 220, an output processing unit 230, a frame rate notification unit 240, and a transmission unit 250.
The receiving unit 210 receives the compressed data transmitted by the transmitting unit 180. The receiving unit 210 outputs the received compressed data to the decoding unit 220.

  The decoding unit 220 decodes moving image compressed data or still image compressed data included in the compressed data acquired from the receiving unit 210. The receiving unit 210 includes a moving image region decoding unit 221 and a still image region decoding unit 222.

  The moving image area decoding unit 221 decodes moving image compressed data. For example, when the moving image compressed data is in the Motion JPEG format, the moving image region decoding unit 221 sequentially performs Huffman decoding processing, inverse quantization processing, and inverse DCT conversion on the moving image compressed data, and obtains the decoded image data of the moving image region. obtain. At this time, the moving image region decoding unit 221 can acquire the Huffman code table and the quantization table used at the time of encoding from the moving image compressed data and use them for decoding. The moving image region decoding unit 221 outputs the decoded image data of the moving image region to the output processing unit 230.

  The still image area decoding unit 222 decodes still image compressed data. For example, when the still image compressed data is in the Motion JPEG format, decoded image data of the still image region is obtained by the same processing as the moving image region decoding unit 221. However, since the moving image region decoding unit 221 and the still image region decoding unit 222 have different image quality, the quantization table and the Huffman code table used for the decoding process are different. The still image area decoding unit 222 outputs the decoded image data of the still image area to the output processing unit 230.

  The output processing unit 230 outputs a decoded image obtained by synthesizing the decoded image data of the moving image area and the decoded image data of the still image area acquired from the decoding unit 220 to the projection unit 205 and causes the projection unit 205 to project the decoded image. For example, the output processing unit 230 appropriately synthesizes the decoded image data in the moving image compression area and the decoded image data in the still image area based on information indicating the position of each mesh included in the moving image compression data or the still image compression data. A decoded image can be obtained.

  The frame rate notification unit 240 acquires a frame rate when the output processing unit 230 outputs (reproduces) the decoded image. When the frame rate is changed, the frame rate notification unit 240 outputs information on the new frame rate to the transmission unit 250. The change of the frame rate is received by the output processing unit 230 when the user performs a predetermined operation on the operation unit 207, for example. It can also be accepted by receiving a frame rate setting change instruction from the computer 100.

The transmission unit 250 transmits the frame rate information acquired from the frame rate notification unit 240 to the computer 100.
FIG. 6 is a diagram illustrating a specific example of a mesh configuration. The image data 300 has a still image area 310 and a moving image area 320. The still image region 310 and the moving image region 320 are configured by a plurality of meshes 330 in which each region is divided into rectangular regions. The mesh 330 includes a plurality of pixels 331 and is, for example, a set of 16 × 16 pixels or 8 × 8 pixels.

The image data 300 has an x coordinate on the horizontal axis and ay coordinate on the vertical axis with the upper left vertex as the origin. The position of each mesh can be specified by (x, y) coordinates.
The change detection unit 145 detects whether each mesh included in the image data 300 has changed from the previous mesh. The region determination unit 150 sets a region that has not been changed by the change detection unit 145 as a still image region 310. The area determination unit 150 sets the area changed by the change detection unit 145 as the moving image area 320.

  FIG. 7 is a diagram illustrating an example of the data structure of the data output control table. The data output control tables 111, 112, and 113 are stored in advance in the control information storage unit 110 in accordance with transmission channel transmission capacity candidate values. The data output control table 111 is control information in a communication environment where the upper limit of the transmission capacity is 20 Mbps (Megabits Per Second). In addition, for example, it is conceivable to prepare control information in a communication environment in which the upper limit of the transmission capacity is 60 Mbps, assuming a wireless USB connection. In the following description, the data output control table 111 will be described, but the data output control tables 112, 113,.

  In the data output control table 111, an item indicating the frame rate, an item indicating the transmittable data amount, and an item indicating the current setting flag are set. Information arranged in the horizontal direction of each item is associated with each other to indicate information related to one transmission environment.

In the item indicating the frame rate, a value indicating the frame rate when the display device 200 reproduces an image is set in units of fps (Frame Per Second).
In the item indicating the amount of transmittable data, the amount of image data that is allowed to be transmitted is set in units of KB / f (Kilo Byte / Frame).

  In the item indicating the current setting flag, a flag indicating which of the transmission capacity and the frame rate of the currently used transmission line is set. In the item indicating the current setting flag, “1” is set when the setting is appropriate, and “0” is set when the setting is not appropriate.

  In the data output control table 111, for example, information that the frame rate is “30” (fps), the transmittable data amount is “28” (KB / f), and the current setting flag is “1” is set. This indicates that the transmission capacity of the current transmission path by the wireless LAN is 20 Mbps, and the frame rate when the display device 200 reproduces an image is set to “30” (fps). Further, it is indicated that the amount of data that can be used for transmission of image data in the transmission capacity is “28” (KB / f).

  The current setting flag is updated by the receiving unit 185 and the frame rate setting receiving unit 195 according to the change in the frame rate setting of the display device 200. In addition, the communication band monitoring unit 190 updates the transmission capacity according to a change in transmission capacity due to a change in the environment of the transmission path.

  The compression rate determination unit 171 refers to the data output control table 111, acquires the transmittable data amount, and uses it to calculate the compression rate. However, the compression rate determination unit 171 may calculate a value corresponding to the transmittable data amount according to the communication environment.

  FIG. 8 is a diagram illustrating a data structure example of the mesh management table according to the second embodiment. The mesh management table 131 is stored in the mesh management information storage unit 130. The mesh management table 131 includes an item indicating mesh coordinates, an item indicating a moving image area flag, and an item indicating a high-quality image transmission completed flag. Information arranged in the horizontal direction of each item is associated with each other to indicate information about one mesh.

In the item indicating mesh coordinates, the coordinates of each mesh are set.
In the item indicating the moving image area flag, a flag indicating whether or not the corresponding mesh is a moving image mesh is set. In the item indicating the moving image area flag, “1” is set when the corresponding mesh is a moving image mesh, and “0” is set when the mesh is not a moving image mesh, that is, a still image mesh.

  In the item indicating the high quality image transmission flag, a flag indicating whether or not an image having a smaller compression rate (high quality image quality) than the moving image mesh has been transmitted for the still image mesh is set. In the item indicating the high-quality image transmission flag, “1” is set when the high-quality image has been transmitted for the corresponding mesh, and “0” is set when the high-quality image has not been transmitted. In the item indicating the high-quality image transmission flag, “-” indicating that the high-quality image is not a transmission target is set when the corresponding mesh is a moving image mesh.

  In the mesh management table 131, for example, information that the mesh coordinates are (x, y) = “(1, 1)”, the moving image area flag is “0”, and the high-quality image transmitted flag is “1” is set. . This indicates that the mesh specified by the mesh coordinates (x, y) = “(1, 1)” is a still image mesh, and a high-quality image has already been transmitted.

  In the mesh management table 131, for example, information that the mesh coordinates are (x, y) = “(71, 45)”, the moving image area flag is “1”, and the high-quality image transmitted flag is “−”. Is set. This indicates that the mesh specified by the mesh coordinates (x, y) = “(71, 45)” is a moving image mesh and is not a transmission target of a high-quality image.

Next, processing procedures of the computer 100 and the display device 200 realized by the above configuration will be described.
First, image data transmission processing by the computer 100 will be described.

FIG. 9 is a flowchart illustrating image data transmission processing according to the second embodiment. In the following, the process shown in FIG. 9 will be described along the step numbers in the flowchart.
[Step S1] The mesh generation unit 140 receives image data for one frame.

[Step S <b> 2] The mesh generation unit 140 generates a mesh by dividing each pixel included in the received image data into rectangular regions, and stores the mesh in the mesh storage unit 120.
[Step S3] The change detection unit 145 has a change in the pixel value of each pixel stored in the mesh storage unit 120 as compared to each mesh of the same coordinates included in the image data input immediately before. To detect. The region determination unit 150 divides the image data into a still image region and a moving image region according to the detection result of the change detection unit 145.

  [Step S <b> 4] The transmission region specifying unit 160 specifies all meshes that are classified as moving image regions by the region determination unit 150 as transmission regions. When transmitting a mesh that has been classified as a still image region by the region determining unit 150, the transmission region specifying unit 160 specifies a part of the meshes obtained by further dividing the still image region as a transmission region.

  [Step S5] The encoding unit 170 encodes the mesh of the transmission area specified by the transmission area specifying unit 160. The encoding unit 170 encodes the moving image mesh and the still image mesh in the transmission area at the respective compression rates, and generates moving image compressed data and still image compressed data. At this time, the encoding unit 170 encodes the still image mesh at a compression rate lower than that of the moving image region.

[Step S <b> 6] The transmission unit 180 transmits the moving image compression data and still image compression data generated by the encoding unit 170 to the display device 200.
[Step S7] The mesh generation unit 140 determines whether image data of the next frame has been received. If accepted, the process proceeds to step S2. If not, complete the process.

  In this way, the mesh generation unit 140 generates a mesh from sequentially received image data. The change detection unit 145 detects a change in the pixel value of each mesh. The region determination unit 150 divides each mesh included in the image data into a still image region and a moving image region according to the change detection result by the change detection unit 145. The transmission area specifying unit 160 specifies a mesh as a transmission area from the still image area and the moving image area. The encoding unit 170 encodes the moving image region with the compression rate of the moving image region and the still image region with the compression rate of the still image region in the mesh set as the transmission region. The transmission unit 180 transmits the moving image compressed data and the still image compressed data generated by the encoding unit 170 to the display device 200.

Next, the processing of steps S3 to S5 will be described in more detail.
FIG. 10 is a flowchart illustrating region determination processing according to the second embodiment. In the following, the process illustrated in FIG. 10 will be described along the step numbers in the flowchart. Note that the process shown in FIG. 10 shows step S3 in FIG. 9 in detail.

  [Step S11] The change detection unit 145 extracts one mesh from the meshes stored in the mesh storage unit 120 and included in the latest input image data. For example, the mesh may be extracted sequentially so as to scan in the positive direction of the x coordinate starting from the origin of the mesh coordinate. For example, when the x coordinate reaches the maximum value, 1 is added to the y coordinate, and then the minimum value of the x coordinate is extracted in the positive direction.

  [Step S12] The change detection unit 145 has the same coordinates as the mesh extracted in step S11 among the meshes (hereinafter referred to as the immediately preceding mesh) included in the image data input immediately before the latest input image data. The mesh just before is extracted. The change detection unit 145 detects the presence / absence of a change in the corresponding mesh by comparing the pixel value of each pixel included in the mesh extracted in step S11 with the pixel value of each corresponding pixel included in the immediately preceding mesh. . If there is a pixel having a difference in pixel value, the process proceeds to step S13. If there is no pixel having a difference in pixel value, the process proceeds to step S14.

  [Step S13] The region determination unit 150 sets the corresponding mesh as a moving image region. The region determination unit 150 sets “1” to the moving image region flag of the mesh corresponding to the corresponding mesh coordinate in the mesh management table 131 stored in the mesh management information storage unit 130.

  [Step S14] The region determination unit 150 sets the corresponding mesh as a still image region. The region determination unit 150 sets “0” to the moving image region flag of the mesh corresponding to the corresponding mesh coordinate in the mesh management table 131 stored in the mesh management information storage unit 130. For the mesh newly changed from the moving image region flag “1” to “0”, the region determination unit 150 sets “0” in the high-quality image transmission completed flag. The area determination unit 150 does not change the setting of the high-quality image transmitted flag for the mesh that maintains the moving image area flag with the setting “0”.

  [Step S15] The change detection unit 145 determines whether there is a mesh that has not been subjected to change detection in the mesh included in the input image data. If there is a mesh for which change detection has not been performed, the process proceeds to step S11. If there is no mesh for which change detection is not performed, the process is completed.

  In this way, the change detection unit 145 detects a change from the immediately preceding mesh having the same coordinates for each mesh. A set of meshes detected as having changed by the area determination unit 150 and the change detection unit 145 is specified as a moving image area. In addition, the region determination unit 150 identifies a set of meshes detected as having no change by the change detection unit 145 as a still image region.

  In step S12, the change detection unit 145 refers to the image data of the immediately preceding frame in order to detect change. Therefore, there is no comparison target for the image data of the first frame received first. For this reason, the change detection unit 145 assumes that there is a change in all meshes for the first frame. Then, the region determination unit 150 executes the subsequent processing for all the meshes as the moving image region for the first frame.

  FIG. 11 is a flowchart illustrating a transmission area specifying process according to the second embodiment. In the following, the process illustrated in FIG. 11 will be described along the step numbers in the flowchart. In addition, the process shown in FIG. 11 shows step S4 of FIG. 9 in detail.

  [Step S21] The transmission region specifying unit 160 refers to the moving image region flag in the mesh management table 131 stored in the mesh management information storage unit 130, and determines whether or not the still image region is included in the image data to be transmitted this time. Determine whether. If a still image area is included, the process proceeds to step S22. If a still image area is not included, the process proceeds to step S25.

  [Step S22] The transmission region specifying unit 160 refers to the high-quality image transmission flag in the mesh management table 131 and determines whether there is a mesh that has not yet transmitted a high-quality image among the still image meshes. . If there is a mesh for which a high-quality image has not been transmitted, the process proceeds to step S23. If there is no mesh that has not yet received a high-quality image, the process proceeds to step S25.

  [Step S23] The transmission area specifying unit 160 specifies the number of divisions of the still image area. Here, the number of divisions of the still image area is determined by the number of transmissions of the still image area. For example, when a still image area is divided into three areas and transmitted at different timings, the number of divisions is three. The smaller the number of divisions, the lower the image quality of the still image area. The lower limit of the image quality is when the number of divisions is 1, and at this time, the image quality is the same as that of the moving image area. Also, the larger the number of divisions, the better the image quality of the still image area. The number of divisions is stored in advance in, for example, the mesh information storage unit 130 according to the usage environment. The transmission area specifying unit 160 calculates the number of transmission meshes of the still image area to be transmitted based on the specified number of divisions.

  [Step S24] The transmission area specifying unit 160 refers to the mesh management table 131, extracts meshes for the calculated number of transmission meshes among meshes that have not yet been transmitted with high-quality images, and selects the extracted still image mesh group. This is the transmission target area. The transmission area specifying unit 160 updates the high-quality image transmission completion flag in the mesh management table 131. That is, the high-quality image transmission flag “0” set in the still image mesh extracted as the transmission target this time is changed to “1”. The transmission area specifying unit 160 outputs the extracted still image mesh group to the encoding unit 170.

  [Step S25] The transmission area specifying unit 160 refers to the moving image area flag of the mesh management table 131 and determines whether or not the moving image area is included in the image data to be transmitted this time. If a moving image area is included, the process proceeds to step S26. If the moving image area is not included, the process is completed.

  [Step S26] The transmission region specifying unit 160 refers to the mesh management table 131, extracts a moving image mesh, and sets the extracted moving image mesh group as a transmission target region. The transmission area specifying unit 160 outputs the extracted moving image mesh group to the encoding unit 170.

  In this way, the transmission area specifying unit 160 specifies a part of the still image area as a transmission target area. The transmission area specifying unit 160 calculates the number of meshes to be selected as the transmission target area according to the number of divisions of the still image area. Further, the transmission area specifying unit 160 specifies the entire moving picture area as a transmission target area.

  The processing for the still image area shown in steps S21 to S24 and the processing for the moving picture area shown in steps S25 and S26 may be reversed. For example, the transmission area specifying unit 160 may perform the processes (steps S21 to S24) for the still image area after executing the processes (steps S25 and S26) for the moving image area first.

  FIG. 12 is a flowchart illustrating the encoding process according to the second embodiment. In the following, the process illustrated in FIG. 12 will be described along the step numbers in the flowchart. The process shown in FIG. 12 shows step S5 in FIG. 9 in detail.

[Step S31] The encoding unit 170 acquires a mesh group (a moving image mesh group and a still image mesh group) to be transmitted that is specified by the transmission region specifying unit 160.
[Step S32] The compression rate determination unit 171 refers to the current setting flags of the data output control tables 111, 112, 113,... Stored in the control information storage unit 110, and determines the current transmittable data amount D. get. For example, in the environment of this display system, the entire band is set to 20 Mbps and the frame rate is set to 30 fps. Therefore, the compression rate determination unit 171 acquires D = 28 (KB / f).

[Step S33] The compression rate determination unit 171 refers to the mesh management table 131 stored in the mesh management information storage unit 130, and acquires the number of meshes X of the still image region.
[Step S34] The compression rate determination unit 171 refers to the mesh management table 131 and obtains the number of meshes Y in the moving image area.

[Step S35] The compression rate determination unit 171 calculates the moving image mesh rate Z = Y / (X + Y).
[Step S36] The compression rate determination unit 171 determines whether the still image mesh group and the moving image mesh group are mixed in the mesh group acquired in Step S31. If so, the process proceeds to step S37. If not, the process proceeds to step S39.

[Step S37] The compression rate determination unit 171 calculates a data amount D1 = D × Z that can be allocated to the moving image mesh group.
[Step S38] The compression rate determination unit 171 calculates a data amount D2 = D−D1 that can be allocated to the still image mesh group. Then, the process proceeds to step S42.

  [Step S39] The compression rate determination unit 171 determines whether the mesh group acquired in step S31 is only a moving image mesh group. If there is only a moving image mesh group, the process proceeds to step S40. If it is not only the moving image mesh group, that is, if it is only the still image mesh group, the process proceeds to step S41.

[Step S40] The compression rate determination unit 171 acquires the data amount D1 = D that can be allocated to the moving image area. Then, the process proceeds to step S42.
[Step S41] The compression rate determination unit 171 acquires a data amount D2 = D that can be allocated to a still image area. Then, the process proceeds to step S42.

  [Step S42] The moving image region encoding unit 172 encodes the moving image mesh group obtained in step S31 so as to approach the moving image transmittable data amount D1 obtained by the compression rate determination unit 171 to obtain moving image compressed data. . Further, the still image region encoding unit 173 encodes the still image mesh group acquired in step S31 so as to approach the still image transmittable data amount D2 obtained by the compression rate determination unit 171 and generates still image compressed data. Get. Note that the encoding unit 170 applies the tables used for encoding (in the case of Motion JPEG, the quantization table and the Huffman encoding table) and the position of each mesh (on the screen) to the compressed video data and the still image compressed data. Include information indicating the coordinates).

[Step S43] The encoding unit 170 outputs the generated moving image compression data and still image compression data to the transmission unit 180.
In this way, the compression rate determination unit 171 calculates the transmission data amount allocated to the still image region and the moving image region based on the ratio of the number of still image meshes to the number of moving image meshes in the image data. Here, the ratio between the number of still image meshes and the number of moving image meshes in the image data corresponds to the ratio of the area between the still image region and the moving image region. Then, the moving image region encoding unit 172 and the still image region encoding unit 173 encode the moving image mesh group and the still image mesh group so as to be close to each data amount allocated by the compression rate determination unit 171.

  Specifically, the compression rate determination unit 171 acquires, for example, the number of still image meshes X = 300. Also, for example, the moving image mesh count Y = 100 is acquired. At this time, the moving image mesh ratio Z = 100 / (300 + 100) = 0.25. Therefore, for the transmittable data D = 28 KB / f, the transmittable video data amount D1 = 28 × 0.25 = 7 KB / f. Further, the amount of transmittable data for still image D2 = D−D1 = 21 KB / f.

  For this reason, the moving image region encoding unit 172 performs encoding so that the encoded data amount of the moving image mesh group approaches 7 KB. The still image area encoding unit 173 performs encoding so that the encoded data amount of the still image mesh group approaches 21 KB. For example, in the case of the Motion JPEG format, the moving image region encoding unit 172 uses tables (quantization table and Huffman code table) in which the amount of data after encoding of the moving image mesh group does not exceed 7 KB. To encode. Alternatively, the moving image region encoding unit 172 performs encoding using the tables whose data amount after encoding of the moving image mesh group is closest to 7 KB. Similarly, the still image region encoding unit 173 encodes the still image mesh group.

  Note that the compression rate determination unit 171 multiplies (or adds or subtracts) a predetermined coefficient corresponding to an area to which attention is focused on the moving image mesh rate Z to obtain the moving image transmittable data amount D1 and the still image transmittable data amount D2. It is also possible to change the ratio. For example, in the above specific example, when it is desired to improve the image quality of the moving image area, the moving image mesh rate Z is multiplied by the coefficient 2 to be Z = 0.5, the moving image transmittable data amount D1 = 14 KB / f, It is also conceivable that the transmittable data amount D2 = 14 KB / f. Further, when it is desired to improve the image quality of the still image area, the moving image mesh rate Z is multiplied by a coefficient 0.5 to obtain Z = 0.25, and D1 = 3.5 KB / f and D2 = 24.5 KB / f. It is also possible to do. It is also conceivable to dynamically change the coefficient by which the moving image mesh rate Z is multiplied according to the availability of the communication band.

Next, the reception processing of moving image compressed data and still image compressed data in the display device 200 will be described.
FIG. 13 is a flowchart illustrating image data reception processing according to the second embodiment. In the following, the process illustrated in FIG. 13 will be described along the step numbers in the flowchart.

[Step S51] The receiving unit 210 receives compressed data from the computer 100. The receiving unit 210 outputs the received compressed data to the decoding unit 220.
[Step S <b> 52] The decoding unit 220 decodes the moving image compressed data or the still image compressed data included in the compressed data acquired from the receiving unit 210. The moving image region decoding unit 221 decodes the moving image compression data using a quantization table or a code table included in the moving image compression data, and obtains decoded image data of the moving image region. The still image region decoding unit 222 decodes the still image compressed data using a quantization table or a code table included in the still image compressed data, and obtains decoded image data of the still image region. The decoding unit 220 outputs the decoded image data of the moving image region or the still image region to the output processing unit 230.

  [Step S53] The output processing unit 230 synthesizes a mesh included in the decoded image data of the moving image region or the still image region acquired from the decoding unit 220. The output processing unit 230 appropriately synthesizes the decoded image data in the moving image compression area and the decoded image data in the still image area based on information indicating the position of each mesh included in the moving image compression data or the still image compression data. Can do. Note that the output processing unit 230 synthesizes still image meshes that have not been updated this time by using still image mesh images acquired up to the previous time.

[Step S <b> 54] The output processing unit 230 outputs the decoded image obtained by the synthesis in step S <b> 53 to the projection unit 205 and projects it on the screen 20.
In this way, the decoding unit 220 decodes the moving image compressed data and still image compressed data received from the computer 100. The output processing unit 230 synthesizes the decoded image data of the moving image area and the decoded image data of the still image area obtained by decoding, and causes the projection unit 205 to project them onto the screen 20.

Next, a processing procedure for changing the setting of the communication environment in this display system will be described.
FIG. 14 is a flowchart illustrating environment setting change processing according to the second embodiment. In the following, the process illustrated in FIG. 14 will be described along the step numbers in the flowchart.

  [Step S61] The communication band monitoring unit 190 monitors the communication status of the transmission unit 180 and the reception unit 185, and determines whether there is a change in the communication band. If the communication band has changed, the process proceeds to step S62. If the communication band has not changed, the process proceeds to step S63.

  [Step S62] The communication band monitoring unit 190 changes the data output control table stored in the control information storage unit 110. At this time, the current setting flag corresponding to the current setting is changed from “1” to “0”. Then, the current setting flag corresponding to the corresponding frame rate in the new data output control table after the change is changed from “0” to “1”.

  [Step S63] The frame rate setting accepting unit 195 determines whether or not the user has accepted a frame rate setting change. If a frame rate setting change is accepted, the process proceeds to step S64. If no frame rate setting change has been received, the process proceeds to step S65.

  [Step S64] The frame rate setting reception unit 195 changes the current setting flag corresponding to the current setting of the data output control table stored in the control information storage unit 110 from “1” to “0”. Then, the frame rate setting receiving unit 195 changes the current setting flag corresponding to the new frame rate after the setting change from “0” to “1”.

  [Step S65] The receiving unit 185 determines whether or not new frame rate information has been received from the display device 200. If new frame rate information is received, the process proceeds to step S66. If no new frame rate information has been received, the process is completed.

  [Step S66] The receiving unit 185 changes the current setting flag corresponding to the current setting of the data output control table 111 stored in the control information storage unit 110 from “1” to “0”. Then, the reception unit 185 changes the current setting flag corresponding to the new frame rate after the setting change from “0” to “1”.

  In this way, the reception unit 185 and the frame rate setting reception unit 195 reflect the settings according to the current communication environment in the data output control tables 111, 112, 113,... Stored in the control information storage unit 110. can do.

Thereby, the encoding unit 170 can perform encoding at an optimal compression rate for the current communication environment.
Next, a method for transmitting image data by the computer 100 will be described more specifically. In the following description, it is assumed that 3 is set as the number of divisions of the still image area.

  FIG. 15 is a diagram illustrating a first specific example of the image data transmission method according to the second embodiment. In the following, the process illustrated in FIG. 15 will be described in order of step number. Note that step ST1 is the processing timing of the first frame that the computer 100 transmits to the display device 200. Step ST2 is the processing timing of the next frame after step ST1. Step ST3 is the processing timing of the next frame after step ST2.

  [Step ST1] The area determination unit 150 sets the entire image 300 as a moving image area. The transmission area specifying unit 160 specifies the entire image data 300 as a transmission area. The encoding unit 170 encodes the entire image data 300 to generate moving image compressed data. The transmission unit 180 transmits the compressed moving image data generated by the encoding unit 170 to the display device 200. In this case, the display device 200 decodes the compressed moving image data and displays a decoded image for the entire image data 300.

  [Step ST2] The region determination unit 150 determines the still image region 311 and the moving image region 321 from the change between each mesh of the image data 300a and each mesh corresponding to the image data 300. The transmission region specifying unit 160 specifies a partial region 311a including the number of meshes obtained by dividing the number of meshes of the still image region 311 into three of the still image regions 311 as a transmission target region. The transmission area specifying unit 160 specifies the moving picture area 321 as a transmission target area. The encoding unit 170 encodes the mesh of the partial region 311a and generates still image compressed data. Also, the encoding unit 170 encodes the mesh of the moving image area 321 to generate moving image compressed data. The transmission unit 180 transmits the still image compressed data and the moving image compressed data generated by the encoding unit 170 to the display device 200.

  In this case, the display device 200 decodes the still image compressed data to obtain decoded image data of the partial area 311a. In addition, the display device 200 decodes the moving image compressed data to obtain decoded image data of the moving image area 321. The display device 200 combines these decoded image data and displays a decoded image for the image data 300a. However, in the partial area 311b which is an area other than the partial area 311a in the still image area 311, the decoded image acquired in step ST1 is continuously displayed. In this case, the image quality of the partial area 311a is higher than that of the partial area 311b and the moving image area 321.

  [Step ST3] The region determination unit 150 determines the still image region 312 and the moving image region 322 from changes between the meshes of the image data 300b and the corresponding meshes of the image data 300a. The transmission area specifying unit 160 specifies a partial area 312 a of the still image area 312 as a transmission target area. In addition, the transmission area specifying unit 160 specifies the moving picture area 322 as a transmission target area. The encoding unit 170 encodes the mesh of the partial region 312a and generates still image compressed data. The encoding unit 170 also encodes the mesh of the moving image area 322 to generate moving image compressed data. The transmission unit 180 transmits the still image compressed data and the moving image compressed data generated by the encoding unit 170 to the display device 200.

  In this case, the display device 200 decodes the still image compressed data to obtain decoded image data of the partial area 312a. In addition, the display device 200 decodes the moving image compressed data to obtain decoded image data of the moving image area 322. The display device 200 combines these decoded image data and displays a decoded image for the image data 300b. However, in the still image region 312, the partial image 312b that has been transmitted in step ST2 continuously displays the decoded image acquired in step ST2. In the still image region 312, the partial image 312c, which is a region other than the partial regions 312a and 312b, continuously displays the decoded image acquired in step ST1. In this case, the image quality of the partial areas 312a and 312b is higher than that of the partial area 312c and the moving image area 322.

As described above, the computer 100 can improve the image quality of the still image region portion by dividing the still image region into a plurality of regions and transmitting the divided still image regions at different timings.
Next, the data of each area encoded and transmitted by the computer 100 at each timing will be described in more detail.

  FIG. 16 is a diagram illustrating a second specific example of the image data transmission method for each region according to the second embodiment. Hereinafter, the process illustrated in FIG. 16 will be described in order of step number. Note that steps ST1 to ST3 in FIG. 16 correspond to steps ST1 to ST3 in FIG. In addition, a region divided into three of the original image is denoted as regions R1, R2, and R3. The regions R1, R2, and R3 are regions that are specified as still image regions after the first frame. In addition, a region that is continuously specified as a moving image region after the first frame in the original image is referred to as a moving image region M.

  [Step ST1] The computer 100 encodes the meshes of the regions R1, R2, and R3 and the moving image region M in the image data to generate moving image compressed data R11, R21, R31, and M11, and transmits them to the display device 200. Note that the moving image compressed data R11, R21, R31, and M11 do not have to be particularly divided.

  [Step ST2] The computer 100 encodes the mesh of the region R1 in the image data to generate still image compression data R12, and transmits the still image compression data R12 to the display device 200. In addition, the computer 100 encodes the mesh of the moving image area M to generate the moving image compressed data M12 and transmits it to the display device 200. Here, the computer 100 can use the amount used for the areas R2 and R3 in step ST1 as the amount of transmission data that can be allocated to the still image area R1, so that the area R1 is compressed at a lower compression rate than the moving image area M. Can be encoded.

  [Step ST3] The computer 100 encodes the mesh of the region R2 in the image data to generate still image compression data R22, and transmits the still image compression data R22 to the display device 200. In addition, the computer 100 encodes the mesh of the moving image area M to generate moving image compressed data M <b> 13 and transmits it to the display device 200. The computer 100 can encode the region R2 at a compression rate lower than that of the moving image region M as in step ST2.

  [Step ST <b> 4] The computer 100 encodes the mesh in the region R <b> 3 in the image data to generate still image compression data R <b> 32 and transmits the still image compression data R <b> 32 to the display device 200. Further, the computer 100 encodes the mesh of the moving image area M to generate moving image compressed data M <b> 14 and transmits it to the display device 200. The computer 100 can encode the region R3 at a compression rate lower than that of the moving image region M as in step ST2.

  [Step ST5] The computer 100 encodes the mesh of the moving image area M in the image data to generate moving image compressed data M15, and transmits it to the display device 200. Since high-quality images have already been transmitted for the regions R1, R2, and R3, the computer 100 does not transmit image data for the still image region.

  [Step ST6] The computer 100 encodes the mesh of the moving image area M in the image data to generate moving image compressed data M16, and transmits it to the display device 200. Since high-quality images have already been transmitted for the regions R1, R2, and R3, the computer 100 does not transmit image data for the still image region.

  [Step ST7] The computer 100 encodes the mesh of the moving image area M in the image data to generate moving image compressed data M17, and transmits it to the display device 200. Since high-quality images have already been transmitted for the regions R1, R2, and R3, the computer 100 does not transmit image data for the still image region. Thereafter, only image data transmission of the moving image area M is performed in the same manner.

  As described above, the computer 100 divides the image data into the still image region and the moving image region based on the temporal change of the mesh included in the image data. Then, the computer 100 makes the number of transmissions of the still image mesh smaller than the number of transmissions of the moving image mesh.

This makes it possible to transmit image data by efficiently using the transmission capacity of the network.
After step ST7, the meshes in the regions R1, R2, and R3 are not updated until a change is detected. However, the images in the regions R1, R2, and R3 may be appropriately updated at a predetermined timing after step ST7.

  Further, by dividing the still image region into a plurality of regions and transmitting them at different timings, it is possible to allocate a larger amount of transmission data per unit area than that of the moving image region.

  Thereby, the image quality of the still image area can be made higher than the image quality of the moving image area. As a result, the visibility of the image displayed on the screen 20 by the display device 200 can be improved.

[Third Embodiment]
Next, a third embodiment will be described. Differences from the second embodiment will be mainly described, and description of similar matters will be omitted.

  Here, in the display system according to the second embodiment, since the mesh located at the boundary between the still image region and the moving image region is determined as the moving image region, the image quality of the mesh at the boundary is lower than that of the still image region. To do. That is, the image quality of the still image included in the boundary portion is reduced. For this reason, the boundary between the still image region and the moving image region is conspicuous, making it difficult to see the image. In addition, for example, when it is desired to browse an interface (for example, an interface for moving image operation) arranged in the vicinity of a moving image, it becomes difficult to see the image if a point of interest is adjacent to the moving image area.

  On the other hand, in the third embodiment, a third mesh (boundary mesh) located at the boundary between the still image region and the moving image region described in the second embodiment is introduced. Thereby, the visibility of the image displayed on the display device 200 is further improved.

  The overall configuration of the display system according to the third embodiment is the same as the overall configuration of the display system according to the second embodiment described with reference to FIG. The hardware configurations of the computer and the display device included in the display system of the third embodiment are the same as the hardware configurations of the computer 100 and the display device 200 of the second embodiment shown in FIGS. Therefore, the description is omitted.

  FIG. 17 is a diagram illustrating a functional configuration of each device according to the third embodiment. The computer 100a includes a control information storage unit 110, a mesh storage unit 120, a mesh management information storage unit 130a, a mesh generation unit 140, a change detection unit 145, a region determination unit 150a, a transmission region specification unit 160, an encoding unit 170a, and a transmission unit. 180, a receiving unit 185, a communication band monitoring unit 190, and a frame rate setting receiving unit 195. These functions are realized by the CPU 101 executing the image processing program. However, some or all of these functions may be realized by dedicated hardware.

  Note that the control information storage unit 110, the mesh storage unit 120, the mesh generation unit 140, the change detection unit 145, the transmission region specification unit 160, the transmission unit 180, the reception unit 185, the communication band monitoring unit 190, and the frame rate setting reception unit 195. Is the same as the configuration described with the same reference numerals in FIG. Further, the configuration of the display device 200 is the same as the configuration described with reference to FIG.

  The mesh management information storage unit 130a stores mesh management information including the position of the mesh in the image, the moving image, the still image, and the boundary area. Note that the boundary region refers to a set of meshes divided into moving images out of two meshes adjacent to each other between the moving image mesh and the still image mesh.

  The region determination unit 150a classifies each mesh included in the image data into a moving image region, a still image region, and a boundary region based on the change detection result of the change detection unit 145. Specifically, the area detected by the change detection unit 145 as having changed is specified as the moving image area. Further, the area detected by the change detection unit 145 as no change is identified as a still image area. Further, the region determination unit 150a sets a mesh that is separated from the moving image region as a boundary mesh among meshes located at the boundary where the moving image region and the still image region are in contact with each other. A set of boundary meshes is a boundary region. The region determination unit 150a generates mesh management information based on the moving image, the still image, and the boundary mesh of each mesh specified in this way, and stores the mesh management information in the mesh management information storage unit 130a.

  The encoding unit 170a sequentially encodes the meshes of the transmission area specified by the transmission area specifying unit 160 to reduce the data amount (compression). The encoding unit 170a includes a compression rate determination unit 171a, a moving image region encoding unit 172a, and a still image region encoding unit 173. Note that the still image region encoding unit 173 is the same as the still image region encoding unit 173 described with reference to FIG.

  The compression rate determination unit 171a determines the encoded data amount of the moving image mesh group, the still image mesh group, and the boundary mesh group. The compression rate determination unit 171a sets the compression rate of the still image mesh lower than the compression rate of the moving image mesh. Also, the compression rate determination unit 171a determines that the compression rate of the boundary mesh is lower than the compression rate for the moving image mesh and higher than the compression rate for the still image mesh.

  The moving image region encoding unit 172a encodes the moving image mesh in the transmission region so as to be close to the encoded data amount of the moving image mesh group determined by the compression rate determination unit 171a. The moving image region encoding unit 172a encodes the boundary mesh in the transmission region so as to be close to the encoded data amount of the boundary mesh group determined by the compression rate determining unit 171a.

The moving image region encoding unit 172a outputs the moving image compressed data and the boundary compressed data generated by encoding to the transmitting unit 180.
Note that the encoding unit 170a includes information on reproducing an image on the display device 200 in each compressed data. Such information includes, for example, encoding in the Motion JPEG format, information indicating the quantization table used for quantization, the Huffman code table used for Huffman encoding, and the position of each mesh (such as coordinates on the screen). It is.

  FIG. 18 is a diagram illustrating a data structure example of the mesh management table according to the third embodiment. The mesh management table 131a is stored in the mesh management information storage unit 130a. The mesh management table 131a includes an item indicating mesh coordinates, an item indicating a moving image area flag, an item indicating a boundary flag, and an item indicating a high-quality image transmitted flag. Information arranged in the horizontal direction of each item is associated with each other to indicate information about one mesh.

  Note that the item indicating the mesh coordinates, the item indicating the moving image region flag, and the item indicating the high-quality image transmitted flag are the same as the items having the same name in the mesh management table 131 described in FIG.

  In the item indicating the boundary flag, a flag indicating whether or not the corresponding mesh is a boundary mesh is set. In the item indicating the boundary flag, “1” is set when the corresponding mesh is a boundary mesh, and “0” is set when the mesh is not a boundary mesh. Since the boundary mesh is specified from the mesh belonging to the moving image area, “−” indicating that the boundary mesh is excluded is set for the mesh belonging to the still image area. That is, when the moving image area flag is “1” and the boundary flag is “0”, the mesh is a moving image mesh. When the moving image area flag is “1” and the boundary flag is “1”, the mesh is a boundary mesh.

  In the mesh management table 131a, for example, the mesh coordinates are (x, y) = “(72, 45)”, the moving image area flag is “1”, the boundary flag is “1”, and the high-quality image transmitted flag is set. Information “-” is set. This indicates that the mesh specified by mesh coordinates (x, y) = “(72, 45)” is a boundary mesh and is not a transmission target of a high-quality image.

  In the mesh management table 131a, for example, the mesh coordinates are (x, y) = “(73, 45)”, the moving image area flag is “1”, the boundary flag is “0”, and a high-quality image has been transmitted. Information that the flag is "-" is set. This indicates that the mesh specified by the mesh coordinates (x, y) = “(73, 45)” is a moving image mesh and is not a transmission target of a high-quality image.

Next, processing procedures of the computer 100a and the display device 200 realized by the above configuration will be described.
Note that the procedure of the image data transmission process of the computer 100a is the same as the procedure described with reference to FIG. Hereinafter, each process included in the image data transmission process of the computer 100a will be described in detail.

  FIG. 19 is a flowchart illustrating region determination processing according to the third embodiment. In the following, the process shown in FIG. 19 will be described along the step numbers in the flowchart. The following process corresponds to the area determination process in step S3 of FIG.

[Step S71] The change detection unit 145 extracts one mesh from the meshes stored in the mesh storage unit 120 and included in the latest input image data.
[Step S72] The change detection unit 145 extracts a previous mesh having the same coordinates as the mesh extracted in Step S71 from the previous mesh included in the image data input immediately before the latest input image data. . The change detection unit 145 detects the presence / absence of a change in the corresponding mesh by comparing the pixel value of each pixel included in the mesh extracted in step S71 with the pixel value of each corresponding pixel included in the immediately preceding mesh. . If there is a pixel having a difference in density, the process proceeds to step S73. If there is no pixel having a difference in pixel value, the process proceeds to step S74.

  [Step S73] The area determination unit 150a sets the corresponding mesh as a moving image area. The region determination unit 150a sets “1” to the moving image region flag of the mesh corresponding to the corresponding mesh coordinate in the mesh management table 131a stored in the mesh management information storage unit 130a.

  [Step S74] The region determination unit 150a sets the corresponding mesh as a still image region. The region determination unit 150a sets “0” to the moving image region flag of the mesh corresponding to the corresponding mesh coordinate in the mesh management table 131a stored in the mesh management information storage unit 130a. For the mesh newly changed from the moving image region flag “1” to “0”, the region determination unit 150a sets “0” in the high-quality image transmission completed flag. The area determination unit 150a does not change the setting of the high-quality image transmission flag for the mesh that maintains the moving image area flag with the setting “0”.

  [Step S75] The change detection unit 145 determines whether there is a mesh that has not been subjected to change detection in the mesh included in the input image data. If there is a mesh for which change detection has not been performed, the process proceeds to step S71. If there is no mesh for which change detection has not been performed, the process proceeds to step S76.

  [Step S76] The area determination unit 150a extracts a mesh adjacent to the mesh set as the still image area in Step S74 from the meshes set as the moving image area in Step S73.

  [Step S77] The area determination unit 150a sets a mesh of a moving image area adjacent to a mesh of a still image area as a boundary mesh. That is, the area determination unit 150a sets “1” to the boundary flag of the mesh management table 131a stored in the mesh management information storage unit 130a for the corresponding mesh. Further, “0” is set in the boundary flag for a mesh in a moving image area other than the boundary mesh.

In this manner, the area determination unit 150a sets the moving image area mesh adjacent to the still image area mesh as the boundary mesh.
20 and 21 are flowcharts illustrating the encoding process according to the third embodiment. Hereinafter, the processing illustrated in FIGS. 20 and 21 will be described in accordance with step numbers in the flowcharts. The following process corresponds to the encoding process in step S5 of FIG.

[Step S81] The encoding unit 170a acquires the transmission target mesh group (the still image mesh group, the still image mesh group, and the boundary mesh group) specified by the transmission region specifying unit 160.
[Step S82] The compression rate determination unit 171a refers to the current setting flags of the data output control tables 111, 112, 113,... Stored in the control information storage unit 110, and determines the current transmittable data amount D. get. For example, in the environment of this display system, the entire band is set to 20 Mbps and the frame rate is set to 30 fps.

  [Step S83] The compression rate determination unit 171a refers to the mesh management table 131a stored in the mesh management information storage unit 130a, and acquires the number of meshes X of the still image region.

  [Step S84] The compression rate determination unit 171a refers to the mesh management table 131a to obtain the number Y of meshes in the moving image area. Here, the number Y of moving image meshes includes the number of moving image meshes and the number of boundary meshes.

[Step S85] The compression rate determination unit 171a obtains the number K of boundary meshes among the meshes in the moving image area.
[Step S86] The compression rate determination unit 171a specifies the quality improvement coefficient a of the boundary mesh. Here, the quality improvement coefficient g (> 1) is a coefficient indicating how much transmission data amount is allocated as compared with the moving image mesh. The quality improvement coefficient g is preset in the control information storage unit 110, for example.

[Step S87] The compression rate determination unit 171a calculates the moving image mesh rate Za = (Y−K) / (X + Y).
[Step S88] The compression rate determination unit 171a calculates a boundary mesh rate Zb = gK / (X + Y).

  [Step S89] The compression rate determination unit 171a determines whether the still image mesh group and the moving image mesh group are mixed in the mesh group acquired in Step S81. If so, the process proceeds to step S90. If not, the process proceeds to step S93.

[Step S90] The compression rate determination unit 171a calculates a data amount D1a = D × Za that can be allocated to the moving image mesh group.
[Step S91] The compression rate determination unit 171a calculates a data amount D1b = D × Zb that can be allocated to the boundary mesh group.

  [Step S92] The compression rate determination unit 171a calculates a data amount D2 = D−D1a−D1b that can be allocated to the still image mesh group. Then, the process proceeds to step S96.

  [Step S93] The compression rate determination unit 171a determines whether or not only the moving image mesh group is included in the mesh group acquired in Step S81. If there is only a moving image mesh group, the process proceeds to step S94. If it is not only the moving image mesh group, that is, if it is only the still image mesh group, the process proceeds to step S95.

[Step S94] The compression rate determination unit 171a obtains a data amount D1 = D that can be allocated to the moving image mesh group. Then, the process proceeds to step S96.
[Step S95] The compression rate determination unit 171a obtains the data amount D2 = D that can be allocated to the still image mesh group. Then, the process proceeds to step S96.

  [Step S96] The moving image region encoding unit 172a encodes the moving image mesh group acquired in step S81 so as to approach the moving image transmittable data amount D1a obtained by the compression rate determination unit 171a to obtain moving image compressed data. In addition, the moving image region encoding unit 172a encodes the boundary mesh group acquired in step S31 so as to approach the boundary transmittable data amount D1b obtained by the compression rate determination unit 171a to obtain boundary compressed data. The still image region encoding unit 173 encodes the still image mesh group acquired in step S81 so as to approach the still image transmittable data amount D2 obtained by the compression rate determination unit 171a, and compresses the still image compressed data. Get. It should be noted that the encoding unit 170a uses the table used for encoding the moving image compressed data, boundary compressed data, and still image compressed data (in the case of Motion JPEG, a quantization table or a Huffman encoding table) and the position of each mesh. Include information indicating (such as coordinates on the screen).

[Step S97] The encoding unit 170a outputs the generated moving image compression data, boundary compression data, and still image compression data to the transmission unit 180.
In this way, the compression rate determination unit 171a calculates the amount of transmission data allocated to the still image region and the moving image region based on the ratio of the number of still image meshes, the number of moving image meshes, and the number of boundary meshes of the image data. Here, the boundary mesh is assigned more data bandwidth than the moving image mesh to improve the quality. For this reason, the number of boundary meshes is multiplied by a quality improvement coefficient g (> 1), and the ratio of the number of still image meshes and the number of moving image meshes is calculated to calculate the transmittable data amount of the boundary mesh group.

  FIG. 22 is a diagram illustrating each region of an image according to the third embodiment. The image data 300 has a still image area 310 and a moving image area 320. Further, the moving image area 320 includes a boundary area 340.

The boundary region 340 is a set of meshes adjacent to the still image region 310 among the meshes included in the moving image region 320 as described above.
The compression rate determination unit 171a determines that the transmission data amount allocated per unit area per boundary mesh group included in the boundary region 340 is a moving image mesh group included in the moving image region 320 (from the mesh group included in the moving image region 320 to the boundary mesh group). The amount of transmission data to be allocated per unit area per mesh group is excluded.

  Thereby, the effect similar to 2nd Embodiment can be acquired. In addition, the image quality of the boundary area 340 can be improved over the area other than the boundary area 340 in the moving image area 320. As a result, an abrupt change in image quality at the boundary between the still image area 310 and the moving image area 320 can be suppressed. For this reason, the display device 200 can display an image that is easier to view while suppressing a sense of discomfort given to the viewer.

  In addition, even when the point of interest in the displayed image is a still image in the still image region 310 adjacent to the moving image region 320, it is possible to suppress degradation of the quality of the image and display an easier-to-view image. It becomes.

[Fourth Embodiment]
Next, a fourth embodiment will be described. Differences from the second and third embodiments will be mainly described, and description of similar matters will be omitted.

  Here, in the display systems of the second and third embodiments, when moving the moving image area, the still image area existing behind the moving image area is newly displayed. In this case, the viewer's attention is easily directed to the area related to the movement.

  On the other hand, in the display system according to the fourth embodiment, a high-quality image is preferentially transmitted with respect to a mesh that is switched from a moving image region to a still image region. Thereby, the visibility of the area | region which concerns on the movement of a moving image area | region is improved.

  The overall configuration of the display system according to the fourth embodiment is the same as the overall configuration of the display system according to the second embodiment shown in FIG. The hardware configurations of the computer and the display device included in the display system of the fourth embodiment are the same as the hardware configurations of the computer 100 and the display device 200 of the second embodiment shown in FIGS. Therefore, the description is omitted.

  FIG. 23 is a diagram illustrating a functional configuration of each device according to the fourth embodiment. The computer 100b includes a control information storage unit 110, a mesh storage unit 120, a mesh management information storage unit 130b, a mesh generation unit 140, a change detection unit 145, a region determination unit 150b, a transmission region specification unit 160b, a coding unit 170, and a transmission unit. 180, a receiving unit 185, a communication band monitoring unit 190, and a frame rate setting receiving unit 195. These functions are realized by the CPU 101 executing the image processing program. However, some or all of these functions may be realized by dedicated hardware.

  The control information storage unit 110, the mesh storage unit 120, the mesh generation unit 140, the change detection unit 145, the encoding unit 170, the transmission unit 180, the reception unit 185, the communication band monitoring unit 190, and the frame rate setting reception unit 195 are described. Since it is the same as the structure which attached | subjected and demonstrated the same code | symbol in FIG. 5, description is abbreviate | omitted. Further, the configuration of the display device 200 is the same as the configuration described with reference to FIG.

  The mesh management information storage unit 130b includes mesh management information including the position of the mesh in the image, the number of continuing frames indicating how long each state of the moving image and the still image, the moving image and the still image has continued, and the like. Remember.

  The area determination unit 150b divides each mesh included in the image data into a moving image area and a still image area based on the change detection result of the change detection unit 145. Specifically, the area detected by the change detection unit 145 as having changed is specified as the moving image area. Further, the area detected by the change detection unit 145 as no change is identified as a still image area. Furthermore, the area determination unit 150b counts the number of frames (the number of continuous frames) that are continuously specified as the same segment area for the corresponding mesh. Note that the area determination unit 150b resets the count and starts again from 1 when the division of the area of the corresponding mesh is changed. The area determination unit 150b generates mesh management information based on the moving image and still image classification and the number of continuation frames of each mesh specified in this way, and stores the mesh management information in the mesh management information storage unit 130b.

  The transmission area specifying unit 160b specifies an area (transmission area) to be transmitted to the display device 200 based on the mesh management information stored in the mesh management information storage unit 130b. The transmission area specifying unit 160b specifies the entire moving image area as the transmission area.

  The transmission area specifying unit 160b specifies a part of the still image area as the transmission area. That is, the transmission area specifying unit 160b further divides the still image area, and sequentially specifies the divided areas as transmission areas at a plurality of timings. In this way, it is possible to allocate a larger amount of transmission data per unit area of the still image region than the amount of transmission data per unit area of the moving image region. That is, it is possible to improve the image quality of the still image region over the image quality of the moving image region by controlling the compression rate accompanying the subsequent encoding.

  Furthermore, the transmission area specifying unit 160b changes the transmission area selection method based on the mesh management information. Specifically, the transmission area specifying unit 160b preferentially specifies the transmission area with respect to the mesh switched from the moving image area to the still image area.

  FIG. 24 is a diagram illustrating an example of a data structure of a mesh management table according to the fourth embodiment. The mesh management table 131b is stored in the mesh management information storage unit 130b. The mesh management table 131b is provided with an item indicating mesh coordinates, an item indicating a moving image area flag, an item indicating a high-quality image transmission flag, and an item indicating the number of continuous frames. Information arranged in the horizontal direction of each item is associated with each other to indicate information about one mesh.

  Note that the item indicating the mesh coordinates, the item indicating the moving image region flag, and the item indicating the high-quality image transmitted flag are the same as the items having the same name in the mesh management table 131 described in FIG.

In the item indicating the number of continuation frames, the number of frames in which the corresponding mesh is continuously specified as a moving image region or a still image region is set.
In the mesh management table 131b, for example, the mesh coordinates are (x, y) = “(1, 1)”, the moving image area flag is “0”, the high-quality image transmitted flag is “1”, and the number of continuation frames is “ Information "10" is set. This is because the mesh specified by mesh coordinates (x, y) = “(1, 1)” is a still image mesh, a high-quality image has already been transmitted, and the state of the still image mesh continues for “10” frames. It shows that.

  In the mesh management table 131b, for example, the mesh coordinates are (x, y) = “(51, 31)”, the moving image area flag is “0”, the high-quality image transmitted flag is “0”, and the number of continuous frames Is set to information “10”. This is because the mesh specified by the mesh coordinates (x, y) = “(51, 31)” is a still image mesh, a high-quality image has not been transmitted, and the state of the still image mesh continues for “10” frames. It shows that.

  In the mesh management table 131b, for example, the mesh coordinates are (x, y) = “(70, 45)”, the moving image area flag is “0”, the high-quality image transmitted flag is “1”, and the number of continuous frames Is set to information “2”. This is because the mesh specified by mesh coordinates (x, y) = “(70, 45)” is a still image mesh, a high-quality image has already been transmitted, and the state of the still image mesh continues for “2” frames. It shows that.

  In the mesh management table 131b, for example, the mesh coordinates are (x, y) = “(72, 45)”, the moving image area flag is “0”, the high-quality image transmitted flag is “0”, and the number of continuous frames Is set to information “1”. This is because the mesh designated by the mesh coordinates (x, y) = “(72, 45)” is a still image mesh, a high-quality image has not been transmitted, and the state of the still image mesh is “1” frames, That is, it indicates that it is immediately after switching from the moving image mesh to the still image mesh.

  In the mesh management table 131b, for example, the mesh coordinates are (x, y) = “(73, 45)”, the moving image area flag is “1”, the high-quality image transmitted flag is “−”, and the number of continuous frames Is set to information “10”. This is because the mesh specified by the mesh coordinates (x, y) = “(73, 45)” is a moving image mesh, is not a transmission target of a high-quality image, and the state of the moving image mesh is “1” frame continuation. This indicates that it is immediately after switching from the still image mesh to the moving image mesh.

Next, processing procedures of the computer 100b and the display device 200 realized by the above configuration will be described.
The procedure of the image data transmission process of the computer 100b is the same as the procedure described with reference to FIG. Hereinafter, each process included in the image data transmission process of the computer 100b will be described in detail.

FIG. 25 is a flowchart illustrating transmission area specifying processing according to the fourth embodiment. In the following, the process illustrated in FIG. 25 will be described along the step numbers in the flowchart.
[Step S101] The transmission region specifying unit 160b refers to the moving image region flag of the mesh management table 131b stored in the mesh management information storage unit 130b, and determines whether or not the still image region is included in the image data to be transmitted this time. Determine whether. If a still image area is included, the process proceeds to step S102. If no still image area is included, the process proceeds to step S107.

  [Step S102] The transmission area specifying unit 160b refers to the high-quality image transmission completion flag in the mesh management table 131b and determines whether there is a mesh that has not yet transmitted a high-quality image among the still image meshes. . If there is a mesh for which a high-quality image has not been transmitted, the process proceeds to step S103. If there is no mesh that has not yet received a high-quality image, the process proceeds to step S107.

  [Step S103] The transmission area specifying unit 160b specifies the number of divisions of the still image area. The number of divisions of the still image area is stored in advance in, for example, the mesh management information storage unit 130b according to the usage environment. The transmission area specifying unit 160b calculates the number of transmission meshes of the still image area to be transmitted based on the specified number of divisions.

  [Step S104] With reference to the mesh management table 131b, the transmission area specifying unit 160b switches from a moving picture mesh to a still picture mesh with a moving picture area flag of “0” and a continuous frame number of “1”. It is determined whether or not the immediately following mesh exists. If it exists, the process proceeds to step S105. If not, the process proceeds to step S106.

  [Step S105] The transmission area specifying unit 160b selects a mesh that has not yet been transmitted as a transmission target mesh of a still image area with a mesh immediately after switching from a moving image mesh to a still image mesh (the number of continuous frames is “1”). Extract.

  [Step S106] The transmission region specifying unit 160b refers to the mesh management table 131b, and extracts meshes for the number of transmission meshes calculated in step S103 from meshes that have not yet received a high-quality image. If the mesh immediately after switching from the moving image mesh to the still image mesh has been extracted in step S105, the remaining number of transmission meshes is extracted. The transmission area specifying unit 160b updates the high-quality image transmitted flag in the mesh management table 131b. That is, the high-quality image transmission flag “0” set in the still image mesh extracted as the transmission target this time is changed to “1”. The transmission area specifying unit 160 b outputs the extracted still image mesh group to the encoding unit 170.

  [Step S107] The transmission area specifying unit 160b refers to the moving image area flag in the mesh management table 131b to determine whether or not the moving image area is included in the image data to be transmitted this time. If a moving image area is included, the process proceeds to step S108. If the moving image area is not included, the process is completed.

  [Step S108] The transmission area specifying unit 160b refers to the mesh management table 131b, extracts a moving image mesh, and sets the extracted moving image mesh group as a transmission target area. The transmission area specifying unit 160 outputs the extracted moving image mesh group to the encoding unit 170.

In this way, the transmission region specifying unit 160b preferentially extracts the mesh immediately after switching from the moving image mesh to the still image mesh and sets it as a transmission target to the display device 200.
The processing for the still image area shown in steps S101 to S106 and the processing for the moving picture area shown in steps S107 and S108 may be reversed. For example, it is also conceivable that the transmission area specifying unit 160b performs the process for the still image area (steps S101 to S106) after executing the process for the moving image area (steps S107 and S108) first.

  FIG. 26 is a diagram illustrating a specific example of the image data transmission method according to the fourth embodiment. In the following, the process illustrated in FIG. 26 will be described in order of step number. In each step of FIG. 26, the number of continuous frames is illustrated on the mesh constituting the image. Step ST11 is the transmission timing of the first frame to the display device 200 by the computer 100b. Step ST12 is the transmission timing of the next frame after step ST11. Thereafter, the same applies to steps ST13 and ST14.

  [Step ST <b> 11] The computer 100 b encodes the entire image 400 as a moving image area and transmits it to the display device 200. At this time, since all meshes are newly specified as moving image areas, the number of continuous frames of all meshes is “1”.

  [Step ST12] The computer 100b specifies the still image region 410 and the moving image region 420 from the change of each mesh. Then, a partial area 411 in the still image area 410 is encoded and transmitted to the display device 200. In addition, the entire moving image area 420 is encoded and transmitted to the display device 200. The partial area 411 is a higher quality image than the moving image area 420. In addition, regarding the area other than the partial area 411 in the still image area 410, the display device 200 continues to display the image data received in step ST11.

  [Step ST13] The computer 100b detects that the moving image area 420 in step ST12 has been changed to the position of the moving image area 421. At this time, the partial area 422 is detected as a moving image area. The computer 100b encodes a partial area 412 in the still image area and transmits it to the display device 200. Also, the entire moving image area 421 and partial area 422 are encoded and transmitted to the display device 200. The partial area 412 is a higher quality image than the moving image area 421 and the partial area 422. Note that for the partial area 411, the display device 200 continues to display the image data received in step ST12. In addition, for the areas other than the partial areas 411 and 412 among the still image areas, the display device 200 continues to display the image data received in step S11.

  [Step ST14] The computer 100b detects that the moving image area 421 in step ST12 has been changed to the position of the moving image area 423. At this time, the partial area 424 is specified as a moving image area. The partial area 415 is identified as a still image area because there is no change from the partial area 422. The computer 100b encodes the partial areas 414 and 415 in the still image area and transmits them to the display device 200. In addition, the moving image area 423 and the partial area 424 are encoded and transmitted to the display device 200. The partial areas 414 and 415 are higher quality images than the moving image area 423 and the partial area 424. Note that for the partial area 413, the display device 200 continues to display the image data received in steps ST12 and ST13. In addition, the display device 200 continues to display the image data received in step ST11 for regions other than the partial regions 413, 414, and 415 in the still image region.

  As described above, the computer 100b preferentially sets the mesh of an area (partial area 415 in the above example) where the moving image area is switched from the moving image area to the still image area as the moving image area moves.

  Thereby, the effect similar to 2nd Embodiment can be acquired. In addition, the viewer's attention tends to be directed to the area related to the movement of the moving image area. For this reason, it is possible to reduce the sense of incongruity that can be given to the viewer by improving the image quality preferentially with respect to such an area, and to further improve the visibility.

[Fifth Embodiment]
Next, a fifth embodiment will be described. Differences from the above-described second to fourth embodiments will be mainly described, and description of similar matters will be omitted.

  Here, in the display systems of the second to fourth embodiments, there is a possibility that a usable bandwidth can be provided depending on the use state. For this reason, in the display system according to the fifth embodiment, the visibility of an image displayed by the display device is further improved by using such an extra band.

  The overall configuration of the display system of the fifth embodiment is the same as the overall configuration of the display system of the second embodiment shown in FIG. The hardware configurations of the computer and the display device included in the display system of the fifth embodiment are the same as the hardware configurations of the computer 100 and the display device 200 of the second embodiment shown in FIGS. Therefore, the description is omitted.

  FIG. 27 is a diagram illustrating a functional configuration of each device according to the fifth embodiment. The computer 100c includes a control information storage unit 110c, a mesh storage unit 120, a mesh management information storage unit 130c, a mesh generation unit 140, a change detection unit 145, a region determination unit 150c, a transmission region specification unit 160c, and an encoding unit 170c. These functions are realized by the CPU 101 executing the image processing program. However, some or all of these functions may be realized by dedicated hardware. In FIG. 27, the relation line between the mesh management information storage unit 130c and the encoding unit 170c is omitted.

  Further, the mesh storage unit 120, the mesh generation unit 140, the change detection unit 145, the area determination unit 150c, the transmission unit 180, the reception unit 185, the communication band monitoring unit 190, and the frame rate setting reception unit 195 are the same in FIG. Since the configuration is the same as that described with reference numerals, the description is omitted. Furthermore, the configuration of the display device 200 is the same as that described with reference to FIG.

  The control information storage unit 110c stores data output control information for specifying a compression rate (or image quality) when encoding image data. Further, the control information storage unit 110c stores a threshold value for the number of continuous frames until the image data in the still image area is retransmitted.

  The mesh management information storage unit 130c includes mesh management information including the mesh position in the image, the number of continuous frames indicating how long each state of the moving image and the still image, the moving image and the still image has continued, and the like. Remember.

  The region determination unit 150c classifies each mesh included in the image data into a moving image region and a still image region based on the change detection result of the change detection unit 145. Specifically, the area detected by the change detection unit 145 as having changed is specified as the moving image area. Further, the area detected by the change detection unit 145 as no change is identified as a still image area. Furthermore, the area determination unit 150c counts the number of frames (the number of continuous frames) that are continuously specified as the same segment area for the corresponding mesh. Note that the area determination unit 150c resets the count and starts again from 1 when the division of the area of the corresponding mesh is changed. The area determination unit 150c generates mesh management information based on the moving image and still image classification of each mesh and information on the number of continuous frames specified in this way, and stores the mesh management information in the mesh management information storage unit 130c.

  The transmission area specifying unit 160c specifies an area (transmission area) to be transmitted to the display device 200 based on the mesh management information stored in the mesh management information storage unit 130c. The transmission area specifying unit 160c specifies the entire moving picture area as the transmission area.

  The transmission area specifying unit 160c specifies a part of the still image area as the transmission area. That is, the transmission area specifying unit 160c divides the still image area (primary division), and sequentially specifies the divided areas as transmission areas at a plurality of timings. In this way, it is possible to assign a larger amount of transmission image data per unit area of the still image region than a transmission image data amount per unit area of the moving image region. That is, it is possible to improve the image quality of the still image region over the image quality of the moving image region by controlling the compression rate accompanying the subsequent encoding.

  Furthermore, the transmission area specifying unit 160c refers to the mesh management information and the continuous frame number threshold stored in the control information storage unit 110c, and exceeds the continuous frame number threshold to determine the state of the still image area. Retransmission of still image mesh that continues. At this time, the transmission area specifying unit 160c divides the still image area by a larger number of divisions than the number of divisions at the time of primary division (secondary division). As described above, by dividing more than the primary division and sequentially retransmitting the still image area, it is possible to allocate a larger amount of transmission image data per unit area of the still image area than at the time of the primary division. Thereby, the image quality of the still image area transmitted by the primary division can be further improved.

  The encoding unit 170c sequentially encodes the meshes of the transmission area specified by the transmission area specifying unit 160c to reduce the data amount. The encoding unit 170c includes a compression rate determination unit 171c, a moving image region encoding unit 172, and a still image region encoding unit 173c. Here, the moving image region encoding unit 172 has the same configuration as that described with the same reference numeral in FIG.

  The compression rate determination unit 171c determines the encoded data amount of the moving image mesh group or the still image mesh group. Here, the compression rate determination unit 171c sets the compression rate of the still image mesh lower than the compression rate of the moving image mesh, and the still image region can display a higher quality image than the moving image region. Furthermore, the compression rate determination unit 171c sets a lower compression rate for the still image area when transmitting in the secondary division than when transmitting in the primary division. Thereby, the image quality of the still image area displayed by the secondary division can be further improved.

  The still image region encoding unit 173c encodes the still image mesh group in the transmission region so as to be close to the encoded data amount of the still image region determined by the compression rate determining unit 171c. When the still image area encoding unit 173c obtains the encoded data amount at the time of primary division from the compression rate determining unit 171c, the still image region encoding unit 173c encodes the still image mesh group so that the data amount is close to the encoded data amount at the time of primary division. To do. In addition, when the still image region encoding unit 173c acquires the encoded data amount at the time of the secondary division from the compression rate determination unit 171c, the still image mesh encoding unit 173c has a data amount close to the encoded data amount at the time of the secondary division. Encode the group.

The still image region encoding unit 173c outputs the still image compressed data generated by encoding to the transmission unit 180.
FIG. 28 is a diagram illustrating an example of a data structure of a continuation frame number threshold setting table according to the fifth embodiment. The continuation frame number threshold setting table 111c is stored in advance in the control information storage unit 110c. A continuation frame number threshold value is set in the continuation frame number threshold setting table 111c.

  The continuation frame number threshold indicates the threshold value of the number of continuation frames that should pass while the corresponding mesh remains in the still image mesh state before transmitting the image that is secondarily divided for the still image region. That is, the transmission region specifying unit 160c specifies a still image mesh group that is equal to or greater than the threshold value for the number of continuous frames as a transmission target by secondary division, and outputs it to the encoding unit 170c. As a result, the image quality of the still image mesh in which the period equal to or greater than the number of continuing frames has elapsed is improved.

  FIG. 29 is a diagram illustrating a data structure example of a mesh management table according to the fifth embodiment. The mesh management table 131c is stored in the mesh management information storage unit 130c. The mesh management table 131c includes items indicating mesh coordinates, items indicating moving image area flags, items indicating primary high-quality image transmitted flags, items indicating secondary high-quality image transmitted flags, and items indicating the number of continuous frames. Is provided. Information arranged in the horizontal direction of each item is associated with each other to indicate information about one mesh.

The items indicating mesh coordinates and the items indicating moving image area flags are the same as the items having the same names in the mesh management table 131 described with reference to FIG.
In the item indicating the primary high-quality image transmitted flag, a flag indicating whether or not the primary high-quality image by the primary division has been transmitted for the still image mesh is set. In the item indicating the primary high-quality image transmission flag, “1” is set when the primary high-quality image has been transmitted for the corresponding mesh, and “0” is set when the mesh has not been transmitted yet. In the item indicating the primary high-quality image transmission flag, “-” indicating that the primary high-quality image is not a transmission target is set when the corresponding mesh is a moving image mesh.

  In the item indicating the secondary high-quality image transmitted flag, a flag indicating whether or not the secondary high-quality image by the secondary division has been transmitted for the still image mesh is set. In the item indicating the secondary high-quality image transmission flag, “1” is set when the secondary high-quality image has been transmitted for the corresponding mesh, and “0” is set when the mesh is not yet transmitted. . Note that in the item indicating the secondary high-quality image transmission flag, “-” indicating that the secondary high-quality image is not a transmission target is set when the corresponding mesh is a moving image mesh.

In the item indicating the number of continuous frames, the number of frames in which the corresponding mesh is continuously specified as a still image region is set.
In the mesh management table 131c, for example, the mesh coordinates are (x, y) = “(1, 1)”, the moving image area flag is “0”, the primary high-quality image transmitted flag is “1”, and the secondary high-quality Information that the image transmission flag is “1” and the number of continuing frames is “35” is set. This indicates that the mesh specified by the mesh coordinates (x, y) = “(1, 1)” is a still image mesh, and “35” frames are continued in the state of the still image mesh. Further, it is shown that both the primary high-quality image and the secondary high-quality image have been transmitted to the display device 200 for the mesh.

  In the mesh management table 131c, for example, the mesh coordinates are (x, y) = “(51, 31)”, the moving image area flag is “0”, the primary high-quality image transmitted flag is “1”, and the secondary Information that the high-quality image transmission flag is “0” and the number of continuing frames is “28” is set. This indicates that the mesh designated by the mesh coordinates (x, y) = “(51, 31)” is a still image mesh, and “28” frames are continued in the state of the still image mesh. In addition, for the mesh, the primary high-quality image has been transmitted to the display device 200, but the secondary high-quality image has not been transmitted to the display device 200.

  The area determination unit 150c sets “0” to the primary high-quality image transmitted flag and the secondary high-quality image transmitted flag at the timing when the corresponding mesh is newly determined as a still image area.

Next, processing procedures of the computer 100c and the display device 200 realized by the above configuration will be described.
The procedure of the image data transmission process of the computer 100c is the same as the procedure described with reference to FIG. Hereinafter, each process included in the image data transmission process of the computer 100c will be described in detail.

FIG. 30 is a flowchart illustrating a transmission area specifying process according to the fifth embodiment. In the following, the process illustrated in FIG. 30 will be described along the step numbers in the flowchart.
[Step S111] The transmission region specifying unit 160c refers to the moving image region flag of the mesh management table 131c stored in the mesh management information storage unit 130c, and determines whether or not the still image region is included in the image data to be transmitted this time. Determine whether. If a still image area is included, the process proceeds to step S112. If a still image area is not included, the process proceeds to step S119.

  [Step S112] The transmission area specifying unit 160c refers to the primary high-quality image transmitted flag of the mesh management table 131c, and determines whether or not there is a mesh that has not transmitted the primary high-quality image among the still image meshes. judge. If there is a mesh for which a primary high-quality image has not been transmitted, the process proceeds to step S113. If there is no mesh that has not yet received the primary high-quality image, the process proceeds to step S115.

  [Step S113] The transmission area specifying unit 160c specifies the primary division number of the still image area. The primary division number of the still image area is stored in advance in, for example, the mesh management information storage unit 130c according to the use environment. The transmission area specifying unit 160c calculates the number of primary transmission meshes of the still image area to be transmitted based on the specified number of primary divisions.

  [Step S114] The transmission region specifying unit 160c refers to the mesh management table 131c, and extracts meshes corresponding to the number of primary transmission meshes calculated in step S113 from meshes that have not yet received a primary high-quality image. The transmission area specifying unit 160c updates the primary high-quality image transmitted flag of the mesh management table 131c. That is, the primary high-quality image transmission flag “0” set in the still image mesh extracted as the transmission target this time is changed to “1”. The transmission area specifying unit 160c outputs the extracted still image mesh group to the encoding unit 170c. The transmission area specifying unit 160c notifies the encoding unit 170c that the output still image mesh group corresponds to the primary division.

  [Step S115] The transmission area specifying unit 160c refers to the control information storage unit 110c to obtain a threshold value for the number of continuous frames. Then, the transmission region specifying unit 160c refers to the mesh management table 131c to determine whether there is a still image mesh having a continuation frame number equal to or greater than the continuation frame number threshold value. If it exists, the process proceeds to step S116. If not, the process proceeds to step S119.

  [Step S116] The transmission region specifying unit 160c refers to the secondary high-quality image transmitted flag in the mesh management table 131c to determine whether there is a mesh that has not yet transmitted the secondary high-quality image. If there is a mesh from which a secondary high-quality image has not been transmitted, the process proceeds to step S117. If there is no mesh that has not yet received the secondary high-quality image, the process proceeds to step S119.

  [Step S117] The transmission area specifying unit 160c specifies the number of secondary divisions of the still image area. The secondary division number of the still image area is stored in advance in, for example, the mesh management information storage unit 130c according to the use environment. The transmission region specifying unit 160c calculates the number of secondary transmission meshes of the still image region to be transmitted based on the specified number of secondary divisions.

  [Step S118] The transmission region specifying unit 160c refers to the mesh management table 131c, and extracts meshes for the number of secondary transmission meshes calculated in step S117 from meshes that have not yet received the secondary high-quality image. The transmission area specifying unit 160c updates the secondary high-quality image transmitted flag in the mesh management table 131c. That is, the secondary high-quality image transmitted flag “0” set in the still image mesh extracted as the transmission target this time is changed to “1”. The transmission area specifying unit 160c outputs the extracted still image mesh group to the encoding unit 170c. The transmission area specifying unit 160c notifies the encoding unit 170c that the output still image mesh group corresponds to the secondary division.

  [Step S119] The transmission region specifying unit 160c refers to the moving image region flag in the mesh management table 131c to determine whether or not the moving image region is included in the image data to be transmitted this time. If a moving image area is included, the process proceeds to step S120. If the moving image area is not included, the process is completed.

  [Step S120] The transmission region specifying unit 160c extracts a moving image mesh with reference to the mesh management table 131c, and sets the extracted moving image mesh group as a transmission target region. The transmission area specifying unit 160c outputs the extracted moving image mesh group to the encoding unit 170c.

  In this way, the transmission area specifying unit 160c specifies a part of the still image area as a transmission target area. The transmission area specifying unit 160c calculates the number of meshes to be selected as the transmission target area according to the number of divided still image areas. Further, the transmission area specifying unit 160c increases the number of divisions of the still image area in the secondary division than in the primary division.

  As a result, in the subsequent encoding process, a larger amount of transmission data of the same area can be allocated to the still image mesh group at the time of the secondary division than the still image mesh group at the time of the primary division.

  The processing for the still image area shown in steps S111 to S118 and the processing for the moving picture area shown in steps S119 and S120 may be reversed. For example, the transmission area specifying unit 160c may perform the processes (steps S111 to S118) for the still image area after executing the processes for the moving image area (steps S119 and S120) first.

  FIG. 31 is a diagram illustrating a specific example of the secondary high-quality image transmission method according to the fifth embodiment. In the following, the process illustrated in FIG. 31 will be described in order of step number. In each step of FIG. 31, the number of continuous frames is illustrated on the mesh constituting the image. Step ST21 is the transmission timing of the first frame to the display device 200 by the computer 100c. Step ST22 is the transmission timing of the next frame after step ST21. Thereafter, the same applies to steps ST23 and ST24.

  [Step ST <b> 21] The computer 100 c encodes the moving image area 520 in the image 500 and transmits it to the display device 200. At this time, it is assumed that the number of continuous frames of all meshes is “29”. Regarding the still image area 510, since the primary high-quality image has already been transmitted for all the still image meshes, the image data is not transmitted to the display device 200.

  [Step ST <b> 22] The computer 100 c encodes the moving image area 521 in the image 500 and transmits it to the display device 200. At this time, the number of continuous frames of all meshes is “30”. Regarding the still image area 511, since the temporary high-quality image has already been transmitted for all the still image meshes, the image data is not transmitted to the display device 200.

  [Step ST <b> 23] The computer 100 c encodes the moving image area 522 in the image 500 and transmits it to the display device 200. In addition, the computer 100 c encodes a partial area 512 that is one unit when the still image area is secondarily divided in the image 500 and transmits the encoded partial area 512 to the display device 200. At this time, for example, if the number of secondary divisions is set to twice the number of primary divisions, a transmission data amount twice as large as that during primary division can be assigned to the partial area 512. That is, the image quality of the partial area 512 can be improved by reducing the compression rate. The number of continuous frames of all meshes is “31”.

  [Step ST24] The computer 100c encodes the moving image area 523 in the image 500 and transmits it to the display device 200. In addition, the computer 100 c encodes a partial area 514 that is one unit when the still image area is secondarily divided in the image 500 and transmits the encoded partial area 514 to the display device 200. Partial area 513 corresponds to partial area 512 in step ST23. The display device 200 continuously displays the image received in step ST23 for the partial region 513. Note that the number of continuous frames of all meshes is “32”.

  As described above, the computer 100c transmits an image by primary division for a still image region, and then performs secondary division with a smaller number of divisions than the primary division for a still image mesh having a predetermined number of frames. Send image data to

Thereby, the same effect as that of the second embodiment can be realized. In addition, it is possible to further improve the image quality of the still image area and realize an image display that is easier to view.
As described in the second to fifth embodiments, the computers 100, 100a, 100b, and 100c divide the image data into still image regions and moving image regions based on temporal changes of meshes included in the image data. To do. Then, the computer 100 makes the number of transmissions of the still image mesh smaller than the number of transmissions of the moving image mesh.

This makes it possible to transmit image data by efficiently using the transmission capacity of the network.
Further, by dividing the still image region into a plurality of regions and transmitting them at different timings, it is possible to allocate a larger amount of transmission data per unit area than that of the moving image region.

  Thereby, the image quality of the still image area can be made higher than the image quality of the moving image area. As a result, the visibility of the image displayed on the screen 20 by the display device 200 can be improved.

  Further, as described in the third embodiment, a boundary region that is a boundary between the still image region and the moving image region is introduced. Then, the transmission data amount assigned per unit area for the boundary mesh group is set to be larger than the transmission data amount assigned per unit area for the moving image mesh group.

  Thereby, the image quality of the boundary mesh group can be improved over the image quality of the moving image mesh group. As a result, an abrupt change in image quality at the boundary between the still image region and the moving image region can be suppressed. For this reason, the display device 200 can display an image that is easier to view while suppressing a sense of discomfort given to the viewer.

  In addition, even when the point of interest in the displayed image is a still image in a still image area adjacent to the moving image area, it is possible to suppress degradation of the quality of the image and display an easier-to-view image. .

Further, as described in the fourth embodiment, a mesh in an area in which the moving image area is switched from the moving image area to the still image area as the moving image area moves is preferentially set as a transmission target.
In other words, the image quality is preferentially enhanced with respect to the area related to the movement of the moving image area that is easy for the viewer to pay attention. Thereby, the uncomfortable feeling which can be given with respect to a viewer can be reduced, and legibility can be improved more.

  In addition, as described in the fifth embodiment, after the image is transmitted by primary division for the still image area, the still image mesh having a predetermined number of frames is subjected to secondary division with a smaller number of divisions than the primary division. The image data is transmitted to the display device 200.

As a result, it is possible to further improve the image quality of the still image region and realize an image display that is easier to view.
By using these methods, it is possible to efficiently transmit image data. Specifically, the number of transmissions of image data in a still image area with little change is made smaller than the number of transmissions of image data in a moving picture area with much change. As a result, unnecessary use of transmission capacity can be suppressed. In addition, the image quality can be further improved by using the excess transmission capacity. In addition, the transmission capacity can be used for other data communications.

  The image processing program, display system, image processing apparatus, and image processing method of the present invention have been described based on the illustrated embodiment. However, the present invention is not limited to this, and the configuration of each unit is the same. Any structure having a function can be substituted. Moreover, other arbitrary structures and processes may be added to the present invention. Further, the present invention may be a combination of any two or more configurations (features) of the above-described embodiments.

DESCRIPTION OF SYMBOLS 1 Computer 1a Detection means 1b Transmission means 2 Display apparatus 10, 10a, 10b Image 11 1st type area | region 12 2nd type area | region 11a, 11b, 12a, 12b Image information

Claims (14)

Computer
In accordance with a change between a plurality of images input in time series, a first type region included in each image and a second type region having a larger amount of change than the first type region are detected. Detecting means to
The number of transmissions of the image information of the first type area in a predetermined time width is set to be smaller than the number of transmissions of the image information of the second type area in the predetermined time width. Transmitting means for sequentially transmitting image information of the area and image information of the second type area;
An image processing program that functions as an image processing program. The detection means detects a plurality of areas as the first type area, sequentially specifies a transmission target area for each predetermined number of the plurality of areas,
The transmitting unit transmits image information of the transmission target area specified by the detecting unit;
The image processing program according to claim 1. The computer further encodes the image information of the transmission target area specified by the detection means at a first compression rate, and sets the image information of the second type area higher than the first compression rate. Function as an encoding means for encoding at a compression rate of 2,
The transmission means transmits the image information of the transmission target area and the image information of the second type area after being encoded by the encoding means.
The image processing program according to claim 2, wherein:   The encoding means includes a transmission capacity of a network to which the computer is connected, an area ratio between the first type area and the second type area, and image information of the transmission target area to be transmitted. 4. The image processing according to claim 3, wherein the first compression rate and the second compression rate are determined based on a data amount and a data amount of image information of the second type area. program. The detecting means detects a third region that is adjacent to the first type region among the plurality of regions detected as the second type region;
The encoding means encodes the image information of the third region at a compression rate higher than the first compression rate and lower than the compression rate of the second type region,
The transmitting unit transmits the image information of the third region after the encoding by the encoding unit together with the image information of the second type region;
The image processing program according to any one of claims 3 and 4. If the area detected as the second type of area by the detecting means is detected as the first type of area, the encoding means detects the type of the area in the first type of area. The image information of the change area is encoded in preference to the area other than the change area that is
The transmission means transmits the image information of the change area after encoding by the encoding means;
The image processing program according to claim 3, wherein the image processing program is characterized in that: The detection means identifies a retransmission target area that has been detected as the first type area for a predetermined period or longer,
The encoding means encodes the image information of the retransmission target area at a compression rate lower than the first compression rate,
The transmission means transmits image information of the retransmission target area after encoding by the encoding means;
The image processing program according to claim 3, wherein the image processing program is any one of claims 3 to 6.   The transmission means transmits image information encoded at the second compression rate by the encoding means for the entire area of the image, and then the first type area and the second by the detection means. Depending on the detection result of the type of area, the image information encoded at the first compression rate by the encoding unit is transmitted as data for replacing the already transmitted image information for the first type of area. The image processing program according to claim 3, wherein the image processing program is executed.   The transmission means does not transmit until the detection means detects a change in a transmitted area among a plurality of areas detected as the first type area by the detection means. Item 9. The image processing program according to any one of Items 3 to 8.   The transmission means transmits the image information of the second type area at a predetermined cycle, and transmits the image information of the first type area to any of the transmission timings of the image information of the second type area. The image processing program according to any one of claims 1 to 9, wherein the image processing program is transmitted at such timing. The detection means detects the change at an input rate of the plurality of images input in the time series,
The transmission means transmits the image information of the second type area at the input rate, thins out the image information of the first type area so that the transmission rate is lower than the input rate, Transmitting image information of the type of area at the transmission rate,
The image processing program according to any one of claims 1 to 10, wherein: In accordance with a change between a plurality of images input in time series, a first type region included in each image and a second type region having a larger amount of change than the first type region are detected. Detecting means for
The number of transmissions of the image information of the first type area in a predetermined time width is set to be less than the number of transmissions of the image information of the second type area in the predetermined time width. Transmitting means for sequentially transmitting image information of the area and image information of the second type area;
An image processing apparatus comprising:
Receiving means for sequentially receiving the image information of the first type area and the image information of the second type area transmitted by the transmitting means;
When the receiving means receives the image information of the first type area and the image information of the second type area, the image information of the first type area and the image information of the second type And when the receiving means receives the image information of the second type area and does not receive the image information of the first type area, Display means for displaying image information and image information of the second type of area received this time;
A display device comprising:
A display system comprising: In accordance with a change between a plurality of images input in time series, a first type region included in each image and a second type region having a larger amount of change than the first type region are detected. Detecting means for
The number of transmissions of the image data of the first type area in a predetermined time width is set to be smaller than the number of transmissions of the image data of the first type area in the predetermined time width. Transmitting means for sequentially transmitting image data of the area and image data of the second type of area;
An image processing apparatus comprising: A computer image processing method comprising:
In accordance with a change between a plurality of images input in time series, the detection means includes a first type region included in each image and a second type of a change amount larger than that of the first type region. Detect areas and
The transmission means sets the number of transmissions of the image data of the first type area in a predetermined time width as a number less than the number of transmissions of the image data of the second type area in the predetermined time width. Sequentially transmitting image data of one type of area and image data of the second type of area;
An image processing method.

Images