JP2007281946A - Photograph image processing apparatus, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem nonconformity, wherein when there are a plurality of image processing requests from a network, an output of encoded data is interrupted, a frame rate is lowered, and block noise occurs, as a result of increased processed loads. <P>SOLUTION: The present invention includes a reading method of imaging element data, in response to image processing requested from a network; an image data output method of a development processing part; and a system or the like, capable of network outputting of stabilized encoded data by providing means for making selectable a proper image processing method, in response to a switching system thereof. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a captured image processing apparatus having a network.

  As an invention in which the readout method of the image sensor is associated with the operating frequency of the image processing unit in the subsequent stage, for example, the one disclosed in Patent Document 1 can be cited.

  This is characterized in that in a digital still camera using a so-called high pixel image sensor, the frame rate of the finder display is improved by making the pixel readout clock faster during the finder operation than when all the pixels are captured.

  Further, as an example of performing a process of switching the operation of the subsequent image processing unit in conjunction with the readout method of the image sensor in accordance with the required image size and image quality, for example, the one disclosed in Patent Document 2 is given. It is done.

  This is characterized in that a digital still camera associates image quality processing with a read image to be applied for each thumbnail image or saved image.

JP 2002-142149 A (page 13, FIG. 7) Japanese Patent Laid-Open No. 2003-102026 (page 8, FIG. 1)

  However, as in the invention disclosed in Patent Document 1, when the process of increasing the operating frequency is performed as a solution for improving the processing capability in the finder mode, the power consumption inevitably increases. was there.

  Furthermore, in the invention disclosed in Patent Document 1, the output destination of image data in the finder mode is limited to one display device, and a case where a plurality of images are output to different display means is assumed. In that case, it would be impossible to deal with it simply by increasing the operating frequency.

  Furthermore, in the invention disclosed in Patent Document 1, switching the readout method of the image sensor requires data of different numbers of pixels such as all pixel readout and finder display thinning readout as a still image for storage. This is a process performed in a case, not due to the processing capability of the device.

  Further, as in the invention disclosed in Patent Document 2, when the operation of the image processing unit is switched according to the required image size and image quality, only the field image is certainly used for the thumbnail image. Although the processing bandwidth of the image processing unit is reduced, the processing load is large to create thumbnail images that are originally created by reducing all pixel images, and reducing the image from all images. Since an image output at a timing and a reading method different from a desired image is used because the speed is reduced, there is a problem that all pixel images and thumbnail images are different in a fast-moving image.

  Therefore, an object of the present invention is to prevent processing from stopping network output by performing processing exceeding the processing capacity, image update stagnation due to a decrease in frame rate, occurrence of block noise, etc. It is to provide a network camera system that can be viewed.

The captured image processing apparatus of the present invention determines a request for an encoding process of a plurality of output methods, an image processing unit, an encoding unit, a network communication unit, and an encoding process of a connection partner via a network. A request determination unit; and a processing load determination unit that determines a processing load of the image processing unit and the encoding unit. The processing load determination unit determines an output method of the imaging unit according to the determination of the request determination unit. An instruction is given to the imaging unit and the image processing unit so as to be switched.
The image processing method of the present invention includes an imaging unit having a plurality of output methods, an image processing unit, an encoding unit, a network communication unit, and a request for determining a request for encoding processing of a connection partner via a network. A processing method in a captured image processing apparatus, comprising: a determination unit; and a processing load determination unit that determines a processing load of the image processing unit and the encoding unit, wherein the processing load is determined according to the determination of the request determination unit The judging means includes a step of switching the output method of the image pickup means, and a step of issuing an instruction to the image pickup means and the image processing means based on the step.

  In the present invention, in order to continue outputting encoded data without interrupting image output to the network, the network controller reads the image sensor data according to the image processing required from the network, and the development processor It is possible to select an image data output method and an appropriate image processing method according to the switching method. Therefore, according to the present invention, it is possible to prevent the network output from being stopped, the image update from being delayed in the display device due to the decrease in the frame rate, the occurrence of block noise, and the like by executing processing exceeding the processing capability. It becomes possible to provide a network camera system that can be viewed stably.

  In the present invention, the image pickup device data reading method and the development processing unit according to the compression processing required by the network so that the network control unit can continue to output the compressed data without interrupting the image output to the network. The image data output method, the processing method in the image processing unit corresponding to the switching method, and the compression method required from the network can be selected. Therefore, according to the present invention, it is possible to provide a network camera system that can be viewed stably.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram illustrating an imaging apparatus according to the first embodiment of the present invention.
In FIG. 1, an imaging apparatus 200 includes a dodging processing unit 101 serving as a G increase processing unit, a compression processing unit 205 serving as an encoding unit, an imaging unit 201 serving as an imaging unit capable of switching an output method, The communication unit 206, which is a network communication means, is roughly divided into four components.

The imaging unit 201 is a developing unit that develops a lens 202 on which an image is incident, an imaging element 203 that is an imaging unit that converts light collected by the lens into an electrical signal, and a signal obtained from the imaging element 203. It consists of a certain development processing unit 204.
Note that the resolution of the image sensor 203 in the present embodiment will be described using a so-called VGA size (640 × 480 pixels).

  When the image sensor 203 includes a primary color filter, the image sensor 203 outputs RGBG data, has a function of switching between field readout and frame readout, and has a mechanism for switching the readout method in accordance with an instruction from the network control unit 207.

  Field readout normally outputs an image of 640 × 240 pixels 60 times per second, and frame readout outputs an image of 640 × 480 pixels 30 times per second, and an imaging method in accordance with an instruction from the network control unit 207 The switching unit 214 has a mechanism for switching between the two reading methods.

  The development processing unit 204 converts the element data output from the image sensor 203 into image data of a predetermined format while performing development processing, and outputs the image data.

  As an image output method, when the output from the image sensor 203 is field readout, the same field output (640 × 240 pixels 60 field output) is performed. When the output from the image sensor 203 is frame readout, odd lines are output. Network control with three output methods: interlaced output that alternately outputs ODD images consisting of only images and EVEN images consisting of only even-line images, and progressive output that outputs frame images as they are The imaging method switching unit 214 has a mechanism for switching the three output methods in accordance with an instruction from the unit 207.

  The developed signal is provided as image data to the dodging processor 101, and after undergoing dodging processing, the encoded signal is encoded by the compression processor 205, and then supplied to the communication unit 206 as encoded data.

  The compression processing unit 205 internally buffers the image output from the dodging processing unit 101 and encodes it according to an encoding method such as JPEG, MPEG, H.264, or JPEG2000 specified by the network processing unit 207. Process.

  In the communication unit 206 serving as a network communication unit, a request for the imaging device 200 from the connected display device 241 is determined by the network control unit 207, and the processing information is the dodging processing unit 101, the imaging unit 201, and the compression processing unit. 205 and the imaging method switching unit 214.

  Note that these processes are basically processed by software operating on the CPU 210 which is a process determination unit and a processing load determination unit in the network control unit 207.

  The moving image data encoded for the display device by the processing information is input to the storage unit 208 via the network control unit 207, and the moving image encoded data input to the storage unit 208 by the network control unit 207 is Processing such as packetization is performed according to the form of the network to be transmitted, and the data is transferred to the network interface unit 209 as transmission data.

  The network interface unit 209 sends data to the network 240 in a form corresponding to the network form.

  The storage unit 208 also stores a moving image output from the camera or a moving image distributed to any user. Note that the network 240 can be assumed to be representative of a wired LAN such as Ethernet (registered trademark), a wireless LAN represented by IEEE 802.11b, or a public line network such as ISDN.

  Note that the storage unit 208 may be composed of not only a disk drive type memory such as a hard disk drive or a DVD-RAM but also a built-in RAM, a built-in DRAM, or a DRAM connected to the outside of the IC.

  In the present embodiment, the dodging processing unit 101, the compression processing unit 205, the imaging unit 201, the communication unit 206, and the imaging method switching unit 214 are described as separate blocks in FIG. When configured by hardware, these blocks are realized as separate ICs, or some parts of each part (for example, the development processing unit 204, the imaging method switching unit 214, the dodging processing unit 1, and the compression processing unit) 205 and the network control unit 206) can be realized in various ways, for example, on a single chip, and the form is not limited in this embodiment.

FIG. 2 is a block diagram illustrating an internal configuration of the dodging processing unit 101 of the imaging apparatus according to the first embodiment.
In FIG. 2, 101 is the dodging processor. An input signal 102 represents image data output from the imaging unit 201 and control signals such as a pixel sampling clock and a horizontal / vertical synchronization signal.

  A luminance / color difference separating unit 103 separates the input image 102 into luminance data and color difference data. The data format of the input image 102 input to the luminance / color difference separation unit 103 is the same regardless of whether it is RGB or YC (regardless of 4: 2: 2 or 4: 4: 4). In other words, the data is separated into luminance data and color difference data.

  A buffer processing unit 104 temporarily stores the color difference data and luminance data output from the luminance / color difference separation unit 3 and outputs them according to an instruction from the dodging control unit 119.

  An image reduction unit 106 reduces the luminance data output from the luminance / color difference separation unit 103 at a desired ratio.

  Reference numeral 109 denotes a spatial filter processing unit that creates a spatially filtered image by applying a low-pass filter such as a Gaussian filter to the reduced image output from the image reduction unit 106.

  A filter image buffer processing unit 110 temporarily accumulates the spatial filter processed image output from the spatial filter processing unit 109 and outputs the image according to an instruction from the dodging control unit 119.

  Reference numeral 112 denotes an image enlargement unit that enlarges the spatial filter processed image read from the filter image buffer processing unit 110 to the same resolution as the input image 2.

  An LUT 113 receives luminance data having the same resolution as that of the input image 2 output from the image enlargement unit, and performs arithmetic processing based on a look-up table (hereinafter referred to as LUT) instructed from the dodging control unit 119. An arithmetic processing unit.

  114 reads the input image 102 separated into the color difference data and the luminance data from the buffer processing unit 104, performs the synthesis process based on the image data output from the LUT arithmetic processing unit 113 and a predetermined arithmetic expression, and outputs the image composition Part.

  Reference numeral 118 denotes image data subjected to dodging processing output from the image composition unit 114.

  Reference numeral 119 denotes a control unit that gives appropriate instructions to each block in the dodging processing unit 1 according to parameters instructed by the network control unit 207, and controls the operation timing of each processing unit by inputting an input signal. It is.

  Although not shown in FIG. 2, all the processing units in the dodging processing unit 1 are connected to signals for being controlled by the dodging control unit 119.

  Each processing unit is provided with a FIFO and a RAM that serve as a timing adjustment buffer. In addition, the buffer processing unit 104 may be configured by a built-in RAM, a built-in DRAM, or a DRAM connected to the outside of the IC.

FIG. 3 is a flowchart showing the operation of the display device 241 in the present embodiment.
FIG. 4 is a flowchart showing the operation of the CPU 210 in the network control unit 207 in the present embodiment.
FIG. 5 shows a comparison between the image reading method (501) of the image sensor 203, the output method (502) of the development processing unit 204, the image reading method (503) during the dodging process, and the image quality after the dodging process in this embodiment ( 504), the image data (505) stored in the buffer processing unit 104 and the filter image buffer processing unit 110, and a mode number (506) assigned to all of these combinations.
FIG. 6 is a sequence chart showing the operation of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in mode number 1.

  In the setting of mode number 1, reading (601) of the image sensor 203 is field reading, and output (602) of the development processing unit 204 is also field output. In the dodging processing unit 101, the input image 102 is directly used as a buffer processing unit. The spatial filter processed image is stored in the filtered image buffer processing unit 110 (606). The spatial filtered image generated by the reduction processing and the spatial filtering processing (605) performed on the luminance data of the input image 102 is stored (606). The

  After the creation of the spatial filter processed image is completed, the dodging control unit 119 reads the input image 102 accumulated from the buffer processing unit 104 (604) and simultaneously reads the spatial filter processed image from the filter image buffer processing unit 110 (606). ) After performing the enlargement process in the image enlargement unit 112 and the γ process in the LUT calculation processing unit on the spatial filter processed image, the synthesis process in the image synthesis unit 114 is executed on the read input image 102 (607). An output image 118 is output.

  The output image 118 is input to the compression processing unit 205, and MPEG2, MPEG4, H.264, and so on. H.264 or the like stream system encoding processing (608) is performed.

  FIG. 7 is a sequence chart showing the operation of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in the mode number 7.

  With the setting in mode number 7, the readout (701) of the image sensor 203 is a frame readout, and the output (702) of the development processing unit 204 is an interlaced output.

  In the dodging processing unit 101, only the ODD data of the input image 102 is accumulated in the buffer processing unit 104 (704), and is created by reduction processing and spatial filter processing (705) performed on the luminance data of the input image 102. The spatial filter processed image is accumulated (706) in the filtered image buffer processing unit 110.

  After the creation of the spatial filter processed image is completed, the dodging control unit 119 reads (704) the input image 102 accumulated from the buffer processing unit 104, and at the same time, the spatial filter created by the ODD data from the filter image buffer processing unit 110 The processed image is read (706), and the enlargement process in the image enlargement unit 112 and the γ process in the LUT calculation processing unit are performed on the spatial filter process image.

  Thereafter, the output image 118 is output by executing (707) the combining process in the image combining unit 114 on the read input image 102. The output image 118 is input to the compression processing unit 205, and MPEG2, MPEG4, H.264, and so on. H.264 or the like stream system encoding processing (708) is performed.

  In mode number 7, the process 706 stored in the filter image buffer processing unit 110 is omitted from the mode 1 by applying the spatial filter processed image created with the ODD data to the EVEN data.

  FIG. 8 is a sequence chart showing the operation of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in mode number 2.

  In the setting of mode number 2, the readout 601 of the image sensor 203 is a field readout, and the output (602) of the development processing unit 204 is also a field output. In the dodging processing unit 101, the input image 102 is stored in the buffer processing unit 104. Without performing (604), the filtered image buffer processing unit 110 accumulates (606) the spatially filtered image created by the reduction processing performed on the luminance data of the input image 102 and the spatial filtering (605).

  After completing the creation of the spatial filter processed image, the dodging control unit 119 reads the spatial filtered image from the filtered image buffer processing unit 110 in accordance with the input image 102 output without being stored in the buffer processing unit 104 (606). .

  After the enlargement process in the image enlargement unit 112 and the γ process in the LUT calculation processing unit are performed on the spatial filter processing image, the synthesis process in the image synthesis unit 114 is performed 607 on the read input image 102, whereby the output image 118 is obtained. Is output.

  The output image 118 is input to the compression processing unit 205, and MPEG2, MPEG4, H.264, and so on. H.264 or the like stream system encoding processing (608) is performed.

  In mode number 2, the input image 102 is not subjected to buffer processing, and the image subjected to spatial filter processing is combined with the next field image, so that input / output processing of the input image 102 to the buffer processing unit 104 is performed. (604) is omitted.

  Image pickup apparatus in the case of performing compression by the moving image type encoding method in the first embodiment of the present invention using FIG. 1, FIG. 2, FIG. 3, FIG. 4, FIG. The operation 200 will be described.

  When a certain display device 241 accesses the imaging device 200 (step S301) and establishes communication, the network control unit 207 activates the imaging unit 201 (step S401) and sets each initial setting item (step S402). The imaging unit 201 starts image processing.

  In the setting of each initial setting item at the time of startup (step S402), first, each processing unit starts operation with the setting in mode number 1 shown in FIG. In mode 1, as shown in FIG. 5, the images written in the buffer processing unit 104 and the filtered image buffer processing unit 110 become the field input image 102 and the field space filtered image.

  The encoded data subjected to the encoding process (608) in the mode 1 operation is output from the network I / F 209 to the network 240 (step S403).

  The display device 241 that has received the output image via the network 240 (step S302) restores the encoded image data and displays it on the display means (step S303).

  The operator controls the imaging device 200 by selecting a mode for controlling the imaging device 200 by some control means existing on the display device while viewing the captured image of the imaging device 200 (step S304). Is possible.

  The display device 241 determines a command input by the operator (step S305). If the command is a command for starting the dodging process, a command for starting or stopping the dodging process is set (step S312). If the command is a command related to processing other than the dodging process, the command is set (step S313), and then the command is transmitted to the imaging apparatus 200 (step S315).

  The other processing commands refer to general control of the imaging apparatus 200 such as stop / start of shooting, change of the luminance level of the entire shot image, and ZOOM.

  The imaging device 200 that has received the command transmission from the display device 241 determines the command in the network control unit 207 (step S405), and if the command is not related to dodging or encoding, the instruction is given. Processing is performed (step S406).

  If it is determined that the command is related to dodging or the encoding method, the network control unit 207 determines the required encoding method type and the required encoding amount from the network (step S407), and the request is further determined. By determining which display device the device is from (Step S408), a determination is made to estimate the processing load of the imaging device 200 (Step S409).

  The request from the display device 241 is MPEG2, MPEG4, H.264, which is a moving image encoding system. 264, and when there is no request from another display device, the network control unit 207 continues output with mode number 1, but an image output request from a different display device 242 is generated, and Although it is the same encoding method for image output, if it is determined that an image output with a different compression efficiency is requested (step S408), the compression processing unit 206 increases the processing load in the determination to estimate the processing load (step S409). The amount of processing to be reduced in the dodging processing unit 101 when each reading method is changed in accordance with the amount of processing to be performed is determined (step S411). If it is determined that each reading method needs to be changed (step S411), the network control unit 207 instructs each processing unit to switch to the mode number 7 method (step S412).

  Each processing unit that has received the instruction switches the processing operation in accordance with the instruction of mode number 7 (step S413).

  In the mode number 1 and the mode number 7, since the accumulation and reading of the spatial filter processed image in the filter image buffer processing unit 110 are reduced by one field, the processing load of the imaging apparatus 200 is reduced, and the reduction in the processing load is reduced. Different encoding processes for the display device 242 can be assigned.

However, in terms of the image quality 504, considering a portion that is a moving image encoding method, when the field reading method in reading of the image sensor 203 has a shorter output period than frame reading, a smooth moving image can be obtained. Conceivable.
Therefore, mode number 1 is considered to be superior in terms of moving image quality 504.

  In the case of a single request from the display device 241, when the request from the display device 242 overlaps, the imaging device 200 operating in the mode number 7 method is further different from the display device 243. If an image output request is generated and it is determined that an image output with a different compression efficiency is requested (step S408), although the same encoding method is used for the image output, a determination to again estimate the processing load (step S408) In step S409, when it is determined that each reading method needs to be changed (step S411), the network control unit 207 instructs each processing unit to switch to the mode number 2 method (step S412).

  Each processing unit that has received the instruction switches the processing operation in accordance with the instruction of mode number 2 (step S413).

  In mode number 2, since the input image 102 is not stored in and read from the buffer processing unit 104, the processing load of the imaging apparatus 200 is further reduced as compared with the case of mode number 7, and the reduction in the processing load. Can be assigned to different encoding processes for the display device 243.

  However, in terms of the image quality 504, considering the portion that is a moving image encoding method, the image sensor 203 reads the image using the same field reading method as mode number 1, but the image combined with the image subjected to the spatial filter processing Therefore, mode number 1 is superior as the dodging processing image.

Further, even when compared with mode number 7, it is considered that the influence of the time lag on the image quality is larger than the influence of the difference in the reading method of the image sensor 203 on the image quality.
Accordingly, it is considered that mode number 1 and mode number 7 are superior in terms of moving image quality 504.

  As described above, in this embodiment, each processing unit determines the processing load of the imaging apparatus 200 itself for processing requested from the network, and cancels out the increase in compression processing with the decrease in buffer processing. By actively switching the reading and processing method, the processing exceeding the processing capacity is executed to stop the network output, the image update stagnation due to the decrease in the frame rate, the occurrence of block noise, etc. It becomes possible to prevent.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
In the first embodiment, the operation of the imaging device 200 when performing compression using the moving image encoding method from the display device 241 side has been described. However, in the present embodiment, a stationary operation such as JPEG is performed from the display device 241 side. The same effect can be expected even when there is a request for an image encoding method.

  FIG. 9 shows an operation sequence of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in the mode number 5.

  With the setting in mode number 5, readout of the image sensor 203 (901) is frame readout, and the output (902) of the development processing unit 204 is progressive output.

  In the dodging processing unit 101, the input image 102 is accumulated in the buffer processing unit 104 (904), and a spatial filter created by reduction processing and spatial filter processing (905) performed on the luminance data of the input image 102. The processed image is accumulated (906) in the filter image buffer processing unit 110.

  After the creation of the spatial filter processed image is completed, the dodging control unit 119 reads the input image 102 accumulated from the buffer processing unit 104 (904) and simultaneously reads the spatial filter processed image from the filter image buffer processing unit 110 (906). ), An enlargement process in the image enlargement unit 112 and a γ process in the LUT calculation processing unit are performed on the spatial filter processed image.

  Thereafter, the output image 118 is output by executing (907) the combining process in the image combining unit 114 on the read input image 102.

  The output image 118 is input to the compression processing unit 205, and a still image type encoding process (908) such as MJPEG is performed.

  FIG. 10 is a sequence chart showing the operation of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in mode number 6.

  In the setting of mode number 6, reading (1001) of the image sensor 203 is frame reading, output (1002) of the development processing unit 204 is progressive output, and the dodging processing unit 101 outputs the input image 102 to the buffer processing unit 104. The accumulation (1004) is not performed, and the spatial filter processed image created by the reduction process and the spatial filter process (1005) performed on the luminance data of the input image 102 is accumulated (1006) in the filter image buffer processing unit 110. .

  After the creation of the spatial filter processed image, the dodging control unit 119 reads out the spatial filtered image from the filtered image buffer processing unit 110 in accordance with the input of the next progressive output image after the spatial filtered image (1006). After the enlargement process in the image enlargement unit 112 and the γ process in the LUT calculation processing unit are performed on the spatial filter processed image, the synthesis process in the image synthesis unit 114 is executed on the read input image 102 (1007). An image 118 is output. The output image 118 is input to the compression processing unit 205, and still image type encoding processing (1008) such as MJPEG is performed.

  FIG. 11 shows an operation sequence of each processing unit controlled by the network control unit 207 and the dodging control unit 119 in the mode number 8.

  In the setting of mode number 8, reading (1101) of the image sensor 203 is frame reading, output (1102) of the development processing unit 204 is interlaced output, and the dodging processing unit 101 is a buffer processing unit 104 for the input image 102. Is not stored (1104), and the spatial filter processed image created by the reduction processing and spatial filter processing (1105) performed on the luminance data of the ODD data output from the development processing unit 204 is the filter image buffer processing unit. 110 is accumulated (1106).

  After the creation of the ODD data spatial filter processed image, the dodging control unit 119 performs the spatial filter processed image from the filter image buffer processing unit 110 in accordance with the input of the ODD data image next to the ODD data image subjected to the spatial filter processing. (1006), the spatial filter processing image is subjected to the enlargement process in the image enlargement unit 112 and the γ process in the LUT calculation processing unit, and then the synthesis process in the image composition unit 114 is performed on the read input image 102 (1007). ), The output image 118 is output.

  Further, the spatial filter processing image by ODD data is similarly applied to EVEN data, and the same processing is performed.

  The output image 118 is input to the compression processing unit 205, and after the process of progressively converting the ODD data composite image and the EVEN data composite image is performed again, still image system encoding processing (1008) such as MJPEG is performed.

  Imaging in the case of performing compression by the still image system encoding method in the second embodiment with reference to FIGS. 1, 2, 3, 4, 5, 7, 9, 10, and 11. The operation of the apparatus 200 will be described.

  As in the first embodiment, when the request from the display device 241 is a method such as JPEG, which is a still image encoding method, and there is no request from another display device, the imaging device 200 The initial setting of each processing unit is performed in the state of mode number 5, and the network control unit 207 continues output with mode number 5, but an image output request from a different display device 242 is generated, and the same code is output in the image output. However, if it is determined that an image output with a different compression efficiency is required (step S408), the determination of the processing load is again performed (step S409), and each reading method needs to be changed. Is determined (step S411), the network control unit 207 instructs each processing unit to switch to the mode number 7 method (step S4). 2).

  Each processing unit that has received the instruction switches the processing operation in accordance with the instruction of mode number 7 (step S413).

  When the mode number 5 and the mode number 7 are compared, in terms of processing load, the accumulation processing of the spatial filter processing image in the filter image buffer processing unit 110 is halved. It is done.

  Although it is determined that there is almost no fading in terms of the image quality 504, when the dodging process is applied to the progressive image and the encoding is performed, the dodging process is performed on the ODD and EVEN, and the progressive processing is performed in the compression processing unit. Comparing the case where the dodging process is performed, it is considered that the former is better in terms of image quality.

  Further, when an image output request from a different display device 243 is generated, and it is determined that an image output with a different compression efficiency is requested even though the image output is the same encoding method (step S408), the processing is performed again. When the load estimation is determined (step S409) and it is determined that each reading method needs to be changed (step S411), the network control unit 207 instructs each processing unit to switch to the mode number 6 method. (Step S412).

  Each processing unit that has received the instruction switches the processing operation in accordance with the instruction of mode number 6 (step S413).

  When the mode number 7 and the mode number 6 are compared, in terms of processing load, it is considered that the processing load is reduced by the absence of the storage / reading process (1004) of the input image 102 for the buffer processing unit 104.

  Even when compared in terms of image quality 504, mode number 7 is superior as the dodging processed image because the image to be synthesized and the image to be combined are shifted by about 1/30 second in time. .

  Therefore, by changing from mode number 7 to mode number 6, the processing load of the imaging apparatus 200 is reduced, and a decrease in the processing load can be assigned to different encoding processes for the display device 243.

  Further, when an image output request from a different display device is generated, or when it is determined that the network output is stopped only by changing to mode number 6 (step S411), the network control unit 207 adopts the mode number 8 method. An instruction is given to each processing unit to switch (step S412).

  Each processing unit that has received the instruction switches the processing operation in accordance with the instruction of mode number 8 (step S413).

  When the mode number 6 and the mode number 8 are compared, in terms of processing load, the accumulation processing of the spatial filter processing image in the filter image buffer processing unit 110 is halved. It is done.

  Although it is determined that there is almost no fading in terms of the image quality 504, when the dodging process is applied to the progressive image and the encoding is performed, the dodging process is performed on the ODD and EVEN, and the progressive processing is performed in the compression processing unit. When the dodging process is performed, the former is considered better in terms of image quality.

  Therefore, by changing from mode number 6 to mode number 8, the processing load of the imaging apparatus 200 is reduced, and a decrease in the processing load can be allocated to an increase in the processing amount of the imaging apparatus 200.

  As described above, also in the second embodiment, the processing load of the imaging apparatus 200 itself for the processing required from the network is determined, and each amount is set so as to cancel out the increase in the compression processing with the decrease in the buffer processing. By actively switching the reading and processing method of the processing unit, processing exceeding the processing capacity is executed to stop network output, image update stagnation due to a decrease in frame rate, occurrence of block noise, etc. Can be prevented in advance.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and a computer such as the system reads and executes the program codes from the storage medium. Is also achieved.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  In addition, the case where the functions of the above-described embodiment are realized by performing part or all of the actual processing by an OS or the like running on the computer based on the instruction of the program code read by the computer. It is.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion unit connected to the computer, the CPU or the like performs actual processing based on the instruction of the program code, and the above-described processing is performed. The case where the functions of the embodiment are realized is also included.

It is a figure showing the imaging device which concerns on the 1st Embodiment of this invention. It is a block diagram showing the internal structure of the dodging process part of the imaging device in the 1st Embodiment of this invention. It is a flowchart showing operation | movement of the display apparatus in the 1st Embodiment of this invention. It is a flowchart showing operation | movement of CPU in the network control part in the 1st Embodiment of this invention. Image sensor reading method, development processing unit output method, image reading method during dodging processing, image quality comparison after dodging processing, buffer processing unit, and filter image buffer processing unit in the first embodiment of the present invention It is a figure which shows having added the mode number to the image data accumulate | stored and all these combinations, respectively. It is a sequence chart which shows operation | movement of each process part which a network control part and a dodging control part control in mode number 1. It is a sequence chart which shows operation | movement of each process part which a network control part and a dodging control part control in mode number 7. It is a sequence chart which shows operation | movement of each process part which a network control part and a dodging control part control in mode number 2. 10 is a sequence chart showing the operation of each processing unit controlled by a network control unit and a dodging control unit in mode number 5; It is a sequence chart which shows operation | movement of each process part which a network control part and a dodging control part control in mode number 6. It is a sequence chart which shows operation | movement of each process part which a network control part and a dodging control part control in the mode number 8.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Dodge processing part 200 Imaging device 201 Imaging part 202 Lens 203 Imaging device 204 Development processing part 205 Compression processing part 206 Communication part 207 Network control part 208 Storage 209 Network interface 210 CPU
214 Imaging Method Switching Unit 240 Network 241 Display Device

Claims (7)

Imaging means having a plurality of output methods;
Image processing means;
Encoding means;
Network communication means;
Request determination means for determining a request for encoding processing of a connection partner via a network;
Processing load determination means for determining the processing load of the image processing means and the encoding means,
The captured image processing apparatus, wherein the processing load determination unit issues an instruction to the imaging unit and the image processing unit to switch the output method of the imaging unit in accordance with the determination of the request determination unit.   The captured image processing apparatus according to claim 1, wherein the processing load determination unit determines a different processing load depending on an encoding method of the encoding unit.   2. The captured image processing according to claim 1, wherein an imaging device is included in the imaging unit, and the imaging device includes a plurality of different readout methods capable of reading the same number of pixels within the same period. apparatus.   The image pickup means includes a development processing unit, and the development processing unit includes a plurality of different output methods capable of outputting the same number of pixels within the same period. Captured image processing apparatus.   The captured image processing apparatus according to claim 1, wherein the image processing unit includes a plurality of different processing units corresponding to the reading method of the image sensor and the reading method of the development process. Imaging means having a plurality of output methods, image processing means, encoding means, network communication means, request determination means for determining a request for encoding processing of a connection partner via a network, and the image processing means And a processing load determination unit that determines a processing load of the encoding unit, and a processing method in the captured image processing apparatus,
The process load determining means switching the output method of the imaging means in accordance with the determination of the request determining means;
An image processing method comprising: a step of issuing an instruction to the image pickup means and the image processing means based on this.   A program for causing a computer to execute the image processing method according to claim 6.

Images