JPH1028269A - Picture-encoding device and picture-transmission system using the same and quantization controlling method and mean through-put calculating method used for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a transmission delay time, and to attain smooth dynamic image display, even when the generated information amounts rapidly increase due to the rapid movement of a subject, or a communicating state is fluctuating. SOLUTION: Image data encoded by an encoding device 12 are temporarily stored in a temporary buffer 13 before being transmitted to a transmission buffer 171. When the information amounts of an image frame stored in the temporarily buffer 13 are larger than a preliminary set threshold value, a dropping and quantization controller 16 cancels the image data in the temporarily buffer 13, sets a quantization value with a quantization step width larger than a quantization step width used at the time of encoding the picture frame stored in the temporarily buffer 13, and outputs it to the encoding device 12.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus, and more particularly, to an apparatus for encoding and transmitting a moving image.

[0002]

2. Description of the Related Art (First Conventional Example) Conventionally, as an image compression encoding method for performing moving image communication at a low bit rate, ITU Recommendation H.264 has been proposed.
261. FIG. 261 is a block diagram illustrating the configuration of an image encoding device that implements the image encoding method recommended by H.261. Hereinafter, the configuration and operation of each section of the first conventional example will be described.

[0003] In FIG. 18, an image encoding device 90 comprises:
Image input unit 11, encoding device 12, transmission control device 1
7 and an output terminal 18. Further, the transmission control device 17 includes a transmission buffer 171 and a transmission control unit 172.
And

[0004] The image input unit 11 is constituted by a camera or the like, and inputs an image. The encoding device 12 converts the image input from the image input unit 11 into an H.264 image. The encoding is performed according to the image encoding method recommended in H.261. The transmission buffer 171 stores the image data encoded by the encoding device 12. The transmission control unit 172 transmits the encoded image data stored in the transmission buffer 171 to the image receiving device 1 via the output terminal 18 at a preset transmission rate.
000.

FIG. 19 is a block diagram of the encoding device 12 shown in FIG.
3 is a block diagram showing a more detailed configuration of FIG. 19, the encoding device 12 includes an image input terminal 100, an encoding control unit 101, a SW (switch) 102, a scan conversion unit 103, a difference unit 104, a quantization unit 105, a motion vector detection Unit 106, motion compensation unit 107, frame memory 108, inverse quantization unit 109, and addition unit 10
a, a multiplexing unit 10b, and a code output terminal 10c.

[0006] The SW 102 switches between encoding and not encoding (ie, dropping frames) the image frame input from the image input unit 11. The encoding control unit 101 controls whether the image frame input by the image input unit 11 is encoded and transmitted, or whether the image frame is dropped without encoding. And the quantization control for determining the optimum quantization value from The scan conversion unit 103 scan-converts the moving image data input from the image input terminal 100 and outputs the data as block-based data including a plurality of pixels. The motion vector detection unit 106 detects a motion vector of a data block output from the scan conversion unit 103. The frame memory 108 stores the locally decoded moving image data of the previous frame. The motion compensation unit 107 searches for and outputs a prediction value from the moving image data in the frame memory 108 based on the motion vector information output from the motion vector detection unit 106.

The difference unit 104 outputs a prediction error by calculating the difference between the data block from the scan conversion unit 103 and the prediction value from the motion compensation unit 107. The quantization unit 105 quantizes the prediction error from the difference unit 104. The inverse quantization unit 109 inversely quantizes the prediction error quantized by the quantization unit 105. The adding unit 10a calculates the prediction error dequantized by the dequantizing unit 109 and the motion compensation unit 10a.
7 to calculate the decoded video data, and store it in the frame memory 108. Multiplexing unit 10b
Multiplexes the prediction error quantized by the quantization unit 105 and the motion vector detected by the motion vector detection unit 106.

[0008] The encoding operation of the conventional image encoding apparatus configured as described above will be described below. Note that the drop control and quantization control of the encoding control unit 101 are as follows.
It will be described later.

[0009] The moving image frame input by the image input unit 11 is input to the encoding device 12. In the encoding device 12, a moving image signal input from the image input terminal 100 is scan-converted by the scan conversion unit 103, and then input to the motion vector detection unit 106 for each block composed of a plurality of pixels. . The motion vector detection unit 106
By comparing the locally decoded image data of the previous frame stored in the frame memory 108 with the block input from the scan conversion unit 103, a motion vector is obtained, and the motion vector is obtained by the motion compensation unit 107 and the multiplexing unit 10b. Output. The motion compensation unit 107 searches for a corresponding predicted value from the image data in the frame memory 108 based on the input motion vector information,
0a.

A difference unit 104 calculates a difference between a block output from the scan conversion unit 103 and a prediction value output from the motion compensation unit 107 to calculate a prediction error, and outputs the prediction error to the quantization unit 105. I do. The quantization unit 105 quantizes the prediction error with the quantization value determined by the coding control unit 101, and multiplexes the multiplexing unit 10b and the inverse quantization unit 109.
Output to The inverse quantization unit 109 inversely quantizes the prediction error and outputs the result to the adding unit 10a. The adder 10a decodes the image data by adding the inversely quantized prediction error and the predicted value output from the motion compensator 107,
The data is stored in the frame memory 108. The quantized prediction error and the motion vector are multiplexed by the multiplexing unit 10b, and transmitted to the transmission buffer 171 via the code output terminal 10c.
Is stored in The transmission control unit 172 controls the transmission buffer 17
1 at a transmission rate set in advance through the output terminal 18.
Send to 00.

Next, the frame drop control performed by the encoding control section 101 will be described. Conventionally, as a frame drop control method, there has been proposed a method of determining the necessity of frame drop based on a comparison result between the staying amount of the transmission buffer and a determination threshold (Yasuhiro Takishima et al .: "Low Bit Rate Video" Study on Image Coding Control Method ", IEICE Spring Convention D-
311, PP7-63, 1990). The encoding control unit 101 performs frame drop control using the technique disclosed in this paper.

FIG. 20 is a sequence diagram showing the operation of the conventional frame drop control system. Referring to FIG. 20, encoding control section 101 transmits transmission buffer 17 at the start of encoding.
If the staying amount of the switch 1 is smaller than a predetermined threshold value, the image frame is coded (that is, the terminals 1-2 of the SW 102 are connected). The frame is dropped without encoding (that is, the terminals 1-3 of the SW 102 are connected).
In FIG. 20, at the start of encoding of frames 2 and 4 (T1, T3), the amount of stay in the transmission buffer 171 exceeds the threshold value, so frames 2 and 4 are dropped.

Next, the quantization control performed by the encoding control unit 101 will be described. Conventionally, there has been proposed an encoding control method for determining a quantization value of an encoding target frame using an encoding rate of an image frame encoded in the past (see Japanese Patent Application Laid-Open No. 7-107482). Encoding control unit 101
Performs quantization control so as to improve image quality within a range that does not impair the reproducibility of motion by employing the encoding control method disclosed in this publication. Hereinafter, a conventional encoding control method for determining a quantization value of an encoding target frame using an encoding rate will be described.

FIG. 21 shows an example of coding control in a conventional coding control method. In FIG. 21, line 101
A to 101D are coding rate-quantization accuracy characteristics, a line 102 is a visually optimum upper limit of the coding rate, a line 103 is a visually optimum lower limit of the coding rate, and 104A is the first quantization. Accuracy, 104B is the second quantization accuracy, 104C is the third quantization accuracy, symbol ● represents a visually optimal operating point, and symbol ○ represents a transient operating point. The vertical axis represents the coding rate, and the horizontal axis represents the quantization accuracy. With respect to the horizontal axis, the quantization accuracy decreases and the spatial distortion of the displayed image increases toward the right. For the vertical axis,
The lower the rate, the lower the coding rate and the higher the probability of occurrence of dropped frames. That is, the temporal distortion of the display image increases.

Here, the coding rate is a value indicating a rate at which an image frame is coded. If all the image frames input from the image input unit 11 in FIG. The rate becomes 1.0, and when coding is performed once in two frames (one frame is dropped), the coding rate becomes 0.5.

Next, the coding rate-quantization accuracy characteristic will be briefly described. In video coding, spatial resolution (quantization accuracy) and temporal resolution (coding rate) are in a trade-off relationship, and there are some combinations. The set of combinations of the spatial resolution and the temporal resolution is referred to herein as a coding rate-quantization accuracy characteristic. The coding rate-quantization accuracy characteristics change according to the magnitude of temporal movement of an input image and spatial frequency components. The curve 101 representing the coding rate-quantization accuracy characteristic moves to the lower right on the coordinates shown in FIG. 21 when the display image includes a large movement or a fine picture, and moves to the upper left when the movement is small. .

The upper limit curve 102 is a curve obtained by connecting points indicating the upper limit of the optimum coding rate on the curve 101, and the lower limit curve 103 is connected by a point indicating the lower limit of the optimum coding rate on the curve 101. Is a curve obtained by That is, curve 1
Assuming a visually optimal area on 01, of that area,
The curve obtained by connecting the points at which the coding rate becomes maximum is the upper limit curve 102 of the optimum coding rate, and the curve obtained by connecting the points at which the coding rate becomes minimum is the lower curve 10 of the optimum coding rate.
3.

The coding control unit 101 determines that the coding rate as a result of coding with the first quantization precision 104A is the upper limit 1 of the optimum coding rate in terms of visual characteristics corresponding to the first quantization precision.
If it is larger than 02, the next encoding is performed with the second quantization precision 104B having a higher precision than the first quantization precision. This is performed at the third quantization accuracy 104C which is lower in accuracy than the accuracy. If the coding rate obtained as a result of encoding with the first quantization precision 104A is within the region, the next encoding is performed with the first quantization precision 104A. Similarly, the coding control unit 101 determines the magnitude relationship between the coding rate of the coding result obtained by coding with a certain quantization precision and the area of the coding rate that is optimal for visual characteristics according to the quantization precision. Accordingly, the quantization accuracy of the next encoding is determined.

Next, conventional coding control when a sudden movement of a subject occurs will be described with reference to FIG. In FIG. 22, lines 2231 to 2233 indicate coding rate-quantization accuracy characteristics, lines 2211 to 2214 indicate visually optimum coding rate ranges, 2221 to 2225 indicate quantization accuracy, and the symbol ● indicates visually optimum coding rate. The operating point and the symbol ○ represent a transient operating point. The vertical axis represents the coding rate,
The horizontal axis represents quantization accuracy. With respect to the horizontal axis, the quantization accuracy decreases and the spatial distortion of the displayed image increases toward the right. On the vertical axis, the lower the coding rate and the higher the probability of occurrence of dropped frames, the lower the vertical axis. That is, the temporal distortion of the display image increases.

For example, during operation at operation point A, the coding rate
Assuming that a small motion of the subject such that the quantization accuracy characteristic changes from 2231 to 2232 occurs, the frame in which the small motion of the subject occurs is encoded with the quantization accuracy 2221 and the operation point moves to the operation point B. I do. Since the coding rate at this time is between the upper limit and the lower limit of the coding rate with respect to the quantization precision 2221, the next image frame is also coded with the quantization precision 2221.

Assuming that a sudden movement of the subject such that the coding rate-quantization accuracy characteristic changes from 2231 to 2233 occurs, the frame in which the sudden movement of the subject occurs is encoded with the quantization accuracy 2221. , The operating point moves to the operating point C. The coding rate at this time is the quantization precision 222
Since the coding rate falls below the lower limit of the coding rate for 1, the next image frame is coded with the quantization accuracy 2222, and the operating point moves to the operating point D. Since the coding rate of the image frame coded with the quantization precision 2222 is lower than the lower limit of the coding rate with respect to the quantization precision 2222, the next image frame is coded with the quantization precision 2223 and the operating point is Move to point E. Thereafter, the same operation is repeated until the coding rate falls within the optimal region in terms of visual characteristics (until the quantization accuracy reaches 2225).

The quantization precision in FIGS. 21 and 22 corresponds to the size of the quantization step width. That is, when the quantization accuracy is good (high), the quantization step width is small, and when the quantization accuracy is bad (low), the quantization step width is large.

(Second Conventional Example) The configuration and operation of a conventional image coding apparatus for changing the image data length and error correction code mode according to the communication state will be described below. Conventionally, when the communication condition is good, the image data length is lengthened and an error correction code with low error correction capability is added, and when the communication condition deteriorates, the image data length is shortened and error correction is performed. Such as adding a high-performance error correction code,
A high-efficiency encoding device for television signals has been proposed (see Japanese Patent Application Laid-Open No. 3-78383). FIG. 23 is a block diagram showing a configuration of a television signal high-efficiency encoding apparatus disclosed in this publication. In FIG. 23, the high-efficiency coding apparatus 130 includes a low-pass filter 131, a sub-sampling circuit 132, a predictive coding circuit 133, a buffer memory 134, a terminal control circuit 137, a variable-length coding circuit 135, And a correction encoding circuit 136.

The operation of the high-efficiency television signal encoding apparatus configured as shown in FIG. 23 will be described below. The output of the variable length coding circuit 135 is input to the error correction coding circuit 136. The error correction coding circuit 136 adds an error correction code to the variable-length coded image data. The error correction code length is switched by a normal / fallback mode selection signal. That is, the error correction code is
In the normal mode, the length is shortened (that is, an error correction code with a low error correction capability), and in the fallback mode, the length is lengthened (that is, an error correction code with a high error correction capability). Further, the prediction encoding circuit 133
In the normal mode, encoding is performed to increase the image data length (that is, quantization is performed using a quantization value having a small quantization step width), and in the fallback mode, encoding is performed so that the image data length is reduced (that is, quantization is performed). (Quantization is performed with a quantization value having a large step width.) When the communication state deteriorates, the operation mode is switched from the normal mode to the fallback mode. In the fallback mode, since the image data is shorter than in the normal mode, the error correction code can be lengthened accordingly. As a result, the error rate can be kept low without extending the entire code length (image data + error correction code).

(Third Conventional Example) Conventionally, an image coding apparatus that divides a coding area and performs hierarchical coding is as follows. That is, a priority area and a non-priority area are determined in advance for an encoding area of an image captured from a camera. The image encoding unit performs encoding by setting the quantization precision of the priority area to be higher than that of the non-priority area.

[0026]

[Problems to be solved by the invention]

(Problem According to First Conventional Example) The image coding apparatus according to the first conventional example described above has four problems as described below.

[First Problem] In an image coding apparatus according to a first conventional example, a coding rate is calculated based on a generated information amount and a transmission rate coded one frame before, and the calculated code is calculated. Because the quantization value of the encoding target frame is determined using the encoding rate, when the subject suddenly moves, encoding is performed with an inappropriate quantization value that does not match the encoding target frame. May be threatened. As a result, there is a problem that the amount of generated information increases rapidly, the delay time until the information is displayed on the receiving side increases, and a moving image with awkward motion is displayed on the receiving side. Further, as the amount of generated information increases, significant frame dropout occurs, and the inter-frame correlation decreases. As a result, there is also a problem that a vicious circle occurs in which the amount of generated information also increases in the next image frame to be encoded.

The relationship between the delay time and the skipping of frames by increasing the amount of generated information will be described with reference to FIG. FIG. 24 shows delay time and frame skipping in a conventional image encoding device. Before the point P1 at which the sudden movement of the subject has occurred, the movement of the subject is small, so that the prediction error between frames is small. As a result, encoding is performed with a quantization value having a small quantization step width, and the delay time is 4
00 msec or less. Also, at point P1 where a sudden movement of the subject has occurred, the encoding is performed with a quantization value having a small quantization step width, despite the fact that the prediction error between the frames has increased. 1
275 msec, and frame skipping is 42 frames. In the frame encoded after the point P1, the frame skip is 42 frames, so that the inter-frame correlation is low, the prediction error between the frames is large, the amount of generated information is increased, and the delay is increased. Time is increasing.

[Second Problem] In the image coding apparatus according to the first conventional example, when the subject suddenly moves, the operating point moves in a stepwise manner. Until a quantization value is selected, encoding is performed with an inappropriate quantization value that does not match the encoding target frame. As a result, until a quantization value that matches the encoding target frame is selected, a moving image with awkward motion or a deteriorated image quality is displayed.

In order to solve the above first and second problems, the following three devices or circuits have been conventionally proposed. (1) A video signal encoding device that predicts the amount of generated information based on the sum of the absolute differences between frames or the dynamic range to determine a quantization value according to the characteristics of the encoding target frame (Japanese Patent Laid-Open Publication No. No. 5-336513). (2) The magnitude of the motion is determined based on the magnitude of the motion vector. If the motion vector is large, coding is performed using a quantization value having a large quantization step width, and if the motion vector is small, a quantization value having a small quantization step width is used. Buffer memory control circuit of an image encoding device that performs encoding in
0587). (3) An image coding apparatus that has a plurality of quantization units and performs quantization by changing quantization values to determine a visually good quantization value (see Japanese Patent Application Laid-Open No. 4-323962).

However, in the above three devices or circuits, the following new problems are pointed out. That is, in the video signal encoding device of the above (1), prediction is often missed, and if the prediction is missed, the same problem as pointed out in the first problem occurs. Further, in the buffer memory control circuit of the above (2), the quantization value is determined according to the magnitude of the motion vector. And a large motion vector does not necessarily increase the amount of generated information. Therefore, even when the motion vector is large but the amount of generated information is small, encoding is performed with a quantization value having a large quantization step width, and image quality is unnecessarily reduced. Further, in the image coding device of (3), since a plurality of quantization units must be provided, the circuit scale becomes large and the cost increases.

[Third Problem] In the image coding apparatus according to the first conventional example, the coding rate is calculated using the same transmission rate regardless of whether the communication state is good or bad. Therefore, encoding may be performed with an inappropriate quantization value that does not match the communication state. An equation for calculating the coding rate in the conventional quantization control is shown below. Coding rate = (LO / S) / RO where: LO: Transmission speed (constant) S: Number of frame samplings per second RO: Generated information amount of frame coded before (one frame before) It is. As can be seen from the above equation, when a transmission error occurs due to the deterioration of the communication state, the image coding apparatus on the transmission side retransmits, and the transmission throughput fluctuates, an inappropriate quantization that does not match the communication state is performed. The value is determined.

In order to solve the third problem, the present applicant sets the transmission rate L0 used to calculate the coding rate of the n-th frame as the transmission rate L0 from the time when the coding of the n-th frame is started. Japanese Patent Application No. 6-1040 discloses an encoding control method for correcting using an average communication speed until encoding is started.
No. 07.

However, a study of technology for preventing video coding quality deterioration in mobile communication (Hisashi Ibaraki et al., Journal of the Institute of Image Electronics Engineers of Japan, Vol. 23, No. 5, (1994) P445-
According to P453), in the frequency band of a so-called PHS (Personal Handy Phone System), the Doppler frequency is 5 Hz at a human walking speed (3 Km / h), and the number of times a transmission error degrades by one digit is 3. 5 times /
It describes that the second and the average duration are 20 msec. In other words, it is necessary to measure an average over a time width of at least 300 msec in order to catch burst errors occurring in the PHS on average. On the other hand, when transmitting a moving image at a low bit rate, the number of image frames deteriorates from 30 frames / sec, but even if it deteriorates to 5 frames / sec, the next frame is encoded after encoding the nth frame. The time required to perform the coding is 200 msec. If the coding rate is corrected using the average communication speed during that time, a problem occurs in which the burst error is dominant and the correction is extremely difficult.

Further, in the same coding control method, the frame to be coded is determined by the coding rate of the previously coded frame. In this case, there is no problem if the communication speed fluctuates slowly. If the communication speed drops abruptly, the coding rate cannot be corrected in time, and in fact the next frame is coded before the data in the transmission buffer decreases, resulting in a buffer overflow. there were.

In this case, a retransmission control method based on Go-Back-N is assumed, and as a method for calculating the average throughput, the transmission confirmation is performed from the time when the encoding of the n-th frame is started until the next encoding is started. It was calculated from the number of bits of the removed frame. However, Go-Back-
The retransmission control method by N is, for example, after transmitting 8 frames, waits for an acknowledgment or a retransmission request from the receiving side. In the case of a retransmission request, all frames after the requested frame are retransmitted until the acknowledgment is received. repeat.

In the retransmission control method based on Go-Back-N, when the delay time of the transmission path is long, the time to wait for acknowledgment increases and the transmission efficiency is extremely deteriorated. Therefore, conventionally, a retransmission control method using Selective Reject has been devised. Selective R
In the case of the retransmission control method using eject, the transmitting side transmits, for example, up to 32 frames without waiting for acknowledgment of transmission, and the receiving side selectively returns a reject for a frame in which an error is detected. Upon receiving the rejection, the transmitting side immediately retransmits the frame. Selective R
In the case of the retransmission control method by eject, no acknowledgment is performed.

[Fourth Problem] In the quantization control method performed by the coding control unit 101 of the image coding apparatus according to the first conventional example, the motion of a displayed image is too large or a large number of transmission errors occur. For example, when the transmission throughput is significantly reduced due to, the spatial resolution (quantization accuracy) and the temporal resolution (coding rate) both exceed the permissible limits, and May be unacceptably deteriorated. The coding rate-quantization accuracy characteristic is determined by the degree of non-redundancy, such as the amount of motion and the fineness of a picture, which cannot be compressed in a given image, and the transmission speed at which coded data is transmitted. These factors are given and are not selectable. Considering only the selectable quantization precision as a parameter, it is optimal to operate in a region between the upper curve 102 and the lower curve 103 under given conditions.
However, the optimum here is only relatively best under given conditions, and is not absolutely good. Absolutely, the quantization accuracy and the coding rate improve as the area between the upper curve and the lower curve moves to the upper left, and both the quantization precision and the coding rate decrease as the area moves to the lower right.

(Problem According to Second Conventional Example) A high efficiency television signal encoding apparatus according to a second conventional example determines a normal mode and a fallback mode in accordance with an instantaneous communication state, Length and error correction code length are controlled. Therefore, when the normal / fallback mode selection signal is received, the communication state is good. When the normal / fallback mode selection signal indicating the normal mode is received, the communication state deteriorates at the time of actual transmission. Causes the following problems. That is, in this case, since the normal mode is selected,
Encoding is performed to increase the image data length, and a short (low error correction capability) error correction code is added. Nevertheless, since the communication state is degraded, a transmission error exceeding the error correction capability occurs, and decoding on the receiving side becomes impossible.

Conversely, when the normal / fallback mode selection signal is received, the communication state is poor, and the normal / fallback mode selection signal indicating the fallback mode is received. When it is improved, the following problems occur. That is, in this case, since the fallback mode is selected, encoding is performed so that the image data length is shortened, and a long (high error correction capability) error correction code is added. For this reason, the image displayed on the receiving side is more deteriorated than in the normal mode.

(Problem According to Third Conventional Example) In the image encoding apparatus according to the third conventional example, the priority area and the non-priority area
It was predetermined, and the size of the area was constant and unchanged. When the amount of generated information increases due to the movement of a subject and the communication throughput decreases due to the deterioration of the state of the communication channel, the quantization accuracy must be reduced by encoding control in order to reduce the amount of generated information. At this time, if the size of the priority area is constant, if the amount of generated information increases extremely or the communication throughput decreases, the quantization accuracy of the priority area as well as the non-priority area greatly deteriorates. Inevitably, as a result, the image quality of the priority area is greatly deteriorated.

Therefore, an object of the present invention is to suppress the delay time even when the amount of generated information sharply increases due to a sudden movement of a subject and the transmission throughput fluctuates, and to smooth the movement on the receiving side. It is an object of the present invention to provide an image encoding device capable of displaying a moving image and an image transmission system using the same. Another object of the present invention is to provide an image encoding device capable of selecting an optimal quantization control and an error correction code according to a communication state, and an image transmission system using the same. Still another object of the present invention is to provide an image code capable of immediately determining, by a simple calculation, an optimal quantization value in terms of visual characteristics for an image frame after a sudden increase in the amount of generated information (a frame in which the movement of a subject is large). And an image transmission system using the same. Still another object of the present invention is to control the distribution of the priority area and the non-priority area and the quantization value according to the amount of generated information and the coding rate (movement of the subject). It is an object of the present invention to provide an image encoding device and a quantization control method that determine a quantization value and can always maintain stable image quality.

[0043]

Means for Solving the Problems and Effects of the Invention In order to achieve the above object, the present invention has the following features. A first aspect is an image encoding apparatus for encoding and transmitting a moving image, comprising: an image input unit configured to output an image frame from a video signal; and an image encoding unit configured to encode the image frame configured by the image input unit. Encoding means, a transmission buffer for storing image data of an image frame encoded by the encoding means, a transmission control means for transmitting image data in the transmission buffer at a predetermined transmission rate, and an encoding means. A temporary buffer that is disposed between the transmission buffer and temporarily stores image data of the image frame encoded by the encoding unit; and determines whether to store the image data in the temporary buffer in the transmission buffer. And a frame removal / quantization control means for determining a quantization value of an image frame to be encoded next, wherein the frame removal / quantization control means includes a temporary buffer. When the information amount of the image frame stored in the buffer is larger than the preset frame drop threshold, the frame data is not stored in the transmission buffer, and the frame data is stored in the temporary buffer. A quantization value of a quantization step width larger than the quantization step width used when encoding the image frame being set is notified to the encoding unit, and the encoding unit transmits the image to be encoded next. It is characterized in that the frame is encoded with the quantization value notified from the frame drop / quantization control means.

As described above, according to the first mode, when the subject suddenly moves, the frame of the image frame in which the amount of generated information sharply increases is removed, and the next image frame is removed. Is encoded with a large quantization step width, so that the amount of generated information can be suppressed,
Delay time can be reduced. Therefore, on the receiving side, it is possible to display a moving image with smooth motion.

According to a second aspect, in the first aspect, the transmission control means further performs automatic retransmission when a communication error occurs and image data cannot be normally decoded on the receiving terminal side. When the error information indicating that retransmission has occurred is notified to the frame drop / quantization control means, and the frame drop / quantization control means further receives the error information notification from the transmission control means, the frame drop / quantization control means If the frame drop threshold for determining whether to store the data in the transmission buffer is lowered from a preset value, and if a notification of error information is not received from the transmission control unit for a predetermined fixed time, the relevant It is characterized in that threshold control is performed to return the frame drop threshold to a preset value.

As described above, according to the second aspect, until the communication state is reduced to a favorable state, the frame drop threshold is lowered to perform the frame drop control. Frame drop control can be performed.

According to a third aspect, in the first aspect, the transmission control means further calculates an average throughput within a predetermined period of time and notifies the frame loss / quantization control means of the result. The quantization control unit further calculates the maximum amount of information that can be transmitted within the maximum allowable delay time based on the preset maximum allowable delay time and the average throughput calculated by the transmission control unit, and calculates the maximum information amount. The obtained maximum information amount is used as a frame drop threshold.

As described above, according to the third aspect, the amount of information that can be transmitted within the maximum allowable delay time is calculated from the preset maximum allowable delay time and the average throughput,
Since the calculated information amount is used as a threshold value for determining whether or not to drop a frame of an image frame in the temporary buffer, the dropping control can be performed according to the communication state.

According to a fourth aspect, in the third aspect, the frame removal / quantization control means further includes an average delay information based on an average amount of generated information of image frames transmitted within a predetermined time and an average throughput. The time is calculated, and the calculated average delay time is used as the maximum allowable delay time.

As described above, according to the fourth aspect, the average delay time is calculated from the average amount of information generated and the average throughput of image frames transmitted within a predetermined time, and the information that can be transmitted within the average delay time is calculated. Since the amount is calculated and the calculated information amount is used as a threshold value for the frame drop control, the frame drop control according to the moving image characteristics and the communication state can be performed.

A fifth aspect is an image coding apparatus for coding and transmitting a moving image, comprising: an image input means for forming and outputting an image frame from a video signal; and an image frame comprising the image input means. , A transmission buffer for storing image data coded by the coding unit, and a first quantization control unit for determining an optimum quantization value in terms of both reproducibility of motion and image quality A second quantization control means for controlling a quantization step width according to a communication state; transmitting image data in a transmission buffer at a predetermined transmission rate; a communication error occurs; Transmission control means for performing automatic retransmission when the image data cannot be decoded and notifying error information indicating that retransmission has occurred to the second quantization control means; Means are transmission system Receiving the error information from the means, sets a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means, If the error information is not received from the transmission control means for a predetermined period of time, the quantization value determined by the first quantization control means is notified to the encoding means as it is, Is
The image frame to be coded next is coded with the quantization value notified from the second quantization control means.

As described above, according to the fifth aspect, encoding is performed by increasing the quantization step width until the communication state is reduced to a good state. Also, the delay time can be reduced, and a moving image with smooth movement can be displayed on the receiving side. Therefore, quantization control that is not affected by fluctuations in the communication state can be performed.

According to a sixth aspect, in the fifth aspect, there is provided a temporary buffer which is disposed between the encoding means and the transmission buffer and temporarily stores image data of an image frame encoded by the encoding means. Transmission control means for determining whether or not to store image data in a temporary buffer in a transmission buffer, and determining a quantization value of an image frame to be encoded next, Notifies the frame drop / quantization control unit of error information indicating that retransmission has occurred, and upon receiving the error information notification from the transmission control unit, the frame drop / quantization control unit The frame drop threshold for determining whether to store the image data in the transmission buffer is lowered from a preset value, and a predetermined fixed time,
If no error information notification is received from the transmission control means, the frame drop threshold is returned to a preset value, threshold control is performed, and the information amount of image data in the temporary buffer is controlled by the threshold control. If the value is larger than the controlled frame drop threshold, the frame drop control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the image frame used in encoding the image frame stored in the temporary buffer is used. It is characterized in that a quantization value with a quantization step width larger than the quantization step width is set and notified to the encoding means.

As described above, according to the sixth aspect, the frame-drop control is performed by lowering the frame-drop threshold until the communication state is reduced to a good state. Frame drop control can be performed.

According to a seventh aspect, in the sixth aspect, the transmission control means further calculates an average throughput within a predetermined period of time, notifies the result to the frame drop / quantization control means, and outputs the frame drop / quantity. The quantization control unit further calculates the maximum amount of information that can be transmitted within the maximum allowable delay time based on the preset maximum allowable delay time and the average throughput calculated by the transmission control unit, and calculates the maximum information amount. The obtained maximum information amount is used as a frame drop threshold.

As described above, according to the seventh aspect, the amount of information that can be transmitted within the maximum allowable delay time is calculated from the preset maximum allowable delay time and the average throughput.
Since the calculated information amount is used as a threshold value for determining whether or not to drop a frame of an image frame in the temporary buffer, the dropping control can be performed according to the communication state.

According to an eighth aspect, in the seventh aspect, in the seventh aspect, the frame drop / quantization control means further performs an average delay based on an average generated information amount of image frames transmitted within a predetermined time and an average throughput. The time is calculated, and the calculated average delay time is used as the maximum allowable delay time.

As described above, according to the eighth aspect, the average delay time is calculated from the average amount of information generated and the average throughput of image frames transmitted within a predetermined time, and the information that can be transmitted within the average delay time is calculated. Since the amount is calculated and the calculated information amount is used as a threshold value for the frame drop control, the frame drop control according to the moving image characteristics and the communication state can be performed.

The ninth aspect is that, when a moving image is subjected to high-efficiency compression encoding and transmitted via a low bit rate communication line,
In an image coding apparatus in which an operating point is predetermined as a balance between temporal distortion due to frame dropping according to a coding rate which is a rate of a frame that can be transmitted per unit time and spatial distortion due to a quantization value. An image input unit configured to output an image frame from a video signal; an encoding unit configured to encode the image frame configured by the image input unit; and image data encoded by the encoding unit. A transmission buffer; a quantization control unit for determining a quantization value according to a communication state; and a transmission control unit for transmitting image data in the transmission buffer at a predetermined transmission rate. The calculated average throughput within a given period of time is calculated, and a code is generated based on the calculated average throughput and the amount of information generated per frame as a result of encoding by the encoding unit. Calculating the rate, determining a quantization value corresponding to the calculated coding rate, and notifying the coding means, and when the remaining amount of data in the transmission buffer falls below a certain threshold, the coding means is notified to the coding means. It is characterized in that it is notified to perform encoding of an image frame.

As described above, according to the ninth aspect, the coding rate is calculated using the average throughput within a certain time,
Since the quantization value of the next frame is determined using the coding rate, quantization control according to the communication state can be performed.

According to a tenth aspect, in the ninth aspect, there is provided a temporary buffer which is disposed between the encoding means and the transmission buffer and temporarily stores image data of an image frame encoded by the encoding means. Determine whether to store the image data in the temporary buffer in the transmission buffer,
The image processing apparatus further includes a frame removal / quantization control unit that determines a quantization value of an image frame to be encoded next, wherein the frame removal / quantization control unit includes a preset maximum allowable delay time and a quantization control. The amount of information that can be transmitted within the maximum allowable delay time is calculated based on the average throughput calculated by the means, and the amount of information of the image frame stored in the temporary buffer is information that can be transmitted within the maximum allowable delay time. If it is larger than the amount, while performing frame dropping control not storing the image data in the temporary buffer in the transmission buffer,
It is characterized in that a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer is set and notified to the encoding means.

As described above, according to the tenth aspect, the information amount that can be transmitted within the maximum allowable delay time is calculated from the preset maximum allowable delay time and the average throughput, and the calculated information amount is calculated. Is used as a threshold value for determining whether or not to drop a frame of an image frame in the temporary buffer, so that the dropping control can be performed according to the communication state.

According to an eleventh aspect, in the ninth aspect, there is provided a temporary buffer disposed between the encoding means and the transmission buffer for temporarily storing image data of the image frame encoded by the encoding means. Determine whether to store the image data in the temporary buffer in the transmission buffer,
The image processing apparatus further includes a frame removal / quantization control unit for determining a quantization value of an image frame to be encoded next, wherein the frame removal / quantization control unit includes an average amount of generated information of the image frame transmitted within a predetermined time. Calculating an average delay time based on the average throughput calculated by the quantization control means, calculating an amount of information that can be transmitted within the average delay time based on the average throughput and the average delay time, If the amount of information of the image frames stored in the buffer is larger than the amount of information that can be transmitted within the average delay time, frame drop control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the frame is stored in the temporary buffer. Set a quantization value with a quantization step width larger than the quantization step width used when encoding the stored image frame, and And notifies the means.

As described above, according to the eleventh aspect, the average delay time is calculated from the average amount of information generated and the average throughput of the image frames transmitted within the predetermined time,
The amount of information that can be transmitted within the average delay time is calculated, and the calculated amount of information is used as a threshold value for frame drop control. It can be performed.

A twelfth aspect is the ninth aspect, wherein the quantization control means varies the averaging time in accordance with the presence or absence of a burst error when calculating the average throughput. .

As described above, according to the twelfth aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where the error occurs, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the quantization control and the dropout control which do not vary even when the burst error occurs.

A thirteenth aspect is characterized in that, in the ninth aspect, the predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the thirteenth aspect, by setting the average obtaining time to 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs. .

A fourteenth aspect is an image encoding apparatus for encoding and transmitting a moving image, comprising: an image input means for constructing and outputting an image frame from a video signal; and an image frame constituted by the image input means. Encoding means for encoding, a transmission buffer for storing image data of the image frame encoded by the encoding means, a quantization control means for determining a quantization value used for encoding, and a transmission buffer. Transmission control means for transmitting the image data at a predetermined transmission rate, and motion threshold value storage means for storing a motion threshold value for determining whether the motion of the subject is large or small; Maximum allowable delay time storage means for storing the maximum allowable delay time, which is the limit of the transmission delay, wherein the quantization control means generates information per frame as a result of encoding by the encoding means. , It is greater than the movement threshold,
Calculate the quantization ratio, which is the ratio between the quantization value used to encode an image frame that exceeds the motion threshold and the quantization value for which the amount of generated information is to be predicted. From the amount of generated information of the frame and the quantization ratio, the amount of information generated when the quantization value is changed is predicted, and the estimated amount of generated information, the transmission rate, and the maximum allowable delay time, within the maximum allowable delay time, The quantizing value which can be transmitted to the encoding unit is determined and notified to the encoding unit, and the encoding unit encodes the image frame with the quantization value notified from the quantization control unit.

As described above, according to the fourteenth aspect, when the motion of the subject is large, the characteristic that the amount of generated information of the image frame is almost inversely proportional to the quantization value is used, and the amount of generated information is Since the ratio of quantized values is used, the amount of generated information can be predicted by simple calculation, and quantized values that can be transmitted within the maximum allowable delay time can be determined immediately after a large movement of the subject. it can.

According to a fifteenth aspect, in the fourteenth aspect,
The transmission control unit further calculates an average throughput within a predetermined period of time and notifies the quantization control unit,
When the amount of information generated per frame as a result of encoding by the encoding unit is larger than the motion threshold, the quantization control unit is used to encode an image frame exceeding the motion threshold. Calculate the quantization ratio, which is the ratio between the quantized value and the quantized value for which the amount of generated information is to be predicted.
The amount of information generated when the quantization value is changed is predicted, and the quantized value that can be transmitted within the maximum allowable delay time is determined from the predicted information amount, the transmission rate, the average throughput, and the maximum allowable delay time. And notifies the encoding means.

As described above, according to the fifteenth aspect, the average throughput is used to determine the quantization value when the motion of the subject is large. It can be performed.

According to a sixteenth aspect, in the fifteenth aspect,
When calculating the average throughput, the transmission control means varies the averaging time in accordance with the presence or absence of a burst error.

As described above, according to the sixteenth aspect, if the time for obtaining the average is shortened, it is possible to quickly determine the quantization value in the case where the movement of the subject is large, in response to the change in throughput. Also, in a situation where a burst-like error occurs, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to determine a quantization value that does not fluctuate even when a burst error occurs. .

According to a seventeenth aspect, in the fifteenth aspect,
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the seventeenth aspect, by setting the time for obtaining the average to 300 msec or more, the determination of the quantization value in the case of a large subject which does not fluctuate even when a burst error occurs. It can be carried out.

An eighteenth aspect is an image encoding apparatus for encoding and transmitting a moving image, comprising: an image input means for constructing and outputting an image frame from a video signal; and an image frame constituted by the image input means. Encoding means for encoding, a transmission buffer for storing image data of the image frame encoded by the encoding means, a quantization control means for determining a quantization value used for encoding, and a transmission buffer. Transmission control means for transmitting the image data at a predetermined transmission rate, motion threshold storage means for storing a motion threshold value for determining whether the motion of the subject is large or small, and transmission data per unit time. An ideal curve whose operating point is predetermined is stored as a balance between temporal distortion caused by dropping frames corresponding to a coding rate which is a frame rate and spatial distortion caused by a quantization value. And a curve storage unit. When the amount of information generated per frame as a result of encoding by the encoding unit is larger than the motion threshold, the quantization control unit Calculate the quantization ratio which is the ratio between the quantization value used for encoding and the quantization value for which the amount of generated information is to be predicted, and calculate the amount of generated information and the quantization ratio of the image frame exceeding the motion threshold. , The amount of information generated when the quantization value is changed is predicted, the coding rate is calculated from the predicted amount of generated information and the transmission rate, and the quantization is calculated from the calculated coding rate and the ideal curve. The quantization value is determined and notified to the encoding unit, and the encoding unit encodes the image frame with the quantization value notified from the quantization control unit.

As described above, according to the eighteenth aspect, similarly to the fourteenth aspect, the amount of generated information can be predicted by a simple calculation, and visual information can be obtained from the ideal curve and the predicted amount of generated information. Due to the characteristics, the optimum quantization value can be determined immediately after a large movement of the subject.

A nineteenth aspect is the eighteenth aspect,
The transmission control unit further calculates an average throughput within a predetermined period of time and notifies the quantization control unit,
When the amount of information generated per frame as a result of encoding by the encoding unit is larger than the motion threshold, the quantization control unit is used to encode an image frame exceeding the motion threshold. Calculate the quantization ratio, which is the ratio between the quantized value and the quantized value for which the amount of generated information is to be predicted.
The amount of information generated when the quantization value is changed is predicted, the coding rate is calculated from the predicted generated information amount, the transmission rate, and the average throughput, and the calculated coding rate and the ideal curve are calculated. It is characterized in that the quantization value is determined and notified to the encoding means.

As described above, according to the nineteenth aspect, by using the average throughput and the ideal curve to determine the quantization value when the motion of the subject is large, the quantization value considering the fluctuation of the throughput can be obtained. Can be determined.

According to a twentieth aspect, in the nineteenth aspect,
When calculating the average throughput, the transmission control means varies the averaging time in accordance with the presence or absence of a burst error.

As described above, according to the twentieth aspect, if the time for obtaining the average is shortened, it is possible to quickly determine the quantization value in the case where the movement of the subject is large, in response to the change in throughput. Also, in a situation where a burst-like error occurs, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to determine a quantization value that does not fluctuate even when a burst error occurs. .

The twenty-first aspect is the nineteenth aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the twenty-first aspect, by setting the time for obtaining the average to 300 msec or more, the quantization value of the case in which the movement of the subject is large, which does not fluctuate even when a burst error occurs. A decision can be made.

A twenty-second aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, and when the calculated error rate exceeds a predetermined threshold, transmits an error signal to the image encoding apparatus, and Includes an image input unit configured to output an image frame from a video signal, an encoding unit configured to encode the image frame configured by the image input unit, and a transmission unit configured to store the image data encoded by the encoding unit. A buffer, transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate, first quantization control means for determining an optimal quantization value in terms of both motion reproducibility and image quality, Error correction coding means for adding error correction codes having different error correction capabilities to the image data encoded in the stage, and a quantization value according to the communication state is determined, and the error correction coding means is determined according to the error signal. And a second quantization control unit for controlling the second quantization control unit. When the second quantization control unit receives the error signal from the image receiving device, the second quantization control unit corresponds to the quantization value determined by the first quantization control unit. A quantization signal having a quantization step width larger than the quantization step width is set, and a selection signal for notifying the encoding means and instructing the error correction encoding means to add an error correction code having a high error correction capability is provided. If the error signal is not received for a predetermined period of time, the quantization value determined by the first quantization control unit is notified to the encoding unit as it is, and the error correction encoding unit is notified of the error. correction The encoding means outputs a selection signal instructing to add a low error correction code, the encoding means encodes the image frame with the quantization value notified from the second quantization control means, and the error correction encoding means , An error correction code specified by the selection signal is added to the image data encoded by the encoding means.

As described above, according to the twenty-second aspect, until the communication state is reduced to a good state, the coding is performed by increasing the quantization step width, and the coding is further performed using the error correction code having a high error correction capability. Because the communication state is good, even though the communication state is good, a quantization value with a large quantization step width, encoding with an error correction code having a high error correction capability, or conversely, despite the bad communication state In addition, there is no problem such as encoding with a quantization value having a small quantization step width and an error correction code having a low error correction capability. Accordingly, encoding control according to the communication state can be performed.

A twenty-third aspect is a method according to the twenty-second aspect, wherein
The image receiving apparatus calculates an average of the error rates calculated within a predetermined period of time, and transmits an error signal to the image encoding apparatus when the calculated average error rate exceeds a predetermined threshold. It is characterized by the following.

As described above, according to the twenty-third aspect, the image receiving apparatus calculates the average of the error rate, and if the average error rate exceeds a preset threshold value, the error signal is converted to the image code. By transmitting the image data to the encoding device, the image encoding device can perform encoding control that is not affected by the temporary good / bad communication state.

According to a twenty-fourth aspect, in the twenty-second or twenty-third aspect, the image encoding device is disposed between the encoding means and the transmission buffer, and converts the image data of the image frame encoded by the encoding means. A temporary buffer for temporarily storing the image data, and a frame removal / quantization control unit for determining whether to store the image data in the temporary buffer in the transmission buffer and determining a quantization value of an image frame to be encoded next When receiving an error signal from the image receiving device, the frame drop / quantization control means sets a frame drop threshold for determining whether to store image data in the temporary buffer in the transmission buffer. The threshold value is reduced from a preset value, and if no error signal notification is received for a predetermined fixed time, the frame drop threshold is returned to a preset value. If the information amount of the image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, the frame buffer control that does not store the image data in the temporary buffer in the transmission buffer is performed. The present invention is characterized in that a quantization value having a quantization step width larger than the quantization step width used when encoding an image frame stored in the temporary buffer is set and notified to the encoding means.

As described above, according to the twenty-fourth aspect, the frame dropping control is performed by lowering the frame drop threshold until the communication state is reduced to a good state. Frame drop control can be performed.

A twenty-fifth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, transmits the calculated error rate to the image encoding apparatus, and the image encoding apparatus constructs an image frame from the video signal and outputs the image frame. Encoding means for encoding an image frame constituted by the input means, a transmission buffer for storing the image data encoded by the encoding means, and transmission control for transmitting the image data in the transmission buffer at a predetermined transmission rate Means, and first quantization control means for determining an optimal quantization value in terms of both reproducibility of motion and image quality;
Error correction coding means for adding error correction codes having different error correction capabilities to the image data coded by the coding means, and a quantization value determined according to a communication state, and an error correction code determined according to an error rate. Second quantization control means for controlling the quantization means, wherein the second quantization control means, when the error rate received from the image receiving device exceeds a predetermined threshold value, Set a quantization value of a quantization step width larger than the quantization step width corresponding to the quantization value determined by the quantization control means, and notify the encoding means,
A selection signal for instructing the error correction coding means to add an error correction code having a high error correction capability is output. If the error rate does not exceed a threshold for a predetermined time, the first quantization control is performed. Notifying the encoding means as it is of the quantized value determined by the means, and outputting a selection signal instructing the error correction encoding means to add an error correction code having a low error correction capability, the encoding means Encoding the image frame with the quantization value notified from the second quantization control means,
The error correction encoding means adds the error correction code specified by the selection signal to the image data encoded by the encoding means.

As described above, according to the twenty-fifth aspect, the good / bad communication state depends on the error rate from the image receiving apparatus.
Deterioration is judged, and until the communication condition is settled to a good condition.
Encoding is performed with a large quantization step width, and encoding is performed using an error correction code having a high error correction capability. Therefore, encoding control can be performed according to a communication state.

[0093] A twenty-sixth aspect is a method according to the twenty-fifth aspect, wherein
The second quantization control means calculates an average of the error rates received from the image receiving device within a predetermined time, and when the calculated average error rate exceeds a predetermined threshold value, A quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means is set and notified to the encoding means,
A selection signal for instructing the error correction coding means to add an error correction code having a high error correction capability is output. If the average error rate does not exceed a threshold for a predetermined period of time, the first quantization is performed. The quantization value determined by the control means is notified to the encoding means as it is, and a selection signal for instructing the error correction encoding means to add an error correction code having a low error correction capability is output.

As described above, according to the twenty-sixth aspect, the average of the error rate from the image receiving apparatus is calculated by the image coding apparatus, and the determination of good / bad communication state is made based on the calculated average error rate. Until the communication state is set to a good state, encoding is performed by increasing the quantization step width, and encoding is performed with an error correction code having a high error correction capability. Control can be performed.

The twenty-seventh aspect is the twenty-sixth aspect,
The image encoding device is disposed between the encoding unit and the transmission buffer, and temporarily transmits the image data of the image frame encoded by the encoding unit, and transmits the image data in the temporary buffer. A frame drop / quantization control unit that determines whether to store the image data in the buffer and determines a quantization value of an image frame to be encoded next; When the average error rate calculated by the quantization control means exceeds a predetermined threshold, a frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is set in advance. If the average error rate does not exceed the threshold for a predetermined period of time,
The threshold control is performed to return the frame drop threshold to a preset value. If the information amount of the image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, the temporary buffer is temporarily controlled. Performs frame drop control so that image data in the buffer is not stored in the transmission buffer, and performs quantization with a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer. It is characterized in that a value is set and notified to the encoding means.

As described above, according to the twenty-seventh aspect, the average of the error rate from the image receiving device is calculated, and the communication state is determined to be good or bad based on the calculated average error rate. Until the communication state is reduced to a good state, the frame-drop control is performed by lowering the frame-drop threshold, so that the frame-drop control can be performed in consideration of the communication state.

A twenty-eighth aspect is a method according to the twenty-fifth aspect,
The image encoding device is disposed between the encoding unit and the transmission buffer, and temporarily transmits image data of the image frame encoded by the encoding unit, and transmits the image data in the temporary buffer. A frame removal / quantization control unit for determining whether to store the image data in the buffer and determining a quantization value of an image frame to be encoded next; When the error rate received from the receiver exceeds a predetermined threshold, the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is reduced from the predetermined value. , A predetermined period of time,
If the error rate does not exceed the threshold, the frame drop threshold is returned to a preset value, threshold control is performed, and the information amount of image data in the temporary buffer is controlled by the threshold control. If the value is larger than the frame drop threshold, the frame drop control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the quantization step used when encoding the image frame stored in the temporary buffer is performed. It is characterized in that a quantization value having a quantization step width larger than the width is set and notified to the encoding means.

As described above, according to the twenty-eighth aspect, based on the error rate from the image receiving apparatus, the image coding apparatus determines whether the communication state is good or bad, and continues until the communication state becomes good. Since the frame-drop control is performed by lowering the frame-drop threshold, the frame-drop control can be performed in consideration of the communication state.

A twenty-ninth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, calculates an average of the error rates obtained within a predetermined time, and transmits the calculated average error rate to the image encoding apparatus. , The image receiving device,
Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and a transmission buffer for storing the image data encoded by the encoding means. Transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate,
A first quantization control unit that determines an optimal quantization value in terms of both the reproducibility of motion and the image quality; and an error that adds an error correction code having a different error correction capability to image data encoded by the encoding unit. Correction encoding means, and a second quantization control means for determining a quantization value according to the communication state and controlling the error correction encoding means according to the average error rate, wherein the second quantization means When the average error rate received from the image receiving apparatus exceeds a predetermined threshold, the quantization step size larger than the quantization step width corresponding to the quantization value determined by the first quantization control means. Set the quantization value of the width, and notify the encoding means, and output a selection signal instructing the error correction encoding means to add an error correction code having a high error correction capability, a predetermined time, Average error rate If not, the quantization signal determined by the first quantization control means is notified to the encoding means as it is, and the selection signal instructing the error correction encoding means to add an error correction code having a low error correction capability. And output
The encoding unit encodes the image frame with the quantization value notified from the second quantization control unit, and the error correction encoding unit encodes the error correction code specified by the selection symbol by the encoding unit. It is characterized in that it is added to the obtained image data.

As described above, according to the twenty-ninth aspect, whether the communication state is good or bad is determined based on the average error rate from the image receiving apparatus, and the quantization step width is determined until the communication state is reduced to a good state. , And encoding is performed using an error correction code having a high error correction capability, so that encoding control that is not affected by a temporary communication state can be performed.

A thirtieth aspect is the twenty-ninth aspect, wherein
The image encoding device is disposed between the encoding unit and the transmission buffer, and temporarily transmits the image data of the image frame encoded by the encoding unit, and transmits the image data in the temporary buffer. A frame removal / quantization control unit that determines whether to store the image data in the buffer and determines a quantization value of an image frame to be encoded next; When the average error rate received from the receiver exceeds a predetermined threshold, the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is determined from a predetermined value. If the average error rate does not exceed the threshold for a predetermined period of time, the frame drop threshold is returned to a predetermined value. If the information amount of the image data in the file is larger than the frame removal threshold value controlled by the threshold value control, the frame buffering control that does not store the image data in the temporary buffer in the transmission buffer is performed, and the temporary buffer is stored in the temporary buffer. It is characterized in that a quantization value with a quantization step width larger than the quantization step width used when encoding the stored image frame is set and notified to the encoding means.

As described above, according to the thirtieth aspect, the image coding apparatus determines whether the communication state is good or bad based on the average error rate from the image receiving apparatus, and the communication state is settled into a good state. Until the above, the frame drop control is performed by lowering the frame drop threshold, so that the frame drop control that is not affected by the temporary communication state can be performed.

A thirty-first aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, and when the calculated error rate exceeds a predetermined threshold, transmits an error signal to the image encoding apparatus, Includes an image input unit configured to output an image frame from a video signal, an encoding unit configured to encode the image frame configured by the image input unit, and a transmission unit configured to store the image data encoded by the encoding unit. A buffer, transmission control means for transmitting image data in the transmission buffer at a predetermined transmission rate, and an image frame arranged between the encoding means and the transmission buffer and encoded by the encoding means. A temporary buffer for temporarily storing image data; a frame drop / decision unit for determining whether to store the image data in the temporary buffer in a transmission buffer and determining a quantization value of an image frame to be encoded next; Quantization control means, and adding an error correction code having a different error correction capability to the image data in the temporary buffer,
Error correction encoding means for storing in a transmission buffer,
The frame drop / quantization control means, when receiving the error signal, lowers the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer from a preset value. While performing the threshold control, and outputs a selection signal instructing the error correction encoding means to add a high error correction code of error correction capability, a predetermined time, if not notified of the error signal, In addition to performing threshold control for returning the frame drop threshold to a preset value, the temporary buffer outputs a selection signal instructing the error correction coding means to add an error correction code having a low error correction capability. If the information amount of the image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, the frame in which the image data in the temporary buffer is not stored in the transmission buffer is set. Performs the control, by setting the quantization value larger quantization step width than the quantization step width used when encoding an image frame stored in the temporary buffer and notifies the encoding means,
The error correction encoding means adds the error correction code specified by the selection signal to the image data encoded by the encoding means.

As described above, according to the thirty-first aspect, the frame dropping control is performed by lowering the frame drop threshold until the communication state is reduced to a good state. Frame drop control can be performed.

The thirty-second aspect is the thirty-first aspect, wherein
The image receiving apparatus calculates an average of the error rates calculated within a predetermined period of time, and transmits an error signal to the image encoding apparatus when the calculated average error rate exceeds a predetermined threshold. It is characterized by the following.

As described above, according to the thirty-second mode, the image receiving apparatus calculates the average of the error rate, and if the average error rate exceeds a preset threshold, the error signal is converted to the image code. By transmitting the image data to the image encoding device, the image encoding device can perform frame dropping control that is not affected by temporary good / bad communication conditions.

A thirty-third mode is an image transmission system in which an image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, transmits the calculated error rate to the image encoding apparatus, and the image encoding apparatus constructs an image frame from the video signal and outputs the image frame. Encoding means for encoding an image frame constituted by the input means, a transmission buffer for storing the image data encoded by the encoding means, and transmission control for transmitting the image data in the transmission buffer at a predetermined transmission rate Means, a temporary buffer disposed between the encoding means and the transmission buffer, for temporarily storing image data of the image frame encoded by the encoding means, and a temporary buffer. Frame removal / quantization control means for determining whether to store the image data in the buffer in the transmission buffer, and determining the quantization value of the image frame to be encoded next; Error correction encoding means for adding error correction codes having different error correction capabilities and storing the error correction codes in a transmission buffer;
When the error rate exceeds a predetermined threshold value, the quantization control means sets a frame drop threshold value for determining whether to store the image data in the temporary buffer in the transmission buffer. In addition to performing threshold control to decrease the error rate from the set value, a selection signal instructing the error correction encoding means to add an error correction code having a high error correction capability is output, and the error rate is set to a predetermined value for a predetermined time. If the predetermined threshold value is not exceeded, threshold control is performed to return the frame drop threshold to a predetermined value, and an error correction code having a low error correction capability is added to the error correction encoding means. Is output to the temporary buffer, if the information amount of the image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, the image data in the temporary buffer is output. In addition to performing frame removal control that is not stored in the transmission buffer, a code is set by setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer. And the error correction encoding means adds the error correction code specified by the selection signal to the image data encoded by the encoding means.

As described above, according to the thirty-third mode, the image coding apparatus determines whether the communication state is good or bad based on the error rate from the image receiving apparatus, and continues until the communication state is set to a good state. Since the frame-drop control is performed by lowering the frame-drop threshold, the frame-drop control can be performed in consideration of the communication state.

[0109] A thirty-fourth aspect is a method according to the thirty-third aspect, wherein
The image encoding apparatus further includes an average error rate calculation unit that calculates an average error rate that is an average of the error rates received from the image reception apparatus within a predetermined time, and the frame drop / quantization control unit includes: When the average error rate exceeds a predetermined threshold, the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is reduced from the predetermined value. In addition to performing threshold control, it outputs a selection signal instructing the error correction coding means to add an error correction code having a high error correction capability, so that the average error rate is determined for a predetermined period of time. If the threshold value is not exceeded, threshold control is performed to return the frame drop threshold to a preset value, and an error correction code having a low error correction capability is added to the error correction encoding means. If the information amount of the image data in the temporary buffer is larger than the frame drop threshold value controlled by the threshold value control, a frame in which the image data in the temporary buffer is not stored in the transmission buffer is output. Along with performing the drop control, set a quantization value of a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notify the encoding unit, The error correction encoding means adds the error correction code specified by the selection signal to the image data encoded by the encoding means.

As described above, according to the thirty-fourth aspect, the average of the error rate from the image receiving apparatus is calculated, and the communication state is determined to be good or bad based on the calculated average error rate. Until the communication state is reduced to a good state, the frame-drop control is performed by lowering the frame-drop threshold, so that the frame-drop control can be performed in consideration of the communication state.

A thirty-fifth aspect is an image transmission system in which an image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data are communicably connected. The image receiving apparatus calculates an error rate of the received image data, calculates an average of the error rates obtained within a predetermined time, and transmits the calculated average error rate to the image encoding apparatus. An image encoding device configured to output an image frame from a video signal, an image encoding device encoding an image frame configured by the image input device, and an image encoded by the encoding device. A transmission buffer for storing data, transmission control means for transmitting image data in the transmission buffer at a predetermined transmission rate, and transmission means arranged between the encoding means and the transmission buffer, and encoded by the encoding means. A temporary buffer for temporarily storing the image data of the selected image frame, and determining whether to store the image data in the temporary buffer in the transmission buffer, and determining the quantization value of the next encoded image frame. A frame drop / quantization control means for determining the image data in the temporary buffer; and an error correction code means for adding an error correction code having a different error correction capability to the image data in the temporary buffer and storing the image data in a transmission buffer. The means is configured to, when the average error rate exceeds a predetermined threshold, set a frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer to a predetermined value. Output a selection signal that instructs the error correction coding means to add an error correction code with a high error correction capability to the error correction coding means. If the average error rate does not exceed a predetermined threshold for a predetermined period of time, threshold control is performed to return the frame drop threshold to a predetermined value. Output a selection signal instructing to add an error correction code having a low error correction capability to the temporary buffer, if the information amount of the image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, In addition to performing frame drop control in which image data in the temporary buffer is not stored in the transmission buffer, the quantization step width is larger than the quantization step width used when encoding the image frame stored in the temporary buffer. The encoding means sets the encoding value and notifies the encoding means, and the error correction encoding means converts the error correction code designated by the selection signal into image data encoded by the encoding means. Is added.

As described above, according to the thirty-fifth aspect, the image coding apparatus determines whether the communication state is good or bad based on the average error rate from the image receiving apparatus, and the communication state is settled to a good state. Until the above, the frame drop control is performed by lowering the frame drop threshold, so that the frame drop control that is not affected by the temporary communication state can be performed.

A thirty-sixth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. Then, the image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts the video signal from the video signal. Image input means for composing and outputting an image frame, encoding means for encoding the image frame constituted by the image input means, a transmission buffer for storing the image data encoded by the encoding means, and a communication state A quantization control unit that determines a quantization value according to the image data, transmits image data in the transmission buffer at a predetermined transmission rate, and determines the number of bits received from the image reception device by the quantization control unit. And a transmission control means for outputting,
The quantization control means calculates an average throughput within a certain time period based on the number of bits received from the transmission control means, and calculates the calculated average throughput and a result per frame of a result of encoding by the encoding means. The coding rate is calculated based on the generated information amount, a quantization value corresponding to the calculated coding rate is determined, and the coding value is notified to the coding unit.

As described above, according to the thirty-sixth aspect, the image encoding device side knows the number of bits that have correctly arrived at the other party's image receiving device within a fixed time, and thus, on the communication path between the two terminals. Even if there are a plurality of transmission paths having different throughputs, optimal quantization control can be performed for the throughput between both terminals.

A thirty-seventh aspect is the thirty-sixth aspect, wherein
The quantization control means, when calculating the average throughput,
The averaging time is variable according to the change in the number of bits received from the transmission control means.

As described above, according to the thirty-seventh aspect, if the time for obtaining the average is shortened, quantization control can be quickly performed in response to a change in throughput, and a situation in which a burst-like error occurs may occur. In, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not vary even when the burst error occurs.

A thirty-eighth aspect is the thirty-sixth aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the thirty-eighth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control that does not vary even when a burst error occurs.

A thirty-ninth embodiment is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus counts the number of bits of data received correctly within a predetermined time, calculates an average throughput from the counted number of bits, transmits the average throughput to the image encoding apparatus, and performs image encoding. The equipment is
Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and a transmission buffer for storing the image data encoded by the encoding means. A quantization control unit for determining a quantization value according to a communication state, transmitting image data in a transmission buffer at a predetermined transmission rate, and outputting an average throughput received from the image receiving device to the quantization control unit. And transmission control means, wherein the quantization control means comprises:
A coding rate is calculated based on the average throughput received from the transmission control means and the amount of information generated per frame as a result of coding by the coding means, and a quantization value corresponding to the calculated coding rate is calculated. Is determined and notified to the encoding means.

As described above, according to the thirty-ninth aspect, the image receiving apparatus calculates the average throughput from the number of bits successfully received in a fixed time, and sends the calculated average throughput to the image encoding apparatus. Even if there are a plurality of transmission paths with different throughputs on the communication path, optimal quantization control can be performed for the throughput between both terminals.

In a fortieth aspect, an image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data communicate via one or more relay stations. An image transmission system operatively connected,
Upon detecting an error in the received data, the image receiving apparatus performs automatic retransmission with an adjacent relay station, and transmits retransmission occurrence information indicating that retransmission has occurred to the image encoding apparatus. Includes an image input unit configured to output an image frame from a video signal, an encoding unit configured to encode the image frame configured by the image input unit, and a transmission unit configured to store the image data encoded by the encoding unit. A buffer, first quantization control means for determining an optimal quantization value in terms of both motion reproducibility and image quality, and second quantization control means for controlling a quantization step width according to a communication state; The image data in the transmission buffer is transmitted at a predetermined transmission rate, and when a communication error occurs with an adjacent relay station, an automatic retransmission is performed with the adjacent relay station, and the self-retransmission is performed. If Or, when retransmission occurrence information is received from the image receiving apparatus, the transmission control unit notifies error information indicating that retransmission has occurred to the second quantization control unit, and the second quantization control unit includes: Upon receiving the error information notification from the transmission control means, a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means is set, and the code is set. If the error information is not received from the transmission control means for a predetermined period of time, the quantization value determined by the first quantization control means is notified to the encoding means as it is, The encoding unit encodes an image frame to be encoded next with the quantization value notified from the second quantization control unit.

As described above, according to the fortieth aspect, there are a plurality of transmission paths between both terminals, and no direct retransmission control is performed between the image encoding device and the image receiving device. If retransmission control is performed with a relay station that performs retransmission, the delay time can be reduced and a smooth moving image can be displayed on the receiving side even if retransmission is performed in one of the relay stations and the communication condition deteriorates. Can be.

The forty-first aspect is the forty-fourth aspect,
The image encoding device is disposed between the encoding unit and the transmission buffer, and temporarily transmits the image data of the image frame encoded by the encoding unit, and transmits the image data in the temporary buffer. A frame removal / quantization control unit for determining whether to store the image data in the buffer and determining a quantization value of an image frame to be encoded next, the transmission control unit indicating that retransmission has occurred The error information is further notified to the frame drop / quantization control means,
Upon receiving the error information from the transmission control unit, the frame drop / quantization control unit sets a frame drop threshold for determining whether or not to store the image data in the temporary buffer in the transmission buffer. If the transmission control means does not receive notification of error information for a predetermined period of time, the frame drop threshold is returned to a predetermined value, threshold control is performed, and a temporary buffer If the information amount of the image data is larger than the frame removal threshold value controlled by the threshold value control, the image data in the temporary buffer is not stored in the transmission buffer and the frame removal control is performed, and the image data in the temporary buffer is stored in the temporary buffer. A quantization step width larger than the quantization step width used when encoding the image frame that is present and notifies the encoding means And wherein the door.

As described above, according to the forty-first mode, the frame dropping threshold is lowered and the frame dropping control is performed until the communication state is reduced to a favorable state. Frame drop control can be performed.

A forty-second aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. There, the image receiving device counts the number of bits of data received correctly within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts the video signal from the video signal. Image input means for composing and outputting an image frame, encoding means for encoding the image frame constituted by the image input means, and a transmission buffer for storing image data of the image frame encoded by the encoding means. A temporary buffer disposed between the encoding unit and the transmission buffer, and temporarily storing image data of the image frame encoded by the encoding unit;
Frame removal / quantization control means for determining whether to store the image data in the temporary buffer in the transmission buffer, and determining the quantization value of the image frame to be encoded next; At a predetermined transmission rate, and transmission control means for calculating an average throughput from the number of bits received from the image receiving apparatus. The frame drop / quantization control means includes a preset maximum allowable delay time and , Based on the average throughput and
The amount of information that can be transmitted within the maximum allowable delay time is calculated. If the information amount of the image frame stored in the temporary buffer is larger than the information amount that can be transmitted within the maximum allowable delay time, the image data in the temporary buffer is calculated. And perform a frame drop control that does not store the frame in the transmission buffer, and set a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer. It is characterized by notifying the encoding means.

As described above, according to the forty-second aspect, the image encoding device knows the number of bits that have correctly arrived at the other party's image receiving device within a predetermined time, and thus, on the communication path between the two terminals. Even if there are a plurality of transmission paths having different throughputs, it is possible to perform optimal dropout control in consideration of the throughput between both terminals.

[0127] A forty-third aspect is the forty-sixth aspect,
The frame drop / quantization control means further calculates an average delay time based on the average amount of generated information of the image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time as a maximum allowable delay. It is characterized in that it is used as time.

As described above, according to the forty-third aspect, the average delay time is calculated from the average amount of information generated and the average throughput of the image frames transmitted within a predetermined time,
The amount of information that can be transmitted within the average delay time is calculated, and the calculated amount of information is used as a threshold value for frame drop control. It can be performed.

[0129] A forty-fourth aspect is a method according to the forty-second aspect, wherein
The transmission control means is characterized in that, when calculating the average throughput, the averaging time is variable according to the change in the number of bits.

As described above, according to the forty-fourth aspect, if the time for obtaining the average is shortened, frame dropping control can be performed quickly in response to a change in throughput. In, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the frame drop control that does not vary even when the burst error occurs.

[0131] A forty-fifth aspect is a method according to the forty-second aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the forty-fifth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform frame drop control that does not fluctuate even when a burst error occurs.

A forty-sixth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus counts the number of bits of data received correctly within a predetermined time, calculates an average throughput from the counted number of bits, transmits the average throughput to the image encoding apparatus, and performs image encoding. The equipment is
Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and image data of the image frame encoded by the encoding means are stored. A transmission buffer, a temporary buffer disposed between the encoding unit and the transmission buffer and temporarily storing image data of the image frame encoded by the encoding unit, and a transmission buffer for storing image data in the temporary buffer. And a frame drop / quantization control means for determining a quantization value of an image frame to be encoded next, and transmitting image data in a transmission buffer at a predetermined transmission rate. Transmission control means for outputting the average throughput received from the image receiving apparatus to the frame drop / quantization control means. Calculates the amount of information that can be transmitted within the maximum allowable delay time based on the preset maximum allowable delay time and the average throughput, and determines that the information amount of the image frame stored in the temporary buffer is the maximum allowable amount. If the amount of information that can be transmitted within the delay time is larger than the amount of information that can be transmitted, the frame buffering control that does not store the image data in the temporary buffer in the transmission buffer is performed, and it is used for encoding the image frame stored in the temporary buffer. It is characterized in that a quantization value having a quantization step width larger than the quantization step width is set and notified to the encoding means.

As described above, according to the forty-sixth aspect, the image receiving apparatus calculates the average throughput from the number of bits that can be correctly received in a certain period of time, and sends the average throughput to the image encoding apparatus. Even if there are a plurality of transmission paths having different throughputs on the communication path, it is possible to perform the optimal frame drop control in consideration of the throughput between the two terminals.

A forty-seventh aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. Then, the image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts the video signal from the video signal. An image input unit configured to output an image frame, an encoding unit configured to encode the image frame configured by the image input unit, and a transmission buffer configured to store image data of the image frame encoded by the encoding unit. And a quantization control means for determining a quantization value to be used for encoding, transmitting image data in a transmission buffer at a predetermined transmission rate, and transmitting a destination bit received from an image receiving apparatus. It calculates an average throughput from the number,
Transmission control means for outputting to the quantization control means, motion threshold value storage means for storing a motion threshold value for determining whether the motion of the subject is large or small, and a transmission delay limit which does not cause visual discomfort And a maximum allowable delay time storing means for storing the maximum allowable delay time, wherein the quantization control means determines that the amount of information generated per frame as a result of encoding by the encoding means is larger than the motion threshold. In this case, a quantization ratio, which is a ratio between a quantization value used for encoding an image frame exceeding the motion threshold and a quantization value for which the amount of generated information is to be predicted, is calculated, and a value exceeding the motion threshold is obtained. From the generated information amount of the image frame and the quantization ratio, the amount of information generated when the quantization value is changed is predicted.From the predicted generated amount of information, the average throughput, the transmission rate, and the maximum allowable delay time, Maximum allowed A quantized value that can be transmitted within the delay time is determined and notified to the encoding unit, and the encoding unit encodes the image frame with the quantized value notified from the quantization control unit. .

As described above, according to the forty-seventh aspect, the image coding apparatus can calculate the predicted occurrence information amount of the image frame in which the motion of the subject is large and the average throughput based on the number of bits received from the image receiving apparatus. By determining a quantizing value that can be transmitted within the maximum allowable delay time, even if there are a plurality of transmission paths having different throughputs on a communication path between the two terminals, an optimum quantization value for the throughput between the two terminals is obtained. Control can be performed.

[0137] A forty-eighth embodiment is a method according to the forty-seventh embodiment, wherein
The transmission control means is characterized in that, when calculating the average throughput, the averaging time is variable according to the change in the number of bits.

As described above, according to the forty-eighth aspect, if the time for obtaining the average is shortened, quantization control can be quickly performed in response to a change in throughput, and a situation in which a burst-like error occurs may occur. In, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not vary even when the burst error occurs.

[0139] A forty-ninth aspect is a method according to the forty-seventh aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the forty-ninth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control that does not vary even when a burst error occurs.

A fiftieth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus counts the number of bits of data received correctly within a predetermined time, calculates an average throughput from the counted number of bits, transmits the average throughput to the image encoding apparatus, and performs image encoding. The equipment is
Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and image data of the image frame encoded by the encoding means are stored. A transmission buffer, quantization control means for determining a quantization value used for encoding,
Transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate and outputting the average throughput received from the image receiving apparatus to the quantization control means, and for determining whether the movement of the subject is large or small. A motion threshold storage means for storing a motion threshold, and a maximum allowable delay time storage means for storing a maximum allowable delay time that is a limit of a transmission delay that does not cause visual discomfort; If the amount of information generated per frame as a result of encoding by the encoding means is greater than the motion threshold, the quantization value used when encoding the image frame exceeding the motion threshold is determined by the Calculate the quantization ratio, which is the ratio of the quantization value for which the amount of information is to be predicted, and generate it when the quantization value is changed based on the amount of information generated and the quantization ratio of the image frame that exceeds the motion threshold. The information amount is predicted, a quantized value that can be transmitted within the maximum allowable delay time is determined from the predicted generated information amount, the average throughput, the transmission rate, and the maximum allowable delay time, and the encoding value is notified to the encoding unit, and the encoding is performed. The quantization means encodes the image frame with the quantization value notified from the quantization control means.

As described above, according to the fiftieth aspect, the image coding apparatus calculates the quantization value from the predicted generation information amount of the image frame in which the subject moves greatly and the average throughput received from the image receiving apparatus. By determining, even if there are a plurality of transmission paths having different throughputs on the communication path between the two terminals, it is possible to perform optimal quantization control on the throughput between the two terminals.

A fifty-first aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. Then, the image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts the video signal from the video signal. An image input unit configured to output an image frame, an encoding unit configured to encode the image frame configured by the image input unit, and a transmission buffer configured to store image data of the image frame encoded by the encoding unit. And a quantization control means for determining a quantization value to be used for encoding, transmitting image data in a transmission buffer at a predetermined transmission rate, and transmitting a destination bit received from an image receiving apparatus. It calculates an average throughput from the number,
A transmission control means for outputting to the quantization control means, a motion threshold value storing means for storing a motion threshold value for determining whether the motion of the subject is large or small, and a rate of frames which can be transmitted per unit time. As a balance between temporal distortion caused by dropping frames according to a certain coding rate and spatial distortion caused by a quantization value, an ideal curve storing means for storing an ideal curve whose operating point is predetermined is provided, and quantization control is performed. Means are
If the amount of information generated per frame as a result of encoding by the encoding means is greater than the motion threshold, the quantization value used when encoding the image frame exceeding the motion threshold is determined by the Calculate the quantization ratio, which is the ratio of the quantization value for which the amount of information is to be predicted, and calculate the information generated when the quantization value is changed based on the amount of information generated and the quantization ratio of the image frame exceeding the motion threshold. The amount is predicted, the coding rate is calculated from the predicted generated information amount, the transmission rate, and the average throughput, the quantization value is determined from the calculated coding rate and the ideal curve, and The encoding unit encodes the image frame with the quantization value notified from the quantization control unit.

As described above, according to the fifty-first aspect, the image coding apparatus can calculate the predicted generation information amount of the image frame in which the subject moves greatly, the average throughput based on the number of bits received from the image receiving apparatus, and By determining the quantization value from the ideal curve, even if there are a plurality of transmission paths having different throughputs on the communication path between the two terminals, optimal quantization control is performed for the throughput between the two terminals. be able to.

The fifty-second aspect is the fifty-first aspect, wherein
The transmission control means is characterized in that, when calculating the average throughput, the averaging time is variable according to the change in the number of bits.

As described above, according to the fifty-second aspect, if the time for obtaining the average is shortened, quantization control can be quickly performed in response to a change in throughput, and a situation in which a burst-like error occurs may occur. In, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not vary even when the burst error occurs.

[0147] A fifty-third aspect is a modification of the fifty-first aspect, wherein
The predetermined constant time for calculating the average throughput is set to 300 msec or more.

As described above, according to the fifty-third aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control that does not vary even when a burst error occurs.

A fifty-fourth aspect is an image transmission system in which an image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected. The image receiving apparatus counts the number of bits of data received correctly within a predetermined time, calculates an average throughput from the counted number of bits, transmits the average throughput to the image encoding apparatus, and performs image encoding. The equipment is
Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and image data of the image frame encoded by the encoding means are stored. A transmission buffer, quantization control means for determining a quantization value used for encoding,
Transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate and outputting the average throughput received from the image receiving apparatus to the quantization control means, and for determining whether the movement of the subject is large or small. A motion threshold storage means for storing a motion threshold, and a balance between temporal distortion due to frame dropping according to a coding rate which is a rate of a frame that can be transmitted per unit time and spatial distortion due to a quantization value. And an ideal curve storage means for storing an ideal curve whose operating point is predetermined. The quantization control means determines that the amount of information generated per frame as a result of encoding by the encoding means is a motion threshold value. If the value is larger than the threshold value, the quantization ratio, which is the ratio between the quantization value used to encode the image frame exceeding the motion threshold value and the quantization value for which the amount of generated information is to be predicted, is calculated. Therefore, the amount of information generated when the quantization value is changed is predicted from the amount of generated information of the image frame exceeding the motion threshold and the quantization ratio, and the predicted amount of generated information, the transmission rate, and the average throughput are calculated. From the above, calculate the coding rate, from the calculated coding rate and the ideal curve, determine the quantization value and notify the encoding means, encoding means,
The image frame is encoded with the quantization value notified from the quantization control unit.

As described above, according to the fifty-fourth aspect, the image coding apparatus calculates the quantization value from the predicted occurrence information amount of the image frame in which the subject moves greatly and the average throughput received from the image receiving apparatus. By determining, even if there are a plurality of transmission paths having different throughputs on the communication path between the two terminals, it is possible to perform optimal quantization control on the throughput between the two terminals.

In a fifty-fifth aspect, an image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data are communicated via one or more relay stations. An image transmission system operatively connected,
The image receiving apparatus automatically performs retransmission with an adjacent relay station when detecting an error in received data, and performs automatic retransmission with an adjacent relay station when detecting an error in received data. At the same time, retransmission occurrence information indicating that retransmission has occurred is transmitted to the image encoding device, and the image encoding device includes image input means for forming and outputting an image frame from a video signal, and image input means. Encoding means for encoding an image frame, a transmission buffer for storing image data encoded by the encoding means, and first quantization for determining an optimal quantization value in terms of both the reproducibility of motion and the image quality Control means for controlling a quantization step width according to a communication state;
And the image data in the transmission buffer is transmitted at a predetermined transmission rate, and when a communication error occurs between adjacent relay stations, automatic retransmission is performed between the adjacent relay stations. Transmission control means for notifying the second quantization control means of error information indicating that retransmission has occurred when the retransmission has been performed by itself or when retransmission occurrence information has been received from the image receiving apparatus. , The second quantization control unit, upon receiving the notification of the error information from the transmission control unit,
A quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means is set and notified to the encoding means, and a predetermined fixed time, If the error information is not notified from the transmission control unit, the quantization value determined by the first quantization control unit is notified to the encoding unit as it is, and the encoding unit transmits the next image frame to be encoded. The encoding is performed using the quantization value notified from the second quantization control unit.

As described above, according to the fifty-fifth aspect, a relay station performs automatic retransmission between an adjacent relay station and an image receiving apparatus, and transmits retransmission occurrence information indicating that retransmission has occurred to an image. By transmitting the data to the encoding device, it is possible to perform quantization control in consideration of the communication state between terminals.

In a fifty-sixth aspect, an image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data communicate via one or more relay stations. An image transmission system operatively connected,
The image receiving apparatus automatically performs retransmission with an adjacent relay station when detecting an error in received data, and performs automatic retransmission with an adjacent relay station when detecting an error in received data. At the same time, retransmission occurrence information indicating that retransmission has occurred is transmitted to the image encoding device, and the image encoding device includes image input means for forming and outputting an image frame from a video signal, and image input means. Encoding means for encoding the image frame, a transmission buffer for storing the image data of the image frame encoded by the encoding means, and transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate. A temporary buffer disposed between the encoding unit and the transmission buffer and temporarily storing image data of an image frame encoded by the encoding unit; and a temporary buffer. Frame determination / quantization control means for determining whether to store the image data in the transmission buffer and determining the quantization value of the image frame to be encoded next, When receiving notification of error information from the transmission control means, the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is lowered from a preset value, and If the error information is not notified from the transmission control means for a given period of time, the frame drop threshold is returned to a preset value, threshold control is performed, and the information of the image frame stored in the temporary buffer is performed. If the amount is larger than the frame drop threshold, frame drop control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the temporary buffer is A quantization value having a quantization step width larger than the quantization step width used when encoding the stored image frame is set and notified to the encoding unit, and the encoding unit performs encoding next. An image transmission system characterized by encoding an image frame with a quantization value notified from a frame drop / quantization control unit.

As described above, according to the fifty-sixth aspect, a relay station performs automatic retransmission between an adjacent relay station and an image receiving apparatus, and transmits retransmission occurrence information indicating that retransmission has occurred to an image. By transmitting the data to the encoding device, encoding control can be performed in consideration of the communication state between the terminals.

A fifty-seventh aspect is an image coding apparatus for coding and transmitting a moving picture, wherein, of the image areas to be coded, a priority area is set to a first quantization accuracy and a non-priority area is set to a second quantization accuracy. And a priority area and a non-priority area are determined according to the amount of generated information of an image encoded by the image encoding means. And a priority area / non-priority area changing means for changing, and a transmission control means for transmitting the image coding bit string.

As described above, according to the fifty-seventh aspect, when the priority area is more clearly coded than the non-priority area, if the amount of generated information increases or the communication throughput decreases, the priority area change occurs. Since the means reduces the priority area,
As a result, the amount of generated information can be suppressed, and the image quality of the priority area does not significantly deteriorate.

In the fifty-eighth aspect, when a moving image is subjected to high-efficiency compression encoding and transmitted through a low-bit-rate communication line, a frame corresponding to the encoding rate, which is the ratio of frames that can be transmitted per unit time, is used. A method for controlling a quantization operation when encoding an image frame in an image encoding device in which an operating point is predetermined as a balance between temporal distortion due to dropping and spatial distortion due to quantization accuracy. According to the magnitude relationship between the coding rate of the result of encoding the entire image frame with a certain quantization precision and the coding rate of the operating point according to the quantization precision, the quantization at the next encoding is performed. A first operation mode for determining the encoding accuracy, an image region to be encoded is divided into a priority region and a non-priority region, and the priority region is encoded with a relatively high first quantization accuracy, and the non-priority region is encoded. Make the area relatively accurate There was encoded in the second quantization accuracy,
Depending on the magnitude relationship between the coding rate of the result of coding with the first and second quantization precisions and the coding rate of the operating point according to the quantization precision, the quantization precision at the next coding is A second operation mode for determining the size of the priority area and the size of the non-priority area, wherein the quantization precision determined in the first operation mode is lower than a preset allowable lower limit of the quantization precision. Becomes, the operation mode shifts from the first operation mode to the second operation mode, and when the size of the priority area determined in the second operation mode becomes maximum, the operation mode becomes the second operation mode. It is characterized by shifting from the second operation mode to the first operation mode.

As described above, according to the fifty-eighth aspect, when spatial image quality due to an error in quantization precision and accompanying frame skipping are within an allowable range, quantization based on conventional visual characteristics is considered. If the spatial image quality and frame skipping exceed the permissible limits by performing the control, the visual characteristics are considered for the important spatial regions by sacrificing the image quality of the less important spatial regions in the image. By performing the above-described quantization control, it is possible to maintain the encoding parameter for the changing input image at a value such that the balance between the reproducibility of motion and the spatial resolution is in an optimal state in terms of visual characteristics.

The fifty-ninth aspect is the fifty-eighth aspect according to the fifteenth aspect.
A coding rate corresponding to each quantization precision is defined as a set of points in an area between the upper limit and the lower limit, and in the first and second operation modes, at a certain quantization precision, If the coding rate of the coding result exceeds the upper limit,
Improve the quantization accuracy at the next encoding, when the coding rate of the result of encoding with a certain quantization accuracy is below the lower limit,
If the quantization accuracy at the next encoding is reduced and the coding rate of the result of encoding at a certain quantization accuracy falls within the range between the upper limit and the lower limit, the quantization at the next encoding is reduced. It is characterized in that the conversion accuracy is not changed.

As described above, according to the fifty-ninth aspect, by providing a range of visually optimum coding rates for each quantization precision, stable coding can be performed even when the input image changes quickly. Control can be realized.

A sixtieth aspect is a method according to the fifty-ninth aspect, wherein
The minimum width of the quantization accuracy when changing in the first and second operation modes is characterized in that the difference in spatial distortion is selected to be a value larger than a visually recognizable value.

As described above, according to the sixtieth aspect, the change in the quantization accuracy given per unit time becomes large,
It is possible to quickly follow a visually optimal change in the operating point due to a change in the input image.

The sixty-first aspect is characterized in that, in the third aspect, when calculating the average throughput, the transmission control means makes the averaging time variable depending on whether or not a burst error has occurred.

As described above, according to the sixty-first aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where the error occurs, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the quantization control and the dropout control which do not vary even when the burst error occurs.

[0165] The 62nd mode is characterized in that, in the 3rd mode, the predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the 62nd aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs. .

A sixty-third aspect is characterized in that, in the seventh aspect, when calculating the average throughput, the transmission control means makes the averaging time variable depending on whether or not a burst error has occurred.

As described above, according to the sixty-third aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where the error occurs, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the quantization control and the dropout control which do not vary even when the burst error occurs.

The sixty-fourth aspect is characterized in that, in the seventh aspect, the predetermined fixed time for calculating the average throughput is set to 300 msec or more.

As described above, according to the sixty-fourth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs. .

The sixty-fifth aspect is the fourty-sixth aspect, wherein
The frame drop / quantization control means further calculates an average delay time based on the average amount of generated information of the image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time as a maximum allowable delay. It is characterized in that it is used as time.

As described above, according to the sixty-fifth aspect, the average delay time is calculated from the average amount of information generated and the average throughput of image frames transmitted within a predetermined time,
The amount of information that can be transmitted within the average delay time is calculated, and the calculated amount of information is used as a threshold value for frame drop control. It can be performed.

The sixty-sixth aspect is the sixty-sixth aspect, wherein
The image receiving apparatus is characterized in that, when calculating the average throughput, the averaging time is variable in accordance with the change in the number of bits of correctly received data.

As described above, according to the sixty-sixth aspect, if the time for obtaining the average is shortened, it is possible to perform frame dropping control that quickly responds to a change in throughput, and to reduce the likelihood of burst errors. In, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the frame drop control that does not vary even when the burst error occurs.

The sixty-seventh aspect is the sixty-sixth aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the sixty-seventh aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform frame drop control that does not vary even when a burst error occurs.

[0177] The 68th aspect is a process according to the 50th aspect, wherein
The image receiving apparatus is characterized in that, when calculating the average throughput, the averaging time is variable in accordance with the change in the number of bits of correctly received data.

As described above, according to the sixty-eighth aspect, if the time for obtaining the average is shortened, it is possible to perform the quantization control quickly corresponding to the change in the throughput, and to obtain a situation in which a burst-like error occurs. In, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not vary even when the burst error occurs.

The sixty-ninth aspect is the fifty-third aspect,
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the sixty-ninth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control that does not vary even when a burst error occurs.

[0181] A seventieth aspect is the fifty-fourth aspect, wherein
The image receiving apparatus is characterized in that, when calculating the average throughput, the averaging time is variable in accordance with the change in the number of bits of correctly received data.

As described above, according to the seventy-seventh aspect, if the time for obtaining the average is reduced, quantization control can be performed quickly in response to a change in throughput, and a situation in which a burst-like error occurs may occur. In, if the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not vary even when the burst error occurs.

A seventy-first embodiment is a modification of the fifty-fourth embodiment wherein:
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the seventy-first aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to prevent the occurrence of a burst error and to obtain a quantized value in the case of a large subject movement. A decision can be made.

[0185] A seventy-second aspect is the tenth aspect according to the tenth aspect.
The quantization control means, when calculating the average throughput,
The averaging time is variable depending on whether or not a burst error has occurred.

As described above, according to the seventy-second aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where the error occurs, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, it is possible to perform the quantization control and the dropout control which do not vary even when the burst error occurs.

[0187] A thirty-seventh aspect is the eleventh aspect, wherein
The quantization control means, when calculating the average throughput,
The averaging time is variable depending on whether or not a burst error has occurred.

As described above, according to the seventy-third aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where occurs, if the time for obtaining the average is set longer than the cycle in which the burst error occurs, quantization control and dropout control that do not vary even when a burst error occurs can be performed.

[0189] A seventy-fourth aspect is a method according to the tenth aspect, wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the seventy-fourth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs.

[0191] A seventy-fifth aspect is the eleventh aspect according to the eleventh aspect.
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the seventy-fifth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs. .

[0193] A seventy-sixth aspect is characterized in that, in the fourth aspect, when calculating the average throughput, the transmission control means varies the averaging time in accordance with the presence or absence of a burst error.

As described above, according to the seventy-sixth aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. If the time for obtaining the average is set to be longer than the cycle in which the burst error occurs in a situation where the burst error occurs, the quantization control and the frame dropping control that do not vary even when the burst error occurs can be performed.

A seventy-seventh aspect is characterized in that, in the fourth aspect, the predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the seventy-seventh aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and dropout control that do not vary even when a burst error occurs. .

The 78th aspect is characterized in that, in the eighth aspect, when calculating the average throughput, the transmission control means makes the averaging time variable depending on whether or not a burst error has occurred.

As described above, according to the seventy-eighth aspect, if the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. If the time for obtaining the average is set to be longer than the cycle in which the burst error occurs in a situation where the burst error occurs, the quantization control and the frame dropping control that do not vary even when the burst error occurs can be performed.

The seventy-ninth embodiment is characterized in that, in the ninth embodiment, the predetermined fixed time for calculating the average throughput is set to 300 msec or more.

As described above, according to the seventy-ninth aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs. .

The 80th embodiment is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, based on the amount of data output per unit time t from a transmission buffer that temporarily stores encoded data. It is characterized in that an average throughput per fixed time is calculated.

As described above, according to the 80th mode, the average throughput is calculated based on the number of bits read from the transmission buffer.
-Back-N also Selective Reject
But you can.

The eighty-first mode is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, in a certain period of time obtained by adding a data amount output per unit time t for a certain time every time a certain time elapses By dividing the amount of data output to a predetermined time period, an average throughput per certain time period is calculated every time a certain time period elapses.

As described above, according to the eighty-first aspect, an average value obtained by dividing the total number of bits in a certain time period by a certain time period is calculated every certain time period, and is output every certain time period. Thus, the average throughput can be calculated by a simple process, and further, as a retransmission method, Go-back-
Both N and Selective Reject can be supported.

The 82nd mode is the third, fourth, seventh to thirteenth,
In any one of 5 to 17, 19 to 21, 61 to 64, and 72 to 79, every time the unit time t elapses, the data amount output to the unit time t from a certain time before to that time is added. By dividing the amount of data output per unit time obtained by the unit time by the unit time, an average throughput per unit time is calculated every unit time t.

As described above, according to the eighty-second aspect, an average value obtained by dividing the total number of bits in a certain time period by a certain time period is calculated for each unit time t, and is output for each unit time t. Thus, the average throughput can be updated every unit time t, and precise control can be performed. Furthermore, as a retransmission method, SelectiveR even in Go-back-N
eject can also be used.

The eighty-third mode is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, the transmission unit temporarily stores encoded data from the transmission buffer to the same unit time t as the image frame period of the image to be encoded. It is characterized in that an average throughput per fixed time is calculated based on the output data amount.

As described above, according to the 83rd mode, by setting the unit time t to be the same as the image frame period of the image to be coded, it is possible to synchronize with the coding timing.

The eighty-fourth aspect is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, the amount of data output in the same unit time t as the image frame period of the encoded image every time a certain time elapses A characteristic feature is that an average throughput per fixed time is calculated each time a certain time elapses by dividing a data amount output per certain time obtained by adding for a certain time by a certain time.

As described above, according to the eighty-fourth aspect, by setting the unit time t to be the same as the image frame period of the image to be coded, synchronization with the coding timing can be achieved.
Furthermore, an average value obtained by dividing the total number of bits in a certain period of time by a certain period of time is calculated every certain period of time, and is output every certain period of time, so that the average throughput can be calculated by simple processing. .

The eighty-fifth aspect is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, each time the same unit time t as the image frame period of the image to be coded elapses, the unit time from a fixed time before to that time The average throughput per fixed time is calculated each time the unit time t elapses by dividing the data amount output per certain time obtained by adding the data amount output to t to a certain time. And

As described above, according to the eighty-fifth aspect, by setting the unit time t to be the same as the image frame period of the image to be coded, synchronization with the coding timing can be achieved.
Furthermore, the average value obtained by dividing the total number of bits in a certain time period by a certain time period is calculated for each unit time t, and is output for each unit time t, so that the average throughput can be updated for each unit time t. And precise control becomes possible.

The 86th mode is the third, fourth, seventh to thirteenth,
In any one of the modes 5 to 17, 19 to 21, 61 to 64, and 72 to 79, the encoded data is output from the transmission buffer for temporarily storing the encoded data at the same unit time t as one frame time of the transmission frame. It is characterized in that an average throughput per fixed time is calculated on the basis of the data amount.

As described above, according to the eighty-sixth aspect, by setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame. As a retransmission method, Go-ba
Both ck-N and Selective Reject can be supported.

The eighty-seventh aspect is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, the amount of data output in the same unit time t as one frame time of a transmission frame every time a certain time elapses An average throughput per fixed time is calculated every time a certain time elapses by dividing a data amount output per certain time obtained by the addition by a certain time.

As described above, according to the eighty-seventh aspect, by setting the unit time t to be equal to one frame time of a transmission frame, synchronization with the transmission timing of the transmission frame can be easily achieved. By calculating an average value obtained by dividing the total number of bits in a certain time period by a certain time period and outputting the average value every certain time period, the average throughput can be calculated by a simple process.

The eighty-eighth aspect is the third, fourth, seventh to thirteenth,
In any one of 5 to 17, 19 to 21, 61 to 64, and 72 to 79, every time the same unit time t as one frame time of a transmission frame elapses, the unit time t from a certain time before to that time is changed to An average throughput per fixed time is calculated each time a unit time t elapses by dividing a data amount output per certain time obtained by adding the output data amount by a certain time. .

As described above, according to the eighty-eighth aspect, by setting the unit time t to be equal to one frame time of a transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame. An average value obtained by dividing the total number of bits in a certain time period by a certain time period is calculated for each unit time t, and is output for each unit time t, so that the average throughput can be updated for each unit time t, Close control is possible.

The 89th embodiment is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, the image frame period of the encoded image and one frame of the transmission frame from the transmission buffer for temporarily storing the encoded data. It is characterized in that an average throughput per fixed time is calculated based on the amount of data output in the same unit time t as the least common multiple of time.

As described above, according to the 89.sup.th mode, if the unit time t is set to the least common multiple of the transmission frame time and the image period, synchronization with both can be easily achieved, and the retransmission method is Go. -Back-N is also Selective
Reject can also be used.

The 90th embodiment is the third, fourth, seventh to thirteenth,
In any one of the embodiments 5 to 17, 19 to 21, 61 to 64, and 72 to 79, the same as the least common multiple of the image frame period of the encoded image and one frame time of the transmission frame every time a predetermined time elapses The average throughput per fixed time is calculated each time a certain time elapses by dividing the data amount output per certain time obtained by adding the data amount output per unit time for a certain time by a certain time. It is characterized by doing.

As described above, according to the ninety-ninth aspect, if the unit time t is set to the least common multiple of the transmission frame time and the image period, synchronization can be easily achieved with both the transmission time and the bit for a certain time T. By calculating an average value obtained by dividing the sum of the numbers by a certain time period and outputting the average value every certain time period, the average throughput can be calculated by a simple process.

The ninety-first embodiment is the third, fourth, seventh to thirteenth,
5-17, 19-21, 61-64, 72-79, each time a unit time t equal to the least common multiple of the image frame period of the image to be coded and one frame time of the transmission frame elapses The average throughput per unit time is calculated by dividing the amount of data output per unit time obtained by adding the amount of data output to the unit time t from the previous period to the point in time by the unit time. It is characterized in that it is calculated every time t elapses.

As described above, according to the ninety-first aspect, if the unit time t is the least common multiple of the transmission frame time and the image period, synchronization can be easily achieved with both, and the number of bits in a certain fixed time can be easily obtained. Is calculated for each unit time t, and the average throughput is updated for each unit time t, so that the average throughput can be updated for each unit time t, and precise control is possible. Become.

[0225] A ninth aspect is the thirty-ninth aspect, wherein
The image receiving apparatus is characterized in that, when calculating the average throughput, the averaging time is variable in accordance with the change in the number of bits of correctly received data.

As described above, according to the ninety-ninth aspect, if the time for obtaining the average is shortened, quantization control can be quickly performed in response to a change in throughput, and a situation in which a burst-like error occurs may occur. If the time for obtaining the average is set to be longer than the cycle in which the burst error occurs, it is possible to perform quantization control that does not fluctuate even when the burst error occurs.

[0227] A thirty-ninth embodiment is a modification of the thirty-ninth embodiment wherein
It is characterized in that a predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the ninety-third aspect, by setting the time for obtaining the average to be 300 msec or more, it is possible to perform quantization control that does not vary even when a burst error occurs.

A ninety-fourth aspect is to calculate an average throughput per fixed time T based on the amount of data output per unit time t from a transmission buffer that temporarily stores encoded data. Features.

As described above, according to the ninety-fourth aspect, the average throughput is calculated based on the number of bits read from the transmission buffer.
-Back-N also Selective Reject
But you can.

The ninety-fifth aspect is the ninety-fourth aspect, wherein
A cycle in which the amount of data output per unit time t from the transmission buffer becomes smaller than a predetermined threshold is detected, and the cycle is set to a fixed time T.

As described above, according to the ninety-fifth aspect, by setting the cycle of the burst error to the first place time T, it is possible to calculate the average throughput which does not fluctuate with respect to the occurrence of the burst error.

[0233] A ninety-sixth aspect is the ninety-fourth aspect, wherein
Every time the fixed time T elapses, the data amount output per unit time t obtained by adding the data amount output per unit time t for the fixed time T is divided by the fixed time T to obtain a value per unit time T. It is characterized in that the average throughput is calculated every time a certain time T elapses.

As described above, according to the ninety-sixth aspect, an average value obtained by dividing the total number of bits in a certain time T by a certain time T is calculated for each certain time T, and
By outputting each time, the average throughput can be calculated by simple processing.

[0235] The 97th embodiment is a modification of the 96th embodiment, wherein
A cycle in which the amount of data output per unit time t from the transmission buffer becomes smaller than a predetermined threshold is detected, and the cycle is set to a fixed time T.

As described above, according to the ninety-seventh aspect, by setting the cycle of the burst error to the fixed time T, it is possible to calculate an average throughput that does not fluctuate even when a burst error occurs.

[0237] A ninety-ninth aspect is the ninety-fourth aspect according to the ninety-fourth aspect.
Each time the unit time t elapses, the amount of data output per unit time T obtained by adding the amount of data output to the unit time t from the time before the certain time T to that time is calculated as the certain time T.
The average throughput per fixed time T is calculated every time the unit time t elapses by dividing by.

As described above, according to the ninety-eighth aspect, an average value obtained by dividing the total number of bits in a certain time period T by a certain time period T is calculated for each unit time t, and is output for each unit time t. By doing so, the average throughput can be updated every unit time t, and precise control can be performed.

[0239] A ninety-ninth aspect is the ninety-ninth aspect, wherein a cycle in which the amount of data output from the transmission buffer per unit time t is smaller than a predetermined threshold is detected, and the cycle is set to a predetermined time T It is characterized by the following.

As described above, according to the ninety-ninth aspect, by setting the cycle of the burst error to the fixed time T, it is possible to calculate an average throughput that does not fluctuate even when a burst error occurs.

The 100th mode is characterized in that in any one of the 94th to 99th modes, the unit time t is an image frame period of the image to be coded.

As described above, according to the 100th aspect,
By setting the unit time t to be the same as the image frame period of the image to be encoded, the encoding timing and the period can be set.

The 101st mode is characterized in that in any of the 94th to 99th modes, the unit time t is one frame time of a transmission frame.

As described above, according to the 101st aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame.

The 102nd mode is the one of the 94th to 99th modes, wherein the unit time t is the least common multiple of the image frame period of the encoded image and one frame time of the transmission frame. .

As described above, according to the 102nd aspect,
If the unit time t is the least common multiple of the transmission frame time and the image period, synchronization with both can be easily achieved.

[0247] The thirty-third aspect is a twelfth, sixteenth, twentieth,
61, 63, 72, 73, 76, 78, a predetermined amount of data output per unit time t from a transmission buffer for temporarily storing encoded data. It is characterized in that an average time used for calculating an average throughput by detecting a cycle smaller than the average throughput is used.

As described above, according to the 103rd aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

The tenth aspect is a twelfth, sixteenth, twentieth,
61, 63, 72, 73, 76, 78, the amount of data output from a transmission buffer for temporarily storing encoded data in the same unit time t as one frame time of a transmission frame Is detected, and a cycle in which the cycle is smaller than a predetermined threshold value is detected, and the cycle is set as an averaging time used in calculating an average throughput.

As described above, according to the 104th aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

[0251] The 105th mode is a twelfth, sixteenth, twentieth,
In any of the modes 61, 63, 72, 73, 76, and 78, the encoded data is output from the transmission buffer that temporarily stores the encoded data at the same unit time t as the image frame period of the image to be encoded. A cycle in which the data amount becomes smaller than a predetermined threshold is detected, and the cycle is set as an averaging time used in calculating an average throughput.

As described above, according to the 105th aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

The 106th mode is a twelfth, sixteenth, twentieth,
61, 63, 72, 73, 76, 78, the transmission buffer that temporarily stores the encoded data is used to determine the image frame period of the encoded image and the one-frame time of the transmission frame. A cycle in which the amount of data output in the same unit time t as the least common multiple is smaller than a predetermined threshold is detected, and the cycle is set as an averaging time used in calculating an average throughput. I do.

As described above, according to the 106th aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

[0255] According to a seventy-seventh aspect, in the thirty-ninth aspect, the image receiving apparatus is configured to calculate the average throughput according to a change in the number of bits of correctly received data.
The averaging time is variable.

As described above, according to the tenth aspect,
By shortening the time required for averaging, quantization control can be performed quickly in response to changes in throughput.In situations where burst errors occur, the time required for averaging is set to be longer than the period during which burst errors occur. This makes it possible to perform quantization control that does not fluctuate even when a burst error occurs.

[0257] The 108th mode is the 39th mode, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more.

As described above, according to the 108th aspect,
By setting the average time to 300 msec or more,
It is possible to perform quantization control that does not fluctuate even when a burst error occurs.

[0259] The 109th mode is the 36th to 38th, 42th to 42th modes.
44, 45, 47 to 49, 51 to 53, every time a predetermined time elapses, the number of bits that can be received without error during a predetermined unit time t is added for a predetermined time. It is characterized in that the obtained number of bits that can be received without error per fixed time is transmitted to the image coding apparatus every time the fixed time elapses.

As described above, according to the 109th aspect,
The number of bits that can be received without error per fixed time can be calculated by simple processing, and the image encoding device can determine the average throughput per fixed time.

[0261] The 110th mode is the 39th, 46th, 50th,
In any one of modes 54, 65 to 71, 107, and 108, every time a predetermined time elapses, the number of bits that can be received without error during a predetermined unit time t is obtained by adding a predetermined time. By dividing the number of bits that can be received without error per fixed time by the fixed time, the average throughput per fixed time is transmitted to the image encoding device every time the fixed time elapses.

As described above, according to the 110th aspect,
The average throughput per fixed time can be calculated by simple processing, and the image coding device can know the average throughput per fixed time.

[0263] The eleventh mode is the thirty-sixth to thirty-eighth,
45, 47 to 49, 51 to 53, every time a predetermined unit time t elapses, the number of bits that can be received without error during a unit time t from a certain time before to that time. Is transmitted to the image encoding apparatus every time the unit time t elapses, the number of bits that can be received without error per fixed time obtained by adding.

As described above, according to the eleventh aspect,
By transmitting the number of bits that can be received without error per fixed time to the image encoding device for each unit time t, the image encoding device can obtain the average throughput for each unit time t. Becomes possible.

The twelfth aspect is the thirty-ninth, 46, 50,
54, 65-71, 107, or 108, the number of bits that can be received without error during a unit time t from a certain time before to the time each time a predetermined unit time t elapses The average throughput per fixed time is transmitted to the image encoding device every time the unit time t elapses by dividing the number of bits received without error per fixed time obtained by adding It is characterized by.

As described above, according to the 112nd aspect,
The average throughput per fixed time per unit time t,
By transmitting the image data to the image encoding device, the image encoding device can perform detailed control.

[0267] The 113th mode is the 36th to 38th, 42th to 42th
45, 47 to 49, in any one of 51 to 53, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and every time a predetermined time elapses, The number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the same unit time t as the image frame period of the encoded image for a certain time elapses, and a certain time elapses. The data is transmitted to the image coding apparatus every time.

As described above, according to the 113th mode,
By setting the unit time t to be the same as the image frame period of the image to be encoded, synchronization with the encoding timing can be easily achieved. Furthermore, the number of bits that can be received without error per fixed time can be calculated by simple processing, and the image encoding device can determine the average throughput per fixed time.

[0269] The 114th embodiment is a modification of the 39th, 46th, 50th,
54, 65 to 71, 107, or 108, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and each time a predetermined time elapses, By dividing the number of bits received without error per fixed time obtained by adding the number of bits received without error during the same unit time t as the image frame period of the coded image for a certain period of time by the certain period of time An average throughput per fixed time is transmitted to the image coding device every time the fixed time elapses.

As described above, according to the 114th aspect,
By setting the unit time t to be the same as the image frame period of the image to be encoded, synchronization with the encoding timing can be easily achieved. Furthermore, the average throughput per fixed time can be calculated by simple processing, and the image encoding device can know the average throughput per fixed time.

[0271] The 115th mode is the 36th to 38th, 42th to 42th modes.
45, 47 to 49, or 51 to 53, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and the image frame period of the encoded image. Every time the same unit time t elapses,
The number of bits received without error per unit time obtained by adding the number of bits received without error during the unit time t from a certain time before to that time is calculated by the image code every unit time t. The data is transmitted to a conversion device.

As described above, according to the fifteenth aspect,
By setting the unit time t to be the same as the image frame period of the image to be encoded, synchronization with the encoding timing can be easily achieved. Furthermore, by transmitting the number of bits that can be received without error per fixed time to the image encoding device for each unit time t, the image encoding device can determine the average throughput for each unit time t, and Control becomes possible.

[0273] The sixty-sixth aspect is the thirty-ninth, sixty-fifth, fifty-sixth,
54, 65 to 71, 107, or 108, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and the image frame period of the encoded image. Each time the same unit time t elapses, the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the unit time t from a certain time before to that time is calculated. , By dividing by a certain period of time,
Each time the unit time t elapses, it is transmitted to the image encoding device.

As described above, according to the 116th mode,
By setting the unit time t to be the same as the image frame period of the image to be encoded, synchronization with the encoding timing can be easily achieved. Further, by transmitting the average throughput per fixed time to the image encoding device for each unit time t,
The image encoding device enables precise control.

[0275] The seventy-seventh aspect is directed to the thirty-sixth to thirty-eighth,
45, 47 to 49, 51 to 53, every time a predetermined time elapses, the number of bits that can be received without error during the same unit time t as one frame time of a transmission frame is added for a predetermined time. The number of bits that can be received without error per fixed time obtained as described above is transmitted to the image encoding apparatus every time the fixed time elapses.

As described above, according to the seventeenth aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame, and further, it is possible to reduce the number of bits received without error per fixed time by a simple process. The image coding apparatus can calculate the average throughput for each fixed time.

[0277] The eighteenth mode is the thirty-ninth, the 46th, the 50th,
In any one of modes 54, 65 to 71, 107, and 108, every time a predetermined time elapses, the number of bits that can be received without error during the same unit time t as one frame time of a transmission frame is added for a predetermined time. The average throughput per fixed time is transmitted to the image coding apparatus every time the certain time elapses by dividing the number of bits that can be received without error per certain time obtained by the above method by the certain time.

As described above, according to the eighteenth aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame, and it is possible to calculate the average throughput per fixed time by a simple process. In the image coding apparatus, the average throughput for each fixed time can be known.

[0279] The ninety-first mode is the thirty-sixth to thirty-eighth,
45, 47 to 49, or 51 to 53, every time the same unit time t as one frame time of a transmission frame elapses, reception is performed without error during a unit time t from a certain time before to that time. It is characterized in that the number of bits received without error per fixed time obtained by adding the number of bits obtained is transmitted to the image encoding device every unit time t.

As described above, according to the 119th aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame. In addition, by transmitting the image data to the image encoding device, the image encoding device can determine the average throughput for each unit time t, and can perform detailed control.

[0281] The 120th aspect is based on the 39th, 46th, 50th,
54, 65 to 71, 107 or 108, the unit time t equal to one frame time of the transmission frame.
Every time elapses, the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during a unit time t from a certain time before to that time,
By dividing by a certain time, an average throughput per a certain time is transmitted to the image coding apparatus every time the unit time t elapses.

As described above, according to the 120th aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame. By transmitting the image data to the device, the image coding device can perform detailed control.

[0283] The twenty-first mode is the thirty-sixth to thirty-eighth,
45, 47 to 49, in any one of 51 to 53, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and every time a predetermined time elapses, No error per fixed time obtained by adding a certain number of bits that can be received without error during the same unit time t as the least common multiple of the image frame period of the encoded image and one frame time of the transmission frame per fixed time The number of bits thus generated is transmitted to the image encoding device every time a predetermined time elapses.

As described above, according to the 121st aspect,
If the unit time t is the least common multiple of the transmission frame time and the image period, synchronization can be easily achieved with both, and the number of bits that can be received without error per fixed time can be calculated by simple processing. In the image encoding device, it is possible to obtain an average throughput for each fixed time.

[0285] The 122nd mode is the 39th, 46th, 50th,
54, 65 to 71, 107, or 108, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and each time a predetermined time elapses, No error per fixed time obtained by adding a certain number of bits that can be received without error during the same unit time t as the least common multiple of the image frame period of the encoded image and one frame time of the transmission frame per fixed time The number of bits obtained is divided by a certain time to transmit an average throughput per a certain time to the image encoding device every time a certain time elapses.

As described above, according to the 122nd aspect,
If the unit time t is the least common multiple of the transmission frame time and the image period, synchronization can be easily achieved with both, and the average throughput per fixed time can be calculated by simple processing. , An average throughput for each fixed time can be obtained.

[0287] The 123rd embodiment is the 36th to 38th, 42th to 42th
45, 47 to 49, or 51 to 53, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and the image frame period of the encoded image. Every time a unit time t equal to the least common multiple of the time and one frame time of the transmission frame elapses, a fixed number obtained by adding the number of bits that can be received without error during a unit time t from a fixed time before to that time. It is characterized in that the number of bits that can be received without error per time is transmitted to the image encoding device every time the unit time t elapses.

As described above, according to the 123rd aspect,
By setting the unit time t to be the least common multiple of the transmission frame time and the image period, synchronization with both can be easily achieved, and the number of bits that can be received without error per fixed time can be determined for each unit time t by the image encoding device , The image coding apparatus can determine the average throughput for each unit time t, and can perform detailed control.

The twenty-fourth aspect is the thirty-ninth, sixty-fifth, fifty-sixth,
54, 65 to 71, 107, or 108, the image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and the image frame period of the encoded image. Every time a unit time t equal to the least common multiple of the time and one frame time of the transmission frame elapses, a fixed number obtained by adding the number of bits that can be received without error during a unit time t from a fixed time before to that time. By dividing the number of bits that can be received without error per time by a certain time, an average throughput per a certain time is transmitted to the image coding apparatus every unit time t.

As described above, according to the 124th aspect,
By setting the unit time t to be the least common multiple of the transmission frame time and the image period, synchronization can be easily achieved with both, and the average throughput per fixed time can be transmitted to the image encoding device for each unit time t. The image encoding device enables precise control.

[0291] In a 125th mode, the average throughput is calculated in a data transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on the received data are communicably connected. The receiving device adds the number of bits received without error during a predetermined unit time t for the predetermined time T every time a predetermined time T elapses. The obtained number of bits that can be received without error per fixed time T is transmitted to the encoding device every time the fixed time T elapses, and the encoding device uses the number of bits received from the receiving device, The method is characterized in that an average throughput per the predetermined time T is calculated.

As described above, according to the 125th aspect,
The number of bits that can be received without error per fixed time T can be calculated by a simple process, and the encoding device can determine the average throughput for each fixed time T.

[0293] In the 126th mode, the average throughput is calculated in a data transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on the received data are communicably connected. The receiving device adds the number of bits received without error during a predetermined unit time t for the predetermined time T every time a predetermined time T elapses. An average throughput per the fixed time T is calculated every time the fixed time T elapses by dividing the obtained number of bits received without error per the fixed time T by the fixed time T. .

As described above, according to the 126th aspect,
The average throughput per fixed time T can be calculated by a simple process.

[0295] A 127th mode is for calculating an average throughput in a data transmission system in which an encoding device for encoding and transmitting data and a receiving device for performing predetermined processing on received data are communicably connected. The receiving device, every time a predetermined unit time t elapses, a bit that can be received without error during the unit time t from a predetermined time T before the predetermined time T to the time. The number of bits that can be received without error per the fixed time T obtained by adding the numbers is transmitted to the encoding device every time the unit time t elapses, and the encoding device receives the number of bits from the receiving device. The average throughput per fixed time T is calculated based on the determined number of bits.

As described above, according to the 127th aspect,
By transmitting the number of bits that can be received without error per fixed time T to the encoding device for each unit time t, the encoding device can determine the average throughput for each unit time t.

In the 128th aspect, an average throughput is calculated in a data transmission system in which an encoding device for encoding and transmitting data and a receiving device for performing predetermined processing on received data are communicably connected. The receiving device, every time a predetermined unit time t elapses, a bit that can be received without error during the unit time t from a predetermined time T before the predetermined time T to the time The average throughput per fixed time T is calculated every time the unit time t elapses, by dividing the number of bits that can be received without error per fixed time T obtained by adding the numbers, by the fixed time T. It is characterized by doing.

[0298] As described above, according to the 128th aspect,
An average throughput per fixed time T can be calculated for each unit time t.

[0299] A 129th aspect is the transmission apparatus according to any one of the 125th to 128th aspects, wherein the unit time t is set to 1
It is characterized by a frame time.

As described above, according to the 129th aspect,
By setting the unit time t to be the same as the data frame period of the encoded data, it is possible to synchronize with the encoding timing.

[0301] In a thirtieth aspect, the receiver according to any one of the 125th to 128th aspects further comprises a function of encoding and transmitting data, similar to the encoding apparatus.
The unit time t is a data frame period of the data to be encoded.

As described above, according to the thirtieth aspect,
By setting the unit time t to be equal to one frame time of the transmission frame, it is possible to easily synchronize with the transmission timing of the transmission frame.

[0303] A thirty-first mode is the receiving apparatus according to any one of the 125th to 128th modes, further comprising a function of encoding and transmitting data, similar to the encoding apparatus.
The unit time t is a least common multiple of a data frame period of encoded data and one frame time of a transmission frame.

As described above, according to the 131st aspect,
If the unit time t is the least common multiple of the transmission frame time and the image period, synchronization with both can be easily achieved.

The 132nd mode is the 39th mode, wherein the image receiving apparatus changes the averaging time in accordance with the presence or absence of a burst error when calculating the average throughput.

As described above, according to the 132nd aspect,
If the time for obtaining the average is reduced, quantization control can be performed quickly in response to a change in throughput.In a situation where a burst-like error occurs, the time for obtaining the average is longer than the cycle in which the burst error occurs. If set, quantization control that does not fluctuate even when a burst error occurs can be performed.

[0307] A 133rd mode is the image processing apparatus according to the 46th mode, wherein when calculating the average throughput, the image receiving apparatus changes the averaging time depending on whether or not a burst error has occurred.

As described above, according to the 133rd aspect,
By shortening the time to calculate the average, it is possible to perform frame dropping control that quickly responds to changes in throughput.In a situation where a burst-like error occurs, the time to calculate the average is longer than the cycle in which a burst error occurs. If the setting is made, it is possible to perform frame dropping control that does not fluctuate even when a burst error occurs.

[0309] The 134th mode is the video processing apparatus according to the 50th mode, wherein the image receiving apparatus varies the averaging time in accordance with the presence / absence of a burst error when calculating the average throughput.

[0310] As described above, according to the 134th aspect,
If the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where a burst-like error occurs, the time for obtaining the average causes a burst error. If the period is set to be longer than the period, it is possible to perform quantization control and dropout control that do not fluctuate even when a burst error occurs.

[0311] The 135th mode is the 54th mode, wherein the image receiving apparatus changes the averaging time in accordance with the presence or absence of the burst error when calculating the average throughput.

[0312] As described above, according to the 135th mode,
If the time for obtaining the average is shortened, quantization control and dropout control can be performed quickly in response to a change in throughput. In a situation where a burst-like error occurs, the time for obtaining the average causes a burst error. If the period is set to be longer than the period, it is possible to perform quantization control and dropout control that do not fluctuate even when a burst error occurs.

[0313] In a thirty-sixth aspect, in any one of the thirty-second to thirty-fifth aspects, the number of bits successfully received during a predetermined unit time t is smaller than a predetermined threshold value. A period is detected, and the period is set as an averaging time used in calculating an average throughput.

As described above, according to the thirty-sixth aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

[0315] A 137th mode is a mode according to any one of the 132nd to 135th modes, wherein the number of bits received without error during the same unit time t as one frame time of the transmission frame is equal to a predetermined threshold value. It is characterized in that a cycle that is smaller than that is detected, and that cycle is set as an averaging time used in calculating the average throughput.

As described above, according to the thirty-seventh aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

[0317] A 138th aspect is the image processing apparatus according to any one of the 132nd to 135th aspects, wherein the image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus. , A period in which the number of bits received without error during the same unit time t as the image frame period is smaller than a predetermined threshold is detected, and the period is averaged for use in calculating the average throughput. It is characterized by time.

According to the 138th aspect, by setting the cycle of the burst error as the average time used for obtaining the average throughput, it is possible to perform the quantization control and the frame dropping control which do not vary even when the burst error occurs. Can be.

[0319] A 139th aspect is the image processing apparatus according to any of the 132nd to 135th aspects, wherein the image receiving apparatus further has a function of encoding and transmitting an image, similar to the image encoding apparatus, and And a period in which the number of bits received without error during the same unit time t as the least common multiple of the image frame period and one frame time of the transmission frame becomes smaller than a predetermined threshold is detected. Is the averaging time used in calculating the average throughput.

[0320] As described above, according to the 139th aspect,
By setting the cycle of the burst error to the average time used for obtaining the average throughput, it is possible to perform quantization control and frame dropping control that do not vary even when a burst error occurs.

[0321]

BEST MODE FOR CARRYING OUT THE INVENTION

Description of a Preferred Embodiment (First Embodiment) FIG. 1 is a block diagram showing a configuration of an image encoding apparatus according to a first embodiment of the present invention. In FIG. 1, an image encoding device 10 includes an image input unit 11
, An encoding device 12, a temporary buffer 13, a first SW (switch) 14, and a generated information amount calculation unit 15
, Frame drop / quantization control device 16 and transmission control device 1
7 and an output terminal 18. Further, the frame drop / quantization control device 16 includes a frame drop control unit 161, a threshold value storage unit 162, and a first quantization control unit 163. Further, the transmission control device 17 includes a transmission buffer 171.
And a transmission control unit 172.

The encoding device 12 performs the same encoding processing as that of the encoding device 12 in the first conventional example described above, and also converts the quantization value used for the currently encoded image frame into the first quantization value. Output to the control unit 163. The temporary buffer 13 is a buffer memory and temporarily stores the image data encoded by the encoding device 12. The generated information amount calculation unit 15 calculates the information amount of the image data from the temporary buffer storage pointer before storing the image data and the temporary buffer storage pointer after storing the image data. Note that the temporary buffer storage pointer refers to an address indicating the position where the image data is written to the temporary buffer 13. Frame drop control unit 16
1 is a predetermined threshold value and a generated information amount calculation unit 1
The information amount of the image data calculated in step 5 is compared, and if the threshold value is larger, the image data in the temporary buffer 13 is dropped. The threshold value storage unit 162 is constituted by a memory, for example, and
Stores a threshold value for determining whether or not to perform frame dropping of image data in the image. The first quantization control unit 163 controls the quantization value according to the frame removal information from the frame removal control unit 161.

Since the image input unit 11 and the transmission control device 17 have the same configuration and function as the corresponding blocks in the first conventional example shown in FIG. 18, their description is omitted here.

[0324] Next, the frame drop control operation and the quantization control operation in the image encoding apparatus shown in Fig. 1 will be described.
However, since the coding operation and the frame removal control operation and the quantization control operation other than the case where the amount of information sharply increases are the same as those of the first conventional example, here, the case where the amount of information sharply increases Only the frame removal control operation and quantization control operation will be described.

[0325] The image frame input from the image input unit 11 is encoded by the encoding device 12 and stored in the temporary buffer 13. The encoding device 12 outputs a quantization value used when quantizing the image frame stored in the temporary buffer 13 to the first quantization control unit 163. The generated information amount calculation unit 15 calculates the generated information amount of the image data in the temporary buffer 13 based on the temporary buffer storage pointer before storing the image data and the temporary buffer storage pointer after storing the image data, and Output to the control unit 161.

The frame drop control unit 161 calculates the amount of generated information of image data in the temporary buffer 13 calculated by the generated information amount calculation unit 15 and a predetermined threshold stored in the threshold value storage unit 162. If the threshold value is higher, the connection state of the first SW 14 is changed to the terminal 1
-3 are connected. On the other hand, if the threshold value is smaller, that is, if the information amount sharply increases, the dropout control unit 161 changes the connection state of the first SW 14 to a state where the terminals 1-2 are connected. switching,
The image data in the temporary buffer 13 is not transmitted. When the threshold value is smaller, the frame removal control unit 161 outputs a frame removal signal indicating that frame removal of image data in the temporary buffer 13 is to be performed to the first quantization unit 163. .

When the first quantization control section 163 receives the frame removal signal from the frame removal control section 161, the first quantization control section 163 performs quantization on the image frame to be subjected to frame removal (the image frame in the temporary buffer 13). A quantization value having a larger quantization step width than the quantization value (the quantization value output from the encoding device 12) is set and output to the encoding device 12. The encoding device 12 quantizes the image frame to be encoded next with the quantization value given from the first quantization control unit 163.

Next, with reference to FIG. 2, a description will be given of the delay time and the fluctuation of the frame skip. FIG. 2 shows a measurement result of a delay time and a transmission frame interval when frame dropping control and quantization control are performed by the image encoding device 10 according to the first embodiment. Represents the relationship between the delay time and the skipping of frames when the amount of generated information sharply increases. In FIG. 2, the vertical axis represents the delay time, and the horizontal axis represents the transmitted frame number. At the point P2, the image frame is first quantized by the quantization value “2”, and since the amount of generated information exceeds the threshold value, frame dropping control is performed, and the next image frame to be encoded is quantized. It is quantized with the quantization value “12”. As a result, the delay time becomes 257 msec, and the number of skipped frames up to the next image frame to be encoded is 7 after the frame drop control.
It became a frame. Thus, it can be seen that both the delay time and the number of skipped frames are about １ ／ of those of the conventional image encoding apparatus shown in FIG.

As described above, according to the first embodiment,
If the amount of generated information exceeds a predetermined threshold, the image frame is discarded, and a step width larger than the quantization step width of the quantization value used when encoding the image frame is used. Since the next image frame is encoded using the quantized value, the delay time and the number of skipped frames can be reduced to about 1/5 of the conventional image encoding device. Therefore, on the receiving side, it is possible to display a smooth moving image with little delay and skipping frames.

In the first embodiment, the frame drop / quantization control device 16 controls the frame drop and quantization based on the information amount of the image data in the temporary buffer 13. However, as in the frame drop / quantization control device 16 'shown in FIG.
Generated information amount storage unit 3 for storing the information amount of image data in the storage
2 and a difference unit 31 for calculating a difference between the image data in the temporary buffer 13 and the information amount stored in the generated information amount storage unit 32 are newly added. If the difference from the amount of generated information of the image frame to be sent is larger than the threshold, the frame drop control,
A configuration for performing quantization control may be adopted.

Also, in the frame drop / quantization control device 16, the image frame is separated into a priority area where the subject is displayed and a non-priority area where the background is displayed. A function for setting a quantization value is newly provided, and when a frame is dropped, the quantization value of the non-priority area of the next image frame is set to a value smaller than the quantization value determined by the first quantization control unit 163. The encoding step may be notified to the encoding device 12 as a quantization value having a large encoding step width. Further, in this case, when a frame drop occurs, the encoding device 12 may be controlled so that the encoding operation and the transmission operation are not performed in the non-priority area, or the high frequency of the input image may be controlled by the pre-filter. The component may be dropped, or only the low frequency component of the DCT coefficient may be encoded.

(Second Embodiment) FIG. 4 shows a second embodiment of the present invention.
It is a block diagram showing the composition of the picture coding device concerning an embodiment. 4, the image encoding device 40 includes an image input unit 11, an encoding device 12, a temporary buffer 13, a first SW 14, a generated information amount calculation unit 15,
Quantization control device 41 and frame drop / quantization control device 42
, A transmission control device 43 and the output terminal 18. Further, the quantization control device 41 includes a second quantization control unit 411, a third quantization control unit 412, and a mode selection unit 413. Also, the frame drop / quantization control device 4
2 is a frame drop control unit 161 and a threshold value switching unit 4
21, the second SW 422, and the first threshold value storage unit 4
23, a second threshold value storage unit 424, and a first quantization control unit 163. Further, the transmission control device 43 includes a transmission buffer 171, a retransmission buffer 431, and a transmission control unit 432.

The coding device 12 performs the same operation as that described in the first conventional example, but when coding an image frame, the quantization control device 41 or the frame drop / quantization control device 42 is used. The difference is that encoding is performed using the quantization value determined in (1), and the quantization value determined by the frame drop / quantization control device 42 has priority as the quantization value.

[0334] The second quantization control unit 411 performs the above-described first quantization control.
The same quantization control as described in the conventional example (see FIG. 21) is performed. The third quantization control unit 412 changes the quantization step width of the quantization value set by the second quantization control unit 411 according to the communication state, and outputs the result to the encoding device 12. The mode selection unit 413 determines an error free mode and an error mode according to the communication state. The error-free mode means that retransmission due to a communication error has not been performed by the transmission control device 43 for a predetermined period of time, and the error mode means that retransmission is performed for a predetermined period of time. Means that

The threshold switching unit 421 determines the error free mode or the error mode according to the communication state, and switches the connection state of the second SW 422. The first threshold value storage unit 423 is configured by a memory, and stores the temporary buffer 13 in the error free mode.
The threshold value for determining whether to store the image data in the transmission buffer 171 is stored. The second threshold value storage unit 424 is configured by a memory, and stores a threshold value for determining whether to store the image data in the temporary buffer 13 in the transmission buffer 171 in the error mode. . The threshold value stored in the first threshold value storage unit 423 is the second threshold value storage unit 42
4 has a value greater than the threshold value stored in.

The retransmission buffer 431 is a buffer memory for storing image data being transmitted. Transmission control unit 432
Is the transmission buffer 17 at a preset transmission rate.
1 and outputs the image data being transmitted to the retransmission buffer 431. When receiving a retransmission request from image receiving apparatus 1000, the image data in retransmission buffer 431 is output to image receiving apparatus 1000. In addition, an error notification indicating that retransmission has been
Output to mode selection section 413 and threshold value switching section 421.

[0337] Here, the procedure of retransmission will be briefly described. Image decoding apparatus 1000 outputs a retransmission request when the received image data cannot be decoded due to a communication error or the like. The transmission control unit 432 transmits the image data in the retransmission buffer 431 to the image receiving device 1000 in response to the received retransmission request.

Since the image input unit 11 and the transmission buffer 171 have the same functions as the corresponding blocks in the conventional image encoding device shown in FIG. 18, their description is omitted here. Further, the temporary buffer 13 and the first S
W14, generated information amount calculation section 15, frame drop control section 161
Since the first quantization control unit 163 has the same function as the corresponding block of the first embodiment, the description is omitted here.

FIG. 5 shows the timing at which the mode selection unit 413 and the threshold value switching unit 421 switch the operation mode to one of the error mode and the error free mode based on the error notification from the transmission control unit 432. It is a timing chart. The symbol (a) in FIG.
Represents a time axis. The symbol (b) in FIG.
Represents the timing at which an error notification is sent to the mode selection unit 413 and the threshold value switching unit 421. That is, a communication error occurs and the image receiving apparatus 10
00 indicates that the received image data could not be decoded and a retransmission request was made, and in response to this retransmission request, the transmission control unit 432 performed retransmission. The symbol (c) in FIG. 5 indicates the timing at which the state transitions to the error mode. The symbol (d) in FIG. 5 indicates the timing at which the state transitions to the error-free mode.

In the following, at the time points A to F in FIG.
The quantization control operation in the quantization control device 41 and the threshold value control operation in the frame drop / quantization control device 42 will be described. Note that the encoding operation in the encoding device 12 and the quantization control operation in the second quantization control unit 411 are the same as those in the first conventional example described above.
The description is omitted.

(Operation at Time A in FIG. 5) When a communication error occurs and the image receiving apparatus 1000 transmits a retransmission request to the image encoding apparatus 40, the transmission control section 432 sends an error notification to the mode selection section 413. And to the threshold switching unit 421.

Upon receiving the error notification, mode selection section 413 outputs an identification signal indicating the error mode to third quantization control section 412. Accordingly, the third quantization control unit 412 sets a quantization value having a quantization step width larger than the quantization step width represented by the quantization value determined by the second quantization control unit 411, Output to the encoding device 12. Thereafter, the third quantization control unit 412 operates in the error mode until the identification signal indicating the error-free mode is received from the mode selection unit 413. The encoding device 12 quantizes the image frame using the quantization value set by the third quantization control unit 412.

On the other hand, upon receiving the error notification, the threshold value switching unit 421 changes the connection state of the second SW 422 from the state where the terminals 1-3 are connected to the terminal SW 2
The state is switched to a state in which the three are connected. Thereafter, the second SW 422 maintains the connected state at this time until it is switched by the threshold value switching unit 421. The frame drop control unit 161 compares the threshold value stored in the second threshold value storage unit 424 with the information amount of the image data stored in the temporary buffer 13, and compares the threshold value with the first embodiment. Similar dropout control and quantization control are performed.

(Operations at time points B and C in FIG. 5) As in time point A, upon receiving the retransmission request, transmission control section 432 notifies mode selection section 413 and threshold value switching section 421 of an error notification. Is output. Mode selector 41
3, the time width between the time points A and B and the time width between the time points B and C do not exceed the predetermined time T, so that the third quantization control unit 412 sends the identification signal indicating the error mode. Output. Therefore, the third quantization control unit 412 performs the same operation as at the time point A described above. Further, the frame drop / quantization control device 42 performs the same operation as at the time point A.

(Operation at Time D in FIG. 5) Since the mode selection unit 413 has not received the error notification until a predetermined time T has elapsed after receiving the error notification at time C, the mode selection unit 413 outputs the identification signal indicating the error free mode. , To the third quantization control unit 412. Accordingly, the third quantization control unit 41
2 outputs the quantization value determined by the second quantization control unit 411 to the encoding device 12 as it is. Thereafter, the third quantization control unit 412 operates in the error-free mode until receiving the identification signal indicating the error mode from the mode selection unit 413. Therefore, the encoding device 12
The image frame is quantized using the quantization value set by the second quantization control unit 411.

On the other hand, the threshold value switching unit 421 does not receive the error notification until a predetermined time T has elapsed after receiving the error notification at the time point C, and changes the connection state of the second SW 422 between the terminals 2-3. Is connected to terminal 1
-3 are connected. Therefore, the frame drop control unit 161 compares the threshold value stored in the first threshold value storage unit 423 with the information amount of the image data stored in the temporary buffer 13, and performs the first execution. The same frame drop control and quantization control as in the embodiment are performed.

(Operation at Time E in FIG. 5) A communication error has occurred, and the image receiving apparatus 1000 transmits a retransmission request to the image encoding apparatus 40. Upon receiving the retransmission request, transmission control section 432 outputs an error notification to mode selection section 413 and threshold value switching section 421.

The mode selection section 413 outputs an identification signal indicating the error mode to the third quantization control section 412, similarly to the time point A in FIG. Accordingly, the third quantization control unit 412 sets a quantization value having a quantization step width larger than the quantization step width represented by the quantization value determined by the second quantization control unit 411. , Encoding device 1
Output to 2.

On the other hand, the threshold value switching unit 421 operates as shown in FIG.
As in the time point A, the connection state of the second SW 422 is switched from the state in which the terminals 1-3 are connected to the state in which the terminals 2-3 are connected. Accordingly, the frame drop control unit 161 performs the same frame drop control and quantization control as in the first embodiment.

(Operation at time point F in FIG. 5) Since the time width between the time points E and F does not exceed the fixed time T, the mode selection section 413 sends the third quantization control section 412 an error mode. An identification signal indicating the presence is output. The third quantization control unit 412 performs the same operation as the time point E in FIG. The frame drop / quantization control device 42 also performs the same operation as at the time point E in FIG.

As described above, in the second embodiment, the transmission control unit 432 outputs an error notification indicating that retransmission has been performed to the quantization control device 41. The quantization control device 41 performs an operation in the error mode when receiving an error notification while operating in the error-free mode, and performs an operation in the error-free mode when the interval of the error notification becomes equal to or longer than a predetermined time T. Thus, the quantization control device 41 can easily perform the quantization control according to the communication state.

Further, according to the second embodiment, the frame drop / quantization control device 42 is provided according to the error notification indicating that the transmission control unit 432 has performed retransmission and the error notification interval.
Switches the threshold value for determining whether or not to store the image data in the temporary buffer 13 in the transmission buffer 171. Therefore, it is necessary to perform frame drop control and quantization control according to the communication state. Can be.

In the quantization control device 41, the image frame is divided into a priority area where the subject is displayed and a non-priority area where the background is displayed. Is newly provided, and during the error mode, the quantization value of the non-priority area is set to a quantization value having a quantization step width larger than the quantization value determined by the third quantization control unit 412. Then, the encoding device 12 may be notified. Further, during the error mode, the encoding device 12 may be controlled so that the encoding operation and the transmission operation are not performed for the non-priority area,
The high frequency component of the input image may be dropped by the pre-filter, or only the low frequency component of the DCT coefficient may be encoded.

Also, in the frame drop / quantization control device 42, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided, and frame drop occurs. Then, the quantization value of the non-priority area of the next image frame is set to a quantization value having a larger quantization step width than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may make it. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

In the case where an image frame is divided into a priority area and a non-priority area, if a drop-out occurs, the drop-out / quantization control unit 42 controls to expand the range of the non-priority area. You may.

In the case where an image frame is divided into a priority area and a non-priority area, if a drop-out occurs, the drop-out / quantization control device 42 sets the quantization value of the non-priority area to A quantization value having a larger quantization step width than the quantization value used when encoding the non-priority region of the dropped image frame may be notified to the encoding device 12.

In the case where an image frame is divided into a priority area and a non-priority area, if a frame loss occurs, for the non-priority area, the frame deletion / quantization control device 42 performs an encoding operation and The encoding device 12 may be controlled so as not to perform the transmission operation, the high frequency component of the input image may be dropped by a prefilter, or only the low frequency component of the DCT coefficient may be encoded. No problem.

(Third Embodiment) FIG. 6 shows a third embodiment of the present invention.
It is a block diagram showing the composition of the picture coding device concerning an embodiment. 6, the image encoding device 60 includes an image input unit 11, an encoding device 12, a temporary buffer 13, a first SW 14, a generated information amount calculation unit 15,
The transmission control device 17 includes a transmission control device 17, a quantization control device 41, a frame drop / quantization control device 42, and an error correction coding device 61. Furthermore, the error correction encoding device 61
, A second error correction coding unit 612, and a second SW 613. The transmission control device 17 includes a transmission buffer 171 and a transmission control unit 172.
And

[0359] The transmission control unit 172 is
When an error signal is received from 0, an error notification indicating that the error signal has been received is output to the quantization control device 41 and the frame drop / quantization control device. The first error correction encoding unit 611 adds an error correction code with low error correction capability to the image data encoded by the encoding device 12 and stores the image data in the temporary buffer 13. The second error correction encoding unit 612 adds the first image to the encoded image data.
The error correction code having a higher error correction capability than the error correction code added by the error correction encoding unit 611 is added and stored in the temporary buffer 13. Third SW6
In 13, the connection state is switched by the quantization control device 41. Like the quantization control device 41 in the second embodiment (see FIG. 4), the quantization control device 41 includes a second quantization control unit 411, a third quantization control unit 412,
And a mode selection unit 413. However, the mode selection unit 4
13 is an error free mode or an error mode,
It has a function of switching the connection state of the third SW 613.

Since the image input section 11 and the transmission buffer 171 have the same configuration and function as the corresponding blocks in the first conventional example of FIG. 18, their description is omitted here. Further, the encoding device 12, the temporary buffer 13, the first SW 14, the generated information amount calculation unit 15, and the frame drop / quantization control device 42 have the same configuration and functions as the corresponding blocks of the second embodiment. Here, the description is omitted.

Next, the operation of the image encoding device 60 shown in FIG. 6 will be described. However, the quantization control operation of the quantization control device 41 and the threshold value control operation of the frame drop / quantization control device 42 are the same as those in the second embodiment, and therefore the description is omitted here. In addition, the encoding operation in the encoding device 12 is the same as that of the first conventional example, and the description is omitted. Therefore, only the operation when the image encoding device 60 receives the error signal will be described below.

Here, the image receiving apparatus 1000 connected to the image encoding apparatus 60 wirelessly or by a wire has a function of calculating an error rate of received image data, and the calculated error rate is predetermined. If the threshold value is larger than the threshold value, a function to notify an error signal to the image transmitting terminal is provided. In the following description of the operation, it is assumed that the error rate of the image data is larger than a predetermined threshold.

In the above case, the image receiving device 1000 outputs an error signal to the image encoding device 60. Upon receiving the error signal, the transmission control unit 172 in the image encoding device 60 outputs an error notification indicating that the error signal has been received to the quantization control device 41 and the frame drop / quantization control device 42. Upon receiving the error notification, the quantization control device 41 changes the state from the error free mode to the error mode, and performs the same quantization control as in the error mode in the second embodiment. Further, the quantization control device 41
Switches the connection state of the third SW 613 to a state in which the terminals 1-3 are connected. Therefore, the image frame input from the image input unit 11 is quantized with a quantization value having a larger quantization step width than in the error-free mode,
Further, after the second error correction encoding unit 612 adds an error correction code having a high error correction capability, the image receiving apparatus 1000
Is output to

When the transmission control unit 172 does not receive an error signal from the image receiving terminal 1000 for a predetermined time or more, that is, the transmission control unit 172 sends an error notification to the quantization control device 41 for a predetermined time or more. If not notified, the quantization control device 41 changes the state from the error mode to the error free mode, and performs the same quantization control as in the error free mode in the second embodiment. Further, the quantization control device 41 switches the connection state of the third SW 613 to a state where the terminals 1-2 are connected.
Therefore, the image frame input from the image input unit 11 is quantized with a quantization value having a smaller quantization step width than in the error mode, and the first error correction encoding unit 61
After adding an error correction code having a low error correction capability in step 1,
It is output to the image receiving device 1000.

As described above, according to the third embodiment,
In the error free mode, the quantization control device 41 performs normal quantization control and performs coding control to add an error correction code having a low error correction capability. In the error mode, the quantization control device 41 is set in the error free mode. A quantization value with a larger quantization step width (the image data amount is smaller) than the quantized value is set, and an error correction code with a high error correction capability is added (the error correction code amount is increased). Control. As a result, encoding control according to the communication state can be easily realized, and even when the communication state deteriorates, the delay time until a moving image is displayed on the image receiving apparatus 1000 can be reduced.

In the third embodiment, the image receiving device 1000 determines whether the communication state is good or bad based on the error rate.
From 000, the error rate may be transmitted to the image encoding device 60 instead of the error signal, and the image encoding device 60 may determine the error mode or the error free mode based on the magnitude of the received error rate. . Further, the image coding device 60 may calculate the average of the error rates received from the image receiving device 1000, and determine the error mode or the error free mode based on the calculated average error rate.

The image receiving apparatus 1000 calculates the average of the error rates calculated within a predetermined time, and if the calculated average error rate exceeds a predetermined threshold value, the error signal is output to the image receiving apparatus 1000. The image data may be transmitted to the encoding device 60, and the image encoding device 60 may perform the same operation as that in the third embodiment based on the received error signal.

The image receiving apparatus 1000 transmits the calculated average error rate to the image coding apparatus 60, and the image coding apparatus 60 performs the same processing as in the third embodiment based on the received average error rate. The operation may be performed.

In the quantization control device 41, the image frame is separated into a priority area where the subject is displayed and a non-priority area where the background is displayed, and a quantization value is set for each area. Is newly provided, and during the error mode, the quantization value of the non-priority area is set to a quantization value having a quantization step width larger than the quantization value determined by the third quantization control unit 412. Then, the encoding device 12 may be notified. Further, during the error mode, the encoding device 12 may be controlled so that the encoding operation and the transmission operation are not performed for the non-priority area,
The high frequency component of the input image may be dropped by the pre-filter, or only the low frequency component of the DCT coefficient may be encoded.

Also, in the frame drop / quantization control device 42, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided. , The quantization value of the non-priority area of the next image frame is set as a quantization value having a quantization step width larger than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may do so. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

In the case where an image frame is divided into a priority area and a non-priority area, if a frame loss occurs, the frame deletion / quantization control device 42 performs control to expand the range of the non-priority area. Is also good.

In the case where an image frame is divided into a priority area and a non-priority area, if a frame deletion occurs, the frame deletion / quantization control device 42 sets the quantization value of the non-priority area to the frame priority. A quantization value having a larger quantization step width than the quantization value used when encoding the non-priority region of the dropped image frame may be notified to the encoding device 12.

In the case where an image frame is divided into a priority area and a non-priority area, if a frame loss occurs, the frame-elimination / quantization is performed so that the non-priority area is not coded or operated. The encoding control device 42 may control the encoding device 12, may reduce the high-frequency component of the input image by a pre-filter,
Only the low frequency components of the CT coefficients may be encoded.

(Fourth Embodiment) FIG. 7 shows a fourth embodiment of the present invention.
It is a block diagram showing the composition of the picture coding device concerning an embodiment. 7, the image encoding device 70 includes an image input unit 11, an encoding device 12, a temporary buffer 13, a first SW 14, a generated information amount calculation unit 15,
Quantization control device 71 and frame drop / quantization control device 72
, A transmission control device 43 and the output terminal 18. Further, the quantization control device 71 includes an average throughput calculation unit 711, a coding rate calculation unit 712, and a fourth quantization control unit 713. The frame drop / quantization control device 72 includes an allowable delay time storage unit 721, a transmittable information amount calculation unit 722, a frame drop control unit 161, and a first quantization control unit 163. Also, the transmission control device 43
Includes a transmission buffer 171, a retransmission buffer 431, and a transmission control unit 432.

The coding device 12 performs the same operation as that described in the first conventional example, but when coding an image frame, the quantization control device 71 or the frame drop / quantization control device 72 is used. The difference is that encoding is performed using the quantized value determined in (1), and that an image frame to be encoded is determined in response to a notification from the quantization control device 71. The quantization value determined by the quantization control device 72 has priority.

The average throughput calculation unit 711 receives the number of bits read from the transmission buffer 171 within a predetermined unit time t from the transmission control unit 432, calculates the throughput within a certain time T, and calculates the average throughput and I do. The method of calculating the average throughput may be a method as described in an eleventh embodiment described later. The coding rate calculation unit 712 calculates a coding rate, which is a rate of frames that can be transmitted per unit time, based on the average throughput and the amount of information generated per frame. When determining the quantization value of the next image frame, the fourth quantization control unit 713 uses the coding rate calculated from the average throughput and performs quantization control (FIG. 21) in the same manner as in the first conventional example. When the remaining amount of data in the transmission buffer 171 becomes equal to or less than a certain threshold value, the encoding device 12 is notified to perform encoding of one image frame. Upon receiving the notification, the encoding device 12 converts the input image at that time into the quantization control device 71 or the frame drop / quantization control device 72.
Is encoded with the quantized value determined by
, The frame is dropped until a notification that the remaining amount of data in the transmission buffer 171 has become equal to or less than the threshold value is received.

[0377] The permissible delay time storage section 721 stores the maximum delay time allowed when the moving image is displayed on the image receiving terminal 1000 due to a difference between the image and the sound. The transmittable information amount calculation unit 722 includes an allowable delay time storage unit 721.
Calculate the number of bits that can be transmitted within the maximum allowable delay time stored in. The frame dropping control unit 161 compares the number of bits calculated by the transmittable information amount calculation unit 722 with the information amount of the image data calculated by the generated information amount calculation unit 15, and If the calculated number of bits is larger, the image data in the temporary buffer 13 is dropped. The transmission control unit 432 performs the second operation except that the number of bits read from the transmission buffer within a predetermined unit time t is output to the quantization control device 71.
The same operation as the transmission control unit 432 according to the embodiment is performed.

The image input unit 11 and the transmission buffer 17
1 has a configuration and a function similar to those of the corresponding block in the first conventional example shown in FIG. Also, the temporary buffer 13, the first SW
14, the generated information amount calculation unit 15, the frame drop control unit 161,
The first quantization control unit 163 and the retransmission buffer 431 have the same configurations and functions as the corresponding blocks of the second embodiment, and thus description thereof is omitted here.

The operation of the quantization control device 71 will be described. The generated information amount calculation unit 15 calculates the information amount of the image data stored in the temporary buffer 13, and calculates the coding rate calculation unit 712 and the frame drop control unit 161.
Output to The transmission control unit 432 calculates the number of bits read from the transmission buffer 171 within a predetermined unit time t, and outputs the number of bits to the average throughput calculation unit 711.
Average throughput calculation section 711 calculates an average throughput within a fixed time T based on the number of bits output from transmission control section 432, and outputs the average throughput to coding rate calculation section 712 and transmittable information amount calculation section 722.

The coding rate calculation unit 712 calculates the coding rate from the average throughput and the amount of information in the temporary buffer 13 using the following equation (1), and calculates the coding rate.
13 is output. Coding rate = (LN / S) / RN (1) In the above equation (1), various quantities are as follows. LN: average throughput S: number of frame samplings per second RN: amount of information in the temporary buffer 13

The fourth quantization control unit 713 performs the same quantization control as in the first conventional example (see FIG. 21) using the calculated coding rate, and outputs the determined quantization value to the coding apparatus. 12
Output to At this time, if the time for measuring the average throughput LN is set to 300 msec or more, burst errors of the PHS can be averaged well. However,
Unlike the first conventional example, the method of determining the frame to be encoded is determined based on the remaining amount of data in the transmission buffer 171.

Next, the operation of the frame drop / quantization control device 72 will be described. Transmittable information amount calculation unit 72
2 is an average throughput within a fixed time T calculated by the average throughput calculation unit 711 and an allowable delay time storage unit 7
Based on the maximum allowable delay time within the maximum allowable delay time, the transmittable information amount that can be transmitted within the maximum allowable delay time is calculated and output to the frame drop control unit 161. The frame drop control unit 161 compares the transmittable information amount with the image data in the temporary buffer 13, and if the transmittable information amount is larger, the first S
The connection state of W14 is set to a state where the terminals 1-3 are connected. Therefore, the image data in the temporary buffer 13 is stored in the transmission buffer 171 and the transmission control unit 432
Sent by

On the other hand, when the transmittable information amount is smaller than the image data in the temporary buffer 13, that is, when the image data in the temporary buffer 13 cannot be displayed on the image receiving terminal 1000 within the maximum allowable delay time. The frame drop control unit 161 changes the connection state of the first SW 14 to a state where the terminals 1-2 are connected, and drops the image data in the temporary buffer 13. Further, the frame drop control unit 161 controls the temporary buffer 1
A frame removal signal indicating that frame removal of the image data in 3 is performed is output to the first quantization unit 163. Upon receiving the frame drop signal from the frame drop control unit 161, the first quantization control unit 163 encodes the image frame (frame in the temporary buffer 13) in which the frame is to be dropped, as in the first embodiment. Then, a quantization value having a larger quantization step width than the quantization value used for the encoding is set and output to the encoding device 12. Further, it notifies the encoding device 12 to perform encoding of the immediately next image frame.

As described above, according to the fourth embodiment,
Next, since the quantization value to be encoded is determined using the encoding rate obtained from the average throughput for a predetermined period of time, even when the throughput varies due to retransmission or the like, the optimal value according to the communication state is determined. A quantization value can be determined.

Also, the amount of information that can be transmitted within the permissible delay time is calculated based on the permissible delay time and the average throughput that are set in advance, and this calculated amount of information is used as the information amount of the image frame in the temporary buffer 13. Since the threshold value is used as to whether or not to drop frames, it is possible to perform frame drop control according to the communication state.

Since the average throughput is calculated on the basis of the number of bits read from the transmission buffer 171, the retransmission method may be selected in Go-back-N.
Even in the transient state in which the transmission rate fluctuates, stable coding control can be realized even in a transient state in which the transmission rate fluctuates.

Here, the quantization controller 71 and the frame drop /
The quantization control devices 72 may be used alone and in combination with the other devices shown in FIG. As an ideal curve in the fourth quantization control unit 713 in the quantization control device 71, a curve in which the frame rate is always constant or a curve in which the quantization value is always constant may be used. Further, as a control algorithm of the fourth quantization control unit 713 in the quantization control device 71, a quantization value may be determined using a table of a combination of a coding rate and a quantization value, or a transmission buffer may be determined. The frame rate may be kept constant or the quantization value may be determined so as to operate with an ideal curve by using a calculation formula based on the remaining amount of data in 171 and the coding rate.

In the fourth embodiment, the frame drop /
The quantization control device 72 sets the maximum allowable delay time in advance. However, like the frame drop / quantization control device 81 in FIG. 8, the average generation information amount of the image frame transmitted during a predetermined time and An average delay time calculation unit 811 for calculating an average delay time from the average throughput at that time is further provided, and the average delay time calculated by the average delay time calculation unit 811 is replaced with a maximum allowable delay time, and the temporary buffer 14 The frame removal control of the image frames within the frame may be performed.

In the quantization control device 71, the image frame is separated into a priority area where the subject is displayed and a non-priority area where the background is displayed, and the quantization value is determined for each area. A function for setting may be newly provided. In this case, when the coding rate exceeds a predetermined threshold value, the coding device 12 may be controlled so that the coding operation and the transmission operation are not performed for the non-priority area. The high frequency component of the input image may be reduced by the filter, or only the low frequency component of the DCT coefficient may be encoded.

Also, in the frame drop / quantization control device 72, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided. , The quantization value of the non-priority area of the next image frame is set as a quantization value having a quantization step width larger than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may do so. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

In the case where an image frame is divided into a priority area and a non-priority area, if a frame loss occurs, the frame deletion / quantization control unit 42 performs control to expand the range of the non-priority area. You may.

Also, in the fourth embodiment, the average throughput during the predetermined fixed time T is calculated, but the time T for obtaining the average is made variable in accordance with the error condition of the communication path. It may be. If the time T for obtaining the average is shortened, the quantization control and the dropout control can be performed quickly in response to the fluctuation of the throughput. However, in a situation where burst-like errors occur periodically on the communication path, if control is performed using an average of a very short time, when a burst error occurs, the average throughput value drops significantly, Means that the average throughput value is large and the control is not stable. A Study on Technology for Preventing Video Coding Quality Degradation in Mobile Communications (Hisashi Ibaraki et al., Journal of the Institute of Image Electronics Engineers of Japan Vol.23
According to No. 5 (1994) P445-453), it is described that when the Doppler frequency is constant, the burst error occurs periodically at a cycle of 0.35 times the Doppler frequency. In such a case, a section in which the number of bits read from the transmission buffer 171 within a unit time t is smaller than a certain threshold value is regarded as a burst error section, and a period is detected by measuring an occurrence interval of the burst error section. If this cycle is used as the time T for calculating the average and the average value is obtained, stable quantization control and dropout control can be performed without fluctuation even when a burst error occurs. At this time, if the period is obtained by averaging the occurrence intervals of a plurality of burst error sections, more stable operation can be achieved. If the interval between burst error intervals exceeds a certain length, it is determined that there is no burst error, and if the time T for obtaining the average is shortened, it is possible to quickly respond to a change in throughput.

In the fourth embodiment, the configuration in which the average throughput calculation section 711 is included in the quantization control device 71 has been described. However, the average throughput calculation section 711 is included in the transmission control device 43. You may. Also,
A configuration is also conceivable in which the unit time t is the same as the time T for obtaining the average, and the average is not calculated in the average throughput calculation unit 711.

Also, the transmission control unit 432 sends the output terminal 18
May be multiplexed with audio or other data and transmitted to the image receiving apparatus 1000. In this case, the transmission rate of the image data output from the transmission control unit 432 is multiplexed. May fluctuate depending on percentage. That is, the throughput of the image data varies not only depending on the occurrence state of the error but also according to the presence / absence of the multiplexed voice or other data. Also at this time, if the amount of data read from the transmission buffer 432 is known, the throughput of the image data can be known, so that the present invention can exert the same effect.

(Fifth Embodiment) FIG. 9 shows a fifth embodiment of the present invention.
It is a block diagram showing the composition of the picture transmission system concerning an embodiment. In FIG. 9, an image encoding device 150 and an image decoding device 151 are wirelessly connected to base stations 152 and 153, respectively. The base stations 152 and 153 are connected by wire. The retransmission control described in the second embodiment is performed in the respective radio sections A (between the image encoding device 150 and the base station 152) and B (between the base station 153 and the image decoding device 151). Think.

The configuration of the image coding device 150 is almost the same as that of the image coding device 70 (see FIG. 7) in the fourth embodiment. The base station 152 includes an error control unit 156. The base station 153 includes a retransmission buffer 157 and a transmission control unit 158. Image decoding device 151
Is an error control unit 151a, a bit counting unit 151b,
It comprises a decoding device 151c and an image display unit 151d.

[0397] Image encoding apparatus 150 transmits the encoded data to base station 152, and stores the data being transmitted in retransmission buffer 431. The error control unit 156 of the base station 152 performs error detection on the data block received from the image encoding device 150, and when an error is detected,
The retransmission request of the erroneous data block is sent to the image encoding device 1
Perform for 50. When receiving a retransmission request from the base station 152, the transmission control unit 432 of the image encoding device 150 retransmits the corresponding data block in the retransmission buffer 431.

The transmission control section 158 of the base station 153 transmits the data received from the base station 152 to the image decoding device 151, and stores the data block being transmitted in the retransmission buffer 157. The error control unit 151a of the image decoding device 151 detects an error in the data block received from the base station 153, and when detecting an error, requests the base station 153 to retransmit the erroneous data block.
The transmission control unit 158 of the base station 153
When there is a retransmission request from 51, the corresponding data block in the retransmission buffer 157 is retransmitted. Image decoding device 151
Decrypts the correctly received data with the decoding device 151c and displays it on the image display unit 151d. In addition, the bit counting unit 151b of the image decoding device 151 counts the number of bits of data received without error within a predetermined time T, and sends the data encoding device 150 a return path (wireless section B → base station B). 153 → base station 152 → wireless section A).

The transmission control section 432 of the image encoding device 150
Receives the bit number information from the image decoding device 151 and outputs the information to the quantization control device 71.

[0400] When the image encoding device 150 directly performs retransmission control with the image decoding device 151, the number of bits correctly received by the image decoding device 151 can be known. However, in the case where there are a plurality of retransmission control sections between the image encoding device 150 and the image decoding device 151, as shown in FIG. 9, the image encoding device 150 starts transmitting to the base station from the retransmission control performed by itself. 152
, Only the number of bits received correctly, that is, the throughput of the wireless section A can be known. Wireless sections A and B
Are not necessarily the same, and the radio section B
Is lower than the throughput in the wired section or the wireless section A, the communication throughput between the image encoding device 150 and the image decoding device 151 is regulated by the throughput in the wireless section B.

Therefore, the image decoding device 151 counts the number of bits of data received without error and transmits it to the image encoding device 150 via a feedback path, so that the image encoding device 150 Communication throughput between the decoding device 150 and the image decoding device 151 can be known. Then, the image encoding device 150 can perform accurate quantization control and frame dropping control based on the known communication throughput. The method of calculating the number of bits of image data that can be received without error may be a method as described in the twelfth embodiment.

The base station 153 counts the number of bits correctly received by the image decoding device 151 within a predetermined time T, and transmits this to the image coding device 150 via a feedback path. It may be.

Further, the base station 153 calculates the average throughput based on the number of bits correctly received by the image decoding device 151 within a predetermined time T, and calculates the average throughput via the feedback path. 150.

In the quantization controller 71, the image frame is separated into a priority area where the subject is displayed and a non-priority area where the background is displayed, and the quantization value is set for each area. May be newly provided. In this case, when the coding rate exceeds a predetermined threshold value, the coding apparatus 12 may be controlled so that coding and transmission operations are not performed for the non-priority area, or the input may be performed by a pre-filter. The high-frequency component of the image may be dropped, or only the low-frequency component of the DCT coefficient may be encoded.

Also, in the frame drop / quantization control device 72, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided. , The quantization value of the non-priority area of the next image frame is set as a quantization value having a quantization step width larger than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may do so. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

In the case where the image frame is divided into a priority area and a non-priority area, if a frame loss occurs, the frame deletion / quantization control unit 42 performs control to expand the range of the non-priority area. You may.

Also, in the fifth embodiment, the number of bits of data received without error within a predetermined time T is determined from the image decoding device 151 to the image encoding device 150.
And the average throughput is calculated in the image encoding device 150. However, the average throughput is calculated in the image decoding device 151, and the average throughput is transmitted to the image encoding device 150 via the feedback path. You may do so. The method of calculating the average throughput will be described in a thirteenth embodiment described later.

In the fifth embodiment, the average throughput in the predetermined time T is calculated. However, the time T for obtaining the average is made variable according to the error condition of the communication path. You may do so. If the time T for obtaining the average is shortened, the quantization control and the dropout control can be performed quickly in response to the fluctuation of the throughput. However, in a situation where burst-like errors occur periodically on the communication path, if control is performed using an average of a very short time, when a burst error occurs, the average throughput value drops significantly, Means that the average throughput value is large and the control is not stable. A Study on Technology for Preventing Video Coding Quality Degradation in Mobile Communications (Hisashi Ibaraki et al., Journal of the Institute of Image Electronics Engineers of Japan Vol.23
No. 5 (1994) P445-P453), it is described that when the Doppler frequency is constant, the burst error occurs periodically at a period of 0.35 times the Doppler frequency. In such a case, a period in which the number of bits received without error from the image encoding device 150 within the unit time t is smaller than a certain threshold value is regarded as a burst error period, and the period of occurrence of this burst error period is measured to determine the period. If the average is determined using this period as a period T for calculating the average, stable quantization control and frame dropping control can be performed without fluctuation even when a burst error occurs. At this time, if the period is obtained by averaging the occurrence intervals of a plurality of burst error sections, more stable operation can be achieved. If the interval between burst error intervals exceeds a certain length, it is determined that there is no burst error, and if the time T for obtaining the average is shortened, it is possible to quickly respond to a change in throughput.

[0409] The image data output from the transmission control unit 432 may be multiplexed with audio or other data and transmitted to the image decoding device 151. The transmission rate of the image data to be multiplexed may fluctuate according to the multiplexing ratio. That is, the throughput of the image data varies not only depending on the occurrence state of the error, but also according to the presence / absence of the multiplexed voice or other data. Also at this time, if the number of bits of the image data that can be received without error is known, the throughput of the image data can be known, so that the present invention can exert the same effect.

(Sixth Embodiment) FIG. 10 is a block diagram showing a configuration of an image transmission system according to a sixth embodiment of the present invention. In FIG. 10, the image encoding device 163
The image decoding device 164 is connected to the base stations 152 and 153 by radio, respectively. Base station 15
2 and 153 are connected by wire. As in the fifth embodiment, retransmission control is performed for each wireless section A
(Formation between the image encoding device 163 and the base station 152) and B (between the base station 153 and the image decoding device 164) are considered.

[0411] The configuration of the image encoding device 163 is almost the same as that of the image encoding device 40 (see Fig. 4) in the second embodiment, and the configuration of the base stations 152 and 153 is the same as that of the fifth embodiment.
This is the same as those in the embodiment (see FIG. 9).
The configuration of the image decoding device 164 is almost the same as that of the image decoding device 151 of the fifth embodiment.
4a, when an error in received data is detected and retransmission is performed, retransmission occurrence information is transmitted to the return path (wireless section B → base station 1).
53 → base station 152 → wireless section A) to transmit to image coding apparatus 163.

[0412] Transmission control section 432 of image coding apparatus 163
Receives the retransmission occurrence information from the image decoding apparatus 164 and outputs an error notification when the retransmission has been performed by itself or when the retransmission occurrence information has been received.

[0413] If the image encoding device 163 directly performs retransmission control with the image decoding device 164, the image encoding device 163 determines whether or not the image encoding device 163 itself has performed retransmission. Error occurrence status can be understood. However, in the case where there are a plurality of retransmission control sections between the image coding apparatus 163 and the image decoding apparatus 164, as shown in FIG. 10, the image coding apparatus 163 cannot grasp the error occurrence status of the wireless section B. .

[0414] Therefore, when the image decoding apparatus 164 performs retransmission, it transmits retransmission occurrence information to the image encoding apparatus 163 via a feedback path.
63 denotes an image encoding device 163 and an image decoding device 1
It is possible to know the error occurrence status between 64. And
The image encoding device 163 can perform accurate quantization control and frame dropping control based on the known error occurrence state.

[0415] In the quantization control device 41, the image frame is separated into a priority area where the subject is displayed and a non-priority area where the background is displayed, and the quantization value is determined for each area. A new setting function is provided, and in the error mode, the quantization value of the non-priority area is set to a quantization value having a larger quantization step width than the quantization value determined by the third quantization control unit 412. , The encoding device 12 may be notified. Further, during the error mode, the quantization control device 41 may control the encoding device 12 so as not to perform the encoding operation and the transmission operation in the non-priority area, or the input image may be controlled by a pre-filter. May be dropped, or only the low-frequency component of the DCT coefficient may be encoded.

[0416] In the frame drop / quantization control device 42, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided. , The quantization value of the non-priority area of the next image frame is set as a quantization value having a quantization step width larger than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may do so. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

In the case where the image frame is divided into a priority area and a non-priority area, if a frame loss occurs, the frame deletion / quantization control device 42 performs control for expanding the range of the non-priority area. You may.

When the image frame is divided into a priority area and a non-priority area, and a frame drop occurs, the frame drop / quantization control device 42 sets the quantization value of the non-priority area to the frame priority. A quantization value larger than the quantization value used for encoding the non-priority area of the dropped image frame may be notified to the encoding device 12.

[0419] Also, when a frame is dropped in a case where an image frame is divided into a priority area and a non-priority area, when the frame is dropped in the non-priority area, the encoding and transmission operations are not performed. The quantization control device 42 may control the encoding device 12, may reduce the high-frequency component of the input image by a pre-filter, or may encode only the low-frequency component of the DCT coefficient. I do not care.

In the sixth embodiment, when retransmission occurs, retransmission occurrence information is transmitted from the image decoding device 164 to the image encoding device 163 via a feedback path. If a retransmission has occurred,
The configuration may be such that transmission is performed from the base station 153 (the terminal that has performed retransmission) to the image encoding device 163.

(Seventh Embodiment) FIG. 11 is a block diagram showing a configuration of an image coding apparatus according to a seventh embodiment of the present invention. In FIG. 11, an image encoding device 110
The image input unit 11, the encoding device 12, the temporary buffer 13, the first SW 14, the generated information amount calculation unit 15, the transmission control device 17, the frame drop / quantization control device 111, the error A correction encoding device 61.
Further, the error correction coding device 61 includes a first error correction coding unit 611, a second error correction coding unit 612, and a second SW 613. In addition, the transmission control device 17
It includes a transmission buffer 171 and a transmission control unit 172.

The transmission control section 172 transmits image data to the image receiving apparatus 1000 at a preset transmission rate, and upon receiving an error signal from the image receiving apparatus 1000, sends an error notification indicating that the error signal has been received. Is output to the frame drop / quantization control device 111. First
The error correction encoding unit 611 adds an error correction code having a low error correction capability to the image data encoded by the encoding device 12 and stores the image data in the temporary buffer 13. Second
The error correction coding unit 612 adds an error correction code having an error correction capability higher than the error correction code added by the first error correction coding unit 611 to the coded image data, The data is stored in the buffer 13. The connection state of the third SW 613 is switched by the frame drop / quantization control device 111. The frame removal / quantization control device 111 is the same as the frame removal / quantization control device 42 in the second embodiment shown in FIG.
1, a first quantization control unit 163, a threshold switching unit 421, a second SW 422, and a second threshold switching unit 424. However, the frame drop control unit 161
Has a function of switching the connection state of the third SW 613 according to the error free mode and the error mode in the second embodiment.

Note that the image input unit 11, the encoding device 12,
Since the transmission buffer 171 has the same configuration and function as the corresponding block in the first conventional example of FIG. 18, the description is omitted here. In addition, temporary buffer 1
3, the first SW 14, and the generated information amount calculation unit 15 have the same configurations and functions as the corresponding blocks of the second embodiment (FIG. 4), and thus description thereof is omitted here. Furthermore, the first error correction encoding device 611 and the second error correction encoding device 612 have the same configuration and function as the corresponding blocks of the third embodiment (FIG. 7), and thus description thereof is omitted here. I do.

Next, the image encoding device 110 shown in FIG.
Will be described. However, the encoding operation in the encoding device 12 is the same as that of the first conventional example in FIG.
Description is omitted. Therefore, in the following, the image encoding device 1
Only the operation when the device 10 receives the error signal will be described.

[0425] Here, the image receiving apparatus 1000 connected to the image encoding apparatus 110 wirelessly or by wire has a function of calculating an error rate of received image data, and the calculated error rate is predetermined. If the threshold value is larger than the threshold value, a function to notify an error signal to the image transmitting terminal is provided. In the following description of the operation, it is assumed that the error rate of the image data is larger than a predetermined threshold.

[0426] In the above case, the image receiving apparatus 1000 outputs an error signal to the image encoding apparatus 60. Upon receiving the error signal, the transmission control unit 172 of the image encoding device 110 outputs an error notification indicating that the error signal has been received to the frame drop / quantization control device 111. Upon receiving the error notification, the frame drop / quantization control device 111 changes the state from the error free mode to the error mode, and performs frame drop and quantization control as in the error mode in the second embodiment. Further, the frame drop / quantization control device 111 switches the connection state of the third SW 613 to a state where the terminals 1-3 are connected. Therefore, the image frame input from the image input unit 11 is output to the image receiving apparatus 1000 after the second error correction encoding unit 612 adds an error correction code having a high error correction capability.

When the transmission control unit 172 does not receive an error signal from the image receiving apparatus 1000 for a predetermined time or more, that is, the transmission control unit 172 sends an error notification for a certain time or more to the frame drop / quantization control. If not notified to the device 111, the frame drop / quantization control device 111
Changes the state from the error mode to the error-free mode, and performs the same frame drop and quantization control as in the error-free mode in the second embodiment. In addition, Koma drop /
The quantization control device 111 switches the connection state of the third SW 613 to a state in which the terminals 1-2 are connected. Therefore, the image frame input from the image input unit 11 is output to the image receiving apparatus 1000 after the first error correction encoding unit 611 adds an error correction code having a low error correction capability.

As described above, in the seventh embodiment, in the error-free mode, the frame drop / quantization control device 111 performs coding control so as to add an error correction code having a low error correction capability, and performs error control. In the mode, encoding control is performed so that an error correction code with high error correction capability is added (the amount of error correction code is increased). In addition, when the amount of generated information increases due to the movement of a subject and the like and frame dropping occurs, a quantization value having a larger quantization step width than the quantization value used when encoding the image frame in which the frame dropping occurred. The encoding device 12 is controlled so that the encoding is performed. Therefore, encoding control according to the communication state can be easily realized, and even when the communication state deteriorates, a delay time until a moving image is displayed on the image receiving apparatus 1000 can be reduced.

In the seventh embodiment, the image receiving apparatus 1000 determines whether the communication state is good or bad based on the error rate.
000 may transmit an error rate to the image encoding device 110 instead of the error signal, and the image encoding device 110 may determine the error mode or the error free mode based on the received error rate. . Further, the image coding apparatus 110 may calculate the average of the error rates received from the image receiving apparatus 1000, and determine the error mode or the error-free mode based on the calculated average error rate.

Also, the image receiving apparatus 1000 calculates the average of the error rates calculated within a predetermined time, and if the calculated average error rate exceeds a predetermined threshold, the error signal is transmitted to the image receiving apparatus 1000. Transmitted to the encoding device 110,
The image encoding device 110 may perform the same operation as in the seventh embodiment based on the received error signal.

[0431] The image receiving apparatus 1000 transmits the calculated average error rate to the image encoding apparatus 110, and the image encoding apparatus 110 compares the calculated average error rate with the seventh embodiment based on the received average error rate. A similar operation may be performed.

Also, in the frame drop / quantization control device 111, a function of separating an image frame into a priority region and a non-priority region and setting a quantization value for each region is newly provided. , The quantization value of the non-priority area of the next image frame is set as a quantization value having a quantization step width larger than the quantization value determined by the first quantization control unit 163, and is notified to the encoding device 12. You may do so. Furthermore, when a frame is dropped, the non-priority area is
The encoding device 12 may be controlled so as not to perform the encoding operation and the transmission operation, the high frequency component of the input image may be dropped by a pre-filter, or only the low frequency component of the DCT coefficient may be encoded. You may do it.

(Eighth Embodiment) A quantization value determination method of the first quantization control unit 163 in the first to seventh embodiments will be described with reference to FIG.

FIG. 12 is a graph showing the relationship between the quantization value and the amount of generated information when an image frame in which the movement of the subject becomes large is encoded by changing the quantization value. FIG.
In FIG. 2, the vertical axis represents the amount of generated information, and the horizontal axis represents the quantization value. The test images used in FIG. 12 include table-tenni, which is an evaluation image for a video conference.
The simulation was performed by using s, sushie, and clare and increasing the frame drop width so that the movement of the subject increased.

As is apparent from FIG. 12, for an image frame in which the motion of the subject is large, that is, an image frame whose transmission delay time exceeds the maximum delay time (500 msec) in which the user does not feel a sense of discomfort, the amount of generated information is quantized. You can see that it has a characteristic that is almost inversely proportional to the value ratio

Therefore, the first quantization control unit 163 in the first to third, sixth, and seventh embodiments determines the quantization value of the next image frame using the following equation (2). Next_Q = (Now_Q * Bits) / (Max_Delay_Time * Rate) (2)

In the above equation (2), various quantities are as follows. Next_Q: Quantized value used for the next image frame to be encoded Now_Q: Quantized value used when encoding an image frame with a large subject motion Bits: Generation of a result of encoding an image frame with a large subject motion Information amount Max_Delay_Time: Maximum allowable delay time (transmission delay time that does not cause visual discomfort: 500 msec) Rate: Transmission rate

Also, the first quantization control unit 163 in the fourth and fifth embodiments predicts and predicts the amount of generated information when the quantization value is changed using the following equation (3). The coding rate is calculated from the generated information amount, the transmission rate, and the average throughput by using the equation (1) in the fourth embodiment, and the coding rate is set to the visually optimum upper limit 102 of the coding rate in FIG. And the lower limit 1 of the coding rate that is visually optimal
03 is determined. Pred_Bits = (Quant / Parameter) * Bits (3)

In the above equation (3), various quantities are as follows. Pred_Bits: Estimated generated information amount Quant: Quantized value used to encode an image frame with a large subject movement Parameter: Quantized value for which predicted generated information amount is desired Bits: Result of encoding an image frame with a large subject movement Of information generated

As described above, even when the coding rate-quantization accuracy characteristic greatly changes due to a large movement of a subject, the optimal quantization according to the changed coding rate-quantization accuracy characteristic can be performed by a simple calculation. Value can be determined immediately.

In the first to third, sixth and seventh embodiments, the image frame is divided into a priority area where the subject is displayed and a non-priority area where the background is displayed. When divided, the first quantization control device 163
May determine the quantization value of the non-priority area using the following equation (4). Next_BG_Q = (Now_BG_Q * BG_Bits) / (Max_Delay_Time * Rate-Clear_Bits) (4)

In the above equation (4), the various quantities are as follows. Next_BG_Q: Quantized value used for the next non-priority area to be encoded Now_BG_Q: Quantized value used when encoding the non-priority area of an image frame with a large subject motion BG_Bits: Non-quantity value of the image frame with a large subject motion Amount of information generated as a result of encoding the priority area Max_Delay_Time: maximum allowable delay time (transmission delay time that does not cause visual discomfort: 500 msec) Rate: transmission rate Clear_Bits: the priority area of an image frame with a large subject motion is encoded Resulting information volume

In the fourth and fifth embodiments, when an image frame is divided into a priority area and a non-priority area, the image coding apparatus 70 according to the fourth embodiment is used.
(FIG. 7) and the image encoding device 150 (FIG. 9) in the fifth embodiment may be configured as follows. That is, the image encoding device 70 and the image encoding device 15
0 stores an upper limit of a visually optimum coding rate and a lower limit of a visually optimum coding rate as shown in FIG. 21 when an image frame is divided into a priority area and a non-priority area. Then, the amount of predicted occurrence information for the priority region and the amount of predicted occurrence information for the non-priority region when the quantization value is changed from the following equations (5) and (6) are calculated, and the respective quantization amounts for the respective regions are calculated. It may be configured to determine the optimal quantization value for each region from the predicted occurrence information amount of the value and the upper and lower limits of the coding rate when dividing the image frame into the priority region and the non-priority region. Good.

Clear_Pred_Bits = (Clear_Quant / Parameter) * Clear_Bits (5) BG_Pred_Bits = (BG_Quant / Parameter) * BG_Bits (6)

In the above equation (5), various quantities are as follows. Clear_Pred_Bits: Estimated amount of generated information in the priority area Clear_Quant: Quantized value used to encode the priority area of an image frame in which the motion of the subject is large Parameter: Quantized value to predict the amount of generated information Clear_Bits: Large amount of motion of the subject Amount of information generated as a result of encoding the priority area of the image frame

In the above equation (6), various quantities are as follows. BG_Pred_Bits: Estimated amount of information generated in the non-priority area BG_Quant: Quantized value used to encode the non-priority area of the image frame in which the motion of the subject is large Parameter: Quantized value to predict the generated information amount BG_Bits: Motion of the object Of information generated as a result of encoding non-priority areas of large image frames

The transmission rate Rate in the above equations (2), (3) and (4) is a value obtained by adding the average throughput rate to the actual transmission rate as shown in the following equation (7). The quantization value may be determined in accordance with the throughput. Rate = transmission rate * average throughput rate (7)

(Ninth Embodiment) FIG. 13 is a block diagram showing a configuration of an image coding apparatus according to a ninth embodiment of the present invention. In FIG. 13, the image encoding device according to the present embodiment includes an image input unit 11, an image encoding device 12, a priority area / non-priority area change unit 1301, and a transmission control unit 17
2 is provided.

[0449] The encoding apparatus 12 converts the QCIF-size moving image captured from the image input unit 11 into an ITU recommendation H.264. 2
61 to generate an encoded bit string. Here, the QCIF coded image is divided into a priority region and a non-priority region, and each region is coded with different quantization precision. Priority area / non-priority area change unit 130
1 is based on the amount of information generated in the previous encoded frame,
A priority area and a non-priority area of an F (176 × 144 dots size) encoded image are determined. The transmission control unit 172 transmits the encoded bit string and monitors the communication throughput.

As an example, the operation when the size of the priority area is changed in two steps will be described with reference to the flow of FIG. The initial value of the priority area is the QCI shown in FIG.
This is the size (the size of 112 × 112 dots) of the central portion of the F image. As a result of encoding one frame (step S105), when the amount of generated information exceeds a certain threshold value 1 (step S101), the priority area is changed to the size (80 × 80 dots) of the central portion of the image shown in FIG. ) (Step S102) to reduce the amount of generated information. Conversely, when the amount of generated information decreases below a certain threshold value 2 (step S103), the priority area is changed again.
The size (112 × 112) at the center of the image in FIG.
) (Step S104), and a clear area is expanded. Thereafter, each region is encoded with a different quantization precision (step S105).

Although the priority area and the non-priority area are changed in two steps in the ninth embodiment, the priority area and the non-priority area may be changed in more steps. In the ninth embodiment, the priority area and the non-priority area are determined based on the amount of generated information of the pre-encoded frame. However, the communication throughput monitored by the transmission control unit, the fourth embodiment, As described in the embodiment, the priority area and the non-priority area may be changed based on the coding rate calculated from the generated information amount and the communication throughput. In this case, when the coding rate exceeds a predetermined threshold value, the image coding apparatus 12 may be controlled so that the coding operation and the transmission operation are not performed for the non-priority area, , The high-frequency component of the input image may be dropped, or only the low-frequency component of the DCT coefficient may be encoded.

In the ninth embodiment, the non-priority area is encoded with a different quantization precision from the priority area. However, the non-priority area may not be encoded.
The high frequency component of the input image may be dropped by the pre-filter, or only the low frequency component of the DCT coefficient may be encoded. In this case, the image range to be encoded changes according to the amount of generated information and the communication throughput.

[0453] Also, the amount of generated information of the pre-encoded frame is
When the threshold value exceeds a predetermined threshold value, the coding operation and the transmission operation may not be performed for the non-priority area, the high frequency component of the input image may be reduced by a pre-filter, Only the low frequency components of the coefficients may be encoded.

As described above, according to the ninth embodiment, even if the amount of generated information is extremely increased or the communication throughput is reduced, the image quality of the priority area does not need to be significantly degraded.

(Tenth Embodiment) Hereinafter, the first embodiment of the present invention will be described.
Embodiment 0 will be described with reference to the drawings. FIG.
FIG. 6 is a principle explanatory diagram of an encoding control method according to a tenth embodiment of the present invention. In FIG. 16, reference numeral 101 denotes a coding rate-quantization accuracy characteristic Ss (q) = L / R (q), which varies according to the magnitude of temporal motion of an input image and spatial frequency components. Then, it moves to the lower right when a large movement or a fine pattern is included, and moves to the upper left when there is little movement. The coding rate-quantization accuracy characteristics and the object of the present invention are described in detail in the section of the related art, and thus are omitted here. In FIG. 16, a lower limit value of the quantization accuracy is set. Therefore, the coding rate-quantization accuracy characteristics 101A to 101D move to the upper left as compared with the description of the conventional technique, but the operation principle is the same as that of the conventional technique.

Reference numeral 102 denotes a visually optimum upper limit of the coding rate, and reference numeral 103 denotes a visually optimum lower limit of the coding rate. The objective function may be assumed to be a set of points where the combination of the coding rate 105 and the quantization precision 104 is visually optimal, but in practice this is a subjectively obtained value, It fluctuates depending on the viewing distance and angle, and personal preference. Therefore, a visually optimum area is assumed on the image quality trade-off function, and a collection of points having the highest coding rate among the areas is the visually optimum upper limit 102 of the coding rate. A collection of points having the minimum coding rate is the visually optimal lower limit 103 of the coding rate. Here, a lower limit value 104C of the quantization accuracy is newly set. That is,
Here, the purpose is to place the operating point in a region between the visually optimal upper limit of the coding rate, the visually optimal lower limit of the coding rate, and the lower limit of the quantization accuracy.

FIG. 17 is a flowchart showing the procedure of the coding control method according to the tenth embodiment of the present invention.
Hereinafter, the encoding control method according to the tenth embodiment of the present invention will be described using FIG. 16 and FIG.

Now, it is assumed that the encoding has been performed in step S201, for example, with the first quantization accuracy 104A. At this time, if the coding rate-quantization accuracy characteristic of the input image is 101A or 101B in FIG. 16, the operating point A or B is within the visually optimal region, and the coding rate Does not exceed the visually optimal upper limit of the coding rate (if the determination in step S202 is “NO”) and does not fall below the visually optimal lower limit of the coding rate (the determination in step S203 is negative). "NO"). Therefore, the quantization precision remains unchanged at 104A. Next, a frame is dropped according to the generated code amount (step S204), and then the next coding is performed (step S201).

If the coding rate-quantization accuracy characteristic of the input image becomes, for example, 101C while repeating the processing of steps S201 to S204, the operating point becomes
The transition is made to the transitional operating point C. The coding rate of the operating point C is
Since the upper limit of the visually optimum coding rate for the quantization precision 104A is exceeded (when the determination in step S202 is “YES”), the quantization precision is improved to, for example, 1
04B (step S205). Thereafter, a frame is dropped according to the generated code amount (step S204).
Thereafter, encoding is performed with a quantization accuracy of 104B (step S201). Assuming that the coding rate-quantization accuracy characteristics of the input image remain unchanged at 101C, the next operating point is:
The operating point D is reached, and again enters the visually optimum area.

Similarly, when operating at operating point B,
When the coding rate-quantization accuracy characteristic of the input image becomes 101D, the operating point shifts to a transient operating point E. Since the coding rate at this time is lower than the visually optimal lower limit of the coding rate (when the determination in step S203 is “YES”), the quantization precision is reduced to, for example, 104C (step S206). Next, a frame is dropped according to the generated code amount (step S204), and then the next coding is performed (step S201). Thus, the operating point becomes the visually optimal operating point F.

Next, when operating at the operating point F and the coding rate-quantization accuracy characteristic of the input image becomes 101E, the operating point becomes a transitional operating point G. At this time,
In the case of the conventional algorithm, since the coding rate is lower than the visually optimal lower limit 103 of the coding rate, the quantization accuracy of the next coding is set to 104D and the operating point is set to H. On the other hand, in the tenth embodiment, since the quantization accuracy value 104D is lower than the lower limit value 104C of the quantization accuracy, it is determined in step S207 whether the quantization accuracy may be reduced. NO ". in this case,
A priority area is set, the priority area is quantized without lowering the quantization precision, and the non-priority area is quantized with a lower quantization precision than the priority area (step S208). As a result, the area quantized with the quantization accuracy 104C becomes smaller, so that the amount of generated information decreases, and the coding rate-quantization accuracy characteristic becomes 101F. As a result, the operating point moves to I, and is within the region between the visually optimum upper limit of the coding rate, the visually optimum lower limit of the coding rate, and the lower limit of the quantization accuracy. to go into.

However, if the operating point is less than the visually optimal lower limit of the coding rate in one setting of the priority area,
In the next encoding, the priority area is further reduced (step S
208). If the operating point exceeds the visually upper limit of the optimal coding rate (the determination in step S202 is “YE
S "), the priority area is expanded (step S21).
0). However, when the priority area cannot be expanded any more (when the determination in step S209 is “NO”)
, The quantization precision is moved to 104A.

As described above, according to the tenth embodiment, the coding control in which the balance between the reproducibility of motion, the spatial resolution, and the noise is kept optimal in terms of visual characteristics by simple control. The method can be realized.

Note that as the unit control width of the quantization accuracy becomes smaller, the visually optimal operation region can be narrowed down. However, in this case, if the tracking speed for the change of the input image becomes slow and it is impossible to follow the speed of the change of the input image, the control may diverge. On the other hand, when the unit control width of the quantization accuracy is increased, the tracking speed for the change of the input image is increased, but the operation area is increased, and the control accuracy is deteriorated. Therefore, it is preferable that the unit control width of the quantization accuracy is set so that the control accuracy is near the recognition limit of the visual characteristic.

Although the method of setting the priority area is not described in detail, it may be set in a range where a person's face is captured, or may be set in the middle of the screen divided into three equal parts. Also,
Regarding the encoding of the non-priority region, the above embodiment has been described as lowering the quantization accuracy. However, no information may be sent at all, or a high frequency component of the input image may be reduced by a pre-filter. Alternatively, only the low frequency component of the DCT coefficient may be encoded.
Further, the coding rate may be corrected by the communication throughput. If the coding rate exceeds a predetermined threshold, the coding operation and the transmission operation are not performed for the non-priority area. It may be.

(Eleventh Embodiment) In FIG. 7, the transmission control unit 432 operates at a constant speed, for example, 3
Output data at 2 kbps. No transmission error occurs,
As long as the reject is not returned from the receiving side, the data read from the transmission buffer 171 is all framed and output. However, a frame in which a transmission error has occurred and a reject has been returned from the receiving side is read from the retransmission buffer 431 and output, and is not read from the transmission buffer. Transmission control unit 43
2 notifies the average throughput calculation unit 711 of the number of bits of data read from the transmission buffer in the unit time t.
If the unit time t is set to a time for transmitting one frame when the frame has a fixed length, for example, 20 ms for an 80-byte frame, synchronization can be easily achieved. Also, by synchronizing with the image cycle of the image input unit 11,
It is possible to synchronize with the encoding timing, and if the unit time t is the least common multiple of the transmission frame time and the image period, both can be synchronized. The average throughput calculator 711 calculates an average value of the throughput for a certain time T from the number of bits read from the transmission buffer per unit time t received from the transmission controller 432. As a simple calculation method, a method is conceivable in which an average value obtained by dividing the total number of bits in a certain time T by a certain time T is calculated every certain time T, and output every certain time T. The process is very simple, but it is rough, so in the case of PHS etc. where errors occur in bursts,
Since the decrease in the average throughput in the section where the burst error has occurred affects all the images to be encoded during the certain time T, the certain time T needs to be sufficiently long. As a second calculation method, a method is conceivable in which an average value obtained by dividing the total number of bits in a certain time period T by the certain time period T is calculated for each unit time t, and output for each unit time t. In this case, the processing is complicated, but the average throughput is updated every unit time t, so that more precise control is possible.

(Twelfth Embodiment) In FIG. 9, the transmission control unit 432 operates at a constant speed, for example, 3
Assume that data is output at 2 kbps. The image receiving device 151 calculates the number of bits of image data that can be received without error during the unit time t. The unit time t is a time for receiving one frame when the frame has a fixed length, for example, 20 ms for a frame of 80 bytes.
Synchronization is easy. The image receiving device 1
If 51 has a function of encoding and transmitting an image like the image encoding device 150, if it synchronizes with the sampling period of the image, it is possible to synchronize with the encoding timing, The unit time t is calculated by the image encoding device 15
By setting the transmission frame time to be received from 0 and the least common multiple of the sampling period of the image, synchronization can be established between both.

The following two methods are conceivable as a method of transmitting the number of bits of image data that can be received without error within a predetermined time T.

The first method is that the bit number counting section 15
1b calculates the number of bits of image data that can be received without error during a certain time T from the number of bits of image data that can be received without error per unit time t from the image encoding device 150, and Is transmitted to the image encoding device 150. In this method, although the processing is very simple, since it is rough, P
In the case of HS or the like, the fixed time T needs to be set sufficiently long because the decrease in the average throughput in the section where the burst error has occurred affects all the images encoded during the fixed time T. In this method, the processing can be further simplified if the unit time t is made equal to the fixed time T.

The second calculation method is a method of calculating the total number of bits for a certain time T for each unit time t and outputting the total for each unit time t. In this case, the processing becomes complicated, but the image encoding device 150 updates the average throughput every unit time t, so that more precise control is possible.

(Thirteenth Embodiment) In FIG. 9, the transmission control unit 432 operates at a constant speed, for example, 3
Assume that data is output at 2 kbps. The image receiving device 151 calculates the number of bits of image data that can be received without error during the unit time t. The unit time t is a time for receiving one frame when the frame has a fixed length, for example, 20 ms for a frame of 80 bytes.
Synchronization is easy. The image receiving device 1
If 51 has a function of encoding and transmitting an image like the image encoding device 150, if it synchronizes with the sampling period of the image, it is possible to synchronize with the encoding timing, The unit time t is calculated by the image encoding device 15
By setting the transmission frame time to be received from 0 and the least common multiple of the sampling period of the image, synchronization can be established between both.

The following two methods are conceivable for calculating the average throughput per fixed time T.

[0473] The first method is that the bit number counting section 15
1b calculates the number of bits of image data that can be received without error during a certain time T from the number of bits of image data that can be received without error per unit time t from the image encoding device 150, In this method, an average value obtained by dividing the total number of bits by a certain time T is calculated for each certain time T, and output at each certain time T. In this method, although the processing is very simple, in a rough manner, in the case of a PHS or the like in which an error occurs in a burst manner, a decrease in the average throughput in a section where a burst error occurs is encoded during a certain time T. Therefore, it is necessary to set the certain time T sufficiently long to affect all the images. In this method, the processing can be further simplified if the unit time t is made equal to the fixed time T.

In the second calculation method, an average value obtained by dividing the total number of bits in a certain fixed time T by the certain time T is calculated for each unit time t, and is output for each unit time t. In this case, the processing becomes complicated, but in the image encoding device 150,
Since the average throughput is updated every unit time t, more precise control is possible.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image encoding device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating measurement results of a delay time and a transmission frame interval in the first embodiment of the present invention.

FIG. 3 is a block diagram illustrating another configuration example of the frame drop / quantization control device according to the first embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of an image encoding device according to a second embodiment of the present invention.

FIG. 5 is a timing chart showing a state transition timing according to the second embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration of an image encoding device according to a third embodiment of the present invention.

FIG. 7 is a block diagram illustrating a configuration of an image encoding device according to a fourth embodiment of the present invention.

FIG. 8 is a block diagram illustrating another configuration example of the frame drop / quantization control device according to the fourth embodiment of the present invention.

FIG. 9 is a block diagram illustrating a configuration of an image transmission system according to a fifth embodiment of the present invention.

FIG. 10 is a block diagram illustrating a configuration of an image transmission system according to a sixth embodiment of the present invention.

FIG. 11 is a block diagram illustrating a configuration of an image encoding device according to a seventh embodiment of the present invention.

FIG. 12 is a graph of a generated information amount with respect to a quantization value according to the eighth embodiment of the present invention.

FIG. 13 is a block diagram illustrating a configuration of an image encoding device according to a ninth embodiment of the present invention.

FIG. 14 is an example of dividing an image frame when the size of a priority area is changed in two steps according to the ninth embodiment of the present invention.

FIG. 15 is a processing flow when changing the size of a priority area in the ninth embodiment of the present invention.

FIG. 16 is a diagram for explaining area control and quantization control of a priority area and a non-priority area according to the tenth embodiment of the present invention.

FIG. 17 is a control flow of area control and quantization control of a priority area and a non-priority area in the tenth embodiment of the present invention.

FIG. 18 is a block diagram illustrating an example of a configuration of a conventional image encoding device.

19 is a block diagram illustrating an example of a configuration of the encoding device in FIG.

FIG. 20 is a sequence diagram for explaining conventional frame drop control.

FIG. 21 is a diagram for describing quantization control in a conventional image encoding device.

FIG. 22 is a diagram for explaining quantization control when a sudden movement of a subject occurs in a conventional image encoding device.

FIG. 23 is a block diagram illustrating an example of a configuration of a conventional high-efficiency encoding device for a television signal.

FIG. 24 is a diagram illustrating measurement results of a delay time and a transmission frame interval in a conventional image encoding device.

[Explanation of symbols]

10, 40, 60, 70, 110, 150: Image coding device 151, 164: Image decoding device 152, 153: Base station 11: Image input unit 12: Coding device 13: Temporary buffer 14: First SW 15: Generated information amount calculation unit 16, 16 ', 42, 72, 81, 111 ...
Quantization control device 161 ... frame drop control unit 162 ... threshold value storage unit 163 ... first quantization control unit 31 ... difference unit 32 ... generated information amount storage unit 421 ... threshold value switching unit 422 ... second SW 423: first threshold storage section 424: second threshold storage section 721: allowable delay time storage section 722 ... transmittable information amount calculation section 811 ... average delay time calculation section 17, 43: transmission control device 171 ... transmission buffers 113, 172, 432 ... transmission control unit 431 ... retransmission buffers 41, 71 ... quantization control unit 411 ... second quantization control unit 412 ... third quantization control unit 413 ... mode selection unit 711 ... average Throughput calculation section 712 ... coding rate calculation section 713 ... fourth quantization control section 61 ... error correction coding apparatus 611 ... first error correction coding section 612 ... second error correction Goka 613 ... third SW 1000 ... image receiving apparatus 1301 ... priority area / non-priority area changing portion

Claims (139)

[Claims] An image encoding apparatus for encoding and transmitting a moving image, comprising: an image input unit configured to output an image frame from a video signal; and an image encoding unit configured to encode the image frame configured by the image input unit. A transmission buffer that stores image data of the image frame encoded by the encoding unit; a transmission control unit that transmits image data in the transmission buffer at a predetermined transmission rate; Arranged between the conversion means and the transmission buffer,
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame removal / quantization control means for determining a quantization value of an image frame to be converted, wherein the frame removal / quantization control means sets an information amount of an image frame stored in the temporary buffer in advance. If the value is larger than the dropped frame threshold value, frame drop control is performed so that image data in the temporary buffer is not stored in the transmission buffer, and the image data stored in the temporary buffer is encoded. Setting a quantization value of a quantization step width larger than the quantization step width, and notifying the encoding means, Goka means, then the image frame to be encoded, characterized by coding the notification quantization value from said decimating / quantization control means, the image coding apparatus. 2. The transmission control means according to claim 1, further comprising, when a communication error occurs and the image data cannot be normally decoded on the receiving terminal side, performs automatic retransmission and an error indicating that retransmission has occurred. Information is sent to the frame drop / quantization control means. When the frame drop / quantization control means further receives error information notification from the transmission control means, the image data in the temporary buffer is sent to the transmission buffer. The frame drop threshold for determining whether or not to store the frame is lowered from a preset value. If a notification of error information is not received from the transmission control means for a predetermined time, the frame drop is performed. 2. The image encoding apparatus according to claim 1, wherein a threshold value control for returning the threshold value to a preset value is performed. 3. The transmission control unit further calculates an average throughput within a predetermined period of time and notifies the frame loss / quantization control unit of the average throughput, and the frame loss / quantization control unit further includes: Based on the preset maximum allowable delay time and the average throughput calculated by the transmission control means, calculate the maximum amount of information that can be transmitted within the maximum allowable delay time, and calculate the calculated maximum information amount. 2. The image coding apparatus according to claim 1, wherein the image coding apparatus uses the frame drop threshold value. 4. The frame drop / quantization control means further calculates an average delay time based on an average amount of generated information of image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time. The image encoding apparatus according to claim 3, wherein an average delay time is used as the maximum allowable delay time. 5. An image encoding apparatus for encoding and transmitting a moving image, comprising: an image input unit configured to output an image frame from a video signal; and an image encoding unit configured to encode the image frame configured by the image input unit. A transmission buffer that stores the image data encoded by the encoding unit; a first quantization control unit that determines an optimal quantization value in terms of both motion reproducibility and image quality; A second quantization control means for controlling a quantization step width in accordance with a communication state; transmitting image data in the transmission buffer at a predetermined transmission rate; Transmission control means for performing automatic retransmission when data cannot be decoded, and notifying the second quantization control means of error information indicating that retransmission has occurred, wherein the second quantization Control hand Upon receiving notification of error information from the transmission control means, sets a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means. Notifying the encoding means, and if the error information is not received from the transmission control means for a predetermined period of time, the quantization value determined by the first quantization control means is directly used. The encoding unit notifies the encoding unit, and the encoding unit encodes a next image frame to be encoded with the quantization value notified from the second quantization control unit. Device. 6. A temporary buffer disposed between the encoding unit and the transmission buffer, for temporarily storing image data of an image frame encoded by the encoding unit, and an image in the temporary buffer. A frame removal / quantization control unit that determines whether to store data in the transmission buffer and determines a quantization value of an image frame to be encoded next, the transmission control unit includes: The frame drop / quantization control unit further notifies the frame drop / quantization control unit of error information indicating that the error has occurred. When the frame drop / quantization control unit receives the error information notification from the transmission control unit, the frame drop / quantization control unit reads the error information. The frame drop threshold for determining whether or not to store the image data in the transmission buffer is lowered from a preset value, and the frame drop threshold is determined for a predetermined period of time. If the notification of error information is not received from the transmission control unit, the frame drop threshold is returned to a preset value, threshold control is performed, and the information amount of image data in the temporary buffer is reduced to the threshold. If the value is larger than the frame drop threshold controlled by the control, while performing the frame drop control not storing the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. The image encoding device according to claim 5. 7. The transmission control means further calculates an average throughput within a predetermined period of time and notifies the average to the dropout / quantization control means, and the dropout / quantization control means further comprises: Based on the preset maximum allowable delay time and the average throughput calculated by the transmission control means, calculate the maximum amount of information that can be transmitted within the maximum allowable delay time, and calculate the calculated maximum information amount. 7. The image encoding apparatus according to claim 6, wherein the apparatus is used as a frame drop threshold. 8. The frame drop / quantization control means further calculates an average delay time based on the average generated information amount of image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time. The image encoding apparatus according to claim 7, wherein an average delay time is used as the maximum allowable delay time. 9. When a moving image is encoded by high-efficiency compression and transmitted through a low-bit-rate communication line, time is reduced by dropping frames in accordance with the coding rate, which is the rate of frames that can be transmitted per unit time. An image encoding apparatus in which an operating point is predetermined as a balance between distortion and spatial distortion due to a quantization value, wherein: an image input unit configured to output an image frame from a video signal; Encoding means for encoding an image frame constituted by means, a transmission buffer for storing image data encoded by the encoding means, and a quantization control means for determining a quantization value according to a communication state; Transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate, wherein the quantization control means calculates an average throughput within a predetermined period of time. Calculating a coding rate based on the calculated average throughput and the amount of information generated per frame as a result of coding by the coding means, and calculating a quantization value corresponding to the calculated coding rate. Determined and notified to the encoding means, When the remaining amount of data in the transmission buffer is less than or equal to a certain threshold value, the encoding means is notified to perform encoding of one image frame. Image encoding device. 10. A temporary buffer disposed between said encoding means and said transmission buffer, for temporarily storing image data of an image frame encoded by said encoding means, and an image in said temporary buffer. A frame removal / quantization control unit that determines whether data is to be stored in the transmission buffer and determines a quantization value of an image frame to be encoded next, the frame removal / quantization control unit Calculates the amount of information that can be transmitted within the maximum allowable delay time based on the preset maximum allowable delay time and the average throughput calculated by the quantization control unit, and stores the information amount in the temporary buffer. If the amount of information of the current image frame is larger than the amount of information that can be transmitted within the maximum allowable delay time, the image data in the temporary buffer is transmitted. While performing the frame drop control not stored in the buffer, and setting a quantization value of a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, The image encoding apparatus according to claim 9, wherein a notice is sent to an encoding unit. 11. A temporary buffer disposed between the encoding unit and the transmission buffer, for temporarily storing image data of an image frame encoded by the encoding unit, and an image in the temporary buffer. A frame removal / quantization control unit that determines whether data is to be stored in the transmission buffer and determines a quantization value of an image frame to be encoded next, the frame removal / quantization control unit Calculates an average delay time based on the average amount of generated information of image frames transmitted within a predetermined time and the average throughput calculated by the quantization control unit, and calculates the average delay time and the average delay time. The information amount that can be transmitted within the average delay time is calculated based on the average delay time, and the information amount of the image frame stored in the temporary buffer is calculated as the average delay time. If the amount of information that can be transmitted within the time period is larger than the amount of information that can be transmitted within the time period, the frame data is not stored in the transmission buffer. 10. The image encoding apparatus according to claim 9, wherein a quantization value having a quantization step width larger than the quantization step width used in step (c) is set and notified to the encoding unit. 12. The method according to claim 9, wherein the quantization control unit varies the averaging time according to the presence or absence of a burst error when calculating the average throughput. Image coding device. 13. The image encoding apparatus according to claim 9, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 14. An image encoding apparatus for encoding and transmitting a moving image, comprising: an image input unit configured to output an image frame from a video signal, and encoding the image frame configured by the image input unit. Encoding means, a transmission buffer for storing image data of an image frame encoded by the encoding means, a quantization control means for determining a quantization value to be used for encoding, Transmission control means for transmitting image data at a predetermined transmission rate; motion threshold value storage means for storing a motion threshold value for determining whether the motion of the subject is large or small; A maximum permissible delay time storage unit that stores a maximum permissible delay time that is a limit of the delay, wherein the quantization control unit is configured to perform per frame of a result of encoding by the encoding unit. When the amount of generated information is larger than the motion threshold, the ratio between the quantization value used when encoding the image frame exceeding the motion threshold and the quantized value for which the amount of generated information is to be predicted. From the amount of generated information of the image frame exceeding the motion threshold value and the quantization ratio, the amount of information generated when the quantization value is changed is predicted. From the information amount, the transmission rate, and the maximum allowable delay time, a quantizing value that can be transmitted within the maximum allowable delay time is determined and notified to the encoding unit, and the encoding unit performs the quantization control. An image encoding device, which encodes an image frame with a quantization value notified from a means. 15. The transmission control unit further calculates an average throughput within a predetermined period of time and notifies the quantization control unit of the average throughput, and the quantization control unit performs encoding by the encoding unit. If the amount of generated information per frame as a result of the calculation is larger than the motion threshold, the quantization value used when encoding the image frame exceeding the motion threshold and the amount of generated information are predicted. A quantization ratio, which is a ratio with a desired quantization value, is obtained.From the amount of generated information of the image frame exceeding the motion threshold and the quantization ratio, the amount of information generated when the quantization value is changed is calculated. Predicting, from the predicted generated information amount, the transmission rate, the average throughput, and the maximum allowable delay time, determine a quantizing value that can be transmitted within the maximum allowable delay time, and notify the encoding unit. thing Wherein, the image coding apparatus according to claim 14. 16. The transmission control means, when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 15. 17. The image encoding apparatus according to claim 15, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 18. An image encoding device for encoding and transmitting a moving image, comprising: an image input unit configured to output an image frame from a video signal, and encoding the image frame configured by the image input unit. Encoding means, a transmission buffer for storing image data of the image frame encoded by the encoding means, a quantization control means for determining a quantization value to be used for encoding, Transmission control means for transmitting image data at a predetermined transmission rate; motion threshold value storage means for storing a motion threshold value for determining whether the motion of the subject is large or small; and frames which can be transmitted per unit time An ideal curve whose operating point is predetermined is stored as a balance between temporal distortion caused by dropping frames corresponding to the coding rate, which is the ratio of, and spatial distortion caused by the quantization value. Ideal curve storage means, wherein the quantization control means sets the motion threshold value when the amount of generated information per frame as a result of encoding by the encoding means is larger than the motion threshold value. A quantization ratio which is a ratio between a quantization value used when encoding the exceeded image frame and a quantization value for which the amount of generated information is to be predicted is obtained, and the generation information of the image frame exceeding the motion threshold is calculated. From the amount and the quantization ratio, the amount of information generated when the quantization value is changed is predicted, and the coding rate is calculated from the predicted amount of generated information and the transmission rate, and the calculated code is calculated. From the quantization ratio and the ideal curve, a quantization value is determined and notified to the encoding unit, and the encoding unit encodes the image frame with the quantization value notified from the quantization control unit. Image coding characterized by the following: apparatus. 19. The transmission control unit further calculates an average throughput within a predetermined period of time and notifies the quantization control unit of the average throughput, and the quantization control unit performs encoding by the encoding unit. If the amount of generated information per frame as a result of the calculation is larger than the motion threshold, the quantization value used when encoding the image frame exceeding the motion threshold and the amount of generated information are predicted. A quantization ratio, which is a ratio with a desired quantization value, is obtained.From the amount of generated information of the image frame exceeding the motion threshold and the quantization ratio, the amount of information generated when the quantization value is changed is calculated. Predicting, calculating the coding rate from the predicted generated information amount, the transmission rate and the average throughput, determining a quantization value from the calculated coding rate and the ideal curve, and Notification Characterized Rukoto, the image coding apparatus according to claim 18. 20. The transmission control means, wherein when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 19. 21. The image encoding apparatus according to claim 19, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 22. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving apparatus calculates an error rate of the received image data, and when the calculated error rate exceeds a predetermined threshold, transmits an error signal to the image encoding apparatus. Image input means for constructing and outputting an image frame from a video signal; encoding means for encoding the image frame constituted by the image input means; and image data encoded by the encoding means. A transmission buffer; transmission control means for transmitting image data in the transmission buffer at a predetermined transmission rate; and a first quantization system for determining an optimal quantization value in terms of both motion reproducibility and image quality. Means, error correction coding means for adding an error correction code having a different error correction capability to the image data coded by the coding means, and a quantization value according to a communication state is determined. A second quantization control means for controlling the error correction coding means in response to the first quantization when the second quantization control means receives the error signal from the image receiving apparatus. A quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the control means is set and notified to the encoding means, and the error correction encoding means has an error correction capability. And outputs a selection signal instructing to add a high error correction code. If a notification of the error signal is not received for a predetermined time, the quantization value determined by the first quantization control means is output. That In addition, while notifying the encoding unit, the selection unit outputs a selection signal instructing the error correction encoding unit to add an error correction code having a low error correction capability. The encoding unit outputs the second quantization control Encoding the image frame with the quantized value notified from the means, and the error correction encoding means adds an error correction code specified by the selection signal to the image data encoded by the encoding means. An image transmission system, characterized in that: 23. The image receiving apparatus calculates an average of the error rates calculated within a predetermined period of time, and when the calculated average error rate exceeds a predetermined threshold value, The image transmission system according to claim 22, wherein the image transmission device transmits the image data to the image encoding device. 24. The image encoding device, wherein the image encoding device is disposed between the encoding unit and the transmission buffer,
A temporary buffer that temporarily stores image data of the image frame encoded by the encoding unit, and determines whether to store the image data in the temporary buffer in the transmission buffer, and then determines whether the image data is encoded. Frame removing / quantizing control means for determining a quantization value of an image frame to be read, wherein the frame removing / quantizing control means, upon receiving an error signal from the image receiving apparatus, stores image data in the temporary buffer. Is reduced from a preset value for determining whether or not to store the frame in the transmission buffer. If the error signal is not notified for a predetermined period of time, the frame drop is performed. Threshold value is returned to a preset value, threshold value control is performed, and the information amount of image data in the temporary buffer is controlled by the threshold value control. If it is larger than the frame drop threshold, while performing the frame drop control not storing the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. , Claim 22 or 23
2. The image transmission system according to 1. 25. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, An image receiving device that calculates an error rate of the received image data and transmits the error rate to the image encoding device; the image encoding device constructs an image frame from a video signal and outputs the image frame; Encoding means for encoding the image frame constituted by the input means, a transmission buffer for storing the image data encoded by the encoding means, and transmitting the image data in the transmission buffer at a predetermined transmission rate Transmission control means, first quantization control means for determining an optimal quantization value in terms of both the reproducibility of motion and image quality, and error correction capability for the image data encoded by the encoding means. Error correction coding means for adding different error correction codes, and a second quantization control means for determining a quantization value according to the communication state and controlling the error correction coding means according to the error rate. The second quantization control means, when the error rate received from the image receiving device exceeds a predetermined threshold value, the quantization value determined by the first quantization control means A quantization value having a quantization step width larger than the quantization step width corresponding to is set and notified to the encoding means, and an error correction code having a high error correction capability is added to the error correction encoding means. And outputting a select signal for instructing, if the error rate does not exceed the threshold value for a predetermined period of time, the quantization value determined by the first quantization control means is directly transmitted to the encoding means. When you notify In addition, a selection signal for instructing the error correction encoding means to add an error correction code having a low error correction capability is output, and the encoding means outputs a quantization signal notified from the second quantization control means. Encoding the image frame with a value, the error correction encoding means adding an error correction code specified by the selection signal to the image data encoded by the encoding means, system. 26. The second quantization control means calculates an average of error rates received from the image receiving device within a predetermined time, and calculates the calculated average error rate by a predetermined threshold. If the value exceeds the value, a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means is set and notified to the encoding means. At the same time, a selection signal instructing the error correction encoding means to add an error correction code having a high error correction capability is output, and if the average error rate does not exceed the threshold value for a predetermined period of time, A selection signal for notifying the encoding means as it is of the quantization value determined by the first quantization control means and instructing the error correction encoding means to add an error correction code having a low error correction capability. 26. The image transmission system according to claim 25, wherein the image transmission system outputs a signal. 27. The image encoding device, wherein the image encoding device is disposed between the encoding unit and the transmission buffer,
A temporary buffer that temporarily stores image data of the image frame encoded by the encoding unit, and determines whether to store the image data in the temporary buffer in the transmission buffer, and then determines whether the image data is encoded. Further comprising: a frame removal / quantization control unit for determining a quantization value of an image frame to be decoded, wherein the frame removal / quantization control unit determines in advance the average error rate calculated by the second quantization control unit. When the predetermined threshold value is exceeded, the frame drop threshold value for determining whether to store the image data in the temporary buffer in the transmission buffer is lowered from a predetermined value, and the predetermined value is reduced. If the time and the average error rate do not exceed the threshold, the frame drop threshold is returned to a preset value, threshold control is performed, and the temporary buffer If the information amount of the image data in the frame is larger than the frame drop threshold controlled by the threshold control, while performing the frame drop control that does not store the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. The image transmission system according to claim 26. 28. The image encoding device, wherein the image encoding device is disposed between the encoding unit and the transmission buffer,
A temporary buffer that temporarily stores image data of the image frame encoded by the encoding unit, and determines whether to store the image data in the temporary buffer in the transmission buffer, and then determines whether the image data is encoded. And a frame removal / quantization control unit for determining a quantization value of an image frame to be transmitted, wherein the frame removal / quantization control unit sets an error rate received from the image receiving apparatus to a predetermined threshold value. If it exceeds, the frame drop threshold for determining whether or not to store the image data in the temporary buffer in the transmission buffer is reduced from a preset value, and the error rate is reduced for a predetermined period of time. If the threshold value is not exceeded, the frame drop threshold value is returned to a preset value, threshold value control is performed, and the image data in the temporary buffer is If the information amount is larger than the frame drop threshold controlled by the threshold control, while performing the frame drop control that does not store the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. An image transmission system according to claim 25. 29. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image coding apparatus calculates an error rate of the received image data, calculates an average of the error rates obtained within a predetermined time, and transmits the calculated average error rate to the image coding apparatus. The image receiving device includes: an image input unit configured to output an image frame from a video signal; an encoding unit configured to encode an image frame configured by the image input unit; and an encoding unit configured to encode the image frame. A transmission buffer for storing the image data stored in the transmission buffer, a transmission control unit for transmitting the image data in the transmission buffer at a predetermined transmission rate, and an optimal quantization value in terms of both the reproducibility of the motion and the image quality. First quantization control means for determining an error correction code for adding error correction codes having different error correction capabilities to the image data encoded by the encoding means, and a quantization value according to a communication state. And a second quantization control means for controlling the error correction coding means according to the average error rate, wherein the second quantization means comprises an average error rate received from the image receiving apparatus. Exceeds a predetermined threshold value, a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means is set; Along with notifying the encoding means, outputting a selection signal instructing the error correction encoding means to add an error correction code having a high error correction capability, the predetermined average time, and Exceed the threshold If not, the quantization value determined by the first quantization control unit is reported to the encoding unit as it is, and the error correction encoding unit is instructed to add an error correction code having a low error correction capability. Outputting a selection signal, the encoding unit encodes the image frame with the quantization value notified from the second quantization control unit, and the error correction encoding unit outputs an error indicated by the selection symbol. An image transmission system, wherein a correction code is added to image data encoded by the encoding means. 30. The image encoding device is disposed between the encoding unit and the transmission buffer,
A temporary buffer that temporarily stores the image data of the image frame encoded by the encoding unit, and determines whether to store the image data in the temporary buffer in the transmission buffer, and then determines whether the image data is encoded. Further comprising: a frame removal / quantization control means for determining a quantization value of an image frame to be transmitted, wherein the frame removal / quantization control means comprises a predetermined threshold value for an average error rate received from the image receiving device. Is exceeded, the frame drop threshold for determining whether to store the image data in the temporary buffer in the transmission buffer is lowered from a preset value, and the average error is reduced for a predetermined period of time. If the rate does not exceed the threshold, the frame drop threshold is returned to a preset value, threshold control is performed, and the image in the temporary buffer is If the information amount of the data is larger than the frame drop threshold controlled by the threshold control, while performing the frame drop control not storing the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. An image transmission system according to claim 29. 31. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving apparatus calculates an error rate of the received image data, and when the calculated error rate exceeds a predetermined threshold, transmits an error signal to the image encoding apparatus. Image input means for constructing and outputting an image frame from a video signal; encoding means for encoding the image frame constituted by the image input means; and image data encoded by the encoding means. A transmission buffer, transmission control means for transmitting the image data in the transmission buffer at a predetermined transmission rate, disposed between the encoding means and the transmission buffer,
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame removal / quantization control means for determining a quantization value of an image frame to be converted; and an error correction code for adding an error correction code having a different error correction capability to the image data in the temporary buffer and storing the error correction code in the transmission buffer. The frame dropping / quantization control means, when receiving the error signal, determines whether to store the image data in the temporary buffer in the transmission buffer or not. While performing threshold control for lowering the value from a preset value, the error correction encoding means includes an error correction code having a high error correction capability. And outputs a selection signal for instructing to add a frame, and if a notification of the error signal is not received for a predetermined period of time, performs threshold control for returning the frame drop threshold to a predetermined value. At the same time, it outputs a selection signal instructing the error correction encoding means to add an error correction code having a low error correction capability, and the information amount of image data in the temporary buffer is controlled by the threshold value control. If it is larger than the frame drop threshold, while performing the frame drop control not storing the image data in the temporary buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means of the error correction code; The image transmission system, wherein the encoding unit adds the error correction code specified by the selection signal to the image data encoded by the encoding unit. 32. The image receiving apparatus calculates an average of the error rates calculated within a predetermined period of time, and when the calculated average error rate exceeds a predetermined threshold value, The image transmission system according to claim 31, wherein is transmitted to the image encoding device. 33. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, An image receiving device that calculates an error rate of the received image data and transmits the error rate to the image encoding device; the image encoding device constructs an image frame from a video signal and outputs the image frame; Encoding means for encoding the image frame constituted by the input means, a transmission buffer for storing the image data encoded by the encoding means, and transmitting the image data in the transmission buffer at a predetermined transmission rate Transmission control means, disposed between the encoding means and the transmission buffer,
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame removal / quantization control means for determining a quantization value of an image frame to be converted; and an error correction code for adding an error correction code having a different error correction capability to image data in the temporary buffer and storing the image data in the transmission buffer. The frame dropping / quantization control means, when the error rate exceeds a predetermined threshold value,
While performing threshold control for lowering the frame drop threshold for determining whether or not to store the image data in the temporary buffer in the transmission buffer from a preset value, the error correction encoding means A selection signal for instructing to add an error correction code having a high error correction capability is output. If the error rate does not exceed a predetermined threshold for a predetermined time, the frame drop threshold is output. Performs a threshold control to return the error correction code to a preset value, and outputs a selection signal instructing the error correction encoding means to add an error correction code having a low error correction capability. If the data information amount is larger than the frame drop threshold controlled by the threshold control, the image data in the temporary buffer is stored in the transmission buffer. Performs gastric frame dropping control,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means of the error correction code; The image transmission system, wherein the encoding unit adds the error correction code specified by the selection signal to the image data encoded by the encoding unit. 34. The image encoding apparatus further comprises an average error rate calculating means for calculating an average error rate which is an average of error rates received from the image receiving apparatus within a predetermined time. The drop / quantization control means, when the average error rate exceeds a predetermined threshold, a dropout for determining whether or not to store the image data in the temporary buffer in the transmission buffer. Threshold control for lowering the threshold from a preset value is performed, and a selection signal instructing the error correction encoding means to add an error correction code having a high error correction capability is output, and a predetermined signal is output. If the average error rate does not exceed a predetermined threshold for a certain period of time, perform threshold control for returning the frame drop threshold to a predetermined value,
Outputting a selection signal for instructing the error correction encoding means to add an error correction code having a low error correction capability, and removing a frame whose information amount of image data in the temporary buffer is controlled by the threshold value control. If the value is larger than the threshold value, the frame data is not stored in the transmission buffer in the temporary buffer.
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means of the error correction code; 34. The image transmission system according to claim 33, wherein the encoding unit adds an error correction code specified by the selection signal to the image data encoded by the encoding unit. 35. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving apparatus obtains an error rate of the received image data, calculates an average of the error rates obtained within a predetermined time, and transmits the calculated average error rate to the image encoding apparatus. The image encoding apparatus includes: an image input unit configured to output an image frame from a video signal; an encoding unit configured to encode the image frame configured by the image input unit; and an image encoded by the encoding unit. A transmission buffer for storing the image data stored therein, a transmission control unit for transmitting the image data in the transmission buffer at a predetermined transmission rate, and a transmission buffer disposed between the encoding unit and the transmission buffer. It is,
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame drop / quantization control means for determining a quantization value of an image frame to be converted; and an error correction code for adding an error correction code having a different error correction capability to the image data in the temporary buffer and storing the error correction code in the transmission buffer. When the average error rate exceeds a predetermined threshold value, determines whether or not to store the image data in the temporary buffer in the transmission buffer. The threshold control for lowering the frame drop threshold for determining the threshold value from a preset value is performed, and the error correction encoding means performs error control. A selection signal instructing to add an error correction code having a high correction capability is output. If the average error rate does not exceed a predetermined threshold for a predetermined period of time, the frame drop threshold is output. While performing threshold control to return to a preset value,
Outputting a selection signal for instructing the error correction encoding means to add an error correction code having a low error correction capability, and removing a frame whose information amount of image data in the temporary buffer is controlled by the threshold value control. If the value is larger than the threshold value, the frame data is not stored in the transmission buffer in the temporary buffer.
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means of the error correction code; The image transmission system, wherein the encoding unit adds the error correction code specified by the selection signal to the image data encoded by the encoding unit. 36. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, wherein: The image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts an image from a video signal into an image. Image input means for composing and outputting a frame; encoding means for encoding the image frame constituted by the image input means; transmission buffer for storing the image data encoded by the encoding means; A quantization control unit for determining a quantization value according to a state; transmitting the image data in the transmission buffer at a predetermined transmission rate; Transmission control means for outputting the number of data to the quantization control means, wherein the quantization control means calculates an average throughput within the predetermined time based on the number of bits received from the transmission control means. Calculating a coding rate based on the calculated average throughput and the amount of information generated per frame as a result of coding by the coding means, and calculating a quantization value corresponding to the calculated coding rate. An image transmission system, wherein the image transmission system determines and notifies the encoding unit. 37. The image transmission system according to claim 36, wherein, when calculating the average throughput, the quantization control unit changes a averaging time according to a change in the number of bits. . 38. The image transmission system according to claim 36, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 39. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving apparatus counts the number of bits of data received correctly within a predetermined time, calculates an average throughput from the counted number of bits, and transmits the average throughput to the image encoding apparatus. Image input means for constructing and outputting an image frame from a video signal; encoding means for encoding the image frame constituted by the image input means; and image data encoded by the encoding means. A transmission buffer that performs quantization, a quantization control unit that determines a quantization value according to a communication state, and transmits image data in the transmission buffer at a predetermined transmission rate. The average throughput received from the receiving apparatus and a transmission control means for outputting to the quantization control means, said quantization control means, and the average throughput received from said transmission control means,
A coding rate is calculated based on the amount of information generated per frame as a result of the coding by the coding means, and a quantization value corresponding to the calculated coding rate is determined. An image transmission system, which provides notification. 40. An image encoding device for encoding and transmitting a moving image and an image receiving device for performing predetermined processing on received image data are communicably connected via one or more relay stations. An image transmission system, wherein the image receiving device detects an error in received data,
While performing automatic retransmission with the adjacent relay station, transmits retransmission occurrence information indicating that retransmission has occurred to the image encoding device, the image encoding device forms an image frame from a video signal, Image input means for outputting the image data, encoding means for encoding an image frame constituted by the image input means, a transmission buffer for storing the image data encoded by the encoding means, and motion reproducibility. First quantization control means for determining an optimal quantization value from both aspects of image quality, second quantization control means for controlling a quantization step width according to a communication state, and image data in the transmission buffer. While transmitting at a predetermined transmission rate, if a communication error occurs between adjacent relay stations, perform automatic retransmission with the adjacent relay station and perform retransmission by itself, or Transmission control means for notifying the second quantization control means of error information indicating that retransmission has occurred, when the retransmission occurrence information is received from the image receiving apparatus, wherein the second quantization control The means, upon receiving the error information notification from the transmission control means, sets a quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the first quantization control means. Set and notify the encoding means, and if the error information is not received from the transmission control means for a predetermined period of time, the quantization value determined by the first quantization control means is The encoding unit notifies the encoding unit as it is, and the encoding unit encodes the next image frame to be encoded with the quantization value notified from the second quantization control unit. Transmission system. 41. The image encoding device is disposed between the encoding unit and the transmission buffer,
A temporary buffer that temporarily stores image data of the image frame encoded by the encoding unit, and determines whether to store the image data in the temporary buffer in the transmission buffer, and then determines whether the image data is encoded. And a frame removal / quantization control unit for determining a quantization value of an image frame to be transmitted. The transmission control unit transmits error information indicating that retransmission has occurred to the frame removal / quantization control unit. Notifying, when receiving the error information notification from the transmission control means, the frame drop / quantization control means, frame drop for determining whether to store the image data in the temporary buffer in the transmission buffer The threshold value is lowered from a preset value, and if a notification of error information is not received from the transmission control means for a predetermined period of time, the frame is dropped. Threshold control is performed to return the threshold value to a preset value. If the information amount of image data in the temporary buffer is larger than the frame drop threshold controlled by the threshold control, the temporary While performing frame drop control that does not store the image data in the buffer in the transmission buffer,
Setting a quantization value having a quantization step width larger than the quantization step width used when encoding the image frame stored in the temporary buffer, and notifying the encoding means. The image transmission system according to claim 40. 42. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts an image from a video signal into an image. Image input means for constructing and outputting a frame, encoding means for encoding the image frame constituted by the image input means, and transmission buffer for storing image data of the image frame encoded by the encoding means And, disposed between the encoding means and the transmission buffer,
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame removal / quantization control means for determining a quantization value of an image frame to be converted, transmitting image data in the transmission buffer at a predetermined transmission rate, and averaging from the number of bits received from the image receiving device. Transmission control means for calculating a throughput, wherein the frame drop / quantization control means includes information which can be transmitted within a maximum allowable delay time based on a preset maximum allowable delay time and the average throughput. The amount of information of the image frame stored in the temporary buffer can be transmitted within the maximum allowable delay time. If it is larger than the amount of information, the frame removing control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the quantization step used when encoding the image frame stored in the temporary buffer is performed. An image transmission system, wherein a quantization value having a quantization step width larger than a width is set and notified to the encoding means. 43. The frame drop / quantization control means further calculates an average delay time based on the average amount of generated information of image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time. 43. The image transmission system according to claim 42, wherein an average delay time is used as the maximum allowable delay time. 44. The transmission control means according to a change in the number of bits when calculating the average throughput.
The averaging time is variable.
3. The image transmission system according to 2. 45. The image transmission system according to claim 42, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 46. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data received correctly within a predetermined period of time, calculates an average throughput from the counted number of bits, and transmits the average throughput to the image encoding device. Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and an image of the image frame encoded by the encoding means A transmission buffer for storing data, disposed between the encoding unit and the transmission buffer;
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame drop / quantization control means for determining a quantization value of an image frame to be converted, and transmitting image data in the transmission buffer at a predetermined transmission rate, and calculating the average throughput received from the image receiving device. Transmission control means for outputting to the frame drop / quantization control means, the frame drop / quantization control means comprising a maximum allowable delay time based on a preset maximum allowable delay time and the average throughput. Calculating the amount of information that can be transmitted within the temporary buffer, and determining the amount of information of the image frame stored in the temporary buffer as the maximum allowable delay. If the information amount is larger than the amount of information that can be transmitted within the interval, the frame removal control that does not store the image data in the temporary buffer in the transmission buffer and the encoding of the image frame stored in the temporary buffer An image transmission system, wherein a quantization value having a quantization step width larger than the quantization step width used in step (c) is set and notified to the encoding means. 47. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts an image from a video signal into an image. Image input means for constructing and outputting a frame, encoding means for encoding the image frame constituted by the image input means, and transmission buffer for storing image data of the image frame encoded by the encoding means And a quantization control means for determining a quantization value to be used for encoding; transmitting image data in the transmission buffer at a predetermined transmission rate; A transmission control unit for calculating an average throughput from the number of bits received from the control unit and outputting the average throughput to the quantization control unit; and a motion threshold value for storing a motion threshold value for determining whether the motion of the subject is large or small. Storage means, comprising a maximum allowable delay time storage means for storing a maximum allowable delay time that is a limit of transmission delay that does not feel visually uncomfortable, the quantization control means, the result of the encoding by the encoding means When the amount of generated information per frame is larger than the motion threshold, the quantization value used when encoding an image frame exceeding the motion threshold and the quantization for which the amount of generated information is to be predicted Determine a quantization ratio that is a ratio with the value, from the amount of generated information of the image frame exceeding the motion threshold and the quantization ratio, predict the amount of information generated when the quantization value is changed, This From the predicted amount of generated information, the average throughput, the transmission rate, and the maximum allowable delay time, determine a quantizing value that can be transmitted within the maximum allowable delay time, and notify the encoding unit of the quantization value. The image transmission system, wherein the quantization unit encodes the image frame with the quantization value notified from the quantization control unit. 48. The transmission control means, when calculating the average throughput, according to a change in the number of bits.
The averaging time is variable.
8. The image transmission system according to 7. 49. The image transmission system according to claim 47, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 50. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data received correctly within a predetermined period of time, calculates an average throughput from the counted number of bits, and transmits the average throughput to the image encoding device. Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and an image of the image frame encoded by the encoding means A transmission buffer for storing data, a quantization control means for determining a quantization value used for encoding, and transmitting image data in the transmission buffer at a predetermined transmission rate Transmission control means for outputting the average throughput received from the image receiving device to the quantization control means; and a motion threshold for storing a motion threshold value for determining whether the motion of the subject is large or small. Value storage means, and maximum allowable delay time storage means for storing a maximum allowable delay time which is a limit of transmission delay that does not cause visual discomfort, wherein the quantization control means is a result of encoding by the encoding means. If the amount of generated information per frame is larger than the motion threshold, the quantization value used when encoding the image frame exceeding the motion threshold and the quantum for which the generated information amount is to be predicted. A quantization ratio, which is a ratio to the quantization value, is calculated, and the amount of information generated when the quantization value is changed is predicted from the amount of generated information of the image frame exceeding the motion threshold and the quantization ratio. From the predicted amount of generated information, the average throughput, the transmission rate, and the maximum allowable delay time, determine a quantizing value that can be transmitted within the maximum allowable delay time, and notify the encoding unit, The image transmission system according to claim 1, wherein the encoding unit encodes the image frame with the quantization value notified from the quantization control unit. 51. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data correctly received within a predetermined time, transmits the counted number of bits to the image encoding device, and the image encoding device converts an image from a video signal into an image. Image input means for constructing and outputting a frame, encoding means for encoding the image frame constituted by the image input means, and transmission buffer for storing image data of the image frame encoded by the encoding means And a quantization control unit for determining a quantization value to be used for encoding; and transmitting image data in the transmission buffer at a predetermined transmission rate; A transmission control unit for calculating an average throughput from the number of bits received from the control unit and outputting the average throughput to the quantization control unit; and a motion threshold value for storing a motion threshold value for determining whether the motion of the subject is large or small. An ideal curve in which an operating point is predetermined as a balance between temporal distortion due to frame dropping according to a coding rate which is a rate of a frame that can be transmitted per unit time and spatial distortion due to a quantization value. And an ideal curve storage unit for storing the motion information. If the amount of information generated per frame as a result of encoding by the encoding unit is larger than the motion threshold, Calculate a quantization ratio, which is a ratio between a quantization value used when encoding an image frame exceeding a threshold value and a quantization value for which the amount of generated information is to be predicted, and exceeds the motion threshold. From the amount of generated information of the image frame and the quantization ratio, an amount of information generated when a quantization value is changed is predicted.From the predicted amount of generated information, the transmission rate, and the average throughput, the encoding is performed. Calculating a rate, determining a quantization value from the calculated coding rate and the ideal curve, and notifying the coding means, the coding means receives the quantization notified from the quantization control means. An image transmission system, characterized by encoding an image frame with a value. 52. The transmission control means according to a change in the number of bits when calculating the average throughput.
6. The averaging time is variable.
2. The image transmission system according to 1. 53. The image transmission system according to claim 51, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 54. An image transmission system in which an image encoding device that encodes and transmits a moving image and an image receiving device that performs predetermined processing on received image data are communicably connected, The image receiving device counts the number of bits of data received correctly within a predetermined period of time, calculates an average throughput from the counted number of bits, and transmits the average throughput to the image encoding device. Image input means for constructing and outputting an image frame from a video signal, encoding means for encoding the image frame constituted by the image input means, and an image of the image frame encoded by the encoding means A transmission buffer for storing data, a quantization control means for determining a quantization value used for encoding, and transmitting image data in the transmission buffer at a predetermined transmission rate Transmission control means for outputting the average throughput received from the image receiving device to the quantization control means; and a motion threshold for storing a motion threshold value for determining whether the motion of the subject is large or small. An ideal value whose operating point is predetermined as a balance between temporal distortion due to frame dropping according to a coding rate which is a rate of a frame that can be transmitted per unit time and spatial distortion due to a quantization value. Ideal curve storage means for storing a curve, wherein the quantization control means, when the amount of information generated per frame as a result of encoding by the encoding means is larger than the motion threshold, A quantization ratio, which is a ratio between a quantization value used when encoding an image frame exceeding a threshold value and a quantization value for which the amount of generated information is to be predicted, is obtained, and the motion threshold is calculated. From the generated information amount of the image frame that exceeds and the quantization ratio, the amount of information generated when the quantization value is changed is predicted.From the predicted generated information amount, the transmission rate, and the average throughput, Calculating the coding rate, determining a quantization value from the calculated coding rate and the ideal curve, and notifying the coding means, the coding means is notified from the quantization control means. An image transmission system, characterized in that an image frame is encoded with a quantized value. 55. An image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data are communicably connected via one or more relay stations. An image transmission system, wherein the image receiving device detects an error in received data,
When performing automatic retransmission with the adjacent relay station, the relay station, when detecting an error in the received data, performs automatic retransmission with the adjacent relay station, and performs retransmission indicating that retransmission has occurred. Transmitting the generated information to the image encoding device, the image encoding device comprising: an image input unit configured to output an image frame from a video signal; and a code encoding the image frame configured by the image input unit. Encoding means; a transmission buffer for storing the image data encoded by the encoding means; a first quantization control means for determining an optimal quantization value in terms of both motion reproducibility and image quality; A second quantization control means for controlling a quantization step width in accordance with the following, transmitting image data in the transmission buffer at a predetermined transmission rate, and generating a communication error between the adjacent relay stations. In the case where the retransmission is performed automatically between the adjacent relay station and the retransmission itself, or when the retransmission occurrence information is received from the image receiving apparatus, an error indicating that retransmission has occurred. Transmission control means for notifying information to the second quantization control means, wherein the second quantization control means, upon receiving a notification of error information from the transmission control means, A quantization value having a quantization step width larger than the quantization step width corresponding to the quantization value determined by the means is set and notified to the encoding means. If the notification of the error information is not received, the quantization value determined by the first quantization control unit is notified to the encoding unit as it is, and the encoding unit encodes an image frame to be encoded next, The second quantity Characterized by coding the notification quantization value from the controlling means, an image transmission system. 56. An image encoding apparatus for encoding and transmitting a moving image and an image receiving apparatus for performing predetermined processing on received image data are communicably connected via one or more relay stations. An image transmission system, wherein the image receiving device detects an error in received data,
When performing automatic retransmission with the adjacent relay station, the relay station, when detecting an error in the received data, performs automatic retransmission with the adjacent relay station, and performs retransmission indicating that retransmission has occurred. Transmitting the generated information to the image encoding device, the image encoding device comprising: an image input unit configured to output an image frame from a video signal; and a code encoding the image frame configured by the image input unit. Encoding means; a transmission buffer for storing image data of an image frame encoded by the encoding means; a transmission control means for transmitting image data in the transmission buffer at a predetermined transmission rate; and the encoding means. And between the transmission buffer and
And a temporary buffer for temporarily storing image data of the image frame encoded by the encoding means, and determining whether or not to store the image data in the temporary buffer in the transmission buffer. Frame removal / quantization control means for determining a quantization value of an image frame to be converted, wherein the frame removal / quantization control means, upon receiving notification of error information from the transmission control means, The frame drop threshold for determining whether or not to store the image data in the transmission buffer is lowered from a preset value, and a notification of error information is received from the transmission control means for a predetermined fixed time. If not, the frame drop threshold is returned to a preset value, threshold control is performed, and the image frame stored in the temporary buffer is If the information amount of the frame is larger than the frame drop threshold, the frame buffer control is performed so that the image data in the temporary buffer is not stored in the transmission buffer, and the image frame stored in the temporary buffer is The quantization unit sets a quantization value having a quantization step width larger than the quantization step width used for encoding, and notifies the encoding unit of the quantization value. An image transmission system, wherein encoding is performed using a quantization value notified from a frame drop / quantization control unit. 57. An image encoding apparatus for encoding and transmitting a moving image, wherein, in an image area to be encoded, a priority area has a first quantization accuracy and a non-priority area has a second quantization accuracy. Image encoding means for generating an image encoded bit string, and determining the priority area and the non-priority area in accordance with the amount of generated information of the image encoded by the image encoding means. An image encoding apparatus, comprising: a priority area / non-priority area changing unit for changing; and a transmission control unit for transmitting the image encoding bit sequence. 58. When a moving image is subjected to high-efficiency compression encoding and transmitted through a low-bit-rate communication line, time is reduced by dropping frames in accordance with the encoding rate, which is the rate of frames that can be transmitted per unit time. Distortion, as a balance between spatial distortion due to quantization accuracy, in an image encoding device in which the operating point is predetermined, a method for controlling the quantization operation when encoding an image frame, Determines the quantization precision for the next encoding according to the magnitude relationship between the coding rate of the result of encoding the entire image frame with a certain quantization precision and the coding rate of the operating point according to the quantization precision. Dividing the image area to be encoded into a priority area and a non-priority area, encoding the priority area with a relatively high first quantization precision, and Second quantity with low accuracy The next encoding is performed in accordance with the magnitude relation between the encoding rate of the result of encoding with the first and second quantization precisions and the operating point according to the quantization precisions. A second operation mode for determining the quantization accuracy at the time and the size of the priority region and the non-priority region, wherein the quantization accuracy determined in the first operation mode is a predetermined quantization accuracy. When the accuracy becomes lower than the allowable lower limit, the operation mode shifts from the first operation mode to the second operation mode, and the size of the priority area determined in the second operation mode becomes smaller. When the maximum is reached, the operation mode shifts from the second operation mode to the first operation mode. 59. A coding rate corresponding to each quantization precision is:
Each of them is defined as a set of points in an area sandwiched between an upper limit value and a lower limit value. In the first and second operation modes, a coding rate of a result of encoding with a certain quantization precision is the upper limit. When the value exceeds the value, the quantization accuracy at the next encoding is improved, and when the coding rate of the result of encoding at a certain quantization accuracy falls below the lower limit, the quantization accuracy at the next encoding is increased. When the coding rate of the result of encoding with a certain quantization accuracy falls within the region between the upper limit value and the lower limit value, the quantization accuracy in the next encoding is not changed. The quantization control method according to claim 58, wherein the quantization control method is characterized in that: 60. The minimum width of the quantization accuracy when changing in the first and second operation modes is selected to be a value larger than a value that allows a difference in spatial distortion to be visually recognized. The quantization control method according to claim 59, wherein: 61. The transmission control means, wherein when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 3. 62. The image coding apparatus according to claim 3, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 63. The transmission control means, wherein when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 7. 64. The image encoding apparatus according to claim 7, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 65. The frame drop / quantization control means further calculates an average delay time based on the average generated information amount of image frames transmitted within a predetermined time and the average throughput, and calculates the calculated average delay time. The image transmission system according to claim 46, wherein an average delay time is used as the maximum allowable delay time. 66. The image receiving apparatus according to claim 46, wherein, when calculating the average throughput, the averaging time is variable according to a change in the number of bits of the correctly received data. Image transmission system. 67. The image transmission system according to claim 46, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 68. The image receiving apparatus according to claim 50, wherein, when calculating the average throughput, the averaging time is variable according to a change in the number of bits of the correctly received data. Image transmission system. 69. The image transmission system according to claim 50, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 70. The image receiving apparatus according to claim 54, wherein, when calculating the average throughput, the averaging time is variable according to a change in the number of bits of the correctly received data. Image transmission system. 71. The image transmission system according to claim 54, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 72. The quantizing control means, when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 10. 73. The quantization control means, when calculating the average throughput, varies the averaging time in accordance with the presence or absence of a burst error.
The image encoding device according to claim 11. 74. The image encoding apparatus according to claim 10, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 75. The image coding apparatus according to claim 11, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 76. The image coding apparatus according to claim 4, wherein said transmission control means, when calculating said average throughput, varies the averaging time in accordance with the presence or absence of a burst error. Device. 77. The image encoding apparatus according to claim 4, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 78. The image coding apparatus according to claim 8, wherein said transmission control means, when calculating said average throughput, varies the averaging time in accordance with the presence or absence of a burst error. Device. 79. The image encoding apparatus according to claim 8, wherein the predetermined fixed time for calculating the average throughput is set to 300 msec or more. 80. An average throughput per fixed time is calculated based on an amount of data output per unit time t from the transmission buffer for temporarily storing encoded data. Claims 3, 4, 7 to 1
3,15-17,19-21,61-64,72-79
The image encoding device according to any one of the above. 81. By dividing the amount of data output per unit time obtained by adding the amount of data output per unit time t for each unit time t every time the unit time elapses by the certain period of time, 18. The method according to claim 3, wherein the average throughput per fixed time is calculated every time the fixed time elapses.
The image encoding device according to any one of 19 to 21, 61 to 64, and 72 to 79. 82. Each time the unit time t elapses, the amount of data output per unit time obtained by adding the amount of data output to the unit time t from the time before the certain time to the point in time is calculated. 4. The method according to claim 3, wherein an average throughput per said fixed time is calculated every time the unit time t elapses by dividing by a certain time T.
4,7-13,15-17,19-21,61-64,
80. The image encoding device according to any one of 72 to 79. 83. An average per fixed time based on an amount of data output from the transmission buffer for temporarily storing encoded data in the same unit time t as an image frame period of an image to be encoded. 3. The throughput according to claim 3, wherein the throughput is calculated.
The image encoding device according to any one of 7, 19 to 21, 61 to 64, and 72 to 79. 84. Each time the fixed time elapses, the data amount output per unit time obtained by adding the data amount output in the same unit time t as the image frame period of the encoded image for the fixed time is output. It is characterized in that an average throughput per the certain time is calculated every time the certain time elapses by dividing a data amount by the certain time,
Claims 3, 4, 7-13, 15-17, 19-21, 6
Image encoding apparatus according to any one of 1 to 64 and 72 to 79. 85. Each time when the same unit time t as the image frame period of the image to be encoded elapses, the constant amount obtained by adding the data amount output to the unit time t from the predetermined time period to the time point. The data amount output per unit of time is divided by the certain period of time to calculate the average throughput per unit time t every time the unit time t elapses. 13,15
The image encoding device according to any one of claims 1 to 17, 19 to 21, 61 to 64, and 72 to 79. 86. An average throughput per fixed time based on an amount of data output from the transmission buffer for temporarily storing encoded data in the same unit time t as one frame time of a transmission frame. , Which is characterized by calculating
The image encoding device according to any one of 19 to 21, 61 to 64, and 72 to 79. 87. The data amount output per fixed time obtained by adding the data amount output in the same unit time t as one frame time of the transmission frame every time the fixed time elapses for the fixed time. 22. The average throughput per the fixed time is calculated each time the fixed time elapses by dividing the constant time by the fixed time. , 61
-64, 72-79. 88. Each time the same unit time t equal to one frame time of a transmission frame elapses, the fixed time per unit time obtained by adding the data amount output to the unit time t from the time before the certain time to the time. Wherein the average throughput per said fixed time is calculated every time the unit time t elapses by dividing the amount of data output to the predetermined time by the certain time. 15 ~
The image encoding device according to any one of 17, 19 to 21, 61 to 64, and 72 to 79. 89. An amount of data output from the transmission buffer for temporarily storing encoded data in a unit time t equal to the least common multiple of an image frame period of an image to be encoded and one frame time of a transmission frame. 20. The average throughput per fixed time is calculated based on the following equation.
The image encoding device according to any one of claims 21 to 61, 64 to 72, and 79 to 79. 90. Every time the fixed time elapses, the data amount output in the same unit time t as the least common multiple of the image frame period of the image to be encoded and one frame time of the transmission frame is added for the fixed time. The data amount output per unit time obtained by dividing the data amount by the unit time to calculate an average throughput per unit time each time the unit time elapses, wherein 4,7-13,15-17,19-21,61-6
The image encoding device according to any one of 4, 72 to 79. 91. Every time a unit time t equal to the least common multiple of the image frame period of the image to be coded and one frame time of the transmission frame elapses, it is output during the unit time t from the predetermined time before to that time Dividing the amount of data output per unit time obtained by adding the calculated data amount by the unit time to calculate the average throughput per unit time every time the unit time t elapses. Claims 3, 4, 7-13, 15-1
The image encoding device according to any one of 7, 19 to 21, 61 to 64, and 72 to 79. 92. The image according to claim 39, wherein, when calculating the average throughput, the image receiving apparatus changes the averaging time in accordance with a change in the number of bits of correctly received data. Transmission system. 93. The image transmission system according to claim 39, wherein the predetermined time for calculating the average throughput is set to 300 msec or more. 94. An average throughput per fixed time T is calculated based on an amount of data output per unit time t from a transmission buffer for temporarily storing encoded data. Average throughput calculation method. 95. A cycle in which the amount of data output per unit time t from the transmission buffer is smaller than a predetermined threshold, and the cycle is set to the predetermined time T.
The average throughput calculation method according to claim 94, wherein: 96. The data output per unit time T obtained by adding the amount of data output in the unit time t every time the predetermined time T elapses, to the amount of data output per unit of time T, By dividing by T, the predetermined time T
The average throughput calculation method according to claim 94, wherein an average throughput per hit is calculated each time the predetermined time T elapses. 97. A cycle in which the amount of data output from the transmission buffer during the unit time t is smaller than a predetermined threshold, and the cycle is set to the predetermined time T.
97. The average throughput calculation method according to claim 96, wherein: 98. The amount of data output per unit time T obtained by adding the amount of data output to the unit time t from before the unit time T until the unit time t every time the unit time t elapses Is divided by the predetermined time T,
The average throughput calculation method according to claim 94, wherein said average throughput per said fixed time T is calculated every time said unit time t elapses. 99. A cycle in which the amount of data output from the transmission buffer in the unit time t is smaller than a predetermined threshold, and the cycle is set to the predetermined time T.
The average throughput calculation method according to claim 98, wherein: 100. The apparatus according to claim 94, wherein said unit time t is an image frame period of an image to be coded.
98. The average throughput calculation method according to any of 98. 101. The unit time t is set to one of the transmission frames.
The frame time is a frame time.
8. The average throughput calculation method according to any one of 8. 102. The average throughput according to claim 94, wherein said unit time t is a least common multiple of an image frame period of an image to be coded and one frame time of a transmission frame. Calculation method. 103. A cycle in which the amount of data output per unit time t is smaller than a predetermined threshold from the transmission buffer for temporarily storing encoded data, and The averaging time used in calculating an average throughput is set as the averaging time.
78. The image encoding device according to any one of 78. 104. An amount of data output from the transmission buffer for temporarily storing encoded data in the same unit time t as one frame time of a transmission frame becomes smaller than a predetermined threshold value. The cycle is detected, and the cycle is set as the averaging time used in calculating an average throughput.
The image encoding device according to any one of 0, 61, 63, 72, 73, 76, and 78. 105. A data amount output from the transmission buffer for temporarily storing encoded data in the same unit time t as an image frame period of an image to be encoded is smaller than a predetermined threshold value. 17. The method according to claim 12, wherein a period that becomes smaller is detected, and the period is used as the averaging time used in calculating an average throughput.
The image encoding device according to any one of 20, 61, 63, 72, 73, 76, and 78. 106. Data output from the transmission buffer for temporarily storing encoded data in a unit time t equal to the least common multiple of an image frame period of an image to be encoded and one frame time of a transmission frame. 21. The method according to claim 12, wherein a period in which the amount is smaller than a predetermined threshold is detected, and the period is used as the averaging time used in calculating an average throughput.
61. The image encoding device according to any one of 61, 63, 72, 73, 76, and 78. 107. The image according to claim 39, wherein, when calculating the average throughput, the image receiving apparatus changes the averaging time in accordance with a change in the number of correctly received bits. Transmission system. 108. The image transmission system according to claim 39, wherein the predetermined fixed time for calculating the average throughput is 300 msec or more. 109. Each time the fixed time elapses, reception can be performed without error per fixed time obtained by adding the number of bits received without error during a predetermined unit time t for the fixed time. The transmitted bit number is transmitted to the image encoding device every time the fixed time elapses.
54. The image transmission system according to any one of 1 to 53. 110. Each time the fixed time elapses, the reception can be performed without error per fixed time obtained by adding the number of bits that can be received without error during a predetermined unit time t for the fixed time. By dividing the number of bits by the fixed time, the average throughput per fixed time,
49. The apparatus according to claim 39, 46, 50, or 5, wherein the image data is transmitted to the image encoding apparatus every time the predetermined time elapses.
The image transmission system according to any one of 4, 65 to 71, 107, and 108. 111. Each time a predetermined unit time t elapses, the predetermined time obtained by adding the number of bits that can be received without error during the unit time t from the time before the predetermined time to the time. The number of bits that can be received without error per hit is transmitted to the image encoding apparatus every time the unit time t elapses.
-45, 47-49, or 51-53. 112. Each time a predetermined unit time t elapses, the predetermined time obtained by adding the number of bits that can be received without error during the unit time t from the time before the predetermined time to the time point. By dividing the number of bits received per error per unit by the fixed time, the average throughput per fixed time is transmitted to the image encoding device every time the unit time t elapses, Claims 39, 46, 50, 54, 65-71, 107, 10
9. The image transmission system according to any one of 8. 113. The image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image frame period of the encoded image is changed every time the fixed time elapses. The number of bits that can be received without error per unit time obtained by adding the number of bits that can be received without error during the same unit time t for the certain time is calculated by the image code every time the certain time elapses. 36. The method according to claim 36, wherein
The image transmission system according to any one of 47 to 49, 51 to 53. 114. The image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image frame period of the encoded image is By dividing the number of bits received without error per fixed time obtained by adding the number of bits received without error during the same unit time t for the fixed time by the fixed time, Transmitting the average throughput of the image encoding device to the image encoding device every time the fixed time elapses, wherein
The image transmission system according to any one of 46, 50, 54, 65 to 71, 107, and 108. 115. The image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, and each time a unit time t equal to the image frame period of the encoded image elapses. The number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the unit time t from the previous time to the time is calculated as 4. The image encoding apparatus according to claim 3, wherein the image data is transmitted to the image encoding device every time the time period elapses.
The image transmission system according to any one of 6 to 38, 42 to 45, 47 to 49, and 51 to 53. 116. The image receiving apparatus further comprises, similar to the image encoding apparatus, a function of encoding and transmitting an image, and each time a unit time t equal to the image frame period of the image to be encoded elapses. The number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the unit time t from the time before the certain time to the time is divided by the certain time. Thus, the average throughput per unit time can be calculated as the unit time t
39, 46, 50, 54, 65-7.
The image transmission system according to any one of 1, 107, and 108. 117. Each time the fixed time elapses, the number of bits received without error during the same unit time t as one frame time of the transmission frame is added for the fixed time, and The number of bits that can be received without error is transmitted to the image encoding device every time the predetermined time elapses.
45, The image transmission system according to any one of 47 to 49 and 51 to 53. 118. Every time the fixed time elapses, the number of bits that can be received without error during the same unit time t as one frame time of a transmission frame is added for the fixed time, and The method according to claim 3, wherein the average throughput per fixed time is transmitted to the image encoding device every time the fixed time elapses by dividing the number of bits received without error by the fixed time.
The image transmission system according to any one of 9, 46, 50, 54, 65 to 71, 107, and 108. 119. Each time a unit time t equal to one frame time of a transmission frame elapses, the number of bits that can be received without error during the unit time t from the predetermined time to the time is obtained by adding. The number of bits that can be received without error per the predetermined time period is transmitted to the image encoding device every time the unit time t elapses. 49, 51-53
The image transmission system according to any one of the above. 120. Each time the same unit time t as one frame time of a transmission frame elapses, the number of bits that can be received without error during the unit time t from the predetermined time to the time is obtained by adding. By dividing the number of bits that can be received without error per fixed time by the fixed time, transmitting the average throughput per fixed time to the image encoding device every time the unit time t elapses. Claims 39, 46, 50, 54, 65 characterized by the features.
110, the image transmission system according to any one of 107, 108. 121. The image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image frame period of the encoded image is changed every time the fixed time elapses. And the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the same unit time t as the least common multiple of one frame time of the transmission frame and the fixed time, The transmission to the image encoding device every time the predetermined time elapses, wherein the image encoding device is transmitted to the image encoding device.
49. The image transmission system according to any one of 49, 51 to 53. 122. The image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image frame period of the encoded image is changed every time the fixed time elapses. And the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the same unit time t as the least common multiple of one frame time of the transmission frame and the fixed time, The method according to claim 39, wherein by dividing by the fixed time, the average throughput per fixed time is transmitted to the image encoding apparatus every time the fixed time elapses.
The image transmission system according to any one of 6, 50, 54, 65 to 71, 107, and 108. 123. The image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image receiving apparatus is configured to transmit the image frame period of the encoded image and one frame time of the transmission frame. The same unit time t as the least common multiple
Every time elapses, the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the unit time t from the time before the certain time to the time, The image data is transmitted to the image encoding device every time the unit time t elapses.
The image transmission system according to any one of 8, 42 to 45, 47 to 49, and 51 to 53. 124. The image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image receiving apparatus is configured to transmit the image frame cycle of the image to be encoded and one frame time of the transmission frame. The same unit time t as the least common multiple
Every time elapses, the number of bits that can be received without error per fixed time obtained by adding the number of bits that can be received without error during the unit time t from the time before the certain time to the time, By dividing by a certain time, the average throughput per the certain time is transmitted to the image encoding device every time the time t elapses,
Claims 39, 46, 50, 54, 65-71, 107,
108. The image transmission system according to any of 108. 125. A method for calculating an average throughput in a data transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on received data are communicably connected. Therefore, every time the predetermined period of time T elapses,
The predetermined time T elapses the number of bits that can be received without error per the predetermined time T obtained by adding the number of bits that can be received without error during the predetermined unit time t for the predetermined time T. The average throughput is transmitted to the encoding device every time, and the encoding device calculates the average throughput per the predetermined time T based on the number of bits received from the receiving device. 126. A method for calculating an average throughput in a data transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on received data are communicably connected. Therefore, every time the predetermined period of time T elapses,
By dividing the number of bits that can be received without error per the predetermined time T obtained by adding the number of bits that can be received without error during a predetermined unit time t for the predetermined time T by the certain time T, An average throughput calculation method, wherein the average throughput per predetermined time T is calculated every time the predetermined time T elapses. 127. A method for calculating an average throughput in a data transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on received data are communicably connected. The receiving device adds the number of bits that can be received without error during the unit time t from before the predetermined time T until the predetermined time T every time the predetermined unit time t elapses. The number of bits that can be received without error per fixed time T obtained by the above is transmitted to the encoding device every time the unit time t elapses, and the encoding device calculates the number of bits received from the receiving device. Calculating the average throughput per the predetermined time T. 128. A method for calculating an average throughput in a transmission system in which an encoding device that encodes and transmits data and a receiving device that performs predetermined processing on received data are communicably connected. Each time a predetermined unit time t elapses, the receiving apparatus adds the number of bits that can be received without error during the unit time t from before a predetermined time T to the time. By dividing the obtained number of bits that can be received without error per fixed time T by the fixed time T, an average throughput per fixed time T is calculated every time the unit time t elapses. , Average throughput calculation method. 129. The unit time t is equal to one of the transmission frames.
125. A frame time, which is a frame time.
128. The method of calculating average throughput according to any one of items 128. 130. The receiving apparatus further comprises a function of encoding and transmitting data, similar to the encoding apparatus, wherein the unit time t is a data frame period of encoded data. The method of calculating an average throughput according to any one of claims 125 to 128. 131. The receiving apparatus further comprises a function of encoding and transmitting the same data as the encoding apparatus, wherein the unit time t is a data frame period of encoded data and one frame of a transmission frame. The average throughput calculation method according to any one of claims 125 to 128, wherein the method is a least common multiple of time. 132. The image receiving apparatus, wherein when calculating the average throughput, varies an averaging time in accordance with the presence or absence of a burst error.
The image transmission system according to claim 39. 133. The image receiving apparatus, wherein, when calculating the average throughput, varies an averaging time according to whether or not a burst error has occurred.
The image transmission system according to claim 46. 134. The image receiving apparatus, when calculating the average throughput, varies the averaging time according to the presence or absence of a burst error.
The image transmission system according to claim 50. 135. The image receiving apparatus according to claim 135, wherein, when calculating the average throughput, the averaging time is variable depending on whether or not a burst error has occurred.
The image transmission system according to claim 54. 136. A method for detecting a cycle in which the number of bits received without error during a predetermined unit time t becomes smaller than a predetermined threshold value, and calculating the cycle in calculating an average throughput. The image transmission system according to any one of claims 132 to 135, wherein the averaging time is used. 137. A cycle in which the number of bits received without error during the same unit time t as one frame time of a transmission frame is smaller than a predetermined threshold,
134. The method according to claim 132, wherein the period is the averaging time used in calculating an average throughput.
135. The image transmission system according to any one of 135. 138. The image receiving apparatus further comprises a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein an error occurs during the same unit time t as the image frame period of the image to be encoded. 133. The method according to claim 132, further comprising: detecting a cycle in which the number of bits that can be received without any error is smaller than a predetermined threshold, and setting the cycle as the averaging time used in calculating an average throughput. ~ 135
The image transmission system according to any one of the above. 139. The image receiving apparatus further includes a function of encoding and transmitting an image, similar to the image encoding apparatus, wherein the image receiving apparatus has a function of encoding an image frame period of an encoded image and one frame time of a transmission frame. The same unit time t as the least common multiple
A period in which the number of bits received without error during the period is smaller than a predetermined threshold value, and the period is used as the averaging time used in calculating an average throughput. The image transmission system according to any one of claims 132 to 135.