CN115643425A - Video communication system and video transmission method based on long reference frame - Google Patents
Video communication system and video transmission method based on long reference frame Download PDFInfo
- Publication number
- CN115643425A CN115643425A CN202211211627.2A CN202211211627A CN115643425A CN 115643425 A CN115643425 A CN 115643425A CN 202211211627 A CN202211211627 A CN 202211211627A CN 115643425 A CN115643425 A CN 115643425A
- Authority
- CN
- China
- Prior art keywords
- frame
- sending
- reference frame
- video
- receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a video transmission method based on a long reference frame in a video communication system in the video communication system, which comprises the following steps: caching at least one frame of IDR frame which is coded recently by a sending end in real time; receiving a request for sending an IDR frame from at least one receiving end; sending a long reference frame request to a sending end based on the request; receiving a long reference frame coded by a sending end, wherein the long reference frame is a long reference frame coded by the sending end by referring to the most recently coded IDR frame; and sending the IDR frame and the long reference frame which are coded recently to a receiving end which sends a request, and sending the long reference frame to other receiving ends which do not send the request. The video transmission method based on the long reference frame in the video communication system effectively reduces the calculation load of the sending end and the uplink network bandwidth pressure, and simultaneously provides better guarantee for the image quality of other terminals joining the session, reduces the downlink bandwidth impact, reduces the probability of congestion and packet loss, and reduces the retransmission pressure.
Description
Technical Field
The present invention relates to the field of communication technologies, and in particular, to a video communication system and a video transmission method based on a long reference frame in the video communication system.
Background
Internet-based video communication technology is widely applied to video conference scenes of work and life. In video communication transmission, a cloud + end mode is usually adopted, that is, a video stream generated by a video communication terminal is forwarded through a video conference media Multipoint Control Unit (MCU) at a cloud end, and a sending end → a cloud end Control Unit → a receiving end. Specifically, when the receiving end is accessed to the conference for the first time or the video with serious packet loss cannot be recovered, the control unit sends an "IDR frame request", forwards the "IDR frame request" to the sending end, codes an IDR frame at the sending end, and then forwards the IDR frame to the receiving end through the control unit in a relaying manner. Or the sending end can generate the IDR frame and the P frame regularly and send the IDR frame and the P frame to all the receiving ends. When the receiving end is accessed to the conference for the first time or the video with serious packet loss can not be recovered, the receiving end waits for the next IDR frame to arrive to continue the conference.
Three kinds of frames are defined in the H.264 protocol, the completely coded frame is called IDR frame, the frame which only contains difference part coding and is generated by referring to the previous IDR frame is called P frame, and the frame which is coded by referring to the previous frame and the next frame is called B frame. H.264 is a new generation of coding standard known for high compression, high quality and streaming media supporting a variety of networks. The core algorithms adopted by h.264 are intraframe compression, which is an algorithm for generating IDR frames, and interframe compression, which is an algorithm for generating B and P frames.
The IDR frame is an intra-frame coding frame, represents a key frame, is an independent frame with all information, can be independently decoded without referring to other images, and is always the IDR frame in a video sequence. The integrity of this frame of picture is preserved; only the frame data is needed to complete the decoding.
The P frame is a forward predictive coding frame, the P frame represents the difference between the frame and a previous IDR frame (or P frame), the data of the frame is compressed according to the difference between the frame and the adjacent previous frame (IDR frame or P frame), and the difference defined by the frame is superposed on the previously buffered picture during decoding to generate a final picture. As shown in the upper part of fig. 2, in the encoding method of the prior art video transmission, the first frame P frame is the first frame IDR frame as the reference frame, which only includes the different points generated after comparing with the first frame IDR frame, and when the receiving end receives the first frame P frame, the video frame generated based on the first frame IDR frame and the first frame P frame form the current video frame; the second frame P frame is the reference frame of the first frame P frame, wherein only the different points generated after the comparison with the first frame P frame are included, when the receiving end receives the second frame P frame, the video frame generated based on the first frame IDR frame and the video frame generated by the first frame P frame are combined with the second frame P frame to form the current video frame.
When a frame is compressed into a B frame, the B frame compresses the frame according to the difference of the adjacent previous frame, the current frame and the next frame data, namely only the difference between the current frame and the previous and next frames is recorded. Only with B-frame compression can 200 be achieved: 1, high compression.
In the prior art, when a receiving end is accessed to a conference for the first time or a seriously lost video cannot be recovered, an 'IDR frame request' is sent to a control unit, the control unit forwards the 'IDR frame request' to a sending end, the sending end encodes an IDR frame, and the IDR frame is forwarded to the receiving end through a relay of the control unit, so that the following defects exist:
1. the transmitting end calculates resource consumption: encoding IDR frames typically consumes more computational resources than encoding P/B frames.
2. The uplink bandwidth of a sending end occupies: in the case where the image quality is not significantly affected, the IDR frame is several times or even ten times larger than the general P frame, and therefore, when a receiver transmits an IDR frame request, the transmission bandwidth requirement of the transmitter becomes high.
3. The image quality of other participants is impaired: under the condition that the uplink bandwidth of a sending end is limited or a system has a low-delay requirement, the number of bytes of an IDR frame is generally required to be controlled within a certain range, so that the situation of image quality reduction is inevitable, and other parties participating in communication feel that the image quality is temporarily reduced when an IDR frame request exists.
Disclosure of Invention
The invention provides a video communication system and a video transmission method based on a long reference frame in the video communication system, which can effectively realize the transmission process of saving transmission bandwidth, calculating resources and reserving higher image quality by using a long reference frame technology to replace an IDR frame under the condition that the position of a camera is relatively fixed.
In a first aspect, the present invention provides a method for transmitting video based on long reference frames in a video communication system, wherein the video communication system includes a control unit, and at least one transmitting end and at least one receiving end connected to the control unit, the method comprising:
caching at least one frame of IDR frame which is most recently coded by the sending end in real time;
receiving a request for sending an IDR frame from at least one receiving end;
sending a long reference frame request to the sending end based on the request;
receiving a long reference frame encoded by the transmitting end, wherein the long reference frame is a long reference frame encoded by the transmitting end by referring to a most recently encoded IDR frame;
and sending the IDR frame which is most recently coded and the long reference frame to a receiving end which sends a request, and sending the long reference frame to other receiving ends which do not send the request.
In a second aspect, the present invention further provides a video communication system, which includes: the device comprises a control unit, at least one sending end and at least one receiving end which are connected with the control unit, wherein the control unit comprises a cache module, a receiving module and a sending module
The buffer module is used for buffering at least one frame of IDR frame which is recently coded by the sending end in real time;
the receiving module is used for receiving a request for requesting to send an IDR frame from at least one receiving end;
the sending module is used for sending a long reference frame request to the sending end based on the request;
the receiving module is further configured to receive a long reference frame encoded by the sending end, where the long reference frame is a long reference frame encoded by the sending end with reference to a most recently encoded IDR frame;
the sending module is further configured to send the most recently encoded IDR frame and the long reference frame to a receiving end that sends a request, and send the long reference frame to other receiving ends that do not send a request.
The video communication system and the video transmission method based on the long reference frame emphasize the cache function of the cloud control unit in the video communication system, in multi-party video communication, when a new terminal joins in a session or seriously lost video cannot be recovered, the IDR frame request is replaced by the long reference frame request, the last frame of IDR frame cached by the control unit is referred by the receiving terminal, the long reference frame is decoded, the IDR frame does not need to be re-encoded by the sending terminal aiming at the IDR frame request sent by the receiving terminal while video transmission is achieved, the calculation load and the uplink network bandwidth pressure of the sending terminal are effectively reduced, meanwhile, better guarantee is provided for the image quality of the terminal which joins in the session, the downlink bandwidth impact is smaller, the probability of congestion and packet loss is reduced, and the retransmission pressure is smaller. Especially in large-scale video communication interaction, frequent terminals join in a conference scene, and the benefit is more obvious.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the embodiments or the prior art description will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings may be obtained according to these drawings without inventive labor.
Fig. 1 is a flowchart of a video transmission method based on a long reference frame in a video communication system according to an embodiment of the present invention;
fig. 2 illustrates the video transmission encoding principle provided by the prior art and the embodiment of the present invention;
FIGS. 3A and 3B are schematic diagrams illustrating interaction of a long reference frame request step according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a video communication system according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Invention (I)SUMMARY
As described above, the present invention provides a video communication system and a video transmission method based on a long reference frame in the video communication system, in multi-party video communication, when a new terminal joins a session or a video with a serious packet loss cannot be recovered, the computational load and the uplink network bandwidth pressure of a sending end are effectively reduced, and meanwhile, better guarantee is provided for the image quality of the terminal that has joined the session, the downlink bandwidth impact is smaller, the probability of congestion and packet loss is reduced, the retransmission pressure is smaller, and the stability of service can be significantly improved.
Exemplary method
Fig. 1 is a flowchart of a video transmission method based on a long reference frame in a video communication system according to an embodiment of the present invention, where the video communication system includes a control unit, and at least one sending end and at least one receiving end connected to the control unit.
The video communication system comprises any one of a monitoring system, a live broadcast system, a video conference system and a video recording system.
The control unit is an intermediate unit connected with the sending end and the receiving end and is generally located in the cloud server.
The long reference frame may be a P frame or a B frame. The P frame is a forward predictive coding frame, which represents the difference of the frame and the previous frame, and the difference defined by the frame needs to be superposed by the picture buffered before in decoding to generate a final picture. The B frame is a bidirectional predictive interpolation coded frame, and records the difference between the current frame and the previous and subsequent frames, in other words, to decode the B frame, not only the previous buffer picture but also the decoded picture are obtained, and the final picture is obtained by superimposing the previous and subsequent pictures on the current frame data.
In summary, the compression rate of P-frames and B-frames is high, both cannot be decoded into one picture independently, and must refer to the already buffered video frames.
When the video communication system normally transmits video, namely, under the condition that no receiving end of the video communication system is accessed for the first time or video with serious packet loss can not be recovered:
as an optional embodiment, the video communication system is a live broadcast system or a video recording system, a sending end firstly encodes and sends a first frame IDR frame carrying complete picture information to a control unit, the control unit forwards the first frame IDR frame to a receiving end, and the receiving end decodes the first frame IDR frame to form a current video frame; then the sending end periodically encodes and sends a P frame carrying a part different from the previous frame to the control unit, the control unit forwards the P frame to the receiving end, and the receiving end forms a current video frame according to the previous video frame and the currently received P frame; the sending end codes and sends a first long reference frame based on the first frame IDR frame coding to the control unit, the control unit forwards the first long reference frame to the receiving end, and the receiving end decodes the first long reference frame and the first frame IDR frame to form a current video frame.
Namely, the control unit receives the long reference frames and the P frames (namely common reference frames) which are periodically re-encoded by the sending end according to a specified period, and sends the received re-encoded long reference frames and the received P frames to all receiving ends accessed to the video communication system, wherein the sending period of the long reference frames is longer than the period of sending the P frames carrying the difference information, and each long reference frame period comprises a plurality of P frames. All receiving ends accessing the video communication system buffer the last video frame and form the current video frame based on the common reference frame from the control unit. The transmitting side may re-encode the IDR frame periodically at a predetermined cycle and transmit the re-encoded IDR frame to the control section, and the control section may transmit the received re-encoded IDR frame to all receiving sides accessing the video communication system. The period of the long reference frame recoded by the sending terminal is less than the period of the IDR frame recoded and greater than the period of the P frame recoded. For example, the transmitting end re-encodes and transmits a 30-frame P frame every 1 second, re-encodes and transmits a one-frame long reference frame every 2 minutes, and re-encodes and transmits a one-frame IDR frame every half an hour.
As another alternative embodiment, the video communication system is a monitoring system or a video conference system, and compared with the previous embodiment, the sending end sends the P frame only by periodically coding after coding the first frame IDR frame. Under the normal condition, the shooting positions of cameras in a monitoring system and a video conference system are fixed, the background of the shot content is unchanged for a long time, and the changed part is small, so that a sending end only needs to regularly code and send a P frame carrying difference information under the condition that a receiving end of a video communication system is not accessed for the first time or a video with serious packet loss cannot be recovered.
The invention mainly solves the problem of how to effectively reduce the calculation load and the uplink network bandwidth pressure of a sending end under the condition that a receiving end is accessed into a conference for the first time or a video with serious packet loss cannot be recovered, and simultaneously provides better guarantee for the image quality of a terminal which is added into a session, reduces the downlink bandwidth impact, reduces the probability of congestion and packet loss and reduces the retransmission pressure.
The innovation points of the invention are mainly as follows: firstly, a control unit caches at least one frame of IDR frame which is coded recently for a long time; the second control unit replaces the IDR frame request from the receiving end with the long reference frame request and sends the long reference frame request to the sending end.
The method of implementation will be explained in detail below, and comprises the following steps:
s101: and caching at least one frame of IDR frame which is coded recently by the sending end in real time.
Specifically, the control unit buffers at least one frame of IDR frame which is most recently coded from the sending end in real time. Preferably, the most recently encoded IDR frame from the transmitting end is buffered. In order to avoid that the most recently encoded IDR frame cannot be used normally, two most recently encoded IDR frames can be buffered, but this scheme requires more capacity of the control unit.
The buffering step of the control unit provides necessary IDR frames for the subsequent receiving end to carry out IDR frame request, and avoids the sending end from recoding the IDR frames when receiving the request.
More specifically, if the video communication system is a live broadcast system or a video recording system, the sending end may re-encode the long reference frame periodically and send the long reference frame to the control unit; the sending end can also re-encode the IDR frame and the long reference frame periodically and send the IDR frame and the long reference frame to the control unit. Wherein the period for re-encoding the long reference frame by the transmitting end is less than the period for re-encoding the IDR frame.
If the video communication system is a monitoring system or a video conference system, when the video picture of the sending end is changed greatly, the sending end recodes the IDR frame and sends the coded IDR frame to the control unit.
Under the normal condition, the shooting positions of cameras in a monitoring system and a video conference system are fixed, the background of the shot content is unchanged for a long time, and the changed part is small, so that when the video communication system is the monitoring system or the video conference system, only under the condition that the video picture generated by a sending end is changed greatly, the sending end carries out recoding on an IDR frame carrying complete picture information, and at other times, only a P frame carrying difference value information needs to be sent by regular coding. Therefore, the computing resource consumption and the bandwidth occupation of the sending end can be greatly saved.
S102: a request from at least one receiving end to send an IDR frame is received.
The receiving end sending the request is as follows: accessing the terminal of the video communication system for the first time or accessing the terminal of the video communication system again after disconnection; or a terminal which has accessed the video communication system and cannot recover video due to serious packet loss.
When the video communication system is abnormally transmitting video, namely, under the condition that a receiving end which is accessed into the video communication system for the first time exists or seriously lost video cannot be recovered, if the receiving end only depends on a P frame carrying difference information, the current video frame cannot be formed, and therefore the receiving end sends an IDR frame request to the control unit.
S103: and sending a long reference frame request to the sending end based on the request.
As shown in fig. 2, when the control unit receives an IDR Frame request from the receiving end, the control unit does not send the IDR Frame request to the transmitting end, but sends a Long Reference Frame LTP request (Long-Term Reference Frame) to the transmitting end.
S104: and receiving a long reference frame coded by the sending end, wherein the long reference frame is a long reference frame coded by the sending end by referring to the most recently coded IDR frame.
When the transmitting end receives the long reference frame request, the transmitting end encodes the long reference frame LTP with reference to the most recently encoded IDR frame, as shown in the lower side of fig. 2. Since the long reference frame is encoded by the transmitting end with reference to the previous IDR frame, its difference from the IDR frame is usually larger than the P frame encoded with reference to the previous frame. The height of the frame shown in the figure is the amount of code for that frame, and the long reference frame LTP height shown in yellow is greater than the P frame shown in blue, and less than the IDR frame shown in red.
The most recently encoded IDR frame coincides with the IDR frame buffered by the control unit in step S101. The long reference frame is an independent frame which is coded by the sending end by referring to the latest coded IDR frame, is different from the IDR frame and is not provided with all information, and the data of the frame is compressed only according to the different points of the frame and the latest coded IDR frame, so that compared with the coded IDR frame, the sending end coding long reference frame consumes less computing resources of the sending end, the requirement on the sending bandwidth of the sending end is reduced, and further under the condition that the uplink bandwidth of the sending end is limited or the system has low delay requirement, the sending end can also provide high-quality pictures, so that the condition that the picture quality of other receiving ends is damaged due to the fact that the long reference frame request is received can not be generated.
S105: and sending the IDR frame which is most recently coded and the long reference frame to a receiving end which sends a request, and sending the long reference frame to other receiving ends which do not send the request.
Wherein the step of sending the most recently encoded IDR frame and the long reference frame to a requesting receiver specifically comprises: and the control unit marks and pairs the IDR frame which is coded most recently and the long reference frame and then sends the IDR frame and the long reference frame to the receiving end which sends the request.
In a video communication system, especially in a video conference system, a plurality of participant terminals generally interact with each other as a transmitting terminal and a receiving terminal, so that a control unit generally needs to be responsible for connecting the plurality of transmitting terminals and the plurality of receiving terminals, different IDR frames from different transmitting terminals are cached in the control unit, and the long reference frame must be marked and paired with the corresponding IDR frame to avoid the misdelivery and the mismatching of the long reference frame and the corresponding IDR frame. The corresponding long reference frame and the IDR frame are generated for the same sender encoding.
And when the request sending receiving end receives the long reference frame and the IDR frame after the mark pairing, forming a latest video frame based on the long reference frame and the IDR frame, and presenting the latest video frame on a display of the request sending receiving end. It should be noted that the receiving end must be the latest video frame formed according to both the long reference frame and the IDR frame after the mark pairing, and the display of the receiving end does not display the video frame corresponding to the IDR frame request.
On the other hand, other receiving ends which do not send requests only receive the long reference frame sent by the control unit, and no IDR frame coded recently.
As an alternative embodiment, as shown in fig. 3A, in a video communication session, a sending end S, a receiving end a, and a receiving end B buffer a latest IDR frame of the sending end S in a control unit MCU, and at this time, a receiving end C needs to join in a conference, which is shown in fig. 3B, and includes the following specific steps: the receiving end C sends an IDR request to the control unit MCU, and the control unit MCU converts the IDR request into a long reference frame request and sends the long reference frame request to the sending end S under the condition that the control unit MCU confirms that the latest IDR frame of the sending end S is cached; the sending end S codes a frame of long reference P frame to the control unit MCU; the control unit MCU sends the long reference P frame to the receiving end A and the receiving end B which are in the conversation, and sends the IDR frame and the long reference P frame which are cached locally to the receiving end C.
In summary, the present invention respectively sends the most recently encoded IDR frame, the long reference frame, and the long reference frame to the other receivers that do not need IDR frame and send request, instead of the prior art, which does not distinguish whether the receiver is the receiver that sends request, the control unit uniformly sends the IDR frame encoded by the sending end after the IDR frame request to all receivers; furthermore, the control unit caches the latest coded IDR frame of the sending end in real time to prepare for receiving the IDR frame request from the receiving end, and only needs to send a long reference frame request to the sending end without sending the IDR frame request because the control unit caches the latest coded IDR frame; therefore, when the sending end receives the request, the IDR frame does not need to be coded, only the long reference frame needs to be coded, the calculation load and the uplink network bandwidth pressure of the sending end are effectively reduced, meanwhile, better guarantee is provided for the image quality of other terminals which join in the session, the downlink bandwidth impact is reduced, the probability of congestion and packet loss is reduced, and the retransmission pressure is reduced.
As another alternative, the control unit may pair the received long reference frame and the corresponding IDR frame and then synthesize a new current IDR frame to send to all the receiving ends. The method has the advantage that the method can be applied without upgrading the existing receiving end, namely, the receiving end receives the requested new IDR frame instead of the paired old IDR frame and the corresponding long reference frame after requesting the IDR frame and performs local synthesis.
As another optional embodiment, when the sending end is disconnected and then accesses the conference again, the data of the sending end in the video communication system is emptied, and an IDR frame needs to be re-encoded, and the control unit receives one frame of the IDR frame re-encoded by the sending end and sends the IDR frame to all receiving ends accessing the video communication system.
As another alternative, the video communication system may be a live system or a video recording system that stores an earliest IDR frame and periodically generates a long reference frame based on the earliest IDR frame. When the receiving end plays back the recorded video, if the playback position is the position of a certain long reference frame, the video frame at the position can be generated and played back based on the long reference frame at the position and the earliest IDR frame; and if the playback position is not at the position of the long reference frame, starting to play back the video from a long reference frame near the playback position, wherein the near long reference frame can be a long reference frame before the playback position or a long reference frame after the playback position. Since the long reference frame occupies data shorter than the IDR frame, the required storage space can be significantly compressed and jitter of the picture due to transmission of a larger IDR frame during live broadcasting can be avoided. The transmitting side may re-encode the IDR frame periodically at a predetermined cycle and transmit the re-encoded IDR frame to the control section, and the control section may transmit the received re-encoded IDR frame to all receiving sides accessing the video communication system. Wherein the period for re-encoding the long reference frame by the transmitting end is less than the period for re-encoding the IDR frame.
As another optional embodiment, the video communication system is a live broadcast system or a video recording system, and when the receiving end plays back the recorded video, if the playback position is not at the position of the IDR frame, the receiving end waits for the current P frame and the previous frame of video frame to obtain the current position of the video frame.
Exemplary System
Correspondingly, the embodiment of the invention also provides a video communication system. Fig. 4 is a schematic diagram of a video communication system 100 according to an embodiment of the present invention, and as shown in fig. 4, the system 100 according to the embodiment includes:
a control unit 101, and at least one sending end 102 and at least one receiving end 103 connected to the control unit, wherein the control unit includes a buffer module 111, a receiving module 121, and a sending module 131, wherein
The buffering module 111 is configured to buffer at least one frame of IDR frame that is most recently encoded by the sending end 102 in real time.
The receiving module 121 is configured to receive a request for sending an IDR frame from at least one receiving end 103.
The sending module 131 is configured to send a long reference frame request to the sender 102 based on the request.
The receiving module 121 is further configured to receive a long reference frame encoded by the transmitting end 102, where the long reference frame is a long reference frame encoded by referring to a most recently encoded IDR frame by the transmitting end 102.
The sending module 131 is further configured to send the most recently encoded IDR frame and the long reference frame to a receiving end 103 that sends a request, and send the long reference frame to other receiving ends 103 that do not send a request.
The receiving module 121 is further configured to receive an IDR frame that is re-encoded by the transmitting end 102 periodically according to a specified period.
The sending module 131 is further configured to send the received re-encoded IDR frame to all receiving ends 103 accessing the video communication system 100.
The receiving end 103 sending the request is:
a terminal which accesses the video communication system 100 for the first time or a terminal which accesses the video communication system 100 again after disconnection; or
A terminal having access to the video communication system 100, which cannot recover video due to a severe packet loss.
When the sending end 102 disconnects and then accesses the conference again, the receiving module 121 is further configured to receive a frame of IDR frame recoded by the sending end 102.
The sending module 131 is further configured to send the IDR frame to all receiving ends 103 accessing the video communication system 100.
Wherein the long reference frame is a P frame or a B frame.
The requesting receiver 103 is further configured to form a latest video frame based on the most recently encoded IDR frame and the long reference frame and present the latest video frame on a display of the requesting receiver 103.
The sending module 131 includes a pairing sub-module 1311 configured to mark and pair the most recently encoded IDR frame and the long reference frame, and the sending module 131 is configured to send the two to the requesting receiving end 103.
The receiving module 121 is further configured to periodically receive a normal reference frame from the sender 102.
The sending module 131 is further configured to send the received common reference frame to all receiving ends 103 accessing the video communication system 100.
All receiving ends 103 accessing the video communication system 100 are further configured to buffer a previous video frame and form a current video frame based on a common reference frame from the control unit.
The video communication system 100 includes any one of a monitoring system, a live broadcast system, a video conference system, and a video recording system.
If the video communication system 100 is a monitoring system or a video conference system, when a video picture of the sending end 102 changes greatly, the receiving module 121 is further configured to receive an IDR frame recoded by the sending end 102.
If the video communication system 100 is a live broadcast system or a video recording system, the receiving module 121 is further configured to receive an earliest IDR frame encoded by the transmitting end 102 and a long reference frame that is periodically re-encoded based on the earliest IDR frame.
The receiving end 103 comprises a playback module 113, which is configured to play back the video from a long reference frame near a playback position when the receiving end 103 plays back the recorded video if the video communication system 100 is a live system or a video recording system.
It should be noted that although the operations of the long reference frame based video transmission method of the present invention in a video communication system are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order or that all of the illustrated operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Furthermore, although several devices, units, or modules of a video communication system are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the modules described above may be embodied in one module according to embodiments of the invention. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects cannot be combined to advantage. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The present invention provides:
1. in a video communication system including a control unit and at least one transmitting end and at least one receiving end connected to the control unit, a method for long reference frame based video transmission in a video communication system, the method comprising:
caching at least one frame of IDR frame which is most recently coded by the sending end in real time;
receiving a request for sending an IDR frame from at least one receiving end;
sending a long reference frame request to the sending end based on the request;
receiving a long reference frame encoded from the transmitting end, wherein the long reference frame is a long reference frame encoded by the transmitting end by referring to a most recently encoded IDR frame;
and sending the latest coded IDR frame and the long reference frame to a receiving end sending a request, and sending the long reference frame to other receiving ends not sending the request.
2. The video transmission method according to claim 1, wherein the method further comprises:
receiving an IDR frame which is re-encoded by the sending end periodically according to a specified period;
and sending the received re-encoded IDR frame to all receiving ends accessing the video communication system.
3. The video transmission method according to claim 1 or 2, wherein the receiving end that issues the request is:
accessing the terminal of the video communication system for the first time or accessing the terminal of the video communication system again after disconnection; or alternatively
And a terminal which is accessed into the video communication system and can not recover the video due to serious packet loss.
4. The video transmission method according to any one of items 1 to 3, characterized in that the method further comprises:
when the sending end is disconnected and then accesses the conference again, receiving a frame of IDR frame recoded by the sending end;
and sending the IDR frame to all receiving ends accessed to the video communication system.
5. The video transmission method according to any of claims 1 to 4, wherein the long reference frame is a P frame or a B frame.
6. The video transmission method according to any one of items 1 to 5, wherein after the step of transmitting the most recently encoded IDR frame and the long reference frame to a requesting receiving end, the method further comprises:
the requesting receiver forming a latest video frame based on the most recently encoded IDR frame and the long reference frame;
presenting the latest video frame on a display of the requesting recipient.
7. The video transmission method according to any of claims 1 to 6, wherein the step of sending the most recently encoded IDR frame and the long reference frame to a requesting receiver specifically comprises:
and marking and pairing the IDR frame which is coded most recently and the long reference frame, and then sending the IDR frame and the long reference frame to the receiving end which sends the request.
8. The video transmission method according to any one of items 1 to 7, characterized in that the method further comprises:
regularly receiving a common reference frame from the transmitting end;
and sending the received common reference frame to all receiving ends accessing the video communication system.
9. The video transmission method according to claim 8, wherein the method further comprises:
and all receiving ends accessed into the video communication system buffer the last video frame and form the current video frame based on the common reference frame from the control unit.
10. The video transmission method according to any one of items 1 to 9, wherein the video communication system includes any one of a monitoring system, a live system, a video conference system, and a video recording system.
11. The video transmission method according to claim 10, wherein the method further comprises:
and if the video communication system is a monitoring system or a video conference system, receiving the IDR frame recoded by the sending end when the video picture of the sending end is greatly changed.
12. The video transmission method according to claim 10, wherein the method further comprises:
and if the video communication system is a live broadcast system or a video recording system, receiving the earliest IDR frame coded by the sending end and a long reference frame regularly coded based on the earliest IDR frame.
13. The video transmission method according to claim 12, wherein the method further comprises:
and if the video communication system is a live broadcast system or a video recording system, when the receiving end plays back the recorded video, starting to play back the video from a long reference frame near the playback position.
14. A video communication system, the system comprising: the device comprises a control unit, at least one sending end and at least one receiving end which are connected with the control unit, wherein the control unit comprises a cache module, a receiving module and a sending module
The cache module is used for caching at least one frame of IDR frame recently coded by the sending end in real time;
the receiving module is used for receiving a request for requesting to send an IDR frame from at least one receiving end;
the sending module is used for sending a long reference frame request to the sending end based on the request;
the receiving module is further configured to receive a long reference frame encoded by the transmitting end, where the long reference frame is a long reference frame encoded by the transmitting end with reference to a most recently encoded IDR frame;
the sending module is further configured to send the most recently encoded IDR frame and the long reference frame to a receiving end that sends a request, and send the long reference frame to other receiving ends that do not send a request.
15. The video communication system of claim 14, wherein the receiving module is further configured to receive an IDR frame that is re-encoded by the transmitting end periodically according to a predetermined period;
the sending module is further configured to send the received re-encoded IDR frame to all receiving ends accessing the video communication system.
16. The video communication system according to claim 14 or 15, wherein the requesting receiver is:
accessing the terminal of the video communication system for the first time or accessing the terminal of the video communication system again after disconnection; or alternatively
And a terminal which is accessed into the video communication system and cannot recover the video due to serious packet loss.
17. The video communication system according to any one of claims 14 to 16,
when the sending end is disconnected and then accesses the conference again, the receiving module is also used for receiving a frame of IDR frame recoded by the sending end;
the sending module is further configured to send the IDR frame to all receiving ends accessing the video communication system.
18. The video communication system according to any of claims 14-17, wherein the long reference frame is a P-frame or a B-frame.
19. The video communication system according to any of claims 14-18, wherein said requesting receiver is further configured to form a latest video frame based on said most recently encoded IDR frame and said long reference frame, and present said latest video frame on a display of said requesting receiver.
20. The video communication system according to any of claims 14-19, wherein the sending module comprises a pairing submodule for marking and pairing the most recently encoded IDR frame and the long reference frame, the sending module being configured to send both to the requesting receiving end.
21. The video communication system according to any one of claims 14 to 20,
the receiving module is further configured to periodically receive a common reference frame from the transmitting end;
the sending module is further configured to send the received common reference frame to all receiving ends accessing the video communication system.
22. The video communication system of claim 21, wherein all receiving ends that are accessed to the video communication system are further configured to buffer a previous video frame and form a current video frame based on a common reference frame from the control unit.
23. The video communication system according to any of claims 14-22, wherein the video communication system comprises any of a surveillance system, a live system, a video conference system, a video recording system.
24. The video communication system of claim 23, wherein if the video communication system is a monitoring system or a video conference system, when a video picture at a sending end changes significantly, the receiving module is further configured to receive an IDR frame re-encoded by the sending end.
25. The video communication system of claim 23, wherein if the video communication system is a live broadcast system or a video recording system, the receiving module is further configured to receive an earliest IDR frame encoded by the transmitting end and a long reference frame periodically encoded based on the earliest IDR frame.
26. The video communication system according to claim 25, wherein the receiving end includes a playback module, and if the video communication system is a live system or a video recording system, when the receiving end plays back the recorded video, the playback module is configured to play back the video from a long reference frame near a playback position.
Claims (10)
1. In a video communication system including a control unit and at least one transmitting end and at least one receiving end connected to the control unit, a method for long reference frame based video transmission in a video communication system, the method comprising:
caching at least one frame of IDR frame which is most recently coded by the sending end in real time;
receiving a request for sending an IDR frame from at least one receiving end;
sending a long reference frame request to the sending end based on the request;
receiving a long reference frame encoded by the transmitting end, wherein the long reference frame is a long reference frame encoded by the transmitting end by referring to a most recently encoded IDR frame;
and sending the IDR frame which is most recently coded and the long reference frame to a receiving end which sends a request, and sending the long reference frame to other receiving ends which do not send the request.
2. The video transmission method according to claim 1, wherein the method further comprises:
receiving the IDR frame which is re-encoded by the sending end periodically according to a specified period;
and sending the received re-encoded IDR frame to all receiving ends accessing the video communication system.
3. The video transmission method according to claim 1 or 2, wherein the requesting receiver is:
accessing the terminal of the video communication system for the first time or accessing the terminal of the video communication system again after the disconnection; or
And a terminal which is accessed into the video communication system and cannot recover the video due to serious packet loss.
4. The video transmission method according to any of claims 1-3, characterized in that the method further comprises:
when the sending end is disconnected and then accesses the conference again, receiving a frame of IDR frame recoded by the sending end;
and sending the IDR frame to all receiving ends accessed to the video communication system.
5. The video transmission method according to any of claims 1-4, wherein the long reference frame is a P-frame or a B-frame.
6. The video transmission method according to any of claims 1-5, wherein after the step of sending the most recently encoded IDR frame and the long reference frame to a requesting receiving end, the method further comprises:
the requesting receiver forming a latest video frame based on the most recently encoded IDR frame and the long reference frame;
presenting the latest video frame on a display of the requesting recipient.
7. The video transmission method according to any of claims 1-6, wherein the step of sending the most recently encoded IDR frame and the long reference frame to a requesting receiver specifically comprises:
and marking and pairing the IDR frame which is coded most recently and the long reference frame, and then sending the IDR frame and the long reference frame to the receiving end which sends the request.
8. The video transmission method according to any of claims 1-7, wherein the method further comprises:
regularly receiving a common reference frame from the transmitting end;
and sending the received common reference frame to all receiving ends accessing the video communication system.
9. The video transmission method according to claim 8, wherein the method further comprises:
all receiving ends accessing the video communication system buffer the last video frame and form the current video frame based on the common reference frame from the control unit.
10. A video communication system, the system comprising: the device comprises a control unit, and at least one sending end and at least one receiving end which are connected with the control unit, wherein the control unit comprises a cache module, a receiving module and a sending module, and the control unit comprises a cache module, a receiving module and a sending module
The buffer module is used for buffering at least one frame of IDR frame which is recently coded by the sending end in real time;
the receiving module is used for receiving a request for requesting to send an IDR frame from at least one receiving end;
the sending module is used for sending a long reference frame request to the sending end based on the request;
the receiving module is further configured to receive a long reference frame encoded by the transmitting end, where the long reference frame is a long reference frame encoded by the transmitting end with reference to a most recently encoded IDR frame;
the sending module is further configured to send the most recently encoded IDR frame and the long reference frame to a receiving end that sends a request, and send the long reference frame to other receiving ends that do not send a request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211211627.2A CN115643425A (en) | 2022-09-30 | 2022-09-30 | Video communication system and video transmission method based on long reference frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211211627.2A CN115643425A (en) | 2022-09-30 | 2022-09-30 | Video communication system and video transmission method based on long reference frame |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115643425A true CN115643425A (en) | 2023-01-24 |
Family
ID=84942361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211211627.2A Pending CN115643425A (en) | 2022-09-30 | 2022-09-30 | Video communication system and video transmission method based on long reference frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115643425A (en) |
-
2022
- 2022-09-30 CN CN202211211627.2A patent/CN115643425A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4808161B2 (en) | Method and apparatus for moving image communication error processing | |
| JP4603695B2 (en) | Video coding | |
| US8144764B2 (en) | Video coding | |
| US8971415B2 (en) | Video communication system, device and method based on feedback reference frames | |
| US6357028B1 (en) | Error correction and concealment during data transmission | |
| JP4753204B2 (en) | Encoding processing apparatus and encoding processing method | |
| US8606025B2 (en) | Encoding and decoding images using refreshed image and recovery point | |
| JP2004507178A (en) | Video signal encoding method | |
| CN101370139A (en) | Method and device for switching channels | |
| WO2011038694A1 (en) | Method, device and network system for transmission processing and sending processing of video data | |
| WO2006111087A1 (en) | H.264-based error eliminating method for compressed video transmission | |
| JP2006526908A (en) | A method for encoding a picture, a method for decoding a picture from a bitstream, an encoder for encoding a picture, a transmission device including an encoder for encoding a picture, a decoder for decoding a picture from a bitstream, and a picture from a bitstream Receiving apparatus and system including a decoder for decoding | |
| CN110392284B (en) | Video encoding method, video data processing method, video encoding apparatus, video data processing apparatus, computer device, and storage medium | |
| CN101742289B (en) | Method, system and device for compressing video code stream | |
| JP4983917B2 (en) | Moving image distribution system, conversion device, and moving image distribution method | |
| CN101207813A (en) | Method and system for encoding and decoding video sequence | |
| WO2008095345A1 (en) | A method for encoding a video sequence and a error remedying system using same | |
| JPH0984019A (en) | Image data storage device | |
| US20060015799A1 (en) | Proxy-based error tracking for real-time video transmission in mobile environments | |
| CN115643425A (en) | Video communication system and video transmission method based on long reference frame | |
| Farber et al. | Robust H. 263 compatible transmission for mobile video server access | |
| JP2001086153A (en) | Data communication equipment, data communication system, data communication method and storage medium | |
| JP2003259375A (en) | Moving picture transmission/reception system, moving picture transmission/reception method, and program for them | |
| JPH1070727A (en) | Method and device for transmitting moving picture | |
| JP3660513B2 (en) | Image communication apparatus and local decoding processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |