CN110366033B - Video playing method, device, equipment and storage medium - Google Patents
Video playing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN110366033B CN110366033B CN201910645432.0A CN201910645432A CN110366033B CN 110366033 B CN110366033 B CN 110366033B CN 201910645432 A CN201910645432 A CN 201910645432A CN 110366033 B CN110366033 B CN 110366033B
- Authority
- CN
- China
- Prior art keywords
- image frame
- target image
- frame
- buffer queue
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000004044 response Effects 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims 2
- 230000037431 insertion Effects 0.000 claims 2
- 230000008569 process Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000003139 buffering effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000020169 heat generation Effects 0.000 description 1
- 238000012966 insertion method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application relates to a video playing method, a video playing device, video playing equipment and a storage medium, wherein the method comprises the following steps: responding to a video playing instruction, acquiring a resource to be played, and storing the resource to be played into a first cache queue; determining the currently read image frame from the first buffer queue as a target image frame; storing the target image frame into a second buffer queue, and decoding the target image frame; when the target image frame fails to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue; acquiring corresponding target audio data; repeatedly executing image frame caching and image frame decoding operations until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data. The invention avoids the phenomenon of picture and sound asynchronism caused by decoding failure in the video playing process, and accelerates the display speed of the picture.
Description
Technical Field
The present application relates to the field of video processing technologies, and in particular, to a video playing method, apparatus, device, and storage medium.
Background
The video decoding is a process of performing a reduction decoding operation on a coded digital video, and when video data is played, the video data to be played needs to be decoded by a video decoder, and when the decoding is successful, the video data to be played can be played. Video decoding can be divided into hard decoding and soft decoding, wherein soft decoding refers to an operation process of decoding by using a CPU, and hard decoding refers to an operation process of decoding by using a non-CPU.
When the hard decoding is used for video decoding, the situation of hard decoding failure can occur due to various reasons, so that the phenomenon that a video picture is blocked and still plays sound continuously is caused, and in this situation, if the decoding failure error is not processed, the subsequent decoding failure can be caused if the decoding is continuously performed.
Disclosure of Invention
The technical problem to be solved in the present application is to provide a video playing method, apparatus, device and storage medium, which can implement continuous playing of a video with decoding failure by placing an image frame being decoded into a buffer queue, restarting a decoder and taking out the image frame from the buffer queue for re-decoding when the decoder fails to decode the current image frame.
In order to solve the above technical problem, in one aspect, the present application provides a video playing method, where the method includes:
responding to a video playing instruction, acquiring a resource to be played, and storing the resource to be played into a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, and each image group comprises a plurality of image frames;
determining the currently read image frame from the first buffer queue as a target image frame;
storing the target image frame into a second buffer queue, and decoding the target image frame;
when the target image frame fails to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue; determining an image group in which the target image frame is positioned as a target image group, and acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
In another aspect, the present application provides a video playback device, including:
the playing instruction response module is used for responding to a video playing instruction, acquiring a resource to be played and storing the resource to be played into a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, and each image group comprises a plurality of image frames;
the target image frame determining module is used for determining the currently read image frame from the first cache queue as a target image frame;
the buffer decoding module is used for storing the target image frame into a second buffer queue and decoding the target image frame;
the loop execution module is used for inserting each image frame in the second buffer queue into the head of the first buffer queue when the target image frame fails to be decoded; determining an image group in which the target image frame is positioned as a target image group, and acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
In another aspect, the present application further provides an apparatus comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the video playing method as described above.
In another aspect, the present application further provides a computer storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, the at least one instruction, at least one program, set of codes, or set of instructions being loaded by a processor and executing the video playing method as described above.
The method comprises the steps of determining a current image frame read from a first cache queue as a target image frame, storing the target image frame into a second cache queue and decoding the target image frame; when the target image frame is failed to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue, and acquiring target audio data corresponding to the image group where the target image frame is located; taking the image frame at the head of the first buffer queue as a target image frame, and repeatedly executing the buffering and decoding steps until the target image frame is successfully decoded; and displaying the image after the decoding is successful and playing the target audio data. The invention avoids the phenomenon of picture and sound asynchronism caused by decoding failure in the video playing process, and accelerates the display speed of the picture.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;
fig. 2 is a flowchart of a video playing method provided in an embodiment of the present application;
fig. 3 is a flowchart of an image frame reading method according to an embodiment of the present application;
fig. 4 is a flowchart of a target image frame caching method according to an embodiment of the present disclosure;
fig. 5 is a flowchart of an audio data acquisition method provided in an embodiment of the present application;
FIG. 6 is a first exemplary schematic diagram provided by an embodiment of the present application;
FIG. 7 is a second exemplary schematic diagram provided by an embodiment of the present application;
FIG. 8 is a third exemplary schematic diagram provided by an embodiment of the present application;
fig. 9 is a schematic view of a video playing apparatus according to an embodiment of the present application;
FIG. 10 is a schematic diagram of a target image frame determination module provided by an embodiment of the present application;
fig. 11 is a schematic diagram of a cache decoding module according to an embodiment of the present application;
FIG. 12 is a schematic diagram of a loop execution module provided by an embodiment of the present application;
FIG. 13 is a schematic diagram of an audio processing module provided by an embodiment of the present application;
fig. 14 is a schematic structural diagram of an apparatus provided in an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the present application will be further described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," and the like, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein.
Please refer to fig. 1, which shows a schematic diagram of an application scenario provided in an embodiment of the present application, where the scenario includes: at least one video playing terminal 110 and a video playing server 120, wherein the video playing terminal 110 and the video playing server 120 can perform data communication through a network. Specifically, the video playing terminal 110 may receive the resource to be played sent by the video playing server 120, and the video playing terminal 110 may decode and play the resource to be played; or the video playing server 120 decodes the resource to be played and sends the decoded video data to the video playing terminal 110 for playing.
The video playback terminal 110 may communicate with the video playback Server 120 based on a Browser/Server mode (Browser/Server, B/S) or a Client/Server mode (Client/Server, C/S). The video playback terminal 110 may include: the physical devices may also include software running in the physical devices, such as applications, and the like. For example, the terminal may run a video player software, a social software with a video playing function, and the like. The operating system running on the video playing terminal 110 in the embodiment of the present application may include, but is not limited to, an android system, an IOS system, linux, windows, and the like.
The video playing server 120 and the video playing terminal 110 may establish a communication connection through a wired or wireless connection, and the video playing server 120 may include an independently operating server, or a distributed server, or a server cluster composed of a plurality of servers, where the server may be a cloud server.
In the video playing process, decoding failure of video image frames may be caused for some reasons, so that the phenomenon that sound and picture are not synchronous when the video playing picture is stuck but the sound continues can occur. To this end, an embodiment of the present application provides a video playing method, which may be applied to a video playing terminal side and may also be applied to a video playing server side, please refer to fig. 2, where the method specifically includes:
s210, responding to a video playing instruction, acquiring a resource to be played, and storing the resource to be played into a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, and each image group comprises a plurality of image frames.
The video playing instruction may be generated by a user by clicking and selecting a corresponding playing video, the video playing instruction may include a name or an identifier of a video to be played, and a resource to be played may be found according to the name or the identifier.
It should be noted that the resource to be played in this embodiment may be a local resource or a network resource. When the resource to be played is a local resource, the resource to be played can be understood as offline playing of the video, that is, the resource to be played is downloaded from the playing server in advance and stored locally, or is copied from another way and stored locally, and when the resource to be played is played, only the playing resource needs to be obtained locally, and does not need to be obtained from the playing server through a network; when the resource to be played is a network resource, online playing can be understood, that is, the resource to be played is acquired from the playing server in real time according to the playing progress requirement for playing.
When the resource to be played is a local resource, the subsequent operation on the resource to be played in this embodiment may not involve the video playing server, that is, the subsequent operation may be implemented by the video playing terminal; when the resource to be played is an online resource, the subsequent operation on the resource to be played can be realized by the video playing terminal or the video playing server.
The resources to be played comprise a plurality of groups of Pictures (GOPs) and audio data corresponding to the GOPs, wherein each Group of GOPs comprises a plurality of image frames, specifically, a Group of GOPs is a Group of continuous Pictures, and is composed of an I frame and a B/P frame, the I frame is an intra-coded frame (also called a key frame), the P frame is a forward predicted frame (forward reference frame), and the B frame is a bidirectional interpolated frame (bidirectional reference frame). In brief, an I-frame is a complete picture, while P-frames and B-frames record changes relative to the I-frame, which cannot be decoded without the I-frame. The I frame can be independently coded and decoded, the P frame needs to forward reference the I frame or the P frame image coded and decoded before to complete coding and decoding, and the B frame needs to forward and backward reference two P frame images coded and decoded before to complete coding and decoding. According to the encoding characteristics of h.264, the decoder decoding must start from a key frame.
S220, determining the currently read image frame as a target image frame from the first buffer queue.
The first buffer queue stores a plurality of image frames of resources to be played, and when a video is to be played, the image frames in the first buffer queue need to be sequentially read and decoded, and the video can be played only after the video is successfully decoded, specifically, referring to fig. 3, which shows an image frame reading method, including:
s310, reading one image frame from the head of the first buffer queue each time, and determining the read image frame as the target image frame.
In the actual playing process, the image frames are read out frame by frame and sent to the decoder for decoding, so that when each image frame is read, the image frame read at the current moment is determined as the target image frame.
S320, deleting the image frames which are read from the first buffer queue.
The image frames that have been read out need to be deleted from the first buffer queue to avoid repeated reading.
And S230, storing the target image frame into a second buffer queue, and decoding the target image frame.
As can be seen from the above explanation of the GOP of pictures in this embodiment, each GOP includes a key frame which is located at the position of the first frame of the GOP, and when decoding each GOP, decoding must be started from the key frame of the GOP.
In order to ensure that the second buffer queue only contains one key frame, when the target image frame is stored in the second buffer queue, it is necessary to determine the stored target image frame, specifically, referring to fig. 4, it shows a target image frame buffering method, where the method includes:
and S410, judging whether the target image frame is the key frame.
And S420, when the target image frame is the key frame, emptying the second buffer queue, and storing the target image frame into the second buffer queue after emptying.
And S430, when the target image frame is not the key frame, storing the target image frame to the tail part of the second buffer queue.
It should be noted that, when the target image frame is not a key frame, the second buffer queue inevitably includes key frames in the same GOP as the target image frame, and the target image frames are sequentially stored in the tail of the second buffer queue.
And repeating the judging process for each image frame so as to buffer the image frame into the second buffer queue.
And storing the target image frame into the second buffer queue and simultaneously decoding the target image frame.
S240, when the target image frame is failed to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue; determining an image group in which the target image frame is positioned as a target image group, and acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
And when the target image frame is successfully decoded, displaying the successfully decoded image and playing the audio data corresponding to the target image frame according to a normal playing flow.
In this embodiment, each image frame in the second buffer queue is reinserted into the first buffer queue for re-reading and decoding.
According to the analysis of the image frames in the second buffer queue in this embodiment, the second buffer queue at least includes the key frame in the target group of images, and when each GOP is decoded, the key frame of the GOP is decoded, so when the image frames in the second buffer queue are inserted into the first buffer queue, it is necessary to ensure that the key frame of the GOP where the target image frame is located is at the head of the first buffer queue, and thus when re-decoding is performed, the first read image frame is the key frame; therefore, the specific image frame insertion method includes:
and sequentially inserting the image frames into the head of the first buffer queue from the tail of the second buffer queue, so that the key frames in the target image group are positioned at the head of the first buffer queue when the inserting operation is finished.
Since the image data and the audio data correspond to each other, when the image frame that fails to be decoded is re-decoded so as to replay the image frame, a callback is also required for the corresponding audio data, specifically, referring to fig. 5, it shows an audio data acquisition method, including:
and S510, when the target image frame fails to be decoded, recording the audio position corresponding to the key frame of the target image group.
When a certain frame of image fails to be decoded, the image is decoded and displayed from the key frame of the GOP where the frame of image is located during re-decoding, so that the corresponding audio data is played again from the first frame of the GOP, namely the key frame, and the audio position corresponding to the key frame is recorded.
S520, acquiring the target audio data from the audio position.
After the audio position corresponding to the key frame is determined, the target audio data can be obtained from the audio position for playing.
In this embodiment, as long as decoding of a certain frame of image fails, the image at the beginning of the image group where the frame of image is located and the audio position at the beginning are played during re-decoding and playing, which facilitates synchronization between the picture and the sound and reduces uncertainty of asynchronization of the sound and the picture which may be caused by starting playing from an intermediate frame.
In this embodiment, decoding failure of the image frame may be caused by various reasons, and after the decoding failure, the decoder needs to be restarted to trigger the initialization of the decoding configuration, so that the decoder can perform re-decoding.
In order to clearly illustrate the embodiments of the present application, a specific example is described below.
In the video playing process, decoding failure of the image frame may be caused by various reasons, in this embodiment, hard decoding of the iOS is taken as an example for illustration, the hard decoder of the iOS can reduce CPU consumption of the player, and reduce heat generation and power consumption, and the hard decoding in the iOS system uses a VideoToolBox framework, but when the VideoToolBox is specifically applied, there are many limitations, for example, when a player playing video is switched to a background, context failure therein may be caused. The problem with the failure of the context is that the subsequent decoding process fails and the context must be reinitialized. When the video player is switched from the background to the foreground operation again, the video picture will be stuck but the sound will continue due to the decoding failure. This problem can be solved by the following method:
specifically, referring to fig. 6, it is assumed that the resource to be played includes an image group 1, an image group 2, an image group 3, … …, and an image group N, where the image group 1 includes an image frame 1, an image frame 2, an image frame 3, … …, and an image frame 10, the image group 1 through the image group N are stored in the first buffer queue, and audio data corresponding to each image group is also stored in the first buffer queue. The second buffer queue is empty before decoding has not begun.
When the video starts to be played, reading one frame of image from the image group 1 in the first buffer queue each time, and respectively sending the image into a hard decoder and a second buffer queue; firstly, reading out an image frame 1 in an image group 1 from a first buffer queue, deleting the image frame 1 from the first buffer queue, storing the read image frame 1 into a second buffer queue, sending the image frame 1 into a decoder, rendering, displaying and playing corresponding audio data for the successfully decoded image after the image frame 1 is successfully decoded; similarly, for the image frame 2, the decoding succeeds after repeating the above steps, and after the image rendering, the image is displayed and the corresponding audio data is played. At this time, two image frames 1 and 2 which have been successfully decoded and successfully played are cached in the second cache queue, wherein the image frame 1 is the first frame of the image group 1, so the image frame 1 is the key frame of the image group 1; and, image frame 1 and image frame 2 have been deleted from the first buffer queue.
When the image frame 3 is read from the first buffer queue for decoding, the player is switched to the background to run due to background switching operation, and the player stops playing; and may cause the decoder to fail, i.e., fail to decode image frame 3, due to background operation. Referring to fig. 7, when the player is switched from the background to the foreground to run, the second buffer queue includes an image frame 1, an image frame 2, and an image frame 3, and the image frame in the second buffer queue is inserted into the head of the first buffer queue, which may be implemented as follows: before inserting the image frame 3 into the image frame 4, before inserting the image frame 2 into the image frame 3, and before inserting the image frame 1 into the image frame 2, it is ensured that the key frame image frame 1 of the image group 1 is positioned at the head of the first buffer queue, the hard decoding is restarted, the reading, the buffering and the decoding can be performed from the image frame 1 again, and each frame is played after being decoded; correspondingly, audio data corresponding to the image frame 1 is acquired, and when the decoding is successful, the image is displayed and the corresponding audio data is played, so that normal playing is resumed. The content played here is played from the first frame image content of the image group 1 where the image 3 is located, and from the viewpoint of image display, the image is backed to the image at the beginning of the image group 1; in sound playback, the sound is backed to the audio position at the beginning of the image group 1. Both the image and the sound are played from the beginning of the image group 1, which facilitates the synchronization of the sound and the picture. By recording the sound position corresponding to the key frame, the sound is recalled when the hard decoder is restarted, and the synchronization effect is achieved.
Since each image frame in the second buffer queue is read out, image frame 1, image frame 2, and image frame 3 may be cleared in order to save storage resources. Sequentially decoding each frame of image in the image group 1, wherein the decoding is successful, and at the moment, the second buffer queue comprises image frames 1 to 10; continuing to read each image frame in the image group 2, where the image group 2 includes image frame 11, image frame 12, image frame 13, … …, and image frame 20, and after reading the image frame 11 of the image group 2, it needs to be stored in the second buffer queue; because the key frame of the 11-bit image group 2 of the image frame contains the key frame of the image group 1, the second buffer queue needs to be emptied first at this time, and all the image frames in the second buffer queue belong to the same image group; the image frame 11 is buffered in the second buffer queue, as shown in fig. 8.
And repeating the steps to realize the playing of the resource to be played.
According to the embodiment of the application, the image frames and the corresponding audio data of the recently played resources are cached in the player, and when decoding fails, the cached data can be sent to the player to be played again, specifically: re-decoding the cached image frame and re-playing the image frame in combination with the audio data; the phenomenon of sound-picture asynchronism when the player is switched from the background to the foreground is avoided, the image is displayed immediately when the decoding is successful, and the picture display speed is increased.
The present embodiment further provides a video playing apparatus, please refer to fig. 9, the apparatus includes:
a playing instruction response module 910, configured to respond to a video playing instruction, obtain a resource to be played, and store the resource to be played in a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, and each image group comprises a plurality of image frames.
A target image frame determining module 920, configured to determine, from the first buffer queue, that a currently read image frame is a target image frame.
And a buffer decoding module 930, configured to store the target image frame into a second buffer queue, and decode the target image frame.
A loop execution module 940, configured to insert each image frame in the second buffer queue into the head of the first buffer queue when decoding of the target image frame fails; determining an image group in which the target image frame is positioned as a target image group, and acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
Referring to fig. 10, the target image frame determination module 920 includes:
a reading module 1010, configured to read one image frame from the head of the first buffer queue at a time, and determine that the read image frame is the target image frame.
A deleting module 1020 configured to delete the image frame that has been read from the first buffer queue.
Referring to fig. 11, the buffer decoding module 930 includes:
the first determining module 1110 is configured to determine whether the target image frame is the key frame.
A queue emptying module 1120, configured to empty the second buffer queue when the target image frame is the key frame, and store the target image frame into the second buffer queue after being emptied.
A second buffer module 1130, configured to store the target image frame to the tail of the second buffer queue when the target image frame is not the key frame.
Wherein each group of images comprises a key frame, said key frame being located at the position of the first frame of said group of images.
Referring to fig. 12, the loop execution module 940 includes:
an image frame inserting module 1210, configured to insert the image frames into the head of the first buffer queue in sequence from the tail of the second buffer queue, so that when the inserting operation is completed, the key frame in the target image group is at the head of the first buffer queue.
And the second buffer queue at least comprises key frames in the target image group.
Referring to fig. 13, the apparatus further includes an audio processing module 1300, and the audio processing module 1300 includes:
the audio position recording module 1310 is configured to record an audio position corresponding to a key frame of the target image group when decoding of the target image frame fails.
An audio data obtaining module 1320, configured to obtain the target audio data from the audio position.
The video playback apparatus further includes:
and the initialization module is used for triggering the initialization of the decoding configuration after the target image frame is failed to be decoded.
The device provided in the above embodiments can execute the method provided in any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments may be referred to a method provided in any of the embodiments of the present application.
The method comprises the steps of determining a current image frame read from a first cache queue as a target image frame, storing the target image frame into a second cache queue and decoding the target image frame; when the target image frame is failed to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue, and acquiring target audio data corresponding to the image group where the target image frame is located; taking the image frame at the head of the first buffer queue as a target image frame, and repeatedly executing the buffering and decoding steps until the target image frame is successfully decoded; and displaying the image after the decoding is successful and playing the target audio data. The invention avoids the phenomenon of picture and sound asynchronism caused by decoding failure in the video playing process, and accelerates the display speed of the picture.
The present embodiments also provide a computer-readable storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, which is loaded by a processor and executes any one of the methods described above in the embodiments of the present application.
Referring to fig. 14, the apparatus 1400 may include one or more Central Processing Units (CPUs) 1422 (e.g., one or more processors) and a memory 1432, one or more storage media 1430 (e.g., one or more mass storage devices) for storing applications 1442 or data 1444. Memory 1432 and storage media 1430, among other things, may be transient or persistent storage. The program stored on storage medium 1430 may include one or more modules (not shown), each of which may include a sequence of instructions operating on the device. Still further, a central processor 1422 may be provided in communication with storage medium 1430 for executing a series of instruction operations on storage medium 1430 on device 1400. The apparatus 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input-output interfaces 1458, and/or one or more operating systems 1441, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc. Any of the methods described above in this embodiment can be implemented based on the apparatus shown in fig. 14.
The present specification provides method steps as described in the examples or flowcharts, but may include more or fewer steps based on routine or non-inventive labor. The steps and sequences recited in the embodiments are but one manner of performing the steps in a multitude of sequences and do not represent a unique order of performance. In the actual system or interrupted product execution, it may be performed sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.
The configurations shown in the present embodiment are only partial configurations related to the present application, and do not constitute a limitation on the devices to which the present application is applied, and a specific device may include more or less components than those shown, or combine some components, or have an arrangement of different components. It should be understood that the methods, apparatuses, and the like disclosed in the embodiments may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a division of one logic function, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or unit modules.
Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (9)
1. A video playback method, comprising:
responding to a video playing instruction, acquiring a resource to be played, and storing the resource to be played into a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, wherein each image group comprises a plurality of non-key image frames and a key frame; the key frame is at the position of the first frame of the image group;
determining that the image frame currently read from the head of the first buffer queue is a target image frame, and deleting the read image frame from the first buffer queue;
storing the target image frame into a second buffer queue, and decoding the target image frame; the second buffer queue comprises a key frame;
when the target image frame is failed to be decoded, inserting each image frame in the second buffer queue into the head of the first buffer queue from the tail of the second buffer queue; determining an image group in which the target image frame is located as a target image group, so that when the insertion operation is completed, a key frame in the target image group is located at the head of the first cache queue; acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
2. The video playing method according to claim 1, wherein said determining that the image frame currently read from the head of the first buffer queue is the target image frame comprises:
reading one image frame at a time from the head of the first buffer queue, and determining the read image frame as the target image frame.
3. The video playing method according to claim 1, wherein the method further comprises:
and when the target image frame is successfully decoded, displaying the successfully decoded image and playing the audio data corresponding to the target image frame.
4. The video playing method according to claim 1, wherein said storing the target image frame in a second buffer queue comprises:
judging whether the target image frame is the key frame;
when the target image frame is the key frame, emptying the second cache queue, and storing the target image frame into the emptied second cache queue;
and when the target image frame is not the key frame, storing the target image frame to the tail part of the second buffer queue.
5. The video playing method according to claim 4, wherein the method further comprises:
when the target image frame fails to be decoded, recording the audio position corresponding to the key frame of the target image group;
the acquiring target audio data corresponding to the target image group includes:
the target audio data is acquired starting from the audio position.
6. The video playing method according to claim 1, wherein the method further comprises:
triggering the initialization of the decoding configuration after the target image frame fails to be decoded.
7. A video playback apparatus, comprising:
the playing instruction response module is used for responding to a video playing instruction, acquiring a resource to be played and storing the resource to be played into a first cache queue; the resource to be played comprises a plurality of image groups and corresponding audio data, wherein each image group comprises a plurality of non-key image frames and a key frame; the key frame is at the position of the first frame of the image group;
the target image frame determining module is used for determining the image frame currently read from the head of the first buffer queue as a target image frame and deleting the read image frame from the first buffer queue;
the buffer decoding module is used for storing the target image frame into a second buffer queue and decoding the target image frame; the second buffer queue comprises a key frame;
a loop execution module, configured to insert each image frame in the second buffer queue into the head of the first buffer queue from the tail of the second buffer queue when decoding of the target image frame fails; determining an image group in which the target image frame is located as a target image group, so that when the insertion operation is completed, a key frame in the target image group is located at the head of the first cache queue; acquiring target audio data corresponding to the target image group; taking the image frame at the head of the first buffer queue as the target image frame, and repeatedly executing the following steps: storing the target image frame into a second buffer queue, and decoding the target image frame until the target image frame is successfully decoded; and displaying the successfully decoded image and playing the target audio data.
8. An electronic device, characterized in that the device comprises a processor and a memory, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which is loaded and executed by the processor to implement the video playback method according to any one of claims 1 to 6.
9. A computer storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded by a processor and which performs a video playback method as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910645432.0A CN110366033B (en) | 2019-07-17 | 2019-07-17 | Video playing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910645432.0A CN110366033B (en) | 2019-07-17 | 2019-07-17 | Video playing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110366033A CN110366033A (en) | 2019-10-22 |
| CN110366033B true CN110366033B (en) | 2021-10-22 |
Family
ID=68220836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910645432.0A Active CN110366033B (en) | 2019-07-17 | 2019-07-17 | Video playing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110366033B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112825563B (en) * | 2019-11-20 | 2023-08-22 | 西安诺瓦星云科技股份有限公司 | Video processing method and device and video processing equipment |
| CN111526404B (en) * | 2020-04-27 | 2022-07-08 | 杭州海康威视系统技术有限公司 | Single frame playing method, video playing device, monitoring equipment and storage medium |
| CN111901666B (en) * | 2020-07-01 | 2021-05-11 | 腾讯科技（深圳）有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN112752150A (en) * | 2020-12-29 | 2021-05-04 | 深圳市天视通视觉有限公司 | Video decoding method and device, electronic equipment and storage medium |
| CN112929792B (en) * | 2021-01-21 | 2022-06-28 | 稿定（厦门）科技有限公司 | Sox-based audio processing method, medium, device and apparatus |
| CN113038156B (en) * | 2021-03-04 | 2023-04-25 | 百果园技术(新加坡)有限公司 | Live broadcast data management system, method, equipment and storage medium |
| CN113709574B (en) * | 2021-03-17 | 2024-03-19 | 腾讯科技（北京）有限公司 | Video screenshot method and device, electronic equipment and computer readable storage medium |
| CN113079402B (en) * | 2021-03-30 | 2023-06-23 | 苏州博动戎影医疗科技有限公司 | Image display method, device, equipment and storage medium |
| CN113542856B (en) * | 2021-06-28 | 2023-06-13 | 宜宾中星技术智能系统有限公司 | Method, device, equipment and computer readable medium for reverse playing of online video |
| CN113905196B (en) * | 2021-08-30 | 2024-05-07 | 浙江大华技术股份有限公司 | Video frame management method, video recorder, and computer-readable storage medium |
| CN113961484A (en) * | 2021-10-29 | 2022-01-21 | Oppo广东移动通信有限公司 | Data transmission method and device, electronic equipment and storage medium |
| CN115209223B (en) * | 2022-05-12 | 2024-09-20 | 广州方硅信息技术有限公司 | Video encoding/decoding control processing method, device, terminal and storage medium |
| CN115460469B (en) * | 2022-08-24 | 2024-06-11 | 网易（杭州）网络有限公司 | Processing method and device for multi-video playing, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1440624A (en) * | 2000-05-15 | 2003-09-03 | 诺基亚有限公司 | Flag controlled video concealing method |
| CN105100748A (en) * | 2015-08-31 | 2015-11-25 | 深圳市茁壮网络股份有限公司 | Video monitoring system and video monitoring method |
| US9462219B2 (en) * | 2014-03-26 | 2016-10-04 | Vivotek Inc. | Method for reverse video playback and computer-readable medium |
| CN109348292A (en) * | 2018-11-20 | 2019-02-15 | 广州易方信息科技股份有限公司 | A kind of video segment method based on slice file byte-threshold |
| CN105187895B (en) * | 2015-09-17 | 2019-03-12 | 暴风集团股份有限公司 | For the hardware-accelerated data cache method and system for playing video of mobile platform |
-
2019
- 2019-07-17 CN CN201910645432.0A patent/CN110366033B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1440624A (en) * | 2000-05-15 | 2003-09-03 | 诺基亚有限公司 | Flag controlled video concealing method |
| US9462219B2 (en) * | 2014-03-26 | 2016-10-04 | Vivotek Inc. | Method for reverse video playback and computer-readable medium |
| CN105100748A (en) * | 2015-08-31 | 2015-11-25 | 深圳市茁壮网络股份有限公司 | Video monitoring system and video monitoring method |
| CN105187895B (en) * | 2015-09-17 | 2019-03-12 | 暴风集团股份有限公司 | For the hardware-accelerated data cache method and system for playing video of mobile platform |
| CN109348292A (en) * | 2018-11-20 | 2019-02-15 | 广州易方信息科技股份有限公司 | A kind of video segment method based on slice file byte-threshold |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110366033A (en) | 2019-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110366033B (en) | Video playing method, device, equipment and storage medium | |
| CN106534952B (en) | Source switching back-film source continuous playing method and smart television | |
| US9531983B2 (en) | Decoding interdependent frames of a video for display | |
| WO2020248909A1 (en) | Video decoding method and apparatus, computer device, and storage medium | |
| CN110784740A (en) | Video processing method, device, server and readable storage medium | |
| KR20110022653A (en) | Uniform video decoding and display | |
| CN111836076B (en) | Video resolution switching method and device and electronic equipment | |
| US20190373274A1 (en) | Hybrid decoding | |
| CN111436004B (en) | Video playing method, device and equipment and computer readable storage medium | |
| JP2009543439A (en) | Support for interactive playback devices for performance-aware peer-to-peer content-on-demand services | |
| CN111447455A (en) | Live video stream playback processing method and device and computing equipment | |
| CN111225171B (en) | Video recording method, device, terminal equipment and computer storage medium | |
| EP3410302B1 (en) | Graphic instruction data processing method, apparatus | |
| EP3142381A1 (en) | Network video playing method and device | |
| CN112601127B (en) | Video display method and device, electronic equipment and computer readable storage medium | |
| CN109348279A (en) | A kind of plug-flow method, apparatus, equipment and storage medium | |
| CN110519640B (en) | Video processing method, encoder, CDN server, decoder, device, and medium | |
| US20130223813A1 (en) | Moving image reproduction apparatus, information processing apparatus, and moving image reproduction method | |
| CN112423140A (en) | Video playing method and device, electronic equipment and storage medium | |
| CN109218739B (en) | Method, device and equipment for switching visual angle of video stream and computer storage medium | |
| JP2003111048A (en) | Server and program for contents reproduction | |
| US8391688B2 (en) | Smooth rewind media playback | |
| CN111318012B (en) | Game data transmission method and device | |
| US20220337908A1 (en) | User Interface (UI) Engine for Cloud UI Rendering | |
| CN110677715B (en) | Decoding method, decoder, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |