WO2018171437A1

WO2018171437A1 - Display method and device for preview image

Info

Publication number: WO2018171437A1
Application number: PCT/CN2018/078446
Authority: WO
Inventors: 徐玉丰; 顾迎节; 周爱彬
Original assignee: 华为技术有限公司
Priority date: 2017-03-21
Filing date: 2018-03-08
Publication date: 2018-09-27
Also published as: CN108632644A; CN108632644B

Abstract

The embodiment of the present invention provides a display method for a preview image. The method comprising: a client acquires from a server an index file of a video program, the index file comprising description information of a preview image; the client acquires the preview image according to the description information of the preview image; and the client displays the preview image according to a user operational instruction. According to the method, the description information of the preview image is comprised in the index file of the video program, so that when playbacking the video according to the index file, the client can acquire the preview image directly according to the index file. By combining the implementation and a playback protocol of the preview image, the consistency in user experience is ensured; the complexity for a video service provider to separately develop a preview image implementation protocol can also be reduced; and a video service thus can be quickly launched.

Description

Preview display method and device

The present application claims the priority of the Chinese Patent Application, filed on March 21, 2017, the application Serial No. .

Technical field

The present invention relates to the field of video, and in particular, to a method and device for displaying a preview image.

Background technique

With the development of the Internet, video resources are becoming more and more abundant. In video playback, dragging the progress bar by dragging the progress of the video is a relatively common user experience. In the process of video progress dragging, many video service providers now provide preview images, which can accurately see the video preview of the corresponding playback progress bar position, helping users to quickly locate the desired image. Look at the content.

However, the preview implementation protocol and the video playback protocol provided by each video service provider are independent of each other. In addition, the preview information provided by each video service provider is not uniform, and the implementation difference is large, which is not conducive to the consistency of the user experience.

Summary of the invention

In view of this, it is necessary to provide a preview map implementation method combined with the playback protocol to ensure the consistency of the user experience.

In a first aspect, a method for displaying a preview image is provided, including:

The client obtains an index file of the video program from the server, where the index file includes description information of the preview image, and the index file is a media presentation description MPD file in the DASH protocol;

The client acquires a preview image from the server according to the preview description information;

The client displays the preview image according to a user operation instruction, such as displaying a preview image of the corresponding position when the user drags the play progress bar. The method includes the description information of the preview image in the index file of the video program, so that when the video is played according to the index file, the client can directly obtain the preview image according to the index file. Combining the implementation of the preview with the playback protocol ensures the consistency of the user experience, and also reduces the complexity of the video service provider separately developing the preview implementation protocol, and enables the video service to be quickly launched.

Optionally, the preview description information includes:

The address information of the preview image and the time information corresponding to the preview image, the address information may be location information stored in the preview image, and the time information corresponding to the preview image may include a corresponding time of the first preview image of the video and a time interval between the preview images. In this way, when displaying the preview image, the client can determine which preview image needs to be displayed according to the time information of the preview image and the position of the playback progress bar, thereby obtaining a corresponding preview image for display, and ensuring the preview image and the video display. consistency.

Optionally, the client may download the preview image from the server to the local location according to the address information of the preview image, and obtain a preview image of the corresponding time from the local time when the user drags the play progress bar, so that the speed of the preview image is displayed. Faster and more real-time.

Optionally, when the user drags the play progress bar, the client determines the address information of the corresponding preview image according to the position of the play progress bar and the time information corresponding to the preview image, according to the determined preview image. The address information is downloaded from the server and previewed. This option can reduce storage pressure on the client and save storage space.

In a second aspect, a method for displaying a preview image is provided, which is applied to a server, and includes:

The server sends an index file of the video program to the client, where the index file is a media presentation description MPD file of the DASH protocol, and the index file includes description information of the preview image;

Receiving a client preview download request;

The preview image is sent to the client in response to the client's download request.

Optionally, the preview description information includes:

Preview the address information of the map and the time information corresponding to the preview.

Optionally, the method further includes:

The server generates preview and preview description information according to the preview generation rule.

Optionally, the preview generation rule includes:

Preview interval information and preview resolution information.

Optionally, the server may generate a preview image by directly capturing a video frame from the video file according to the interval information and the resolution information of the preview image.

In a third aspect, a client device is provided, including:

a receiver, configured to acquire an index file of a video program from a server, where the index file is an MPD file of a DASH protocol, and the index file includes description information of the preview image;

And a processor, configured to acquire a preview image from the server according to the preview description information, and display the preview image according to a user operation instruction.

Optionally, the preview description information includes:

Optionally, the processor is specifically configured to:

When the user drags the play progress bar, the corresponding preview image is displayed according to the position of the play progress bar.

Optionally, the processor is further configured to: before the user drags the play progress bar, download the preview image from the server according to the address information of the preview image.

Optionally, the processor is further configured to: when the user drags the play progress bar, determine the address information of the corresponding preview image according to the position of the play progress bar and the time information corresponding to the preview image, according to the The address information of the determined preview is downloaded from the server.

In a fourth aspect, a server is provided, including:

a transmitter, configured to send an index file of a video program to the client, where the index file is an MPD file of a DASH protocol, and the index file includes description information of the preview image;

a receiver, configured to receive a client preview download request;

The transmitter is further configured to send the preview image to the client in response to the download request of the client.

Optionally, the server further includes:

A processor is configured to generate a preview image and preview description information according to the preview generation rule.

Optionally, the preview generation rule includes:

Preview interval information and preview resolution information.

Optionally, the processor may generate a preview image by directly capturing a video frame from the video file according to the interval information and the resolution information of the preview image.

In a fifth aspect, a computer program product is provided, comprising executable program code, wherein the program code includes instructions that, when executed by the processor, cause the server or client to perform as described above The preview display method described in the aspect.

In a sixth aspect, an embodiment of the present application provides a computer storage medium for storing computer software instructions used by the server and the client, and includes a program designed to execute the foregoing aspects.

DRAWINGS

1 is a system architecture diagram of an adaptive HTTP stream transmission for implementing an embodiment of the present application;

2 is a structural block diagram of a network device used to implement an embodiment of the present application;

3 is an interaction flowchart of a preview display method according to an embodiment of the present application;

FIG. 4 is an interaction flowchart of a preview display method according to another embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

FIG. 1 illustrates a typical system architecture for implementing adaptive HTTP streaming for embodiments of the present application. As shown in FIG. 1, the system 100 can include n HTTP servers 110, n can be any integer number, client 120, and network 130. HTTP server 110 and client 120 may be coupled to one another via network 130 and other network components such as an HTTP cache (not shown).

The HTTP server 110 can be any hardware computer server for transmitting and receiving data over HTTP. The HTTP server 110 may include a server DASH module (DM) 111 for transmitting and receiving data via HTTP and according to the DASH standard. The HTTP server may form a content distribution network (CDN) 140, which may refer to a distributed server system deployed across multiple backbones for distributing content over multiple backbones. The CDN 140 can include any number of HTTP servers 110 such that n can be any integer number. Typically, a CDN can include hundreds or thousands of servers.

Client 120 can be any hardware device for transmitting and receiving data over HTTP. For example, client 120 can be a laptop, tablet, desktop, mobile phone, or other device. Client 120 may include a client DASH module (DM) 121, an application 122, and a graphical user interface (GUI) 123.

The client DM121 can be used to send and receive data over HTTP and according to the DASH standard. The client DM121 may further include a DASH access engine (DAE) 1211 and a media export (ME) 1212. The DAE 1211 may be the primary component for receiving raw data from the HTTP server 110 and building the data into a viewing format. For example, the DAE 1211 can format the data in the Moving Picture Experts Group (MPEG) container format along with the timing data, and then output the formatted data to the ME 1212. The ME 1212 may be responsible for initialization, playback, and other functions associated with the content and output the content to the application 122.

Application 122 may be a web browser or other application with an interface for downloading and presenting content. Application 122 can be coupled to GUI 123 such that a user associated with client 120 can view various functions of application 122. The application 122 can include a search function so that the user can search for content. For example, if the application 122 is a media player, the application 122 can include a search bar so that the user can enter a string to search for a movie. The application 122 can display a search hit list, and the user can select the desired movie from the hit list. After selection, the application 122 can send an instruction to download the movie to the client DM121. The DM 121 can download movies and process movies for output to the application 122. The application 122 can provide instructions to the GUI 123 for the GUI 123 to display the movie for viewing by the user. The application 122 can further provide instructions to the GUI 123 for the GUI 123 to display a progress bar that displays the time progress of the movie.

Network 130 may be any network for providing communication between HTTP server 110 and client 120. For example, network 130 is either the Internet, a mobile telephone network, or other network. Network 130 may allow for communication along a wired or wireless channel.

2 is a schematic diagram of a network device 200 for implementing an embodiment of the present invention. Apparatus 200 can include a plurality of ingress ports 210 and/or receiver units (Rx) 220 for receiving data; a logic unit or processor 230 for processing signals; a plurality of egress ports 240 and/or transmitter units (Tx ) 250 for transmitting data to other components; and a memory 260. Apparatus 200 can be adapted to implement any of the disclosed features, methods, and apparatus. For example, device 200 can be adapted to implement HTTP server 110, and client 120.

Logic unit 230 (which may be referred to as a central processing unit (CPU)) may be in communication with ingress port 210, receiver unit 220, egress port 240, transmitter unit 250, and memory 260. Logic unit 230 can be implemented as one or more CPU chips, cores (eg, multi-core processors), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and/or digital signal processors (DSPs), and/ Or it can be part of one or more ASICs.

The memory 260 can be comprised of one or more magnetic disks, tape drives, optical drives, or solid state drives; can be used for non-volatile storage of data and as an overflow data storage device; can be used to store programs when such programs are selected for execution; Can be used to store instructions and data that are read during program execution. Memory 260 can be volatile and/or non-volatile and can be read only memory (ROM), random access memory (RAM), triple content addressed memory (TCAM), static random access memory (SRAM). , other suitable memory types, or any combination thereof. Memory 260 can include DM 270, which can be adapted to implement server DM 111 or client DM 121. Although memory 260 is shown as including DM 270, logic unit 230 or any combination of logic unit 230 and memory 260 can include DM 270.

Based on the foregoing typical system architecture and network device for HTTP streaming, FIG. 3 is a flow interaction diagram of a preview display method according to an embodiment of the present application.

Client 110 may use an HTTP GET request or similar message to download a Media Presentation Description (MPD) from HTTP server 120 or HTTP cache (not shown). The MPD can be an Extensible Markup Language (XML) document. The MPD includes metadata of a format used to encode the media content, such as bit rate, codec, screen resolution, and language. In the embodiment, the MPD further includes description information of the preview image.

In this embodiment, when the video content is injected, the server may read the preview image generation rule of the corresponding video, for example, the interval time of the preview image, the resolution of the preview image, etc., and generate the preview image and the description information of the preview image, and preview the description. Information can be implemented in multiple ways:

In the first implementation, all preview images can be compressed into one file and carried in the form of a compressed package. The attribute parameters of the preview map are defined in MPD Representation, as follows:

The elements in the Representation tag are introduced:

Id: preview group ID.

Width: the width of the preview, in pixels.

Height: the height of the preview, in pixels.

startTime: The first preview corresponds to the time point in the video.

Interval: Interval length of adjacent previews, in seconds.

Number: The total number of previews generated by this video.

BaseURL: The URL of the group preview image packaged into a zip file for download.

In the second embodiment, the preview map is described by a list, and the attribute parameters of the preview map are defined in the MPD Representation, and are described as follows:

The elements in the Representation tag are introduced:

Id: the identifier of the preview (group).

Width: the width of the preview, in pixels.

Height: the height of the preview, in pixels.

startTime: The first preview corresponds to the time point in the video.

Interval: Interval length of adjacent preview pictures, in seconds.

Number: The total number of previews generated by this video.

SegmentURL: Tags support multiple values

Media: The URL of the specific preview file for download by the terminal.

In the third embodiment, the preview image is described by the template, and the attribute parameters of the preview image are defined in the MPD Representation, as follows:

The elements in the Representation tag are introduced:

PreviewTemplate: The rule for previewing the download address (URL).

PreviewTimeline: A rule that describes the video content corresponding to the preview.

startTime: The first preview corresponds to the point in time in the video.

Interval: Interval length of adjacent preview pictures, in seconds.

Number: The total number of previews generated by this video.

Id: Preview map (group) ID.

Width: The width of the preview, in pixels.

Height: the height of the preview, in pixels.

As shown in FIG. 3, the preview display method of this embodiment includes the following steps:

In step 310, the client 120 can send a request for the MPD to the HTTP server 110.

At step 320, the HTTP server 110 can send the MPD to the client 120. The client 120 can receive the MPD through the DASH access engine DAE 1211. The MPD includes description information of the preview image. Specifically, the description information of the preview image may include address information of the preview image, time information, and may also include resolution information of the preview image.

In step 330, the client 120 parses the description information of the preview image from the MPD file, and obtains the address information, the time information, and the resolution information of the preview image, where the time information of the preview image includes at least: the time corresponding to the preview image of the first frame, The time between adjacent previews and the total number of frames in the preview.

In step 340, the client 120 requests a preview image from the HTTP server according to the description information of the preview image.

Specifically, the client 120 may select a suitable preview resolution according to a terminal parameter such as a screen size, and a corresponding preview image xml document, and request a corresponding preview image from the server according to the address of the preview image in the document.

In step 350, the HTTP server sends a preview to the client 120, and the client 120 caches the preview into the local storage.

In step 360, the client 120 displays a preview of the corresponding progress when the user drags the play progress bar.

Specifically, when the user drags the play progress bar, the position and time at which the play progress bar stops are determined, and a preview image of the corresponding time is obtained according to the time for display. It should be noted that dragging the play progress bar may be performed by the user directly using the mouse, or the user directly inputs the time parameter to locate the position of the play progress bar.

Optionally, the client 120 may download all preview images from the server to the local storage according to the address information of the preview image at the beginning of the video playback, and obtain a preview of the corresponding time locally when the user drags the playback progress bar. Display, this way can display the preview in real time when the user drags the play progress bar, showing the experience better.

In addition, the client 120 may also determine the stop position of the progress bar and the corresponding time when the user drags the progress bar, select a preview image of the corresponding time according to the time, and download the corresponding preview image from the server according to the position information of the preview image. Show that this method can reduce the storage pressure of the client and save the storage space of the client.

As shown in FIG. 4, it is an interactive flowchart of a preview display method according to another embodiment of the present application.

In this embodiment, the preview description information may be a separate XML file, and the MPD may include only the description of the XML file, such as the URL of the preview image XML file. After the client can parse the MPD, the preview image XML file is obtained from the server according to the URL of the preview image XML file. Among them, the MPD description of the preview image XML file can be as follows:

Parameter Description:

mimeType: identifies application/xml, non-video audio, preview image belongs to application class, hosted by XML

Value: Clearly used for timeline preview, timelinepreview.

Representation id: When there are multiple sets of resolution previews to provide preview, multiple Representations are distinguished by id.

Width, preview width in pixels

Height, the height of the preview, in pixels

BaseURL: Previews the URL of the XML file. The client can obtain detailed description information of the preview image based on the URL.

Specifically, there are multiple ways to describe the preview image XML file. In this embodiment, in the first implementation manner, the preview image XML file is in a single picture mode, and each preview image is a single picture, which is described as follows:

Parameter Description:

Timemline: The content under the XML node is the scene for the timeline

Image: indicates that the preview object node is described

Time: format hour and minute HHMMSS

Id="preview_240_$time$.jpg": URL of the preview file

In the second implementation manner, the preview image XML file is a merged picture mode, that is, multiple preview images are combined in one picture.

in:

Parameter Description:

Timemline: The content under the XML node is the scene for the timeline

Image: indicates that the preview object node is described

startTime: the start time of the video, format: HHMMSS

endTime: end time of the video, format: HHMMSS

Interval: Interval between previews in this video. Unit: second

Id="preview_$startTime$_$endTime$.jpg": URL of the preview file

The merge picture mode can reduce the number of pictures, and the client can read the merged picture, which can reduce the number of files opened and closed, and improve the display speed.

In this embodiment, the preview display method includes the following steps:

In step 410, the client 120 can send a request for the MPD to the HTTP server 110.

In step 420, the HTTP server 110 can send the MPD to the client 120. The client 120 can receive the MPD through the DAE 1211, where the MPD includes preview description information. Specifically, the preview description information includes: a URL of the preview description file.

Specifically, the preview description information can be described as follows:

The client 120 determines, according to the value=“timelinepreview”, that the video supports the preview feature, wherein multiple Representations indicate that the preview image with multiple resolutions can be obtained, and the terminal can select an appropriate resolution preview image xml according to the size of the screen. In this example, the obtained preview image of 240 resolution is taken as an example.

In step 430, the client 120 parses the URL of the preview description file from the MPD file.

In step 440, the client 120 requests the preview description file from the server according to the URL of the preview description file.

Specifically, the client 120 may request the description file of the preview image from the server according to the selected BaseURL of the preview with a resolution of 240.

In step 450, the HTTP server 110 returns the description file of the preview image to the client 120.

Specifically, the preview description file can be an XML document, and the specific content can be as follows:

In step 460, the client 120 requests a preview image from the HTTP server according to the description file of the preview image.

In step 470, the HTTP server sends a preview to the client 120, and the client 120 caches the preview into the local storage.

In step 480, the client 120 displays a preview of the corresponding progress when the user drags the play progress bar.

Specifically, when the user drags the play progress bar, the position and time at which the play progress bar stops are determined, and a preview image of the corresponding time is obtained according to the time for display.

It should be noted that the solution of the embodiment of the present application can also be applied to other HTTP protocols such as HLS. For example, the HLS protocol is applicable, and the description information of the preview image may be included in the M3U8 file. For the specific implementation manner, refer to the foregoing embodiment.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent person can be easily conceived within the technical scope of the present invention by any person skilled in the art. Modifications or substitutions are intended to be included within the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

A method for displaying a preview image, comprising:

The client obtains an index file of the video program from the server, where the index file is a Media Presentation description (MPD) file of the DASH protocol, and the index file includes description information of the preview image;

The client acquires a preview image according to the preview description information;

The client presents the preview map according to a user operation instruction.
The method of claim 1, wherein the preview description information comprises:

Preview the address information of the map and the time information corresponding to the preview.
The method according to claim 1 or 2, wherein the client displays the preview according to a user operation instruction, including:

When the user drags the play progress bar, the corresponding preview image is displayed according to the position of the play progress bar.
The method according to any one of claims 1-3, wherein the client obtains a preview according to the preview description information, including:

Before the user drags the play progress bar, the preview image is downloaded from the server to the local according to the address information of the preview image.
The method according to any one of claims 1-3, wherein the client obtains a preview according to the preview description information, including:

When the user drags the play progress bar, determining the address information of the corresponding preview image according to the position of the play progress bar and the time information corresponding to the preview image, and downloading the preview from the server according to the determined address information of the preview image. Figure.
A method for displaying a preview image, comprising:

The server sends an index file of the video program to the client, where the index file is an MPD file of the DASH protocol, and the index file includes description information of the preview image;

Receiving a client preview download request;

The preview image is sent to the client in response to the client's download request.
The method of claim 6 wherein said preview description information comprises:

Preview the address information of the map and the time information corresponding to the preview.
The method of claim 6 or claim 7, wherein the method further comprises:

Generate preview and preview description information according to the preview generation rule.
The method of claim 8, wherein the preview generation rule comprises:

Preview interval information and preview resolution information.
A client device comprising:

a receiver, configured to acquire an index file of a video program from a server, where the index file is an MPD file of a DASH protocol, and the index file includes description information of the preview image;

And a processor, configured to acquire a preview image according to the preview description information; and display the preview according to a user operation instruction.
The client device according to claim 10, wherein the preview description information comprises:

Preview the address information of the map and the time information corresponding to the preview.
The client device according to claim 10 or 11, wherein the processor is specifically configured to:

When the user drags the play progress bar, the corresponding preview image is displayed according to the position of the play progress bar.
The client device according to any one of claims 10 to 12, wherein the processor is further configured to: download a preview image from the server according to the address information of the preview image before the user drags the play progress bar .
The method according to any one of claims 10 to 12, wherein the processor is further configured to: when the user drags the play progress bar, according to the position of the play progress bar and the corresponding preview image The time information determines the address information of the corresponding preview image, and downloads the preview image from the server according to the determined address information of the preview image.
A server that includes:

a transmitter, configured to send an index file of a video program to the client, where the index file is an MPD file of a DASH protocol, and the index file includes description information of the preview image;

a receiver, configured to receive a client preview download request;

The transmitter is further configured to send the preview image to the client in response to the download request of the client.
The server according to claim 15, further comprising:

A processor is configured to generate a preview image and preview description information according to the preview generation rule.
The server according to claim 16, wherein the preview generation rule comprises:

Preview interval information and preview resolution information.