CN113973207A - Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream - Google Patents
Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream Download PDFInfo
- Publication number
- CN113973207A CN113973207A CN202111249719.5A CN202111249719A CN113973207A CN 113973207 A CN113973207 A CN 113973207A CN 202111249719 A CN202111249719 A CN 202111249719A CN 113973207 A CN113973207 A CN 113973207A
- Authority
- CN
- China
- Prior art keywords
- frame
- real
- ios
- video stream
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 11
- 230000006835 compression Effects 0.000 claims abstract description 11
- 238000007906 compression Methods 0.000 claims abstract description 11
- 230000005540 biological transmission Effects 0.000 claims abstract description 10
- 238000010586 diagram Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses an IOS remote real machine WEB terminal real-time control playback method based on H264 code stream, which comprises the following steps: the picture is compiled into a video stream in an H264 format in real time by using a built-in screenshot and hardware coding mode of an IOS system, the video stream is transmitted through a TCP/IP protocol, the H264 code stream can form an I frame (an IDR frame is a key frame), a P frame and a B frame according to image change characteristics, wherein the I frame adopts a core algorithm to store complete image information for intraframe compression, and the P frame and the B frame adopt interframe compression to store correlation data of connected frames. The invention skillfully utilizes the H264 difference frame technology, greatly reduces the transmission bandwidth pressure (saves more than 2/3), hard-compiles the IOS picture into the video stream, and reduces the transmission data volume and stably controls the transmission rate by skillfully utilizing the H264 difference frame technology; and the H264 video stream is analyzed into a picture stream at the WEB end, so that the resource consumption of server centralized decoding is reduced, and the remote control fluency is improved.
Description
Technical Field
The invention relates to the field of test management, in particular to an IOS remote real machine WEB end real-time control playback method based on an H264 code stream.
Background
At present, the IOS remote real-time image display mostly adopts a screenshot interface carried by an IOS system to screenshot at a frequency of not less than 60 times per second, and a picture stream is transmitted to a WEB terminal by a TCP/IP or HTTP protocol and is rendered on Canvas in real time for display so as to facilitate gesture and sliding operation. Because of the IOS high-resolution picture and the high-performance screenshot capability, the texture of a real-time picture can be guaranteed, but the scheme has high requirement on bandwidth and is not suitable for large-scale centralized use, and particularly under the condition of limited bandwidth, the pause condition is very obvious; the prior art solution is shown in figure 3.
Disclosure of Invention
The technical problem to be solved by the invention is to overcome the defects of the prior art, provide a real-time control and display method for the IOS remote real machine WEB side based on H264 code stream, solve the problem of real-time control and display blockage of the IOS real machine remote WEB side under the environment of low bandwidth or limited personal network, maintain fluency, ensure a certain image quality standard and provide real machine operability.
The invention provides the following technical scheme:
the invention provides an IOS remote real machine WEB terminal real-time control echoing method based on an H264 code stream, which comprises the following steps:
1) the picture is compiled into a video stream in an H264 format in real time by using a built-in screenshot and hardware coding mode of an IOS (input/output system), the video stream is transmitted through a TCP/IP (transmission control protocol/Internet protocol), the H264 code stream can form an I frame (an IDR frame is a key frame), a P frame and a B frame according to image change characteristics, wherein the I frame adopts a core algorithm to store complete image information for intraframe compression, and the P frame and the B frame adopt interframe compression to store correlation data of connected frames;
2) analyzing the received video stream in the H264 format into pictures in real time at a WEB end;
3) and finally, drawing the image information on the WEB to form an operable smooth dynamic picture stream.
Compared with the prior art, the invention has the following beneficial effects:
the invention skillfully utilizes the H264 difference frame technology, greatly reduces the transmission bandwidth pressure (saves more than 2/3), hard-compiles the IOS picture into the video stream, and reduces the transmission data volume and stably controls the transmission rate by skillfully utilizing the H264 difference frame technology; and the H264 video stream is analyzed into a picture stream at the WEB end, so that the resource consumption of server centralized decoding is reduced, and the remote control fluency is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a framework diagram of the present invention;
FIG. 2 is a schematic diagram of an embodiment of the present invention;
fig. 3 is a prior art frame diagram.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation. Wherein like reference numerals refer to like parts throughout.
Example 1
As shown in fig. 1-3, the present invention provides a real-time control playback method for an IOS remote live machine WEB end based on H264 code stream, comprising the following steps:
1) the picture is compiled into a video stream in an H264 format in real time by using a built-in screenshot and hardware coding mode of an IOS (input/output system), the video stream is transmitted through a TCP/IP (transmission control protocol/Internet protocol), the H264 code stream can form an I frame (an IDR frame is a key frame), a P frame and a B frame according to image change characteristics, wherein the I frame adopts a core algorithm to store complete image information for intraframe compression, and the P frame and the B frame adopt interframe compression to store correlation data of connected frames;
2) analyzing the received video stream in the H264 format into pictures in real time at a WEB end;
3) and finally, drawing the image information on the WEB to form an operable smooth dynamic picture stream.
Furthermore, the method changes a pure picture stream transmission mode, hard codes the IOS picture into a video stream, and decodes the video stream into a picture stream by using a decoding scheme of a WEB end, thereby recovering the operability of the real machine.
Example as shown in fig. 2, the technical points are as follows:
screenshot of the IOS mobile phone: the method is implemented by a private screenshot interface _ XCT _ requestScreensense OfScreenWithID of an XCode self-contained unit test framework XCTest.
Hardware coding H264 code stream: the hardware codec is directly accessible using the XCode's VideoToolbox framework. The video compression and decompression service can be provided, and the format conversion of the image stored in the CoreVideo pixel buffer is also provided;
h264 frame characteristics:
i-frame (intra picture) I-frame is usually the first frame of each GOP (a video compression technique used by MPEG), and is moderately compressed and used as a reference point for random access, which can be regarded as a picture. An I-frame can be viewed as the product of a compression of an image.
P-frames (forward predictive coded frames), also called predictive-frames, compress coded pictures that transmit an amount of data by sufficiently reducing temporal redundancy information below previously coded frames in a sequence of pictures.
The B frame (bi-directionally predicted frame) takes into account both the coded frame preceding the source image sequence and the temporal redundancy information between the coded frames following the source image sequence to compress the coded image of the transmitted data amount, also called bi-directionally predicted frame.
H264 decoding: the video stream is decoded into picture frames using a broadway framework and a picture stream is formed.
Picture rendering: the new characteristic canvas of H5 is adopted to draw pictures and is compatible with most WEB browsers
In the invention.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (1)
1. A real-time control and playback method for an IOS remote real machine WEB end based on an H264 code stream is characterized by comprising the following steps:
1) the picture is compiled into a video stream in an H264 format in real time by using a built-in screenshot and hardware coding mode of an IOS (input/output system), the video stream is transmitted through a TCP/IP (transmission control protocol/Internet protocol), the H264 code stream can form an I frame (an IDR frame is a key frame), a P frame and a B frame according to image change characteristics, wherein the I frame adopts a core algorithm to store complete image information for intraframe compression, and the P frame and the B frame adopt interframe compression to store correlation data of connected frames;
2) analyzing the received video stream in the H264 format into pictures in real time at a WEB end;
3) and finally, drawing the image information on the WEB to form an operable smooth dynamic picture stream.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111249719.5A CN113973207A (en) | 2021-10-26 | 2021-10-26 | Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111249719.5A CN113973207A (en) | 2021-10-26 | 2021-10-26 | Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113973207A true CN113973207A (en) | 2022-01-25 |
Family
ID=79588423
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111249719.5A Pending CN113973207A (en) | 2021-10-26 | 2021-10-26 | Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113973207A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115037732A (en) * | 2022-06-01 | 2022-09-09 | 中国电力科学研究院有限公司 | Method, device, equipment and medium for remote real machine debugging through streaming media |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404768A (en) * | 2008-10-07 | 2009-04-08 | 广东威创视讯科技股份有限公司 | Image-processing method |
| WO2014198319A1 (en) * | 2013-06-13 | 2014-12-18 | M-Way Solutions Gmbh | Method for mobile internet access and system therefor |
| CN111182079A (en) * | 2020-01-06 | 2020-05-19 | 深圳市同洲电子股份有限公司 | Universal system for screenshot and desktop video recording |
| CN112799891A (en) * | 2021-01-15 | 2021-05-14 | 网易（杭州）网络有限公司 | iOS device test method, device, system, storage medium and computer device |
-
2021
- 2021-10-26 CN CN202111249719.5A patent/CN113973207A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404768A (en) * | 2008-10-07 | 2009-04-08 | 广东威创视讯科技股份有限公司 | Image-processing method |
| WO2014198319A1 (en) * | 2013-06-13 | 2014-12-18 | M-Way Solutions Gmbh | Method for mobile internet access and system therefor |
| CN111182079A (en) * | 2020-01-06 | 2020-05-19 | 深圳市同洲电子股份有限公司 | Universal system for screenshot and desktop video recording |
| CN112799891A (en) * | 2021-01-15 | 2021-05-14 | 网易（杭州）网络有限公司 | iOS device test method, device, system, storage medium and computer device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115037732A (en) * | 2022-06-01 | 2022-09-09 | 中国电力科学研究院有限公司 | Method, device, equipment and medium for remote real machine debugging through streaming media |
| CN115037732B (en) * | 2022-06-01 | 2024-04-23 | 中国电力科学研究院有限公司 | Method, device, equipment and medium for remote real machine debugging through streaming media |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9609275B2 (en) | Single-stream transmission method for multi-user video conferencing | |
| CA2737728C (en) | Low latency video encoder | |
| US9635374B2 (en) | Systems and methods for coding video data using switchable encoders and decoders | |
| WO2016126749A1 (en) | Video decoding | |
| US11647223B2 (en) | Dynamic motion vector referencing for video coding | |
| US10999599B2 (en) | System and method for non-uniform video coding | |
| CN102291561A (en) | Reducing use of periodic key frames in video conferencing | |
| JP2005522084A (en) | Coding transform coefficient in image / video encoder and / or decoder | |
| CN104144322A (en) | Method and system for achieving video monitoring on mobile terminal and video processing server | |
| CN112073737A (en) | Re-encoding predicted image frames in live video streaming applications | |
| CN115361582B (en) | Video real-time super-resolution processing method, device, terminal and storage medium | |
| CN113973207A (en) | Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream | |
| CN115514960A (en) | Video coding method and device, electronic equipment and storage medium | |
| CN114339263A (en) | Lossless processing method for video data | |
| CN111757126A (en) | VR-oriented real-time image compression method, system and storage medium | |
| CN103918258A (en) | Reducing amount of data in video encoding | |
| CN117692650A (en) | Image encoding/decoding method, electronic device, and computer-readable storage medium | |
| CN111182310A (en) | Video processing method and device, computer readable medium and electronic equipment | |
| US20060133491A1 (en) | Video codec | |
| CN110798705A (en) | Video image transmission method, device and equipment | |
| CN112672147A (en) | Decoding method, device and system based on screen projection | |
| WO2009122925A1 (en) | Dynamic image conversion device, dynamic image delivery system, method for converting dynamic image and program | |
| US20150078433A1 (en) | Reducing bandwidth and/or storage of video bitstreams | |
| CN117676266A (en) | Video stream processing method and device, storage medium and electronic equipment | |
| CN101765014B (en) | Device and method for generating re-sampling reference picture and image-decoding system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |