CN113660409A

CN113660409A - Image processing method and device, computer equipment and storage medium

Info

Publication number: CN113660409A
Application number: CN202010399363.2A
Authority: CN
Inventors: 张安龙
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2020-05-12
Filing date: 2020-05-12
Publication date: 2021-11-16

Abstract

The application discloses an image processing method and device, computer equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a rotated image frame and a rotation angle of the rotated image frame, wherein the rotation angle represents an angle of the image frame acquired by an image acquisition device rotating relative to a reference direction of the image acquisition device; carrying the rotation angle in a reserved field of the encoded data of the rotated image frame; and sending the encoded data carrying the rotation angle to an image server so that the image server can provide image frames for an image display. The image server or the image display can correct the received rotated image frame conveniently according to the rotation angle.

Description

Image processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, a computer device, and a storage medium.

Background

With the development of terminal technology, more and more scenes are acquired by adopting the terminal. Moreover, when the terminal is used for collecting images, different shooting angles may be adopted for shooting different scenes, so that the shot images have different rotation angles. At this time, if the acquired image is directly sent to other terminals, the images seen by the other terminals may have different rotation angles, and even the situation that the images cannot be normally viewed occurs, so that the user experience is poor.

Disclosure of Invention

The application provides an image processing method and device, computer equipment and a storage medium, which can normally check a high-resolution image after being coded on the basis of not changing coding hardware of a terminal.

In a first aspect, an image processing method is provided, where the method is applied to an image collector, and the method includes: acquiring the rotated image frame and a rotation angle of the rotated image frame, wherein the rotation angle represents an angle of the image frame acquired by the image acquisition device rotating relative to a reference direction of the image acquisition device; carrying the rotation angle in a reserved field of the encoded data of the rotated image frame; and sending the encoded data carrying the rotation angle to an image server so that the image server can provide image frames for an image display.

The image acquisition device comprises a rotation angle acquisition module, an image server and an image display module, wherein the rotation angle acquisition module is used for acquiring a rotation angle of a rotated image and a rotated image frame, carrying the rotation angle in a reserved field of encoded data of the rotated image frame, and then sending the encoded data carrying the rotation angle to the image server so as to provide the image frame acquired by the image acquisition module for the image display module. Compared with the related art, the image server or the image display can correct the received rotated image frame according to the rotation angle by carrying the rotation angle of the image frame in the reserved field of the encoded data.

Optionally, the width of the image frame is smaller than the height of the image frame, and the encoding capability of the image collector on the image width is greater than that of the image collector on the image height.

At the moment, the image collected by the image collector is rotated, the rotated image frame is coded, the coding capability of the image collector on the image width is used for coding in the height direction of the image collected by the image collector, and the coding capability of the image collector on the image height is used for coding in the width direction of the image collected by the image collector, so that the matching degree of the coding capability of the image collector and the dimensionality of the image is improved, and therefore, on the premise of not changing the coding hardware of the terminal, the accuracy of the coded image frame is improved, and the coded image frame can be normally checked.

Optionally, before sending the encoded data carrying the rotation angle to the image server, the method further includes: and carrying an angle use identifier in a reserved field of the encoded data, wherein the angle use identifier is used for indicating whether the reserved field carries the rotation angle or not. Correspondingly, the method for sending the encoded data carrying the rotation angle to the image server includes: and sending the coded data carrying the rotation angle and the angle use identifier to an image server.

In one implementation, the rotation angle can be represented using one or more bits of binary data.

For example, binary data 00 is used to indicate a rotation angle of 0 degrees clockwise; binary data 01 is adopted to represent the rotation angle of 90 degrees clockwise rotation; binary data 10 is used to represent the rotation angle of 180 degrees clockwise; binary data 11 is used to indicate a rotation angle of 270 degrees clockwise.

Optionally, the code stream format of the encoded data of the rotated image frame includes: the format of appendix B, byte alignment format or header description format, and the reserved field is reserved field in code stream of the coded data.

Optionally, the image frame includes: image frames of 4K resolution.

In a second aspect, an image processing method is provided, which is applied to an image server, and includes: receiving encoded data of the rotated image frame sent by the image collector, wherein a reserved field of the encoded data carries a rotation angle of the rotated image frame, and the rotation angle represents an angle of the image frame collected by the image collector rotating relative to a reference direction of the image collector; rotating the rotated image frame based on the rotation angle to obtain an image frame after being rotated; and sending the corrected image frame to an image display.

The image acquisition device receives the coded data of the rotated image frame sent by the image acquisition device, rotates the rotated image frame based on the rotation angle to obtain a corrected image frame, and sends the corrected image frame to the image display device so that the image display device can display the image frame acquired by the image acquisition device. Compared with the related art, the reserved field of the encoded data carries the rotation angle of the rotated image frame, so that the image server can correct the received rotated image frame according to the rotation angle.

In addition, the image server can provide image frames for the plurality of image displays, and the image frames are transferred in the image server, so that the image frames do not need to be transferred in the image displays, and resources consumed by transferring the image frames of each image display in the plurality of image displays can be reduced. Further, since the image server is generally higher in configuration than the image display, the image server can correct the image frame, thereby improving the processing efficiency and processing speed for correcting the image frame. In addition, if the image display is required to correct the image frame, a processing device for identifying the rotation angle and correcting the image frame is required to be arranged in the image display, which increases the arrangement cost of the image display, and if the processing device for identifying the rotation angle and correcting the image frame is not arranged in the image display, the use range of the image frame is limited, so that the image frame can be ensured to be applied to more scenes by correcting the image frame in the image server.

Optionally, the reserved field of the encoded data further carries an angle usage identifier, and the angle usage identifier is used to indicate whether the reserved field of the encoded data carries a rotation angle. At this time, rotating the rotated image frame based on the rotation angle includes: and when the reserved field of the angle use identification indication coded data carries the rotation angle, rotating the rotated image frame based on the rotation angle.

Optionally, the image frame includes: image frames of 4K resolution.

In a third aspect, there is provided an image processing apparatus comprising: the acquisition module is used for acquiring the rotated image frames and the rotation angles of the rotated image frames, wherein the rotation angles represent the rotation angles of the image frames acquired by the image acquisition device relative to the reference direction of the image acquisition device; the processing module is used for carrying the rotation angle in a reserved field of the coded data of the rotated image frame; and the sending module is used for sending the coded data carrying the rotation angle to the image server so that the image server can provide image frames for the image display.

Optionally, the processing module is further configured to carry the angle usage identifier in a reserved field of the encoded data, where the angle usage identifier is used to indicate whether the reserved field carries the rotation angle; and the sending module is specifically used for sending the coded data carrying the rotation angle and the angle use identifier to the image server.

Alternatively, the rotation angle is represented by one or more bits of binary data.

Optionally, binary data 00 is used to represent a rotation angle of 0 degree clockwise rotation; binary data 01 is adopted to represent the rotation angle of 90 degrees clockwise rotation; binary data 10 is used to represent the rotation angle of 180 degrees clockwise; binary data 11 is used to indicate a rotation angle of 270 degrees clockwise.

Optionally, the image frame includes: image frames of 4K resolution.

In a fourth aspect, there is provided an image processing apparatus comprising: the receiving module is used for receiving encoded data of the rotated image frame sent by the image collector, wherein a reserved field of the encoded data carries a rotation angle of the rotated image frame, and the rotation angle represents an angle of the image frame collected by the image collector rotating relative to a reference direction of the image collector; the processing module is used for rotating the rotated image frame based on the rotation angle to obtain a corrected image frame; and the sending module is used for sending the corrected image frame to the image display.

Optionally, the reserved field of the encoded data further carries an angle usage identifier, where the angle usage identifier is used to indicate whether the reserved field of the encoded data carries a rotation angle. At this time, the processing module is specifically configured to: and when the reserved field of the angle use identification indication coded data carries the rotation angle, rotating the rotated image frame based on the rotation angle.

Optionally, the image frame includes: image frames of 4K resolution.

In a fifth aspect, there is provided a first computer device comprising: the system comprises a first processor and a first memory, wherein the first memory stores a computer program; the first computer device implements the method of the first aspect when the first processor executes the computer program.

In a sixth aspect, there is provided a second computer device comprising: a second processor and a second memory, the second memory having stored therein a computer program; the second computer device implements the method of the second aspect when the computer program is executed by the second processor.

In a seventh aspect, a first storage medium is provided, in which instructions, when executed by a processor, implement the method of the first aspect.

In an eighth aspect, there is provided a second storage medium having instructions which, when executed by a processor, implement the method of the second aspect.

In a ninth aspect, there is provided a first computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the first aspect.

In a tenth aspect, there is provided a second computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the second aspect.

Drawings

Fig. 1 is a schematic diagram of an application scenario involved in an image processing method provided in an embodiment of the present application;

fig. 2 is a schematic diagram of an image frame displayed in an image collector according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of an image frame in a video stream output by an image collector according to an embodiment of the present disclosure;

fig. 4 is a flowchart of an image processing method provided in an embodiment of the present application;

FIG. 5 is a flow chart of another image processing method provided by the embodiments of the present application;

fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of another image processing apparatus provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a first computer device according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of a second computer device according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Moreover, when the resolution of the image collected by the terminal is high, but the processing capability of the coding hardware of the terminal is limited, the application range of the high-resolution image collected by the terminal is limited. For example, when the image captured by the terminal is an image with a resolution of 4K (e.g. 2160 × 3840), and the height of the image that can be processed by the encoding hardware of the terminal is less than 3840, the image encoded by the encoding hardware of the terminal may have problems such as screen splash, and the encoded image cannot be normally viewed. Wherein, the resolution x × y of the image refers to: the width of the image is x and the height of the image is y. Where 4K resolution refers to a resolution at or near 4096 pixel values per row in the horizontal direction.

The embodiment of the application provides an image processing method, wherein a rotated image and a rotation angle of a rotated image frame are obtained, the rotation angle is carried in a reserved field of coded data of the rotated image frame, and the coded data with the rotation angle is sent to an image server so that the image server can provide the image frame collected by an image collector for an image display. Compared with the related art, the image server or the image display can correct the received rotated image frame according to the rotation angle by carrying the rotation angle of the image frame in the reserved field of the encoded data.

And when the coding capability of the image collector on the image width is greater than that of the image collector on the image height and the width of the image collected by the image collector is less than that of the image, the image collected by the image collector is rotated and the rotated image frame is coded, the coding capability of the image collector on the image width is used for coding in the height direction of the image collected by the image collector, and the coding capability of the image collector on the image height is used for coding in the width direction of the image collected by the image collector, so that the matching degree of the coding capability of the image collector and the dimensionality of the image is improved, and therefore, on the premise of not changing coding hardware of a terminal, the accuracy of the coded image frame is improved, and the coded image frame can be normally viewed.

For example, when the acquired image is an image with a resolution of 4K (such as 2160 × 3840), and the height of the image that can be processed by the encoding hardware of the image acquirer is less than 3840 (such as 2500), the image is rotated by 90 degrees clockwise, so that the height of the image that needs to be processed by the encoding hardware of the image acquirer becomes 2160, the degree of matching between the encoding capability of the image acquirer and the dimension of the image is improved, and the accuracy of the encoded image frame is improved. And the reserved field of the coded data of the image carries the rotation angle of the image, so that the image server or the image display can correct the received rotated image frame according to the rotation angle, and the image display can normally view the coded image.

Fig. 1 is an application scenario related to an image processing method according to an embodiment of the present application. As shown in fig. 1, the application scenario includes: an image collector 01, an image server 02 and an image display 03. A communication connection is established between the image collector 01 and the image server 02. A communication connection is established between the image server 02 and the image display 03. It should be noted that there may be one or more image collectors 01 connected to the image server 02, and similarly, there may be one or more image displays 03 connected to the image server 02, and fig. 1 is an example in which one image collector 01 is connected to the image server 02, and one image display 03 is connected to the image server 02.

The image collector 01 is configured to collect image frames, acquire the rotated image frames and rotation angles of the rotated image frames relative to a reference direction of the image collector 01, encode the rotated image frames, carry the rotation angles of the rotated image frames in reserved fields of encoded data of the rotated image frames, and send the encoded data carrying the rotation angles to the image server 02, so that the image server 02 provides the image frames collected by the image collector 01 for the image display 03.

The image frame may be an image frame in a picture or a video. Also, the rotation angle of the image frame may be caused by a photographing angle of the image pickup device 01. For example, when an image is captured using a landscape of a mobile terminal, a 90-degree rotation of the captured image may occur. Alternatively, the rotation angle of the image frame may be caused by processing the image frame in the process of outputting the captured image frame by the image capturing device 01. For example, when the image capturing device 01 is used to record a video, the image frames in the video stream output by the image capturing device 01 are usually image frames that are rotated with respect to the image frames displayed in the image capturing device 01. The image frames displayed in the image collector 01 are as shown in fig. 2, and the image frames displayed in the image collector 01 are image frames displayed upright. The image frames in the video stream output by the image collector 01 are as shown in fig. 3, and the image frames in the video stream output by the image collector 01 are the image frames displayed in the image collector 01 and rotated by 90 degrees counterclockwise.

The image server 02 is configured to rotate the rotated image frame based on a rotation angle carried in a reserved field of the encoded data after receiving the encoded data of the rotated image frame sent by the image collector 01, to obtain a corrected image frame, and send the corrected image frame to the image display 03. Alternatively, the image server 02 is configured to receive the encoded data of the rotated image frame transmitted by the image collector 01, and then transmit the encoded data of the rotated image frame to the image display 03.

The image display 03 is configured to display the corrected image frame after receiving the corrected image frame sent by the image server 02. Or, the image display 03 is configured to rotate the rotated image frame based on the rotation angle carried in the reserved field of the encoded data after receiving the encoded data of the rotated image frame sent by the image server 02, to obtain a corrected image frame, and display the corrected image frame.

Alternatively, the image server 02 may be a server, a server cluster composed of several servers, or a cloud computing service center. The image collector 01 may be a mobile phone, a smart phone, a tablet computer, a multimedia player, a smart appliance, an artificial intelligence device, a smart wearable device, a Digital Video (DV), a laptop computer, a desktop computer, or the like having an image collecting function. The image display 03 may be a device having an image display function, such as a mobile phone, a smart phone, a tablet computer, a multimedia player, a smart appliance, an artificial intelligence device, a smart wearable device, a digital video, a laptop computer, a desktop computer, an electronic reader, a smart car device, a smart appliance, an artificial intelligence device, a wearable device, an internet of things device, or a virtual reality/augmented reality/mixed reality device.

The following describes an implementation process of an image processing method provided in an embodiment of the present application. As shown in fig. 4, the implementation process of the network access method may include the following steps:

step 401, an image collector collects image frames.

The image collector can use a camera to take pictures or record videos to obtain image frames. In addition, the image collector can also be provided with a display screen which can display the collected image frames so that a user can preview the collected image frames. For example, a surface view (surface view) software may be configured in the image collector, and the image collector may display the collected image frames in a display screen through the surface view software.

And, when the configuration of the image collector is different, the resolution of the collected image frame is different. For example, the resolution of the image frames acquired by the image acquirer may include: 4K resolution, 720P resolution, or other resolutions, which are not specifically limited in this application.

Step 402, the image collector obtains the rotated image frame and the rotation angle of the rotated image frame.

The rotation angle represents an angle at which an image frame captured by the image capture device rotates with respect to a reference direction of the image capture device.

The rotation angle of the image frame may be caused by a photographing angle of the image collector. For example, when an image is captured using a landscape of a mobile terminal, a 90-degree rotation of the captured image may occur. At this time, the angle at which the photographing angle of the image capturing device is rotated with respect to the reference direction of the image capturing device is a rotation angle, that is, the angle rotated from the reference direction of the image capturing device to the photographing angle of the image capturing device is a rotation angle. Therefore, the rotation angle of the rotated image frame can be obtained by acquiring the shooting angle of the image collector and comparing the shooting angle with the reference direction. The reference direction of the image collector can be determined according to application requirements, and the image collector is not specifically limited in the embodiment of the application. For example, the reference direction of the image collector may be a direction pointing to the earth center from the image collector when the image collector is placed upright.

Alternatively, the rotation angle of the image frame may be caused by processing the image frame in the process of outputting the acquired image frame by the image acquisition device. For example, when an image capturing device is used to record a video, the image frames in the video stream output by the image capturing device are usually image frames that are rotated with respect to the image frames displayed in the image capturing device. At this time, the angle of the image frame rotated by the image collector in the process of processing the image frame is the rotation angle. Therefore, the rotation angle of the rotated image frame can be obtained by obtaining the rotation angle of the image frame in the process of processing the image frame by the image collector.

For example, it is assumed that an image frame acquired by an image acquirer is as shown in fig. 2, and an obtained image frame is as shown in fig. 3 through a process of outputting the acquired image frame by the image acquirer. As can be seen from fig. 2 and 3, the image frame is rotated 90 degrees counterclockwise, that is, 270 degrees clockwise. Assuming that the reference direction of the image acquirer is a 0 degree direction, it may be determined that the rotation angle of the rotated image frame is 90 degrees counterclockwise or 270 degrees clockwise.

And step 403, the image collector carries the rotation angle in a reserved field of the encoded data of the rotated image frame.

Optionally, encoding technologies such as h.264 and High Efficiency Video Coding (HEVC) may be adopted to encode the rotated image frame, and the encoded data is packed in a packing manner such as an Annex B (Annex-B) format, a byte Alignment (AVCC) format, or a header description (HVCC) format, so as to obtain a code stream of the packed encoded data.

The encoded packet is called a Network Abstraction Layer Unit (NALU). The data packed in different formats can be distinguished by different data formats. For example, when packing encoded data using the Annex-B format, the NALU is preceded by a start code of 0x000001 or 0x00000001 of three bytes or four bytes, and thus, if the NALU of the packed data is preceded by a start code of 0x000001 or 0x00000001 of three bytes or four bytes, it can be determined that the data is packed using Annex-B.

Correspondingly, the reserved field for carrying the rotation angle is a reserved field in the packed code stream. When the data of the image frame is packed by using different packing methods, the position of the reserved field in the coded data is different. Referring to tables 1 and 2, the positions of reserved fields for carrying rotation angles in the encoded data will be described below by taking AVCC format and HVCC format as examples.

TABLE 1

Number of bits	Means of	Description of the related Art
			8	Version (version)	0x01
8	Advanced video coding profile (avc profile)	Sequence parameter set (sps) [1 ]]
			8	avc compatibility (avc compatibility)	sps[2]
8	av level (av level)	sps[3]
			6	Reserved field (reserved)	Default set to '111111'
2	NALU prefix length minus one (NALULENGThSizeMinusOne)
			3	Reserved field (reserved)	Default to '111'
5	Number of SPS NALUs (number of SPS NALUs)	Is usually 1
			16	SPS size (SPS size)
N	Variable SPS NALU data (variable SPS NALU) data
			8	Number of picture parameter sets NALU (number of PPS NALUs)	Is usually 1
16	PPS size (PPS size)
			N	Variable PPS NALU data (variable PPS NALU data)

Table 1 shows the size, meaning and related description of each field in the codestream in AVCC format. Where the size of the field is expressed in terms of the number of bits occupied by the field. As can be seen from table 1, the data packed using the AVCC format includes two reserved fields, and either or both of the two reserved fields may be used to carry the rotation angle.

TABLE 2

Table 2 shows the size, meaning and related description of the various fields in the HVCC format. Where the size of the field is expressed in terms of the number of bits occupied by the field. As can be seen from table 2, the data packed using the HVCC format includes five reserved fields, and any one or more of the five reserved fields may be used to carry the rotation angle.

Alternatively, the rotation angle is represented by one or more bits of binary data. The total number of bits of binary data for representing the rotation angle may be determined according to the number of angles to be represented. For example, the rotation angle of an image frame captured by the image capturing device may be 2^NIn this case, the total number of bits of the binary data for representing the rotation angle is N, and one binary data represented by N-bit binary data can be used to represent one rotation angle.

For example, when the rotation angle of the image frame captured by the image capturing device may be one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, the rotation angle may be represented by a binary number, and different binary data composed of the binary number may represent one angle. For example, the rotation angle of 0 degree clockwise is represented by binary data 00, the rotation angle of 90 degrees clockwise is represented by binary data 01, the rotation angle of 180 degrees clockwise is represented by binary data 10, and the rotation angle of 270 degrees clockwise is represented by binary data 11. Accordingly, as per the example in step 402 and as can be seen in fig. 3, 11 may be carried in the reserved field of the encoded data to indicate that the rotation angle of the rotated image frame is 270 degrees clockwise.

And step 404, the image collector sends the encoded data carrying the rotation angle to an image server.

After the image collector carries the rotation angle in the reserved field, the encoded data carrying the rotation angle can be sent to the image server, so that the image server can provide the image frame for the image display according to the encoded data carrying the rotation angle, and the image frame can be displayed in the image display.

It should be noted that, before the encoded data carrying the rotation angle is sent to the image server, the encoded data needs to be encapsulated. Moreover, the encoding data can be encapsulated by adopting an encapsulation format supporting the encapsulation mode of the encoding data so as to ensure the accuracy of the data. For example, when the encoded data is packed in a format such as AVCC or HVCC, the encoded data may be packed in a packing format such as streaming media (FLV) that supports the format such as AVCC or HVCC.

TABLE 3

For example, when the coded data is transmitted by using a Real Time Messaging Protocol (RTMP), the image frame may be encapsulated by using an FLV encapsulation format, and at this Time, an FLV encapsulation header may be added before the coded data of the image frame to indicate that the image frame is encapsulated by using the FLV encapsulation format. The format of the encoded data of the image frame added with the FLV encapsulation header is shown in table 3, where the first column of table 3 represents the size (in terms of bits) of each field, the second column represents the meaning of each field, and 0 to 15 bits are used for representing the FLV encapsulation header, and the rest bits are used for representing the encoded data of the image frame.

Step 405, the image server rotates the rotated image frame based on the rotation angle to obtain the corrected image frame.

After receiving the encoded data carrying the rotation angle, the image server may decode the encoded data, obtain a rotated image frame and the rotation angle thereof according to the decoded data, and perform a correction process on the rotated image frame according to the rotation angle to obtain a corrected image. For example, still taking the example in step 403 as an example, after the image server receives 11 carried in the reserved field of the encoded data, it can be determined that the rotated angle of the image frame is rotated clockwise 270, and then the image server may rotate the rotated image frame counterclockwise by 270 degrees to obtain the image frame after being rotated in the right direction as shown in fig. 2.

Step 406, the image server sends the corrected image frame to the image display.

After the image server obtains the corrected image frame, the image frame can be sent to the image display through a distribution network or a forwarding network, so that the image display can display the image frame. And, before sending the image frame to the image display, the image server may perform encoding, packing, and encapsulating operations on the image frame. The foregoing contents may be referred to correspondingly for implementation of the image server performing operations such as encoding, packaging, and encapsulating on the image frame, and are not described herein again.

Step 407, the image display displays the corrected image frame.

The image display can directly display the image frame after receiving the corrected image frame for the user to view. Therefore, the image frames displayed on the image display are the same as the image frames displayed by the image collector, and the user experience can be guaranteed.

The image server can provide image frames for the plurality of image displays, and the image frames are transferred in the image server, so that the image frames do not need to be transferred in the image displays, and resources consumed by transferring the image frames of each image display in the plurality of image displays can be reduced. Further, since the image server is generally higher in configuration than the image display, the image server can correct the image frame, thereby improving the processing efficiency and processing speed for correcting the image frame. In addition, if the image display is required to correct the image frame, a processing device for identifying the rotation angle and correcting the image frame is required to be arranged in the image display, which increases the arrangement cost of the image display, and if the processing device for identifying the rotation angle and correcting the image frame is not arranged in the image display, the use range of the image frame is limited, so that the image frame can be ensured to be applied to more scenes by correcting the image frame in the image server.

It should be noted that the reserved field may also carry an angle usage identifier, where the angle usage identifier is used to indicate whether the reserved field carries a rotation angle. Accordingly, as shown in fig. 5, the image processing method further includes:

step 408, the image collector carries the angle use identifier in the reserved field of the encoded data.

Wherein the angle usage flag can be represented using binary data. For example, binary data 0 may be used to indicate that the reserved field carries the rotation angle, and binary data 1 may be used to indicate that the reserved field does not carry the rotation angle. Moreover, the implementation manner of the reserved field carrying the angle use identifier can correspondingly refer to the implementation manner of the carried rotation angle. It should be noted that the same reserved field may be used to carry the rotation angle and the angle usage identifier, or different reserved fields may be used to carry the rotation angle and the angle usage identifier, which is not specifically limited in the embodiment of the present application.

Because some reserved fields can be set as default values when not used, whether the reserved fields carry the rotation angle or not is indicated by using the angle use identification, the rotation angle can be distinguished from the default values in the reserved fields, the default values are prevented from being mistaken as the rotation angle, and therefore the image display can normally check the image frame.

Correspondingly, as shown in fig. 5, when the image processing method provided in the embodiment of the present application further includes step 408, the implementation process of step 404 includes: step 4041, the image collector sends the encoded data carrying the rotation angle and the angle usage identifier to the image server.

The implementation manner of step 4041 may refer to the description in step 404, which is not described herein again.

Correspondingly, as shown in fig. 5, when the image processing method provided in the embodiment of the present application further includes step 408, the implementation procedure of step 405 includes: step 4051, when the reserved field of the angle usage identification indication encoded data carries the rotation angle, the image server rotates the rotated image frame based on the rotation angle to obtain the corrected image frame.

When the reserved field also carries the angle use identifier, after receiving the encoded data sent by the image collector, the image server may first determine whether the reserved field carries the rotation angle according to the angle use identifier, and when it is determined that the reserved field carries the rotation angle, decode the encoded data to obtain the rotation angle and the rotated image frame, and rotate the rotated image frame according to the rotation angle to obtain the corrected image frame.

In the foregoing embodiments, the image processing method provided in the embodiment of the present application is described by taking an example in which the image server rotates the image frame according to the rotation angle and transmits the image frame after the rotation to the image display. However, in the embodiment of the present application, after the image server receives the encoded data carrying the rotation angle and the angle usage identifier, the encoded data carrying the rotation angle and the angle usage identifier may also be forwarded to the image display. In this way, the image display device can rotate the rotated image frame based on the rotation angle to obtain the corrected image frame and display the corrected image frame.

In summary, in the image processing method provided in the embodiment of the present application, the rotation angles of the rotated image and the rotated image frame are obtained, the rotation angles are carried in the reserved fields of the encoded data of the rotated image frame, and then the encoded data with the rotation angles are sent to the image server, so that the image server provides the image frame acquired by the image acquirer for the image display. Compared with the related art, the image server or the image display can correct the received rotated image frame according to the rotation angle by carrying the rotation angle of the image frame in the reserved field of the encoded data.

It should be noted that, the order of steps of the network access method provided in the embodiment of the present application may be appropriately adjusted, and the steps may also be correspondingly increased or decreased according to the situation. Any method that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application is covered by the protection scope of the present application, and thus the detailed description thereof is omitted.

The embodiment of the application also provides an image processing device. As shown in fig. 6, the image processing apparatus 60 includes:

the obtaining module 601 is configured to obtain a rotated image frame and a rotation angle of the rotated image frame, where the rotation angle represents an angle at which the image frame acquired by the image acquirer rotates relative to a reference direction of the image acquirer.

A processing module 602, configured to carry the rotation angle in a reserved field of encoded data of the rotated image frame.

The sending module 603 is configured to send the encoded data carrying the rotation angle to the image server, so that the image server provides the image frame to the image display.

Optionally, the processing module 602 is further configured to carry an angle usage identifier in a reserved field of the encoded data, where the angle usage identifier is used to indicate whether the reserved field carries a rotation angle.

The sending module 603 is specifically configured to send the encoded data carrying the rotation angle and the angle usage identifier to the image server.

Alternatively, binary data 00 is used to indicate a rotation angle of 0 degree clockwise. Binary data 01 represents a rotation angle of 90 degrees clockwise. Binary data 10 is used to indicate the rotation angle of 180 degrees clockwise. Binary data 11 is used to indicate a rotation angle of 270 degrees clockwise.

Optionally, the image frame includes: image frames of 4K resolution.

In summary, in the image processing apparatus provided in the embodiment of the present application, the obtaining module obtains the rotation angle of the rotated image and the rotated image frame, the processing module carries the rotation angle in the reserved field of the encoded data of the rotated image frame, and the sending module sends the encoded data carrying the rotation angle to the image server, so that the image server provides the image frame acquired by the image acquirer for the image display. Compared with the related art, the image server or the image display can correct the received rotated image frame according to the rotation angle by carrying the rotation angle of the image frame in the reserved field of the encoded data.

The embodiment of the application also provides another image processing device. As shown in fig. 7, the image processing apparatus 70 includes:

the receiving module 701 is configured to receive encoded data of the rotated image frame sent by the image collector, where a reserved field of the encoded data carries a rotation angle of the rotated image frame, and the rotation angle represents an angle at which the image frame collected by the image collector rotates relative to a reference direction of the image collector.

A processing module 702, configured to rotate the rotated image frame based on the rotation angle, so as to obtain a corrected image frame.

A sending module 703, configured to send the corrected image frame to an image display.

Optionally, the reserved field of the encoded data further carries an angle usage flag, and the angle usage flag is used to indicate whether the reserved field of the encoded data carries the rotation angle flag. At this time, the processing module 702 is specifically configured to: and when the reserved field of the angle use identification indication coded data carries the rotation angle, rotating the rotated image frame based on the rotation angle.

Optionally, the image frame includes: image frames of 4K resolution.

In summary, in the image processing apparatus provided in the embodiment of the application, the receiving module receives encoded data of a rotated image frame sent by the image collector, the processing module rotates the rotated image frame based on the rotation angle to obtain a rotated image frame, and the sending module sends the rotated image frame to the image display, so that the image display displays the image frame collected by the image collector. Compared with the related art, the reserved field of the encoded data carries the rotation angle of the rotated image frame, so that the image server can correct the received rotated image frame according to the rotation angle.

It is clear to those skilled in the art that, for convenience and brevity of description, the configuration and specific working process of the gatekeeper and the module described above may refer to the corresponding contents in the foregoing system embodiment and method embodiment, and are not described herein again.

The embodiment of the application provides a first computer device. Fig. 8 illustratively provides a possible architecture diagram for the first computer device. As shown in fig. 8, the first computer device 80 may include a first processor 801, a first memory 802, a first communication interface 803, and a first bus 804. In the first computer device, the number of the first processors 801 may be one or more, and fig. 8 illustrates only one of the first processors 801. Alternatively, the first processor 801 may be a Central Processing Unit (CPU). If the first computer device has a plurality of first processors 801, the plurality of first processors 801 may be of different types or may be the same type. Optionally, the plurality of first processors of the first computer device may also be integrated into a multi-core processor.

The first memory 802 stores computer instructions and data, and the first memory 802 may store computer instructions and data required to implement the functions of the image collector in the image processing method provided by the present application. The first memory 802 may be any one or any combination of the following storage media: nonvolatile Memory (e.g., Read-Only Memory (ROM), Solid State Disk (SSD), Hard Disk Drive (HDD), optical disc, etc., volatile Memory.

The first communication interface 803 may be any one or any combination of the following devices: network interface (such as Ethernet interface), wireless network card, etc.

The first communication interface 803 is used for data communication of the first computer device with other nodes or other computer devices.

Fig. 8 also illustratively depicts a first bus 804. A first bus 804 may connect the first processor 801 with the first memory 802, the first communication interface 803. Thus, through the first bus 804, the first processor 801 may access the first memory 802 and may also utilize the first communication interface 803 for data interaction with other nodes or other computer devices.

In the present application, the first computer device executes the computer instructions in the first memory 802, so as to implement the function of the image collector in the image processing method provided by the present application. For example, a first computer device executing computer instructions in first memory 802 may perform the following steps performed by an image collector: acquiring the rotated image frame and the rotation angle of the rotated image frame; carrying the rotation angle in a reserved field of the encoded data of the rotated image frame; and sending the encoded data carrying the rotation angle to an image server so that the image server can provide image frames for an image display. Moreover, the implementation process of the first computer device executing the steps executed by the image collector by executing the computer instructions in the first memory 802 may refer to the corresponding description in the above method embodiments.

The embodiment of the application provides a second computer device. FIG. 9 illustratively provides a possible architecture diagram for the second computer device. As shown in fig. 9, the second computer device 90 may include a second processor 901, a second memory 902, a second communication interface 903, and a second bus 904. In the second computer device, the number of the second processors 901 may be one or more, and fig. 9 illustrates only one of the second processors 901. Alternatively, the second processor 901 may be a central processor. If the second computer device has a plurality of second processors 901, the types of the plurality of second processors 901 may be different, or may be the same. Optionally, a plurality of second processors of the second computer device may also be integrated into a multi-core processor.

The second memory 902 stores computer instructions and data, and the second memory 902 may store computer instructions and data required to implement the functions of the image server in the image processing method provided by the present application. The second memory 902 may be any one or any combination of the following storage media: non-volatile memory (e.g., read only memory), solid state drives, hard disks, optical disks, etc., volatile memory.

The second communication interface 903 may be any one or any combination of the following devices: network interface (such as Ethernet interface), wireless network card, etc.

The second communication interface 903 is used for data communication of the second computer device with other nodes or other computer devices.

Fig. 9 also illustratively depicts a second bus 904. The second bus 904 may connect the second processor 901 with the second memory 902 and the second communication interface 903. Thus, through the second bus 904, the second processor 901 can access the second memory 902 and can also utilize the second communication interface 903 for data interaction with other nodes or other computer devices.

In the present application, the second computer device executes the computer instructions in the second memory 902, and can implement the function of the image server in the image processing method provided by the present application. For example, the second computer device executing the computer instructions in second memory 902 may perform the following steps performed by the image server: receiving encoded data of the rotated image frame sent by an image collector; rotating the rotated image frame based on the rotation angle carried in the reserved field of the encoded data to obtain a corrected image frame; and sending the corrected image frame to an image display. Also, the implementation process of the second computer device executing the steps executed by the image server by executing the computer instructions in the second memory 902 may refer to the corresponding description in the above method embodiments.

The embodiment of the present application further provides a first storage medium, where the first storage medium is a nonvolatile computer-readable storage medium, and when an instruction in the first storage medium is executed by a processor, the function implemented by the image collector in the image processing method provided in the embodiment of the present application is implemented.

The embodiment of the present application further provides a second storage medium, which is a nonvolatile computer-readable storage medium, and when instructions in the second storage medium are executed by a processor, the functions implemented by the image server in the image processing method provided in the embodiment of the present application are implemented.

The embodiment of the present application further provides a first computer program product containing instructions, and when the first computer program product runs on a computer, the computer is enabled to execute the functions implemented by the image collector in the image processing method provided by the embodiment of the present application.

The embodiment of the present application further provides a second computer program product containing instructions, which, when running on a computer, causes the computer to execute the functions implemented by the image server in the image processing method provided by the embodiment of the present application.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

In the embodiments of the present application, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The term "at least one" means one or more, and the term "plurality" means two or more, unless expressly defined otherwise.

The term "and/or" in this application is only one kind of association relationship describing the associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

The above description is only exemplary of the present application and is not intended to limit the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. An image processing method is applied to an image collector and comprises the following steps:

acquiring a rotated image frame and a rotation angle of the rotated image frame, wherein the rotation angle represents an angle of the image frame acquired by the image acquisition device rotating relative to a reference direction of the image acquisition device;

carrying the rotation angle in a reserved field of encoded data of the rotated image frame;

and sending the encoded data carrying the rotation angle to an image server so that the image server can provide the image frame for an image display.

2. The method according to claim 1, wherein before the sending the encoded data carrying the rotation angle to an image server, the method further comprises:

carrying an angle use identifier in a reserved field of the encoded data, wherein the angle use identifier is used for indicating whether the reserved field carries the rotation angle or not;

the sending the encoded data carrying the rotation angle to an image server includes:

and sending the coded data carrying the rotation angle and the angle use identifier to the image server.

3. A method according to claim 1 or 2, wherein the rotation angle is represented by one or more bits of binary data.

4. The method of claim 3,

binary data 00 is used for representing the rotation angle of clockwise rotation of 0 degree;

binary data 01 is adopted to represent the rotation angle of 90 degrees clockwise rotation;

binary data 10 is used to represent the rotation angle of 180 degrees clockwise;

binary data 11 is used to indicate a rotation angle of 270 degrees clockwise.

5. The method according to any one of claims 1 to 4, wherein the code stream format of the encoded data of the rotated image frame comprises: and in annex B format, byte alignment format or header description format, the reserved field is a reserved field in the code stream of the encoded data.

6. The method of any of claims 1 to 5, wherein the image frames comprise: image frames of 4K resolution.

7. An image processing method, applied to an image server, the method comprising:

receiving encoded data of a rotated image frame sent by an image collector, wherein a reserved field of the encoded data carries a rotation angle of the rotated image frame, and the rotation angle represents an angle of the image frame collected by the image collector rotating relative to a reference direction of the image collector;

rotating the rotated image frame based on the rotation angle to obtain a corrected image frame;

and sending the corrected image frame to an image display.

8. The method according to claim 7, wherein the reserved field of the encoded data further carries an angle usage identifier, the angle usage identifier is used to indicate whether the reserved field of the encoded data carries the rotation angle, and the rotating the rotated image frame based on the rotation angle includes:

and when the angle use identifier indicates that the reserved field of the encoded data carries the rotation angle, rotating the rotated image frame based on the rotation angle.

9. The method of claim 7 or 8, wherein the image frames comprise: image frames of 4K resolution.

10. An image processing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring the rotated image frames and the rotation angles of the rotated image frames, wherein the rotation angles represent the rotation angles of the image frames acquired by the image acquirer relative to the reference direction of the image acquirer;

the processing module is used for carrying the rotation angle in a reserved field of the coded data of the rotated image frame;

and the sending module is used for sending the coded data carrying the rotation angle to an image server so that the image server can provide the image frame for an image display.

11. The apparatus of claim 10,

the processing module is further configured to carry an angle usage identifier in a reserved field of the encoded data, where the angle usage identifier is used to indicate whether the reserved field carries the rotation angle;

the sending module is specifically configured to send the encoded data carrying the rotation angle and the angle usage identifier to the image server.

12. The apparatus according to claim 10 or 11, wherein the rotation angle is represented by one or more bits of binary data.

13. The apparatus of claim 12,

binary data 11 is used to indicate a rotation angle of 270 degrees clockwise.

14. The apparatus according to any one of claims 10 to 13, wherein a code stream format of the encoded data of the rotated image frame comprises: and in annex B format, byte alignment format or header description format, the reserved field is a reserved field in the code stream of the encoded data.

15. The apparatus of any of claims 10 to 14, wherein the image frames comprise: image frames of 4K resolution.

16. An image processing apparatus, characterized in that the apparatus comprises:

the receiving module is used for receiving encoded data of the rotated image frame sent by the image collector, wherein a reserved field of the encoded data carries a rotation angle of the rotated image frame, and the rotation angle represents an angle of the image frame collected by the image collector rotating relative to a reference direction of the image collector;

the processing module is used for rotating the rotated image frame based on the rotation angle to obtain a corrected image frame;

and the sending module is used for sending the corrected image frame to an image display.

17. The apparatus according to claim 16, wherein the reserved field of the encoded data further carries an angle usage identifier, and the angle usage identifier is used to indicate whether the reserved field of the encoded data carries the rotation angle, and the processing module is specifically configured to:

18. The apparatus of claim 16 or 17, wherein the image frames comprise: image frames of 4K resolution.

19. A first computer device, wherein the first computer device comprises: a first processor and a first memory, the first memory having a computer program stored therein; the first computer device implementing the method of any one of claims 1 to 6 when the first processor executes a computer program.

20. A second computer device, the second computer device comprising: a second processor and a second memory, the second memory having a computer program stored therein; the second computer device implementing the method of any one of claims 7 to 9 when the second processor executes the computer program.

21. A first storage medium, wherein instructions in the first storage medium, when executed by a processor, implement the method of any of claims 1 to 6.

22. A secondary storage medium, wherein instructions in the secondary storage medium, when executed by a processor, implement the method of any of claims 7 to 9.