[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN112214188B - Image processing method, device, storage medium and processor - Google Patents

Image processing method, device, storage medium and processor Download PDF

Info

Publication number
CN112214188B
CN112214188B CN202010956380.1A CN202010956380A CN112214188B CN 112214188 B CN112214188 B CN 112214188B CN 202010956380 A CN202010956380 A CN 202010956380A CN 112214188 B CN112214188 B CN 112214188B
Authority
CN
China
Prior art keywords
area
refresh
refreshing
region
areas
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010956380.1A
Other languages
Chinese (zh)
Other versions
CN112214188A (en
Inventor
王成东
卢涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Wanxiang Electronics Technology Co Ltd
Original Assignee
Xian Wanxiang Electronics Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Wanxiang Electronics Technology Co Ltd filed Critical Xian Wanxiang Electronics Technology Co Ltd
Priority to CN202010956380.1A priority Critical patent/CN112214188B/en
Publication of CN112214188A publication Critical patent/CN112214188A/en
Application granted granted Critical
Publication of CN112214188B publication Critical patent/CN112214188B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G06F3/1462Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/67Circuits for processing colour signals for matrixing

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an image processing method, an image processing device, a storage medium and a processor. The method comprises the following steps: acquiring first color data of a current frame image and second color data of a previous frame image; determining at least one first merge region of the first color data and at least one second merge region of the second color data; determining at least one variation region of the at least one first merge region relative to the at least one second merge region; and converting the image data in the change area from the first color format to the second color format, and sending the image data in the second color format to the client, wherein the image data in the second color format is used for refreshing the image displayed on the client. The invention solves the technical problem of resources consumed in image refreshing.

Description

Image processing method, device, storage medium and processor
Technical Field
The present invention relates to the field of image processing, and in particular, to an image processing method, apparatus, storage medium, and processor.
Background
Currently, the image transmission system comprises a client and an acquisition end, wherein the acquisition end acquires images from image source equipment and sends the images to the client, and the images are displayed to a user by the client. When the image of the image source equipment changes, the image source equipment needs to be acquired again through the acquisition end to refresh the client side picture. The method comprises the steps that an acquisition end determines a refreshing area (original area) of an image in image source equipment, the acquisition end acquires an entire screen image of a current frame, the format of the acquired entire screen image of the current frame is converted, data of an area which is actually changed in the original area is determined based on the original area, data of the entire screen image of the current frame after the conversion and data of the entire screen image of a previous frame after the conversion, then the acquisition end sends the data of the area which is actually changed to a client, and the client refreshes a display image according to the data of the area which is actually changed.
However, in a general case, when a screen in an image source device changes more severely, for example, a user maximizes or minimizes a window, drags a window, plays a video, or opens a web page, a large number of initial areas are generated, so that the workload of determining the initial areas in an image is large; in addition, the data conversion format of the whole screen image (of the current frame and the previous frame) also consumes more resources, so that the waste of calculation resources is caused.
In view of the above-described problem of resources consumed in performing image refresh, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides an image processing method, an image processing device, a storage medium and a processor, which are used for at least solving the technical problem of resources consumed in image refreshing.
According to an aspect of an embodiment of the present invention, there is provided a method of processing an image. The method may include: acquiring first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by image source equipment; determining at least one first merging region of the first color data and at least one second merging region of the second color data, wherein the first merging region is obtained by merging a plurality of first refreshing regions of the first color data and does not comprise a blank region except the plurality of first refreshing regions, and the second merging region is obtained by merging a plurality of second refreshing regions of the second color data and does not comprise a blank region except the plurality of second refreshing regions; determining at least one variation region of the at least one first merge region relative to the at least one second merge region; and converting the image data in the change area from the first color format to the second color format, and sending the image data in the second color format to the client, wherein the image data in the second color format is used for refreshing the image displayed on the client.
Optionally, converting the image data in the change region from the first color format to the second color format includes: determining an effective area based on the at least one variation area; the image data of the active area is converted from a first color format to a second color format.
Optionally, determining the effective area based on the at least one change area includes: merging the at least one change area to obtain at least one rectangular area; at least one rectangular region is determined as an effective region.
Optionally, the active area comprises one change area or a plurality of interconnected change areas.
Optionally, determining the effective area based on the at least one change area includes: and determining a change area at the uppermost layer in the at least one change area as an effective area, wherein the change area at the uppermost layer covers the change areas except the change area at the uppermost layer in the at least one change area.
Optionally, each first merge region includes at least two first refresh regions that overlap each other, or at least two first refresh regions that are connected to each other, or one first refresh region that is not connected to or overlaps any other first refresh region; each second merge region includes at least two second refresh regions that overlap each other, or at least two second refresh regions that are connected to each other, or one first refresh region that is not connected to or overlaps any other second refresh region.
Optionally, the image data in the overlapping area of the at least two first refresh areas that are overlapped is the image data of one first refresh area of the at least two first refresh areas; the image data in the overlapping region of the at least two second refresh regions superimposed is the image data of one of the at least two second refresh regions.
Optionally, determining at least one first merge area of the first color data includes: respectively determining first nodes corresponding to a plurality of corner points of each first refreshing area on a first coordinate axis and a second coordinate axis to obtain a plurality of first nodes; determining a first line segment of each first refreshing region based on a first node corresponding to a first coordinate axis in the first nodes, and determining a second line segment of each first refreshing region based on a first node corresponding to a second coordinate axis in the first nodes; determining that the plurality of first refreshing regions are not overlapped, or determining that each first refreshing region is a first merging region when the first line segments of the plurality of first refreshing regions are not overlapped, or the second line segments of the plurality of first refreshing regions are not overlapped, or the first line segments of the plurality of first refreshing regions are not overlapped; and determining the first merging region based on the first target line segment and the second target line segment under the condition that the first line segment of the first refreshing regions has an overlapped first target line segment and the second line segment of the first refreshing regions has an overlapped second target line segment.
Optionally, determining at least one second merge area of the second color data includes: respectively determining second nodes corresponding to the plurality of corner points of each second refreshing region on the first coordinate axis and the second coordinate axis to obtain a plurality of second nodes; determining a third line segment of each second refreshing region based on a second node corresponding to the first coordinate axis in the plurality of second nodes, and determining a fourth line segment of each second refreshing region based on a second node corresponding to the second coordinate axis in the plurality of second nodes; determining that the plurality of second refresh regions are not overlapped, or determining that each of the plurality of second refresh regions is a second merge region when the third line segments of the plurality of second refresh regions are not overlapped, or the fourth line segments of the plurality of second refresh regions are not overlapped, or the third line segments of the plurality of second refresh regions are not overlapped and the fourth line segments of the plurality of second refresh regions are not overlapped; and determining at least one second merging region based on the third target line segment and the fourth target line segment when the third line segment of the plurality of second refreshing regions has an overlapped third target line segment and the fourth line segment of the plurality of second refreshing regions has an overlapped fourth target line segment.
Optionally, determining at least one first merge area of the first color data includes: dividing at least two overlapped first refreshing areas into a plurality of first sub-refreshing areas under the condition that at least two overlapped first refreshing areas exist in the plurality of first refreshing areas, wherein each first sub-refreshing area is independent; generating at least one first merge region based on the plurality of first sub-refresh regions; determining at least one second merge area of the second color data, comprising: dividing the overlapped at least two second refreshing areas into a plurality of second sub-refreshing areas under the condition that the overlapped at least two second refreshing areas exist in the plurality of second refreshing areas, wherein each second sub-refreshing area is independent; at least one second merge region is generated based on the plurality of second sub-refresh regions.
According to another aspect of the embodiment of the invention, an image processing device is also provided. The apparatus may include: an acquisition unit configured to acquire first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by an image source device; a first determining unit configured to determine at least one first merging area of first color data and at least one second merging area of second color data, where the first merging area is merged by a plurality of first refresh areas of the first color data and does not include a blank area other than the plurality of first refresh areas, and the second merging area is merged by a plurality of second refresh areas of the second color data and does not include a blank area other than the plurality of second refresh areas; a second determining unit configured to determine at least one variation region of the at least one first combining region with respect to the at least one second combining region; and a transmitting unit configured to convert the image data in the change area from the first color format to the second color format, and transmit the image data in the second color format to the client, where the image data in the second color format is used to refresh an image already displayed on the client.
The embodiment of the invention also provides a computer readable storage medium. The computer readable storage medium includes a stored program, wherein the program when executed by a processor controls a device in which the computer readable storage medium resides to execute the image processing method of the embodiment of the present invention.
The embodiment of the invention also provides a processor. The processor is used for running a program, wherein the image processing method of the embodiment of the invention is executed when the program runs.
In the embodiment of the invention, at least one first merging area is determined through at least one first refreshing area of the first color data of the current frame image, and at least one second merging area is determined through at least one second refreshing area of the second color data of the previous frame image, and the first merging area and the second merging area do not have any blank area, so that the merging area calculated by the embodiment is very accurate, and the resources occupied by a CPU (Central processing Unit) in comparison processing can be reduced. In addition, the embodiment determines the change area of at least one first combination area relative to at least one second combination area, converts the image data in the change area from a first color format to a second color format, and sends the image data in the second color format to the client, so that the problem that the whole screen image of the current frame is converted from an RGB format to a YUV format and the whole screen image of the previous frame is converted from RGB to the YUV format is avoided, more resources are consumed, the waste of calculation resources is caused, the technical problem of resources consumed in image refreshing is solved, and the technical effect of reducing the resources consumed in image refreshing is achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a flowchart of a method of processing an image according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a plurality of refresh zones according to an embodiment of the present invention;
FIG. 3 is a schematic illustration of at least one merge area in accordance with an embodiment of the invention;
FIG. 4 is a schematic diagram of determining nodes and corresponding line segments of each corner point of each refresh zone on the X-axis and Y-axis according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a target alternate refresh zone according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of dividing a target alternate refresh zone into actual small zones according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of another division of a target alternate refresh zone into actual zones according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an alternative refresh zone according to an embodiment of the present invention;
FIG. 9 is a schematic illustration of determining a change region according to an embodiment of the invention;
FIG. 10 is a schematic view of a scene of an image refresh according to an embodiment of the invention;
FIG. 11 is a schematic view of another image refresh scenario according to an embodiment of the present invention;
fig. 12 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
According to an embodiment of the present invention, there is provided an embodiment of a method of processing an image, it being noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that herein.
Fig. 1 is a flowchart of a method of processing an image according to an embodiment of the present invention. As shown in fig. 1, the method may include the steps of:
Step S102, acquiring first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by an image source device.
In the technical solution provided in the step S102 of the present invention, the acquisition end may acquire the first color data of the current frame image and the second color data of the previous frame image in the image transmission system, where the first color data of the current frame image may be red, green and blue (RGB) format image data, and the second color data of the previous frame image may be RGB format image data.
In this embodiment, the above-described current frame image and previous frame image may be generated by an image source device, i.e., a local display device.
Step S104, determining at least one first merging area of the first color data and at least one second merging area of the second color data.
In the technical solution provided in the above step S104 of the present invention, after the first color data of the current frame image and the second color data of the previous frame image are acquired, at least one first merging area of the first color data and at least one second merging area of the second color data are determined, where the first merging area is obtained by merging a plurality of first refresh areas of the first color data and does not include a blank area except the plurality of first refresh areas, and the second merging area is obtained by merging a plurality of second refresh areas of the second color data and does not include a blank area except the plurality of second refresh areas.
In this embodiment, the capturing end may first determine a plurality of first refresh areas in the first color data, where the first refresh areas are areas that have been refreshed in the current frame image of the image source device determined by the capturing end through the bottom layer driver, and may be partially overlapped with each other, or may be completely overlapped with each other, or may be connected with each other, and may also be referred to as an original area of the current frame image, where the display image of the client has not yet been updated. After the acquisition end determines a plurality of first refreshing areas in the first color data, at least one first merging area of the first color data is determined according to the at least one first refreshing area, and the first merging area does not include blank areas except the plurality of first refreshing areas, namely, blank areas which do not belong to the plurality of first refreshing areas are not existed, and the blank areas can be irregular areas, so that the purpose of accurately determining the first merging area is achieved, resources occupied by a central processing unit (Central Processing Unit, called CPU for short) are reduced, and processing pressure of the CPU is reduced.
In this embodiment, for the second color data, the capturing end may first determine a plurality of second refresh areas in the second color data, where the second refresh areas are areas that have been refreshed in a previous frame image of the image source device determined by the capturing end through the bottom layer driving, and may be partially superimposed, may also be completely superimposed, may also be connected, and may also be referred to as an original area of the previous frame image. After the acquisition end determines a plurality of second refreshing areas in the second color data, at least one second merging area of the second color data is determined according to the at least one second refreshing area, and the second merging area does not include blank areas except the plurality of second refreshing areas, namely, blank areas which do not belong to the plurality of second refreshing areas are not present and can be irregular areas, so that the purpose of accurately determining the second merging area is achieved, resources occupied by a CPU (Central processing Unit) are reduced, and processing pressure of the CPU is reduced.
Step S106, determining a change area of the at least one first merge area with respect to the at least one second merge area.
In the solution provided in the above step S106 of the present invention, after determining at least one first merging area of the first color data and at least one second merging area of the second color data, a change area of the at least one first merging area with respect to the at least one second merging area is determined.
In this embodiment, the at least one first merge area and the at least one second merge area may be compared to determine a change area of the at least one first merge area with respect to the at least one second merge area, where the change area refers to an area where an actual change of the at least one first merge area with respect to the at least one second merge area occurs, and may be a minimum area where the actual change of the at least one first merge area with respect to the at least one second merge area occurs. The number of change regions of this embodiment may be 0, which represents an image display refresh but no change. Alternatively, in the case where the number of change regions is 0, the flow may be terminated. In the case where the number of the change regions is not 0, step S108 may be continued to be performed.
Step S108, converting the image data in the change area from the first color format to the second color format, and sending the image data in the second color format to the client, wherein the image data in the second color format is used for refreshing the image displayed on the client.
In the technical solution provided in the above step S108 of the present invention, after determining the change area of at least one first merge area relative to at least one second merge area, the image data in the change area may be converted from a first color format into a second color format, for example, the first color format is an RGB format, the second color format is a luminance information and chrominance information (YUV) format, the image data in the change area is converted from the RGB format into the YUV format, and then the image data in the YUV format is sent to the client for refreshing the image already displayed on the client.
Through the above steps S102 to S108 of the present application, at least one first merging area is determined through at least one first refresh area of the first color data of the current frame image, and at least one second merging area is determined through at least one second refresh area of the second color data of the previous frame image, where the first merging area and the second merging area do not have any blank area, so that the merging area calculated by this embodiment is very accurate, and resources occupied by the CPU in comparison processing can be reduced. In addition, the embodiment determines the change area of at least one first combination area relative to at least one second combination area, converts the image data in the change area from a first color format to a second color format, and sends the image data in the second color format to the client, so that the problem that the whole screen image of the current frame is converted from an RGB format to a YUV format and the whole screen image of the previous frame is converted from RGB to the YUV format is avoided, more resources are consumed, the waste of calculation resources is caused, the technical problem of resources consumed in image refreshing is solved, and the technical effect of reducing the resources consumed in image refreshing is achieved.
The above-described method of this embodiment is further described below.
As an optional embodiment, step S108, converting the image data in the change area from the first color format to the second color format, includes: determining an effective area based on the at least one variation area; the image data of the active area is converted from a first color format to a second color format.
In this embodiment, since not all data in at least one change area may be used to refresh an image on the client, in order to further reduce the resources consumed in performing the image refresh, the embodiment may determine an effective area in at least one change area, further convert the image data of the effective area from RGB format to YUV format, and send the image data in YUV format to the client for refreshing an image already displayed on the client. Optionally, the client of this embodiment may superimpose the image data in YUV format of the effective area on the image data in YUV format of the whole screen image of the previous frame, so as to obtain YUV data of the refreshed whole screen image. Wherein, the YUV data of the whole screen image of the previous frame can be saved in advance by the client.
It should be noted that, in the related art, the whole frame of image needs to be converted from RGB format to YUV format, but in this embodiment, only the image data of the effective area in the change area is converted from RGB format to YUV format, and then the image data in YUV format is sent to the client, so that the data needing to be subjected to format conversion is reduced, the amount of data sent to the client is also reduced, and further the resources consumed during image refreshing are reduced.
As an alternative embodiment, determining the effective area based on the at least one change area includes: merging the at least one change area to obtain at least one rectangular area; at least one rectangular region is determined as an effective region.
In this embodiment, in the case that the number of the change areas is not 0, the collecting end may combine at least one change area to form one or more rectangular areas, and determine the one or more rectangular areas as the effective area.
As an alternative embodiment, the active area comprises one change area or a plurality of interconnected change areas.
In this embodiment, the effective area may be rectangular, and one effective area may include at least one variation area, or include a plurality of interconnected variation areas, and the variation areas in one effective area may overlap each other or may be interconnected, which is not particularly limited herein.
As an alternative embodiment, determining the effective area based on the at least one change area includes: and determining a change area at the uppermost layer in the at least one change area as an effective area, wherein the change area at the uppermost layer covers the change areas except the change area at the uppermost layer in the at least one change area.
In this embodiment, in at least one of the change regions, if the change region of the uppermost layer completely covers other change regions than the change region of the uppermost layer in the at least one change region, only the change region of the uppermost layer may be represented, and the change region of the uppermost layer is determined as an effective region, for example, in a plurality of change regions, if the at least one change region X is completely covered by another change region Y, the effective region no longer represents the at least one change region X that is completely covered, but only the change region Y of the uppermost layer, which may also reduce the workload, and the covered change region is not paid attention, thereby achieving the technical effect of reducing the resources consumed in performing the image refresh.
The merge area of this embodiment is further described below.
As an alternative embodiment, each first merge area comprises at least two first refresh areas superimposed on each other, or at least two first refresh areas connected to each other, or one first refresh area not connected to or superimposed on any other first refresh area; each second merge region includes at least two second refresh regions that overlap each other, or at least two second refresh regions that are connected to each other, or one first refresh region that is not connected to or overlaps any other second refresh region.
In this embodiment, the collecting end merges the plurality of first refresh regions to obtain at least one first merged region, where each first merged region may include at least two first refresh regions stacked on each other, or may include at least two first refresh regions connected to each other, or may be a first refresh region that does not overlap any other first refresh region, and use the first refresh region alone as one first merged region.
Optionally, the collecting end merges the plurality of second refreshing areas to obtain at least one second merged area, and each second merged area may include at least two second refreshing areas stacked with each other, or may include at least two second refreshing areas connected with each other, or may be a first refreshing area that does not overlap any other second refreshing area, and uses the first refreshing area alone as one second merged area.
FIG. 2 is a schematic diagram of a plurality of refresh regions according to an embodiment of the present invention. As shown in fig. 2, the area filled with squares is referred to as a refresh area a, the area filled with slashes is referred to as a refresh area B, the area filled with black dots is referred to as a refresh area C, and the refresh areas A, B and C are each set in an XY-axis coordinate system with 0 as an origin. Each small rectangular block is a refresh zone, and it can be seen that some refresh zones are superimposed on each other, some refresh zones are independent, and multiple refresh zones may be far apart. Wherein the refresh area B and the refresh area a are overlapped with each other, and the refresh area C is far from both the refresh areas a and B.
As an alternative embodiment, the image data in the overlapping area of the at least two first refresh areas that are superimposed is the image data of one of the at least two first refresh areas; the image data in the overlapping region of the at least two second refresh regions superimposed is the image data of one of the at least two second refresh regions.
In this embodiment, the superimposed at least two first refresh regions have an overlapping region, and the image data in the overlapping region may be the image data of one of the at least two first refresh regions, and may be the image data of one of the first refresh regions having the largest area, that is, the embodiment calculates only once for the image data of the overlapping region without repeating the calculation, thereby precisely determining the at least one first merge region and reducing the processing pressure of the CPU.
Alternatively, in this embodiment, the superimposed at least two second refresh regions have an overlapping region, and the image data in the overlapping region may be the image data of one of the at least two second refresh regions, and may be the image data of one of the second refresh regions having the largest area, that is, the image data of the overlapping region is calculated only once without repeating the calculation, thereby precisely determining the at least one second merge region and reducing the processing pressure of the CPU.
FIG. 3 is a schematic diagram of at least one merge area in accordance with an embodiment of the invention. As shown in fig. 3, the merging area is two areas with black thick lines as an outer frame, wherein an overlapping area exists between the refresh area a and the refresh area B, one merging area is generated through the refresh area a and the refresh area B, and the other refresh area C which does not overlap with the refresh area a and the refresh area B can be independently used as one merging area.
In this embodiment, in the merged region composed of two refresh regions in which an overlap region exists, the calculation of the overlap region may be performed in the refresh region a filled with squares, and is no longer embodied in the refresh region B filled with slashes, that is, there is no case where the calculation of any pixel point in the merged region is repeatedly calculated.
As an alternative embodiment, determining at least one first merge area of the first color data includes: respectively determining first nodes corresponding to a plurality of corner points of each first refreshing area on a first coordinate axis and a second coordinate axis to obtain a plurality of first nodes; determining a first line segment of each first refreshing region based on a first node corresponding to a first coordinate axis in the first nodes, and determining a second line segment of each first refreshing region based on a first node corresponding to a second coordinate axis in the first nodes; determining that the plurality of first refreshing regions are not overlapped, or determining that each first refreshing region is a first merging region when the first line segments of the plurality of first refreshing regions are not overlapped, or the second line segments of the plurality of first refreshing regions are not overlapped, or the first line segments of the plurality of first refreshing regions are not overlapped; and determining the first merging region based on the first target line segment and the second target line segment under the condition that the first line segment of the first refreshing regions has an overlapped first target line segment and the second line segment of the first refreshing regions has an overlapped second target line segment.
In this embodiment, the first color data has a plurality of first refreshing areas, and a first node corresponding to a plurality of corner points of each first refreshing area on a first coordinate axis and a second coordinate axis can be determined respectively, where an intersection point is a vertex of the first refreshing area, the first coordinate axis may be an X axis in a two-dimensional coordinate system, the second coordinate axis may be a Y axis in the two-dimensional coordinate system, and the first node may be an X coordinate corresponding to each corner point on the first coordinate axis or a Y coordinate corresponding to each corner point on the second coordinate axis. The embodiment may determine the first line segment of each first refresh zone based on a first node of the plurality of first nodes corresponding to the first coordinate axis, and determine the second line segment of each first refresh zone based on a first node of the plurality of first nodes corresponding to the second coordinate axis.
After determining the first line segment of each first refresh region and the second line segment of each first refresh region, the embodiment may determine whether the first line segments of the plurality of first refresh regions overlap, and if it is determined that the first line segments of the plurality of first refresh regions do not overlap, or the second line segments of the plurality of first refresh regions do not overlap, or the first line segments of the plurality of first refresh regions do not overlap and the second line segments of the plurality of first refresh regions do not overlap, it may be explained that the plurality of first refresh regions do not overlap, and then each first refresh region is determined as one first merge region, respectively.
Alternatively, if there is a first target line segment overlapping a first line segment of the plurality of first refresh regions and there is a second target line segment overlapping a second line segment of the plurality of first refresh regions, the first merge region may be determined based on the first target line segment and the second target line segment.
As an alternative embodiment, determining at least one first merge area of the first color data includes: dividing at least two overlapped first refreshing areas into a plurality of first sub-refreshing areas under the condition that at least two overlapped first refreshing areas exist in the plurality of first refreshing areas, wherein each first sub-refreshing area is independent; generating at least one first merge region based on the plurality of first sub-refresh regions; determining at least one second merge area of the second color data, comprising: dividing the overlapped at least two second refreshing areas into a plurality of second sub-refreshing areas under the condition that the overlapped at least two second refreshing areas exist in the plurality of second refreshing areas, wherein each second sub-refreshing area is independent; at least one second merge region is generated based on the plurality of second sub-refresh regions.
In this embodiment, when determining at least one first merge region of the first color data is implemented, for at least two first refresh regions that are superimposed, that is, there is an overlap region between the at least two first refresh regions, the at least two first refresh regions may be divided into a plurality of first sub-refresh regions, which may be referred to as a plurality of actual small regions, may include the actual small regions divided by the regions other than the overlap region in the refresh region a and the refresh region B shown in fig. 3, and then be combined into the first merge region.
Optionally, when determining at least one second merging region of the second color data, for at least two second refresh regions that are superimposed, i.e. there is an overlap region between the at least two second refresh regions, the at least two second refresh regions may be divided into a plurality of second sub-refresh regions, which may be referred to as a plurality of actual small regions, which are further combined into the first merging region.
The above-described method of determining the merge area of this embodiment is further exemplified below.
Fig. 4 is a schematic diagram of determining nodes and corresponding line segments of each corner point of each refresh zone on the X-axis and the Y-axis according to an embodiment of the present invention. As shown in fig. 4, the method of merging regions of this embodiment may include the steps of:
step S1, determining nodes corresponding to four corner points of each refreshing area on an X axis and a Y axis; and determining a first line segment formed by two nodes corresponding to each refreshing region on the X axis and a second line segment formed by two nodes corresponding to each refreshing region on the Y axis.
Alternatively, as shown in fig. 4, first, the nodes corresponding to the four corner points of the refreshing area a filled with squares, the refreshing area B filled with slashes, and the refreshing area C filled with black dots on the X-axis and the Y-axis, and the first line segment and the second line segment respectively composed of the nodes on the X-axis and the Y-axis are determined, respectively.
Optionally, the nodes of the refresh area a corresponding to the X axis and the Y axis are X1, X4, Y1 and Y4, respectively, where a first line segment formed by two nodes on the X axis is A1 (X1, X4), and a second line segment formed by two nodes on the Y axis is A2 (Y1, Y4).
The nodes of the refreshing area B corresponding to the X axis and the Y axis are X3, X2, Y3 and Y2 respectively, wherein a first line segment formed by two nodes on the X axis is B1 (X3, X2), and a second line segment formed by two nodes on the Y axis is B2 (Y3, Y2).
The nodes corresponding to the refreshing area C on the X axis and the Y axis are X5, X6, Y5 and Y6 respectively, wherein a first line segment formed by two nodes on the X axis is C1 (X5, X6), and a second line segment formed by two nodes on the Y axis is C2 (Y5, Y6).
Step S3, judging whether overlapping exists between the first line segments and between the second line segments of each refreshing area; if so, the overlapped part of the first line segment of each refreshing area is marked as a first target line segment, the overlapped part of the second line segment of each refreshing area is marked as a second target line segment, and the step S5 is executed; if not, the fact that the overlapped area does not exist among the refreshing areas is indicated, and the refreshing areas are respectively used as merging areas.
As shown in fig. 4, when there is an overlap between the first line segment B1 (x 3, x 2) of the refresh area B and the first line segment A1 (x 1, x 4) of the refresh area a, the overlapping portion is a first target line segment (x 1, x 2), and there is an overlap between the second line segment B2 (y 3, y 2) of the refresh area B and the second line segment A2 (y 1, y 4) of the refresh area a, the overlapping portion is a second target line segment (y 1, y 2).
In addition, the first line segment C1 (x 5, x 6) of the refresh area C is not overlapped with the first line segment B1 (x 3, x 2) of the refresh area B, the first line segment A1 (x 1, x 4) of the refresh area a, and the second line segment C2 (y 5, y 6) of the refresh area C is not overlapped with the second line segment B2 (y 3, y 2) of the refresh area B, and the second line segment A2 (y 1, y 4) of the refresh area a. Thus, for the refresh zone C, there is no first target line segment and no second target line segment.
In this embodiment, if there is no overlap between the first line segments, or there is no overlap between the second line segments, or there is no overlap between the first line segments and the second line segments, of each refresh zone, it is indicated that there is no overlap region between the refresh zones. Step S106 may be performed with each refresh zone being a respective merge zone.
Each refresh zone of this embodiment determines a first merge zone based on the first target line segment and the second target line segment in the presence of the first target line segment and the second target line segment.
S5, marking the refreshing areas corresponding to the first target line segment and the second target line segment as alternative refreshing areas (alternative initial areas); all the candidate refresh regions are ordered from large to small by area, thereby generating a refresh region list (original region list).
As shown in fig. 4, the refresh area corresponding to the first target line segment and the second target line segment is a refresh area a filled with squares and a refresh area B filled with slashes, that is, the alternative refresh area of this embodiment includes a refresh area a and a refresh area B, and the refresh area list includes: refresh area a and refresh area B.
S7, in the refresh zone list, the first-ordered (i.e. the refresh zone A in the figure 4) alternative refresh zone (the refresh zone A with the largest area) can be moved to the independent zone list; updating the refresh zone list.
It should be noted that this step may be performed only at the first time of execution, and then the step may be skipped directly.
It should be noted that, each independent area in the independent area list is an independent area that is generated after the alternative refresh area is processed by the overlapping area and does not have an overlapping area. In this way, in the subsequent processing, only the independent areas in the independent area list are updated, and because no overlapping area exists between the independent areas, the workload of the image data update can be reduced to the minimum, the processing task amount of the system is reduced, and the resources consumed in the process of image refreshing are reduced.
It will be appreciated that when there is an overlap region in each of the candidate refresh regions, the overlap region may be uniformly counted among the candidate refresh regions having the largest area. In order to avoid repeated calculation of the overlapped area, through the subsequent steps, other alternative refreshing areas can be divided into a plurality of actual small areas based on the overlapped area, and the actual small areas are moved into an independent area list; the small region overlapping the overlapping region is discarded.
The regions in the independent region list described above of this embodiment are ordered in the order of addition, and therefore, the candidate refresh region (refresh region a in fig. 4) of the largest area that is added first to the independent region list is ordered first in the independent region list.
In the step, firstly, the alternative refreshing area with the largest area is moved to an independent area list to be used as a first independent area; then, the refresh area list is updated, that is, the alternative refresh area originally ordered second is modified to be ordered first, and the arrangement sequence numbers of other alternative refresh areas are adjusted accordingly.
And S9, in the updated refresh zone list, marking the first-ordered alternative refresh zone as a target alternative refresh zone.
It should be noted that, when this step is performed for the first time, the target candidate refresh area is the next largest candidate refresh area (for example, refresh area B in fig. 4).
Step S11, determining an overlapping area of a target alternative refreshing area and a target area in an independent area list; and dividing the target alternative refreshing area into a plurality of actual small areas according to the overlapped area.
Wherein, when the step is executed for the first time, the target area in the independent area list is the first alternative refreshing area in sequence, namely, the alternative refreshing area with the largest area (such as refreshing area A in FIG. 4); the step is then sequentially and alternately performed in order, i.e. the second time, the target region being the second alternate refresh region in the list of independent regions, and so on.
First, coordinates of a preset small area except for an overlapping area in a target alternative refresh area (the first alternative refresh area is ordered in an updated refresh area list) are preset.
FIG. 5 is a schematic diagram of a target alternate refresh zone according to an embodiment of the present invention. As shown in fig. 5, the large black frame is a target alternative refreshing area, the middle small black frame is an overlapping area, and the large black frame is divided into four preset small areas based on the extension lines of the upper and lower frames of the middle small black frame, which can be respectively marked as a preset small area 1, a preset small area 2, a preset small area 3 and a preset small area 4.
In this embodiment, four corner points where the overlapping region is set are respectively upper left corner points (xa 1, xb 1), upper right corner points (xa 2, xb 1), lower left corner points (xa 1, xb 2), and lower right corner points (xa 2, xb 2). The four-point corner points of the target alternative refreshing region are respectively an upper left corner point (xx 1, xy 1), an upper right corner point (xx 2, xy 1), a lower left corner point (xx 1, xy 2) and a lower right corner point (xx 2, xy 2).
Then, four corner points of the preset small area 1 are respectively an upper left corner point (xx 1, xb 1), an upper right corner point (xa 1, xb 1), a lower left corner point (xx 1, xb 2) and a lower right corner point (xa 1, xb 2); four corner points of the preset small region 2 are respectively an upper left corner point (xx 1, xy 1), an upper right corner point (xx 2, xy 1), a lower left corner point (xx 1, xb 1) and a lower right corner point (xx 2, xb 1); four corner points of the preset small area 3 are respectively an upper left corner point (xa 2, xb 1), an upper right corner point (xx 2, xb 1), a lower left corner point (xa 2, xb 2) and a lower right corner point (xx 2, xb 2); four corner points of the preset small area 4 are respectively an upper left corner point (xx 1, xb 2), an upper right corner point (xx 2, xb 2), a lower left corner point (xx 1, xy 2) and a lower right corner point (xx 2, xy 2).
It should be noted that, in the first step, the divided four preset small areas are in the most ideal state, it is understood that only when the overlapping area is within the target candidate refresh area, the four preset small areas exist simultaneously, but in practical application, the overlapping area and the target candidate refresh area generally also have overlapping edges, so the four preset small areas generally do not exist simultaneously. The first step is a pre-prepared step, and in a specific implementation, may be performed directly from the second step.
And secondly, determining the actual small areas actually existing after the target alternative refreshing areas are divided according to the coordinates of the 4 preset small areas and the actual coordinates of the four corner points of the target alternative refreshing initial area and the overlapping area.
Taking fig. 4 as an example, as shown in fig. 4, the target alternative digital display area is a refresh area B filled with a slash, and the refresh area B and the overlapping area (black thick frame) have 1 common corner point, that is, two frames corresponding to the corner points coincide. Therefore, the x-axis coordinate values of the upper right corners of the (black thick frame) of the target alternate refresh region (B) and the overlap region are equal, that is, xa2=xx2; the y-axis coordinate values of the lower left corner are equal, that is, xb2=xy 2; the x-axis coordinate value and the y-axis coordinate value of the lower right corner are equal, that is, xx2=xa2, xy 2=xb2.
Then, after the coordinates of the upper right corner and the coordinates of the lower right corner in the four corner points of the four preset small areas are replaced, the coordinates of the four corner points of the four actual small areas can be obtained, which may be:
The four-point corner points of the actual small area 1 are respectively upper left corner points (xx 1, xb 1), upper right corner points (xa 1, xb 1), lower left corner points (xx 1, xb 2) and lower right corner points (xa 1, xb 2); the four-point corner points of the actual small area 2 are respectively an upper left corner point (xx 1, xy 1), an upper right corner point (xa 2, xy 1), a lower left corner point (xx 1, xb 1) and a lower right corner point (xa 2, xb 1); the four-point corner points of the actual small area 3 are respectively an upper left corner point (xa 2, xb 1), an upper right corner point (xa 2, xb 1), a lower left corner point (xa 2, xb 2) and a lower right corner point (xa 2, xb 2); the four-point corner points of the actual small region 4 are respectively an upper left corner point (xx 1, xb 2), an upper right corner point (xa 2, xb 2), a lower left corner point (xx 1, xb 2) and a lower right corner point (xa 2, xb 2).
It can be inferred that no corner points with coincident coordinates exist in the four corner points of the actual small area 1, so that the actual small area 1 exists; the four-point angular points of the actual small area 2 do not have angular points with coincident coordinates, so that the actual small area 2 exists; in the actual small area 3, the coordinates of the upper left corner point and the upper right corner point coincide, and the coordinates of the lower left corner point and the lower right corner point coincide, so that the actual small area 3 does not exist; in the actual small area 4, the coordinates of the upper left corner point and the lower left corner point coincide, and the coordinates of the upper right corner point and the lower right corner point coincide, so that the actual small area 4 does not exist either.
As can be seen from the above, taking fig. 4 as an example, the target alternative refresh area (refresh area B filled by a slash) is divided into 2 actual small areas except for the overlap area, as shown in fig. 6. Fig. 6 is a schematic diagram of dividing a target alternative refresh zone into actual small zones, and target alternative refresh zone B may be divided into actual small zone 2 and actual zone 1 according to an embodiment of the present invention.
Fig. 7 is a schematic diagram of another division of a target alternative refresh zone into actual zones according to an embodiment of the present invention. As shown in fig. 7, for greater clarity, the two actual small areas are the two rectangles of the black thick border in the refresh area B in fig. 7.
As can be seen from fig. 7, there are no overlapping areas between the two actual small areas and all the existing independent areas in the independent list (the current independent area list includes the refresh area a). That is, the actual small area of the target alternate refresh zone is the remainder of the target alternate refresh zone after the overlap zone with all of the individual zones present in the individual list is scratched out. For ease of calculation, the remainder is divided into a plurality of actual small areas.
S13, judging whether each actual small area contained in the target alternative refreshing area is overlapped with each independent area in the independent area list or not in sequence; if not, adding the actual small area into an independent area list, deleting the target alternative refreshing area from the refreshing area list, and executing step S15; if yes, the next area of the current target area in the independent area list is updated to be the target area, and the step S11 is executed.
It should be noted that, since there are fewer alternative refresh regions in fig. 4, only two actual small regions corresponding to the refresh region B and the refresh region a do not already have an overlapping region, two actual small regions corresponding to the refresh region B may be added to the independent region list, so that the independent region list includes three regions, and the refresh region a is ordered in the first, and the two actual small regions are ordered in the second and third.
However, for a larger number of alternative refresh regions, i.e. in case there is a mutual overlap of multiple refresh regions, there may be an overlap region with other regions than the current target region in the independent region list as well for the actual small region of the target alternative refresh region.
FIG. 8 is a schematic diagram of an alternative refresh zone according to an embodiment of the present invention. As shown in fig. 8, the refresh zone list includes three alternative refresh zones, the largest being a bottomless box zone, the next largest being a slash zone and the smallest cross-bar zone. According to the above processing steps, the bottomless frame area is first added to the independent area list, and the slash area is the target alternative refresh area.
In processing the slash area, the divided actual small area may be added to the independent area list, that is, the independent area list includes the first non-base-frame area ordered in the first, and the second first black thick-frame area 1'.
In processing the bar area, the following steps may be performed:
The method comprises the steps that firstly, a target area in an independent area list is a background-free area; according to the overlapping part of the bar area and the bottomless frame area, an actual small area of the bar area is generated, namely, a second black thick frame area 2' (the border lines of the second black thick frame area are thicker than the border lines of the first black thick frame area) in fig. 8.
As can be seen from the above, there is still an overlapping area between the second black thick frame region 2 'and the first black thick frame region 1'.
Step two, updating the first black thick frame region 1' of the second sequence in the independent region column into a target region; according to the overlapping portion of the first black thick frame region 1' and the bar region, an actual small area of the bar region is generated, that is, the third black thick frame region 3' (the step S11 is executed to implement) in fig. 7, and the border line of the third black thick frame region 3' is thicker than the border line of the second black thick frame region.
As can be seen from the above, the third black thick frame region and the other alternative refresh regions no longer have overlapping regions, and therefore, the third black thick frame region 3 'may be added to the independent region list, where the independent region list includes the bottomless frame region ordered in the first, the first black thick frame region ordered in the second, and the third black thick frame region 3' ordered in the third.
It will be appreciated that only the actual small areas, which do not have any overlap with other alternative refresh areas, can be added to the separate area list. That is, the regions in the independent region list are independent of each other.
S15, judging whether an alternative refreshing region exists in the refreshing region list; if yes, returning to the step S9, if no, recording all the areas in the independent area list as merging areas.
It should be noted that, in some special scenarios, one or more refresh regions may be far apart and have no connection or overlap, such as refresh region C in fig. 2, and there are multiple refresh regions that have overlapping regions, such as refresh region a and refresh region B.
In the related art, a circumscribed rectangle where there is an overlap region or a plurality of connected refresh regions is taken as one merge region, and a refresh region where there is no overlap or connection is taken as another merge region. However, taking circumscribed rectangles of the plurality of refreshing areas as a merging area, wherein blank areas which do not belong to the refreshing areas still exist in the merging area; in order to further accurately merge the range of the region, the merge region in the embodiment does not have a blank region which does not belong to the refresh region, so that the area of the merge region is reduced, the merge region can be more accurately determined, the processing pressure of a CPU is reduced, and further, the resources consumed during image refresh are reduced.
As an alternative embodiment, determining at least one second merge area of the second color data comprises: respectively determining second nodes corresponding to the plurality of corner points of each second refreshing region on the first coordinate axis and the second coordinate axis to obtain a plurality of second nodes; determining a third line segment of each second refreshing region based on a second node corresponding to the first coordinate axis in the plurality of second nodes, and determining a fourth line segment of each second refreshing region based on a second node corresponding to the second coordinate axis in the plurality of second nodes; determining that the plurality of second refresh regions are not overlapped, or determining that each of the plurality of second refresh regions is a second merge region when the third line segments of the plurality of second refresh regions are not overlapped, or the fourth line segments of the plurality of second refresh regions are not overlapped, or the third line segments of the plurality of second refresh regions are not overlapped and the fourth line segments of the plurality of second refresh regions are not overlapped; and determining at least one second merging region based on the third target line segment and the fourth target line segment when the third line segment of the plurality of second refreshing regions has an overlapped third target line segment and the fourth line segment of the plurality of second refreshing regions has an overlapped fourth target line segment.
The method of determining at least one second merging region of the second color data in this embodiment may be the same as that of determining at least one first merging region of the first color data, and may be applicable to the above steps S1 to S15, which are not repeated here.
In this embodiment, determining at least one variation area of the at least one first combined area relative to the at least one second combined area may be determining a minimum variation area of the at least one first combined area relative to the at least one second combined area. Fig. 9 is a schematic diagram of determining a change region according to an embodiment of the present invention. As shown in fig. 9, it is a variation region determined based on fig. 3, wherein the variation region includes a region a to a region d, wherein the region a and the region b are superimposed, and the region c and the region d are connected.
The acquisition end of the embodiment combines based on at least one change area to form one or more rectangular areas, the rectangular areas are marked as effective areas, the effective areas are required to be converted from RGB format to YUV format, then image data in YUV format is sent to the client to update a refreshing image displayed on the client, and the whole frame of image is required to be converted from RGB format to YUV format in the prior art, so that the format conversion workload of the embodiment is small, and resources during image refreshing are saved.
In this embodiment, at least one first merging area is determined by at least one first refresh area of the first color data of the current frame image, and at least one second merging area is determined by at least one second refresh area of the second color data of the previous frame image, and the first merging area and the second merging area do not have any blank area, so that the merging area calculated in this embodiment is very accurate, and resources occupied by the CPU in comparison processing can be reduced. In addition, the embodiment determines the change area of at least one first combination area relative to at least one second combination area, converts the image data in the change area from a first color format to a second color format, and sends the image data in the second color format to the client, so that the problem that the whole screen image of the current frame is converted from an RGB format to a YUV format and the whole screen image of the previous frame is converted from RGB to the YUV format is avoided, more resources are consumed, the waste of calculation resources is caused, the technical problem of resources consumed in image refreshing is solved, and the technical effect of reducing the resources consumed in image refreshing is achieved.
Example 2
The preferred embodiment of this example is described by way of example.
In the related art, the image transmission system comprises a client and an acquisition end, wherein the acquisition end sets acquire images from image source equipment and send the images to the client, and the images are displayed to a user by the client. When the image of the image source device changes, the image source device needs to be acquired again through the acquisition end to refresh the client side picture, and the specific flow can be as follows:
S1, the acquisition end confirms all refreshed areas in the image source equipment through bottom drive and records the refreshed areas as initial areas.
S2, the acquisition end acquires the whole screen image of the current frame and converts the acquired whole screen image of the current frame into YUV data from RGB data.
S3, the acquisition end determines YUV data of an area in the initial area, which is actually changed, based on the initial area determined in the S1, YUV data of the whole screen image of the current frame and YUV data of the whole screen image of the previous frame.
And S4, the acquisition end sends the YUV data of the actually changed area determined in the S3 to the client, and the client refreshes the display image according to the YUV data of the actually changed area.
In a general case, when the picture in the image source device changes more severely, for example, the user maximizes or minimizes the window, drags the window, plays the video, or opens the web page, a large amount of initial area is generated, so that the workload of the step 1 is larger; in addition, the whole screen image (of the current frame and the previous frame) is converted from RGB to YUV, and more resources can be consumed, so that the waste of calculation resources is caused.
In this embodiment, the collecting end determines a merging area according to the initial area, particularly, the initial area with the overlapping part, and divides the initial area with the overlapping part into a plurality of actual small areas so as to form the merging area; then, comparing the RGB data of the current frame with the RGB data of the previous frame to determine a change area in the merging area; and finally, converting the effective area of the change area from an RGB format to a YUV format, and sending the YUV format to a client for refreshing the display diagram.
For example, fig. 10 is a schematic view of an image refreshing scene according to an embodiment of the present invention, as shown in fig. 10, which is a page display of an office scene, in which an initial area for driving reporting includes a rectangle 1 (font change drop-down frame, 3), a rectangle 2 (input method window and input text change area, 3), a rectangle 3 (lower left corner word count and chinese-english display, 2) and a word client area rectangle 4 (3), and the initial area may be a plurality of overlapped rectangular areas. A merge area (the same range as the client area rectangle) may be generated from the initial area to regenerate three effectively changing change areas, i.e., the active area. Fig. 11 is a schematic view of another image refresh scenario according to an embodiment of the present invention. As shown in fig. 11, the effective area includes a rectangle M, N, P, and finally, the three effective areas are combined to obtain a final effective area. In this embodiment, the three effective areas are not overlapped, so the number of the combined effective areas is unchanged, the RGB data of the three effective areas are converted into YUV format, and the YUV format is transmitted to the client for superposition display after being encoded.
Since the embodiment can compare the merging area of the RGB data of the current frame with the merging area of the RGB data of the previous frame, the change area in the merging area is determined, the calculated merging area is very accurate, and the resources occupied by the CPU in comparison processing can be reduced; in addition, the acquisition end only converts RGB of the effective area part into YUV and sends the YUV to the client, so that the sent data volume is smaller, the technical problem of resources consumed in image refreshing is solved, and the technical effect of reducing the resources consumed in image refreshing is achieved.
Example 3
The embodiment of the invention also provides an image processing device. The image processing apparatus of this embodiment may be used to execute the image processing method of the embodiment of the present invention.
Fig. 12 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. As shown in fig. 12, the processing device 120 of the image may include: an acquisition unit 121, a first determination unit 122, a second determination unit 123, and a transmission unit 124.
An acquiring unit 121 for acquiring first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by an image source device.
The first determining unit 122 is configured to determine at least one first merging area of the first color data and at least one second merging area of the second color data, where the first merging area is obtained by merging a plurality of first refresh areas of the first color data and does not include a blank area except the plurality of first refresh areas, and the second merging area is obtained by merging a plurality of second refresh areas of the second color data and does not include a blank area except the plurality of second refresh areas.
The second determining unit 123 is configured to determine at least one change area of the at least one first merging area relative to the at least one second merging area.
And a transmitting unit 124 for converting the image data in the change area from the first color format to the second color format and transmitting the image data in the second color format to the client, wherein the image data in the second color format is used for refreshing the image already displayed on the client.
In the image processing apparatus of this embodiment, at least one first merge area is determined by at least one first refresh area of first color data of a current frame image, and at least one second merge area is determined by at least one second refresh area of second color data of a previous frame image, and the first merge area and the second merge area do not have any blank area, so that the merge area calculated by this embodiment is very accurate, and resources occupied by a CPU in comparison processing can be reduced. In addition, the embodiment determines the change area of at least one first combination area relative to at least one second combination area, converts the image data in the change area from a first color format to a second color format, and sends the image data in the second color format to the client, so that the problem that the whole screen image of the current frame is converted from an RGB format to a YUV format and the whole screen image of the previous frame is converted from RGB to the YUV format is avoided, more resources are consumed, the waste of calculation resources is caused, the technical problem of resources consumed in image refreshing is solved, and the technical effect of reducing the resources consumed in image refreshing is achieved.
Example 4
According to an embodiment of the present invention, there is also provided a computer-readable storage medium. The computer readable storage medium includes a stored program, wherein the program when executed by a processor controls a device in which the computer readable storage medium resides to execute the image processing method of the embodiment of the present invention.
Example 5
According to an embodiment of the present invention, there is also provided a processor for running a program, where the program executes the image processing method according to the embodiment of the present invention.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with respect to each other may be an indirect coupling or communication connection via some interfaces, units or models, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims (13)

1. A method of processing an image, comprising:
acquiring first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by image source equipment;
Determining at least one first merging region of the first color data and at least one second merging region of the second color data, wherein the first merging region is obtained by merging a plurality of first refreshing regions of the first color data and does not comprise blank regions except the plurality of first refreshing regions, and the second merging region is obtained by merging a plurality of second refreshing regions of the second color data and does not comprise blank regions except the plurality of second refreshing regions;
determining at least one variation region of the at least one first merge region relative to the at least one second merge region;
converting the image data in the change area from a first color format to a second color format, and sending the image data in the second color format to a client, wherein the image data in the second color format is used for refreshing an image displayed on the client;
Wherein the method further comprises:
the merge area is determined by:
step S1, determining nodes corresponding to four corner points of each refreshing area on an X axis and a Y axis, and determining a first line segment formed by two nodes corresponding to each refreshing area on the X axis and a second line segment formed by two nodes corresponding to each refreshing area on the Y axis;
Step S3, judging whether overlapping exists between the first line segments and between the second line segments of each refreshing area or not respectively; if yes, marking the overlapped part of the first line segment as a first target line segment, marking the overlapped part of the second line segment as a second target line segment, and executing the step S5; if not, taking each refreshing area as the merging area;
Step S5, determining the refresh areas corresponding to the first target line segment and the second target line segment as alternative refresh areas, and sequencing all the alternative refresh areas from large to small according to the area to generate a refresh area list;
step S7, in the refresh zone list, the first-ordered alternative refresh zone is moved to an independent zone list; updating the refreshing region list;
Step S9, in the updated refreshing region list, the first-ordered alternative refreshing region is recorded as a target alternative refreshing region;
Step S11, determining an overlapping area of the target alternative refreshing area and a target area in the independent area list; dividing the target alternative refreshing region into a plurality of small regions based on the overlapping region;
Step S13, judging whether a plurality of small areas are overlapped with each independent area in the independent area list; if not, adding the small area to the independent area list, deleting the target alternative refreshing area from the refreshing area list, and executing step S15; if yes, updating the next area of the current target area in the independent area list to the target area, and executing the step S11;
step S15, determining whether the refresh area list has the alternative refresh area; if yes, executing the step S9, otherwise, recording all areas in the independent area list as the merging areas;
the merging area is the first merging area or the second merging area.
2. The method of claim 1, wherein converting the image data in the change region from a first color format to a second color format comprises:
determining an effective area based on the at least one variation area;
and converting the image data of the effective area from the first color format to the second color format.
3. The method of claim 2, wherein determining an effective area based on the at least one change area comprises:
merging the at least one change area to obtain at least one rectangular area;
the at least one rectangular region is determined as the effective region.
4. The method of claim 2, wherein the active area comprises one of the change areas or a plurality of interconnected change areas.
5. The method of claim 2, wherein determining an effective area based on the at least one change area comprises:
and determining a change area at the uppermost layer in the at least one change area as the effective area, wherein the change area at the uppermost layer covers the change areas except the change area at the uppermost layer in the at least one change area.
6. The method of claim 1, wherein each of the first merge areas comprises at least two first refresh areas that overlap each other, or at least two first refresh areas that are connected to each other, or one first refresh area that is not connected to or overlaps any other first refresh area; each of the second merge areas includes at least two of the second refresh areas superimposed on each other, or at least two of the second refresh areas connected to each other, or one of the first refresh areas not connected to or superimposed on any other of the second refresh areas.
7. The method of claim 6, wherein the image data in the overlapping region of the at least two first refresh regions that overlap is the image data of one of the at least two first refresh regions; the image data in the overlapping area of the at least two second refresh areas that are overlapped is the image data of one of the at least two second refresh areas.
8. The method of claim 1, wherein determining at least one first merge area of the first color data comprises:
respectively determining first nodes corresponding to a plurality of corner points of each first refreshing area on a first coordinate axis and a second coordinate axis to obtain a plurality of first nodes;
Determining a first line segment of each first refreshing region based on a first node corresponding to the first coordinate axis in the first nodes, and determining a second line segment of each first refreshing region based on a first node corresponding to the second coordinate axis in the first nodes;
Determining that the plurality of first refresh regions are not overlapped, or determining that each of the first refresh regions is one of the first merge regions, if the first line segments of the plurality of first refresh regions are not overlapped, or the second line segments of the plurality of first refresh regions are not overlapped, or the first line segments of the plurality of first refresh regions are not overlapped;
And determining the first merging region based on the first target line segment and the second target line segment under the condition that the first target line segment of the first refreshing regions is overlapped and the second target line segment of the second refreshing regions is overlapped.
9. The method of claim 1, wherein determining at least one second merge area of the second color data comprises:
Respectively determining second nodes corresponding to the plurality of corner points of each second refreshing area on the first coordinate axis and the second coordinate axis to obtain a plurality of second nodes;
determining a third line segment of each second refreshing region based on a second node corresponding to the first coordinate axis in the plurality of second nodes, and determining a fourth line segment of each second refreshing region based on a second node corresponding to the second coordinate axis in the plurality of second nodes;
Determining that the plurality of second refresh regions are not overlapped, and determining each of the second refresh regions as one of the second merge regions, if the third line segments of the plurality of second refresh regions are not overlapped, or the fourth line segments of the plurality of second refresh regions are not overlapped, or the third line segments of the plurality of second refresh regions are not overlapped and the fourth line segments of the plurality of second refresh regions are not overlapped;
and determining the second merging region based on the third target line segment and the fourth target line segment when the third target line segment of the second refreshing regions is overlapped and the fourth target line segment of the second refreshing regions is overlapped.
10. The method according to claim 1, characterized in that it comprises:
Determining at least one first merge area of the first color data, comprising: dividing at least two overlapped first refreshing areas into a plurality of first sub-refreshing areas under the condition that the overlapped at least two first refreshing areas exist in the plurality of first refreshing areas, wherein each first sub-refreshing area is independent; generating the at least one first merge region based on the plurality of first sub-refresh regions;
Determining at least one second merge area of the second color data, comprising: dividing the overlapped at least two second refreshing areas into a plurality of second sub-refreshing areas under the condition that the overlapped at least two second refreshing areas exist in the plurality of second refreshing areas, wherein each second sub-refreshing area is independent; the at least one second merge region is generated based on the plurality of second sub-refresh regions.
11. An image processing apparatus, comprising:
an acquisition unit configured to acquire first color data of a current frame image and second color data of a previous frame image, wherein the current frame image and the previous frame image are generated by an image source device;
A first determining unit configured to determine at least one first merge area of the first color data and at least one second merge area of the second color data, where the first merge area is merged by a plurality of first refresh areas of the first color data and does not include a blank area other than the plurality of first refresh areas, and the second merge area is merged by a plurality of second refresh areas of the second color data and does not include a blank area other than the plurality of second refresh areas;
a second determining unit configured to determine at least one variation region of the at least one first combined region with respect to the at least one second combined region;
A transmitting unit, configured to convert image data in the change area from a first color format to a second color format, and transmit the image data in the second color format to a client, where the image data in the second color format is used to refresh an image already displayed on the client;
wherein the first determining unit is further configured to determine the merge area by:
step S1, determining nodes corresponding to four corner points of each refreshing area on an X axis and a Y axis, and determining a first line segment formed by two nodes corresponding to each refreshing area on the X axis and a second line segment formed by two nodes corresponding to each refreshing area on the Y axis;
Step S3, judging whether overlapping exists between the first line segments and between the second line segments of each refreshing area or not respectively; if yes, marking the overlapped part of the first line segment as a first target line segment, marking the overlapped part of the second line segment as a second target line segment, and executing the step S5; if not, taking each refreshing area as the merging area;
Step S5, determining the refresh areas corresponding to the first target line segment and the second target line segment as alternative refresh areas, and sequencing all the alternative refresh areas from large to small according to the area to generate a refresh area list;
step S7, in the refresh zone list, the first-ordered alternative refresh zone is moved to an independent zone list; updating the refreshing region list;
Step S9, in the updated refreshing region list, the first-ordered alternative refreshing region is recorded as a target alternative refreshing region;
Step S11, determining an overlapping area of the target alternative refreshing area and a target area in the independent area list; dividing the target alternative refreshing region into a plurality of small regions based on the overlapping region;
Step S13, judging whether a plurality of small areas are overlapped with each independent area in the independent area list; if not, adding the small area to the independent area list, deleting the target alternative refreshing area from the refreshing area list, and executing step S15; if yes, updating the next area of the current target area in the independent area list to the target area, and executing the step S11;
step S15, determining whether the refresh area list has the alternative refresh area; if yes, executing the step S9, otherwise, recording all areas in the independent area list as the merging areas;
the merging area is the first merging area or the second merging area.
12. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored program, wherein the program, when run by a processor, controls a device in which the computer readable storage medium is located to perform the method of any one of claims 1 to 10.
13. A processor for running a program, wherein the program when run performs the method of any one of claims 1 to 10.
CN202010956380.1A 2020-09-11 2020-09-11 Image processing method, device, storage medium and processor Active CN112214188B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010956380.1A CN112214188B (en) 2020-09-11 2020-09-11 Image processing method, device, storage medium and processor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010956380.1A CN112214188B (en) 2020-09-11 2020-09-11 Image processing method, device, storage medium and processor

Publications (2)

Publication Number Publication Date
CN112214188A CN112214188A (en) 2021-01-12
CN112214188B true CN112214188B (en) 2024-06-07

Family

ID=74049368

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010956380.1A Active CN112214188B (en) 2020-09-11 2020-09-11 Image processing method, device, storage medium and processor

Country Status (1)

Country Link
CN (1) CN112214188B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030002608A (en) * 2001-06-29 2003-01-09 주식회사 성진씨앤씨 Method of adaptive noise smoothing/restoration in spatio-temporal domain and high-definition image capturing device thereof
CN1655214A (en) * 2004-02-11 2005-08-17 乐金电子(中国)研究开发中心有限公司 Mobile communication terminal display drive device and method
CN1694058A (en) * 2005-04-30 2005-11-09 广东威创日新电子有限公司 Remote display processing method based on server end/client end structure
CN101667123A (en) * 2009-09-24 2010-03-10 中兴通讯股份有限公司 Realization method of irregular window and mobile handheld device
CN103426419A (en) * 2012-05-23 2013-12-04 腾讯科技(深圳)有限公司 Method and device for refreshing areas
CN104598018A (en) * 2013-10-31 2015-05-06 株式会社东芝 Image display device and image display method
CN104735449A (en) * 2015-02-27 2015-06-24 成都信息工程学院 Image transmission method and system based on rectangular segmentation and interlaced scanning
CN104994153A (en) * 2015-06-30 2015-10-21 Tcl移动通信科技(宁波)有限公司 Image transmission method and system of WIFI DISPLAY
CN106873929A (en) * 2016-12-19 2017-06-20 奇酷互联网络科技(深圳)有限公司 Image refreshing method and apparatus based on virtual reality technology
CN109168071A (en) * 2018-09-21 2019-01-08 西安万像电子科技有限公司 Image processing method and device

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030002608A (en) * 2001-06-29 2003-01-09 주식회사 성진씨앤씨 Method of adaptive noise smoothing/restoration in spatio-temporal domain and high-definition image capturing device thereof
CN1655214A (en) * 2004-02-11 2005-08-17 乐金电子(中国)研究开发中心有限公司 Mobile communication terminal display drive device and method
CN1694058A (en) * 2005-04-30 2005-11-09 广东威创日新电子有限公司 Remote display processing method based on server end/client end structure
CN101667123A (en) * 2009-09-24 2010-03-10 中兴通讯股份有限公司 Realization method of irregular window and mobile handheld device
CN103426419A (en) * 2012-05-23 2013-12-04 腾讯科技(深圳)有限公司 Method and device for refreshing areas
CN104598018A (en) * 2013-10-31 2015-05-06 株式会社东芝 Image display device and image display method
CN104735449A (en) * 2015-02-27 2015-06-24 成都信息工程学院 Image transmission method and system based on rectangular segmentation and interlaced scanning
CN104994153A (en) * 2015-06-30 2015-10-21 Tcl移动通信科技(宁波)有限公司 Image transmission method and system of WIFI DISPLAY
CN106873929A (en) * 2016-12-19 2017-06-20 奇酷互联网络科技(深圳)有限公司 Image refreshing method and apparatus based on virtual reality technology
CN109168071A (en) * 2018-09-21 2019-01-08 西安万像电子科技有限公司 Image processing method and device

Also Published As

Publication number Publication date
CN112214188A (en) 2021-01-12

Similar Documents

Publication Publication Date Title
CN109388448B (en) Image display method, display system, and computer-readable storage medium
CN105955687B (en) Image processing method, device and system
CN109472839B (en) Image generation method and device, computer equipment and computer storage medium
CN113222874B (en) Data enhancement method, device, equipment and storage medium applied to target detection
CN112999654B (en) Image processing method, image processing device, computer equipment and storage medium
TW200306524A (en) Character display apparatus and character display method, control program for controlling the character display method and recording medium recording the control program
US11321812B2 (en) Display method, display device, virtual reality display device, virtual reality device, and storage medium
CN112214188B (en) Image processing method, device, storage medium and processor
CN108230454B (en) Panorama picture cutting method and device and storage medium
EP3301908A1 (en) Numerical image conversion method and device, and storage medium and device
JP4063918B2 (en) Apparatus and method for creating an image containing graphics information for display
TWI356394B (en) Image data generating device, image data generatin
CN113316018A (en) Method, device and storage medium for overlaying time information on video picture display
US20240037701A1 (en) Image processing and rendering
CN118214817A (en) Method, device, equipment, system and storage medium for layer processing
US7646513B2 (en) Image processing device and method thereof
JP6002263B2 (en) Data display system
CN115379180A (en) Projection picture adjusting method, device, equipment and storage medium
CN114757861A (en) Texture image fusion method and device, computer equipment and readable medium
WO2007135807A1 (en) Image processing system and image processing program
JPWO2020166596A1 (en) Image processing system and program
JP2005149439A (en) Image processor and image processing method
KR101684834B1 (en) Method, device and system for resizing and restoring original depth frame
WO2024150430A1 (en) Image processing device, image processing method, and image processing program
JP2000315073A (en) Device and method for plotting image, and computer- readable recording medium where program making computer implement the method is stored

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant