Image shadow removing method and apparatus
Technical field
The present invention relates to image processing fields, in particular to a kind of image shadow removing method and apparatus.
Background technology
Shade is prevalent among vedio data, although shade can be people perceive scene illumination, the depth of field with
And body form etc. provides auxiliary information, but the presence of shade also increases the difficulty of many image processing tasks, such as scheming
As in segmentation task, shadow edge is often difficult to distinguish with object boundary;In the tasks such as object identification, target tracking, shade
In the presence of often changing to the material striated band of object, the performance of related algorithm is affected.In addition, being needed for vision perception
It asks, people are often desirable to go the associated shadow for influencing visual experience in scene in the post-production of some image/videos
It removes.For this purpose, shadow removing is an important image processing tasks.
Shadow removing generally comprises two tasks of Shadow recognition and illumination restoration, in existing illumination restoration task, one
As using based on region and be based on two class method of shadow edge, the illumination restoration method based on region use Region Matching side
Formula is found non-hatched area progress Lighting information similar with shadow region material and is supplied in the scene, and is based on shadow edge
Illumination restoration method be then using shadow edge both sides illumination variation information to shadow region carry out illumination restoration.It can see
The Lighting information excavation for going out scene is the key that realize image shadow removing, but existing shadow removing method is all only in part
Information excavating is carried out in region to supply, and cannot retain all kinds of material characteristics, treatment effeciency and quality are all difficult to make us meeting.
In illumination restoration task in for the above-mentioned prior art can only localized region information carry out information excavating supply
And the problem of all kinds of material characteristics cannot be retained, currently no effective solution has been proposed.
Invention content
An embodiment of the present invention provides a kind of image shadow removing method and apparatus, at least to solve light in the prior art
According in recovery tasks can only localized region information carry out information excavating and supply and the technologies of all kinds of material characteristics cannot be retained to ask
Topic.
One side according to the ... of the embodiment of the present invention provides a kind of image shadow removing method, including:Determine initial graph
Different types of color line as in;Shadow recognition is carried out to initial pictures, obtains Shadow recognition result;According to Shadow recognition knot
Fruit carries out illumination restoration to the shadow region of different types of color line, obtains shadow removing image.
Another aspect according to the ... of the embodiment of the present invention additionally provides a kind of image shadow removing device, including:First determines
Module, for determining different types of color line in initial pictures;First identification module, for carrying out shade knowledge to initial pictures
Not, Shadow recognition result is obtained;First illumination restoration module is used for according to Shadow recognition as a result, to different types of color line
Shadow region carry out illumination restoration, obtain shadow removing image.
Another aspect according to the ... of the embodiment of the present invention, additionally provides a kind of storage medium, and storage medium includes the journey of storage
Sequence, wherein equipment executes above-mentioned image shadow removing method where controlling storage medium when program is run.
Another aspect according to the ... of the embodiment of the present invention, additionally provides a kind of computer equipment, including memory, processor and
The computer program that can be run on a memory and on a processor is stored, processor realizes above-mentioned image shade when executing program
Removing method.
In embodiments of the present invention, by determining different types of color line in initial pictures;Initial pictures are carried out cloudy
Shadow identifies, obtains Shadow recognition result;According to Shadow recognition as a result, the shadow region to different types of color line carries out illumination
Restore, obtain shadow removing image, illumination restoration is carried out to shade by non local color line, it is high-quality to realize offer
The shadow removing image of amount, targetedly handles all kinds of materials, retains the technique effect of all kinds of material characteristics, and then solves existing
Have in the illumination restoration task in technology can only localized region information carry out information excavating and supply and all kinds of materials cannot be retained
The technical issues of characteristic.
Description of the drawings
Attached drawing described herein is used to provide further understanding of the present invention, and is constituted part of this application, this hair
Bright illustrative embodiments and their description are not constituted improper limitations of the present invention for explaining the present invention.In the accompanying drawings:
Fig. 1 is a kind of schematic diagram of image shadow removing method according to the ... of the embodiment of the present invention;And
Fig. 2 is a kind of schematic diagram of image shadow removing device according to the ... of the embodiment of the present invention.
Specific implementation mode
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The every other embodiment that member is obtained without making creative work should all belong to the model that the present invention protects
It encloses.
It should be noted that term " first " in description and claims of this specification and above-mentioned attached drawing, "
Two " etc. be for distinguishing similar object, without being used to describe specific sequence or precedence.It should be appreciated that using in this way
Data can be interchanged in the appropriate case, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " comprising " and " having " and their any deformation, it is intended that cover
It includes to be not necessarily limited to for example, containing the process of series of steps or unit, method, system, product or equipment to cover non-exclusive
Those of clearly list step or unit, but may include not listing clearly or for these processes, method, product
Or the other steps or unit that equipment is intrinsic.
Embodiment 1
According to embodiments of the present invention, a kind of embodiment of the method for image shadow removing method is provided, it should be noted that
Step shown in the flowchart of the accompanying drawings can execute in the computer system of such as a group of computer-executable instructions, and
It, in some cases, can be to execute institute different from sequence herein and although logical order is shown in flow charts
The step of showing or describing.
Fig. 1 is image shadow removing method according to the ... of the embodiment of the present invention, as shown in Figure 1, this method comprises the following steps:
Step S102 determines different types of color line in initial pictures;
Step S104 carries out Shadow recognition to initial pictures, obtains Shadow recognition result;
Step S106 is obtained according to Shadow recognition as a result, the shadow region to different types of color line carries out illumination restoration
To shadow removing image.
Specifically, for the identical pixel of material in initial pictures, their reflectivity having the same and different illumination
Decay factor, therefore these pixels can form a color line for crossing origin in RGB color, that is, in RGB color
Different color lines is to represent the pixel of different materials, and point different on same color line then illustrates different illumination
Situation.
In embodiments of the present invention, by determining different types of color line in initial pictures;Initial pictures are carried out cloudy
Shadow identifies, obtains Shadow recognition result;According to Shadow recognition as a result, the shadow region to different types of color line carries out illumination
Restore, obtain shadow removing image, illumination restoration is carried out to shade by non local color line, it is high-quality to realize offer
The shadow removing image of amount, targetedly handles all kinds of materials, retains the technique effect of all kinds of material characteristics, and then solves existing
Have in the illumination restoration task in technology can only localized region information carry out information excavating and supply and all kinds of materials cannot be retained
The technical issues of characteristic.
In a kind of optional embodiment, different types of color line in initial pictures is determined in step S102, including:
Step S202 carries out offset correction to the color line in initial pictures, obtains color line skew correction image;
Step S204 corrects the color line in image to color line skew and clusters, obtains different types of color line.
In a kind of optional embodiment, offset correction is carried out to the color line in initial pictures in step S202, is obtained
Color line skew corrects image, including:
Step S302 calculates one of shade in initial pictures and exists with the color line representated by non-shadow juncture area
Offset in RGB color;
Step S304, according to the corresponding color line deviation of each pixel in offset calculating initial pictures;
Step S306 carries out offset correction to the color line in initial pictures according to color line deviation, obtains color line
Offset correction image.
Specifically, calculating one of shade and the face representated by non-shadow juncture area in initial pictures in step S302
When offset of the colo(u)r streak in RGB color, principal direction that can be by the pixel of the juncture area in RGB color
It is worth to pixel in the juncture area, wherein principal direction can analyze to obtain by the method for principal component analysis, it is assumed that
Principal direction indicates that mean value is indicated with p with v, juncture area muIt indicates, the color line representated by juncture area is in RGB color
Offset D (m in spaceu) indicate, then just like following formula 1:
In equation 1 above, ‖ v ‖ are indicated to v modulus.
In step S304 when color line deviation corresponding according to each pixel in offset calculating initial pictures, specifically may be used
Existed with the pixel of offset and the juncture area of the color line representated by above-mentioned juncture area in RGB color
Principal direction in RGB color is calculated, if initial pictures are indicated with I (x), each pixel corresponds in initial pictures I (x)
Color line deviation with D (x) indicate, then just like following formula 2:
Offset correction is carried out to the color line in initial pictures according to color line deviation in step S306, obtains color line
When offset correction image, initial pictures are subtracted into the corresponding color line deviation of each pixel in initial pictures, you can obtain face
Colo(u)r streak offset correction image, if color line skew correction image is usedIt indicates, then just like following formula 3:
Equation 3 above can also be decomposed into such as following formula 4:
In formula 4, S (x) indicates that the illumination decay factor figure corresponding to color line skew correction image, L indicate line skew school
The fixed illumination constant of scene global where positive image, R (x) indicate the corresponding reflectivity of each pixel, wherein in nonshaded area
Domain, S (x) values be equal to or slightly less than 1, in penumbra region, S (x) is between 0 to 1, in umbra region, S (x) be equal to or
Slightly larger than 0.
The color line in image is corrected in a kind of optional embodiment, in step S204 to color line skew to gather
Class obtains different types of color line, specially:The RGB color vector of each pixel in image is corrected using color line skew
Direction color line is clustered, what is clustered different classes of represents different color lines.
In a kind of optional embodiment, after different types of color line is obtained in step S204, further include:Step
S402 determines the illumination decay factor figure of color line skew correction image;Shadow recognition is carried out to initial pictures in step S104,
Shadow recognition is obtained as a result, including:Step S502 carries out Shadow recognition to initial pictures using illumination decay factor figure, obtains
Shadow recognition result.
Specifically, after obtaining different types of color line in step S204, in each classification, it is worth maximum pixel and represents
The case where illumination decay factor is 1, and the reflectivity of each classification is identical, therefore can utilize each in each classification
Ratio between the maximum pixel of the corresponding class label of pixel obtains the illumination decay factor of color line skew correction image
Figure, wherein illumination decay factor figure can use S (x) to indicate.
In a kind of optional embodiment, Shadow recognition result includes umbra region and the penumbra identified in initial pictures
Region, according to Shadow recognition as a result, carrying out illumination restoration to the shadow region of different types of color line and including in step S106:
Step S602, according to Shadow recognition as a result, umbra region and penumbra region to different types of color line carry out illumination restoration.
Specifically, carrying out Shadow recognition to initial pictures in step S104, Shadow recognition is obtained as a result, being specifically as follows:
Selected part shadows pixels and non-shadow pixel, training obtain KNN graders (N=3) and carry out Shadow recognition to initial pictures,
And distinguish the umbra region in image, penumbra region and non-hatched area using illumination decay factor figure S (x), wherein this
Shadow zone domain can indicate that penumbra region can be indicated with P with U, and non-hatched area can be indicated with N.
According to Shadow recognition as a result, umbra region and penumbra region to different types of color line carry out in step S602
Illumination restoration, wherein when carrying out illumination restoration to umbra region, may be used such as following formula 5:
In equation 5 above, E () indicates that mean value is sought, and SD () indicates that standard deviation is sought, and m indicates the index of color line,
Indicate the shadow removing image in the umbra region corresponding to color line m, Nm,UmThe nonshaded area corresponding to color line m is indicated respectively
The pixel set in domain and umbra region,Indicate that color line skew corrects imageIn non-hatched area,It indicates
Color line skew corrects imageIn umbra region.
It, can be right using such as following formula 6 first with illumination decay factor figure S (x) when carrying out illumination restoration to penumbra region domain
Shadow factor K (the P in penumbra regionm) carry out linear interpolation:
Wherein, PmIndicate the pixel set in the penumbra region corresponding to color line m, S (Um) indicate in illumination decay factor figure
Umbra region, S (Pm) indicate illumination decay factor figure in penumbra region, S (Nm) indicate illumination decay factor figure in non-the moon
Shadow zone domain utilizes K (Pm), the illumination restoration result in penumbra region can be obtained as shown in following formula 7
To sum up, according to the illumination restoration result in umbra regionThe illumination restoration result in penumbra regionAnd face
Colo(u)r streak deviates correcting imageIn non-hatched areaColor line skew correcting image can be obtainedIt is corresponding
Shadow removing image
In a kind of optional embodiment, after shadow removing image is obtained in step S106, method further includes:Step
S702 carries out local smoothing method optimization processing to the Lighting information in shadow removing image after illumination restoration, obtains illumination optimization
Image.
Specifically, the non-local information in initial pictures is mainly utilized in illumination restoration part in the embodiment of the present invention, and
Illumination variation often with local smoothing method variation characteristic, use by local smoothing method optimization processing, the light of image can be made
It is more smooth natural according to variation.
In a kind of optional embodiment, to the Lighting information in shadow removing image after illumination restoration in step S702
Local smoothing method optimization processing is carried out, including:Step S802, using minimum energy equation to extensive through illumination in shadow removing image
Lighting information after multiple carries out local smoothing method optimization processing.
Specifically, minimizing energy equation as shown in following formula 8:
In equation 8 above,For shadow factor, K ' (x) indicates the shadow factor after optimization, σs(x) it is every color
The standard deviation of the corresponding Lighting information S (x) of line, λ are smoothing factors, can choose λ=0.001, NxIt is the corresponding four neighbours pictures of x
Plain coordinate set.Finally, the corresponding illumination optimization image I of initial pictures I (x)fIt (x) can be as shown in following formula 9:
Herein it should be noted that the image shadow removing method of above-mentioned all embodiments can be answered as image preprocessing
Use a variety of field of machine vision.
Embodiment 2
According to embodiments of the present invention, a kind of product embodiments of image shadow removing device are provided, Fig. 2 is according to this hair
The image shadow removing device of bright embodiment, as shown in Fig. 2, the device includes the first determining module, the first identification module and the
One illumination restoration module, wherein the first determining module, for determining different types of color line in initial pictures;First identification
Module obtains Shadow recognition result for carrying out Shadow recognition to initial pictures;First illumination restoration module, for according to the moon
Shadow recognition result carries out illumination restoration to the shadow region of different types of color line, obtains shadow removing image.
In embodiments of the present invention, different types of color line in initial pictures is determined by the first determining module;First
Identification module carries out Shadow recognition to initial pictures, obtains Shadow recognition result;First illumination restoration module is according to Shadow recognition
As a result, the shadow region to different types of color line carries out illumination restoration, shadow removing image is obtained, non local face is passed through
Colo(u)r streak carries out illumination restoration to shade, to realize the shadow removing image for providing high quality, targetedly handles all kinds of materials
Matter, retains the technique effect of all kinds of material characteristics, and then solving can only be to part in illumination restoration task in the prior art
Area information carries out the technical issues of information excavating is supplied and cannot retain all kinds of material characteristics.
Herein it should be noted that above-mentioned first determining module, the first identification module and the first illumination restoration module correspond to
Step S102 to step S106 in embodiment 1, the example and application scenarios phase that above-mentioned module and corresponding step are realized
Together, but it is not limited to the above embodiments 1 disclosure of that.It should be noted that above-mentioned module can be with as a part for device
It is executed in the computer system of such as a group of computer-executable instructions.
In a kind of optional embodiment, the first determining module includes the first offset rectification module and cluster module, wherein
First offset rectification module obtains color line skew correction image for carrying out offset correction to the color line in initial pictures;
Cluster module clusters for correcting the color line in image to color line skew, obtains different types of color line.
Herein it should be noted that above-mentioned first offset rectification module and cluster module correspond to the step in embodiment 1
S202 to step S204, above-mentioned module is identical as example and application scenarios that corresponding step is realized, but is not limited to above-mentioned reality
Apply 1 disclosure of that of example.It should be noted that above-mentioned module can be such as one group of computer can as a part of of device
It is executed in the computer system executed instruction.
In a kind of optional embodiment, the first offset correction module include the first computing module, the second computing module and
First offset correction module, wherein the first computing module has a common boundary for calculating one of shade in initial pictures with non-shadow
Offset of the color line in RGB color representated by region;Second computing module, it is initial for being calculated according to offset
The corresponding color line deviation of each pixel in image;First offset correction module is used for according to color line deviation to initial
Color line in image carries out offset correction, obtains color line skew correction image.
Herein it should be noted that above-mentioned first computing module, the second computing module and the first offset correction module correspond to
Step S302 to step S306 in embodiment 1, the example and application scenarios phase that above-mentioned module and corresponding step are realized
Together, but it is not limited to the above embodiments 1 disclosure of that.It should be noted that above-mentioned module can be with as a part for device
It is executed in the computer system of such as a group of computer-executable instructions.
In a kind of optional embodiment, device further includes the second determining module, not of the same race for being obtained in cluster module
After the color line of class, the illumination decay factor figure of color line skew correction image is determined;First identification module, including the second knowledge
Other module obtains Shadow recognition result for carrying out Shadow recognition to initial pictures using illumination decay factor figure.
Herein it should be noted that above-mentioned second determining module and the second identification module correspond to the step in embodiment 1
S402 to step S502, above-mentioned module is identical as example and application scenarios that corresponding step is realized, but is not limited to above-mentioned reality
Apply 1 disclosure of that of example.It should be noted that above-mentioned module can be such as one group of computer can as a part of of device
It is executed in the computer system executed instruction.
In a kind of optional embodiment, Shadow recognition result includes umbra region and the penumbra identified in initial pictures
Region, the first illumination restoration module include:Second illumination restoration module is used for according to Shadow recognition as a result, to different types of
The umbra region of color line and penumbra region carry out illumination restoration.
Herein it should be noted that above-mentioned second illumination restoration module corresponds to the step S602 in embodiment 1, above-mentioned mould
Block is identical as example and application scenarios that corresponding step is realized, but is not limited to the above embodiments 1 disclosure of that.It needs
Illustrate, above-mentioned module can be in the computer system of such as a group of computer-executable instructions as a part of of device
It executes.
In a kind of optional embodiment, device further includes:First optimization module, for being obtained in the first illumination restoration module
To after shadow removing image, the Lighting information in shadow removing image after illumination restoration is carried out at local smoothing method optimization
Reason obtains illumination optimization image.
Herein it should be noted that above-mentioned first optimization module correspond to embodiment 1 in step S702, above-mentioned module with
The example that corresponding step is realized is identical with application scenarios, but is not limited to the above embodiments 1 disclosure of that.It needs to illustrate
, above-mentioned module can hold as a part of of device in the computer system of such as a group of computer-executable instructions
Row.
In a kind of optional embodiment, the first optimization module, including the second optimization module, for using minimum energy
Equation carries out local smoothing method optimization processing to the Lighting information in shadow removing image after illumination restoration.
Herein it should be noted that above-mentioned second optimization module correspond to embodiment 1 in step S802, above-mentioned module with
The example that corresponding step is realized is identical with application scenarios, but is not limited to the above embodiments 1 disclosure of that.It needs to illustrate
, above-mentioned module can hold as a part of of device in the computer system of such as a group of computer-executable instructions
Row.
Embodiment 3
According to embodiments of the present invention, a kind of product embodiments of storage medium are provided, which includes storage
Program, wherein equipment executes above-mentioned image shadow removing method where controlling storage medium when program is run.
Embodiment 4
According to embodiments of the present invention, a kind of product embodiments of processor are provided, which is used to run program,
In, program executes above-mentioned image shadow removing method when running.
Embodiment 5
According to embodiments of the present invention, a kind of product embodiments of computer equipment, the computer equipment are provided, including is deposited
Reservoir, processor and storage on a memory and the computer program that can run on a processor, reality when processor executes program
Existing above-mentioned image shadow removing method.
Embodiment 6
According to embodiments of the present invention, a kind of product embodiments of terminal are provided, which includes the first determining module, the
One identification module, the first illumination restoration module and processor, wherein the first determining module is different in initial pictures for determining
The color line of type;First identification module obtains Shadow recognition result for carrying out Shadow recognition to initial pictures;First light
According to recovery module, it is used for according to Shadow recognition the shadow region progress illumination restoration as a result, to different types of color line, is obtained
Shadow removing image;Processor, processor run program, wherein for being identified from the first determining module, first when program is run
Module and the data of the first illumination restoration module output execute above-mentioned image shadow removing method.
Embodiment 7
According to embodiments of the present invention, a kind of product embodiments of terminal are provided, which includes the first determining module, the
One identification module, the first illumination restoration module and storage medium, wherein the first determining module, for determining in initial pictures not
Congener color line;First identification module obtains Shadow recognition result for carrying out Shadow recognition to initial pictures;First
Illumination restoration module is used for according to Shadow recognition the shadow region progress illumination restoration as a result, to different types of color line, obtains
To shadow removing image;Storage medium, for storing program, wherein program is at runtime for from the first determining module, first
Identification module and the data of the first illumination restoration module output execute above-mentioned image shadow removing method.
The embodiments of the present invention are for illustration only, can not represent the quality of embodiment.
In the above embodiment of the present invention, all emphasizes particularly on different fields to the description of each embodiment, do not have in some embodiment
The part of detailed description may refer to the associated description of other embodiment.
In several embodiments provided herein, it should be understood that disclosed technology contents can pass through others
Mode is realized.Wherein, the apparatus embodiments described above are merely exemplary, for example, the unit division, Ke Yiwei
A kind of division of logic function, formula that in actual implementation, there may be another division manner, such as multiple units or component can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be INDIRECT COUPLING or communication link by some interfaces, unit or module
It connects, can be electrical or other forms.
The unit illustrated as separating component may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple
On unit.Some or all of unit therein can be selected according to the actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it can also
It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.Above-mentioned integrated list
The form that hardware had both may be used in member is realized, can also be realized in the form of SFU software functional unit.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can be stored in a computer read/write memory medium.Based on this understanding, technical scheme of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes:USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic disc or CD etc. are various can to store program code
Medium.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.