CN108681991A - Based on the high dynamic range negative tone mapping method and system for generating confrontation network - Google Patents
Based on the high dynamic range negative tone mapping method and system for generating confrontation network Download PDFInfo
- Publication number
- CN108681991A CN108681991A CN201810299749.9A CN201810299749A CN108681991A CN 108681991 A CN108681991 A CN 108681991A CN 201810299749 A CN201810299749 A CN 201810299749A CN 108681991 A CN108681991 A CN 108681991A
- Authority
- CN
- China
- Prior art keywords
- dynamic range
- high dynamic
- negative tone
- confrontation network
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013507 mapping Methods 0.000 title claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 34
- 230000006870 function Effects 0.000 claims abstract description 27
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 8
- 238000013139 quantization Methods 0.000 claims description 6
- 230000004069 differentiation Effects 0.000 claims description 5
- 238000010200 validation analysis Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000009021 linear effect Effects 0.000 abstract description 4
- 238000011156 evaluation Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 2
- 239000004615 ingredient Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 208000001613 Gambling Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009022 nonlinear effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of high dynamic range negative tone mapping method and system based on generation confrontation network, wherein：Original high dynamic range video is read, cutting, which is converted into, can be used for trained standard dynamic range data set corresponding with high dynamic range；A kind of generation confrontation network connected with great-jump-forward based on convolutional neural networks is established, converts standard dynamic range image to high dynamic range images, i.e. negative tone maps；According to the integrated objective function of setting, continues to optimize entire generate of training and fight network, finally obtained network can complete the mapping from standard dynamic range to high dynamic range.Present invention improves existing non-learning method it is non-linear insufficient, parameter regulation is complicated the problems such as, and in view of the one-dimensional and gradient characteristics of high dynamic range images, the negative tone mapping to high dynamic range has been better achieved.
Description
Technical field
It is specifically a kind of dynamic based on the height for generating confrontation network the present invention relates to a kind of method of technical field of image processing
State range negative tone mapping method and system.
Background technology
Show that high dynamic range (HDR) is undoubtedly a main trend in technique for taking field in current TV.High dynamic
Range is a kind of with the completely different technology of existing display technology, this is mainly reflected in it on the shifting gears of picture.
Under the action of high dynamic range, the color that TV is presented is more lively, and black is deeper, and the object in picture is also more clear
It is illustrated.At the same time, the tone of picture can be also extended, and be expanded by the BT.709 standards under widely used standard dynamic range
To the BT.2020 standards of high dynamic range.Brightness is the key that high dynamic range：Most of TV brightness is 400 on the market
Nit or so, part type have reached 750 nits.But the maximum brightness of high dynamic range TV is up to 1000 nits, brightness
Promotion can allow scene to seem more true, especially Outdoor Scene.Photographing request due to high dynamic range content and shooting generation
Valence is high, it is difficult to by a large amount of high dynamic range content of direct shooting and producing, therefore using in existing standard dynamic range
Hold and make high dynamic range content as an of great value direction, this conversion is known as negative tone mapping.
Due to the method that traditional method is based not on study, essence is subsection compression, the non-linear property of method
It can be insufficient with operability.Method of some occurred in recent years based on study is seldom examined primarily directed to the expansion of brightness domain
Consider the expansion of colour gamut, therefore causes viewing experience bad.
Invention content
The present invention is directed to the defect of existing negative tone mapping techniques, provides a kind of based on the high dynamic for generating confrontation network
Range negative tone mapping method, this method fight net by considering the expansion of brightness range and Color Range using generating
Network makes up the nonlinear deficiency of conventional method, devises a kind of generation confrontation model based on convolutional neural networks, to reach
To visually with better conversion effect on objective evaluation.
The first object of the present invention is to provide a kind of high dynamic range negative tone mapping method based on generation confrontation network,
Including：
S1：Original high dynamic range video is read, standard dynamic range image is sheared and be converted into, with high dynamic range figure
The data set for having supervision as forming, as subsequent training dataset and validation data set；
S2：It establishes the generation connected with great-jump-forward based on convolutional neural networks and fights network；
S3：Network is fought to the generation, the target loss function integrated by one-dimensional characteristic and Gradient Features is established, uses
The training dataset constantly training optimization obtains the generation confrontation network that can complete negative tone mapping；
S4：The validation data set, which is input to, can complete the generation confrontation network model of negative tone mapping, and mapping obtains
High dynamic range images.
Preferably, by HDRTools that existing HDR videos is single by scene extraction when establishing data set in the S1
Vertical frame dimension dynamic image, and piecemeal cuts into the image of low resolution for network training.Further, part of the present invention is real
Apply in example is that the high dynamic range images of several 512 × 512 resolution ratio are intercepted out from the video of 4K resolution ratio as data set
In high dynamic range images, reuse Reinhard tone-mapping algorithms, by high dynamic range images be converted into standard dynamic
Range image forms one-to-one data set for training and verifying.
It is used to train it is highly preferred that the data set includes multigroup standard dynamic range and the corresponding image of high dynamic range
Model, in addition also multigroup for verifying, wherein high dynamic range images are the exr formatted files of 10 bit quantizations, are used
BT.2020 standard color gamuts, maximum brightness are 1000 nits；Standard dynamic range image is the png formatted files of 8 bit quantizations,
Using BT.709 standard color gamuts, maximum brightness is 100 nits.In order to easy to use in the training described in S3, when getImage
It is stored in h5 files using vector.Data above collection uses when being section Example of the present invention, can also choose other high dynamics
The corresponding standard dynamic range image construction data set of range image, but need using same quantization digit, colour gamut
Range and maximum brightness.
Preferably, the target loss function is determined by the one-dimensional characteristic of image and the Gradient Features of all directions；
If L, H indicates standard dynamic range input and high dynamic range output respectively, G, D indicate generator and differentiation respectively
Device, it may be considered that the entire target loss function for generating confrontation network is a Minimax Problems：
Wherein：Indicate the content loss function of generator,Indicate pair of generator and arbiter
Anti- loss function, λ are hyper parameters, control the proportion of two kinds of loss function ingredients.It can be written as：
α indicates the weight of one-dimensional characteristic and Gradient Features, dx、dyThe gradient in transverse and longitudinal direction in image is indicated respectively.It can be expressed as：
The high dynamic range negative tone mapping method based on generation confrontation network that the present invention designs, has in view of image
While some one-dimensional characteristics, it is also contemplated that the Gradient Features of high dynamic range.Specifically, negative tone maps corresponding pixel points
Value not only determined by the standard dynamic range pixel of current location, it is also related to the variation tendency of surrounding pixel point, because
This use calculates Gradient Features to calculated pixel and surrounding pixel point, 2 norms is used in the present invention, while using volume
Product operation is so that this operating effect is more preferable.By this setting, generation pair is trained up in the training dataset using foundation
After anti-network, good visual effect and evaluation score can be reached.
The present invention second is designed to provide a kind of high dynamic range negative tone mapped system based on generation confrontation network,
Including：Processor and memory, have program stored therein in the memory instruction, and the processor is for transferring described program instruction
It is above-described based on the high dynamic range negative tone mapping method for generating confrontation network to execute.
Compared with prior art, the present invention has following advantageous effect：
The method of the invention and system consider it is existing expanded based on brightness in learning method while, also take into account
The expansion of colour gamut, there is a better visual effect.
Compared with widely used non-learning method, the method for the invention and system have better non-linear behaviour and
Multi-scale transform performance.It ensure that the theory of negative tone mapping is complete simultaneously, improving visual effect and the visitor of mapping result
See evaluation index.
Description of the drawings
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the flow chart of one embodiment of the invention method；
Fig. 2 is the network structure block diagram that confrontation network is generated in one embodiment of the invention；
Fig. 3 is the Contrast on effect that result and existing method are generated in one embodiment of the invention.
Specific implementation mode
With reference to specific embodiment, the present invention is described in detail.Following embodiment will be helpful to the technology of this field
Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill of this field
For personnel, without departing from the inventive concept of the premise, various modifications and improvements can be made.These belong to the present invention
Protection domain.
Shown in referring to Fig.1, the flow of the invention based on the high dynamic range negative tone mapping method for generating confrontation network
Figure, mentality of designing are：
1. reading original high dynamic range video, standard dynamic range image is sheared and is converted into, with high dynamic range figure
As composition has the data set of supervision, for training in 3. and 4. in verification；
2. based on convolutional neural networks, establish and generate confrontation network, wherein generator is connected comprising great-jump-forward, is obtained
It generates confrontation network and negative tone map operation can be completed by training 3., the verification result in 4.；
3. establishing by the target loss function of one-dimensional characteristic and Gradient Features synthesis, the training data established in use 1.
Collection, constantly training optimization obtain to complete the generation confrontation network model of negative tone mapping；
4. the data set for verification established in 1. is input to and fights network model through 1.-generation for 3. establishing, reflect
It penetrates and obtains high dynamic range images, and output result is evaluated.
Wherein the 1., 2. step establishes data set and generates confrontation network, the 3. step pass through the target loss function instruction that sets
The generation confrontation network model of negative tone mapping can be completed by getting, the 4. walk to the generation that training obtains fight network model into
Row assessment.High dynamic range negative tone mapping generation confrontation network model is described below establishes process.
The detailed technology operation involved in above-mentioned each step is illustrated below by way of specific embodiment, it should be understood that
, it is only section Example of the present invention below.
1. the foundation of high dynamic range data set corresponding with standard dynamic range
When establishing data set, existing HDR videos are pressed into scene, shears and is converted into standard dynamic range image.Make again
With Reinhard tone-mapping algorithms, it converts high dynamic range images to standard dynamic range image, composition is one-to-one
Data set is for training and verifying.
In section Example of the present invention, in order to obtain the high dynamic range images of high quality, existing height is selected
Quality high dynamic range play pieces, with following feature：10bit quantization exr formatted files, 1000 nit peak brightness,
Colour gamut meets BT.2020 standards, and PQ curve processings are encoded using HEVCMain10, color space YUV, sample rate 4:2:
0.Then video will be selected to be the image of single frames 3840 × 2160 by scene cut above by the tool boxes HDRTools, and cut
At 512 × 512 resolution ratio, color space conversion is rgb space, results in the high dynamic range images in data set.
Further, when establishing corresponding standard dynamic range image in supervised learning data set, in standard dynamic range
Appearance is obtained using Reinhard tone mapping operators, and the standard dynamic range content mapped has following feature：8bit amounts
The png formatted files of change, 100 nit peak brightness, colour gamut meet BT.709 standards, color space RGB.
By obtained standard dynamic range image and high dynamic range images linear normalization between [0,1], for net
It is easy to use in network training, it is stored in h5 files correspondingly using vector when getImage.In obtained data set,
Training dataset is by 2660 groups of image constructions, and validation data set is by 140 groups of image constructions.
2. generating the foundation of confrontation network
It generates confrontation network and constitutes i.e. generator, arbiter by two networks, wherein standard dynamic range image is inputted
Generator, the high dynamic range images output predicted, the image that arbiter is used for differentiating input arbiter is to generate also
It is primary high dynamic range images.
When designing generator, main purpose is the further feature of extraction standard dynamic image to represent whole picture figure
Then picture goes out high dynamic range images by this feature reconstruction again.Therefore the generator in the embodiment of the present invention is solved using volume
Code device structure, wherein the standard dynamic range image inputted constantly extracts feature by continuous convolutional layer, convolution among this
The convolution kernel size of layer is constant, and port number is continuously increased, and the size in each channel is gradually reduced, to reach feature extraction
Operation, process are similar to the coding of image information.
Then, the height of original image is constantly gone back by the layer that deconvolutes corresponding with convolutional layer using obtained further feature
Dynamic range information, image channel are reduced to RGB triple channels.Particularly, it is added between corresponding convolutional layer and the layer that deconvolutes
Great-jump-forward connects, and provides more original information for the layer that deconvolutes to go back original image.
In the preferred embodiment of the present invention, generator is a kind of U-NET structures, wherein the convolution of each convolutional layer and the layer that deconvolutes
The size of core is all 3 × 3, stride 2, has batch normalization layer and leaky-RELU as activation letter after every layer of convolution operation
Number, but the activation primitive of last layer is sigmoid, and stride is 2.The port number of 5 layers of convolutional layer of generator is respectively 64,
128,256,512,1024, in addition the port number of the 4 layers of layer that deconvolutes is respectively 512,256,128,64；In addition, generator is defeated
It is 3 to enter layer port number, and output layer port number is 3.It should be noted that not containing Max- in the network structure of the present invention
Pooling layers.
Another part of confrontation network is generated, arbiter is made of convolutional neural networks and full articulamentum, main complete
Pairs of input picture carries out feature extraction, and judges that input picture is the high dynamic range figure generated according to arbiter final output
Picture or primary high dynamic range images, value is between 0-1, it is considered that value is closer to 1, and input picture is it is more likely that original
Raw high dynamic range images；Input is closer to 0, and input picture is it is more likely that the high dynamic range images generated.It is being preferably implemented
In example, arbiter is made of five layers of convolutional layer and two layers of full articulamentum, wherein in convolutional layer first two layers of convolution kernel size be 5 ×
5, latter three layers of convolution kernel size is 3 × 3；The stride of three first layers is 2, and latter two layers of stride is 1；It is similar with generator,
Each convolutional layer has batch normalization operation and a leaky-RELU activation primitives, the port numbers of 5 layers of convolutional layer is respectively 64,
128、128、256、1.It after convolutional layer, outputs it and vector is converted to by Flatten operations, then pass through two layers of full connection
Layer, number of nodes is respectively 1024 and 1.
For above-mentioned network structure as shown in Fig. 2, wherein conv indicates that convolutional layer, deconv indicate the layer that deconvolutes, k is convolution
Core size, n indicate that port number, s indicate that stride sizes, FC indicate full articulamentum.
3. setting object function and training network, the generation confrontation network model that can complete negative tone mapping is obtained
The training principle for generating confrontation network is that the high dynamic range images of "false" are generated by generator, this is generated
High dynamic range images distinguished with primary high dynamic range images input arbiter.On the one hand training process is to allow life
Grow up to be a useful person generation image closer to primary image, to achieve the purpose that cheat arbiter；On the other hand it is trained arbiter to make
It differentiates that the accuracy rate for generating image and primary image is more preferable.It thus generates confrontation network and forms a gambling process, target
Loss function also thus establish by principle.
If L, H indicates standard dynamic range input and high dynamic range output respectively, G, D indicate generator and differentiation respectively
Device, it may be considered that the entire target loss function for generating confrontation network is a Minimax Problems：
Wherein：Indicate the content loss function of generator,Indicate pair of generator and arbiter
Anti- loss function, λ are hyper parameters, control the proportion of two kinds of loss function ingredients.Indicate the relevant loss letter of content
Number proposes a kind of high dynamic range images in view of generating and primary high dynamic range images in embodiments of the present invention
Between mean square error and differential errors losses by mixture function.In embodiments of the present invention,It can be written as：
Wherein：E indicates the expectation of formula in bracket, (L, H)~pdataIt indicates to sample out data group from training data concentration
(L, H), G (L) indicate the high dynamic range images generated, | | | |FIndicate the F- norms of the tensor, dx、dyImage is indicated respectively
In in horizontal and vertical direction each Color Channel differential value.(2) first item is widely applied mean square deviation MSE losses in formula
Function, it is a kind of computational methods of pixel scale；(2) Section 2 is high dynamic range images and primary that comparison generates in formula
The deeper contrast difference of high dynamic range images, is named as dMSE, and the help that is introduced into of dMSE inhibits image mistake in former problem
In smooth problem.
Be generate confrontation network confrontation loss function, obtained by the principle of GAN networks, the computational item with
Generator and arbiter are all related, and concrete form is：
Wherein H~pdataWith L~pdataIt indicates to sample out high dynamic range images from training data concentration respectively and standard is dynamic
State range image.
(1) Minimax Problems in formula are the games generated between better image and more accurately differentiation generation image
Problem generates confrontation network and needs with the trained parameter for carrying out progressive updating generator and arbiter.Particularly, by (1)
When formula is calculated separately for generator and arbiter, it can be written as：
Wherein GkAnd DkGenerator G when being illustrated respectively in kth time iteration and arbiter D.
Principle among these is when training generator, by the content loss function for minimizing generator
Its high dynamic range images generated can be in pixel scale close to primary high dynamic range images.By pair for minimizing network
Anti- loss functionThe image that generator generates " can deceive " arbiter and do the judgement to make mistake as possible, it is believed that its
The image of generation is primary high dynamic range images；Arbiter is with trained progress simultaneously, the image to generation and primary figure
The differentiation accuracy rate of picture gradually increases, and then in next iteration, is differentiated result for training generator.
After the completion of the above training, the generation confrontation network model that can complete negative tone mapping is obtained.
4. implementation condition and outcome evaluation
In section Example of the present invention, code realization is completed by Python, and frame uses TensorFlow.Training process
In, the batch size of each iteration of selection is 6, and optimization method selects RMSProp, and learning rate is by 10-4Starting, with iteration
The increase of number constantly reduces, and reaches 10 after 80,000 iteration-5.Parameter setting in object function, λ are set as 104, α
It is set as 105。
The evaluation index of high dynamic range images is different from standard dynamic range image, generally using HDR-VDP-2 come into
Row objective evaluation, what mass fraction reflected is the high dynamic range images of generation relative to primary high dynamic range images matter
The degree declined is measured, the objective evaluation index to exporting result is got by a mean subjective opinion.In addition to this, it evaluates
Index mPSNR has also been introduced evaluate quality and the SSIM of pixel scale evaluate generate image structural dependence.
The Contrast on effect of table 1 embodiment of the present invention and existing method
The evaluation of result index comparison of result and existing method that the embodiment of the present invention obtains is given in table 1, wherein
Huo, KO are non-learning methods, and parameter setting is executed according to default setting in addition to image attributes；DrTM is based on study
Method, code, which is realized, derives from author.From the results of view, the result of the embodiment of the present invention all has in three kinds of evaluation indexes
Highest score, it is believed that method proposed by the present invention has promotion relative to existing method on generating quality, as a result example
It is referred to Fig. 3.
Fig. 3 is the result of the embodiment of the present invention and the Comparative result of existing method, it can be seen that result of the invention has
Better color performance, and closer primary high dynamic range images.
The present invention also provides a kind of embodiment based on the high dynamic range negative tone mapped system for generating confrontation network, packets
It includes：Processor and memory, have program stored therein in the memory instruction, the processor for transfer described program instruction with
Execute the high dynamic range negative tone mapping method based on generation confrontation network described in Fig. 1, Fig. 2.
The method of the invention and the system enhancement non-linear deficiency of existing non-learning method, parameter regulation complexity etc.
Problem, and in view of the one-dimensional and gradient characteristics of high dynamic range images, preferably realize to the anti-of high dynamic range
Tone mapping.
It should be noted that provided by the invention described based on the high dynamic range negative tone mapping side for generating confrontation network
Step in method can utilize described based on corresponding mould in the high dynamic range negative tone mapped system for generating confrontation network
Block, device, unit etc. are achieved, and the technical solution that those skilled in the art are referred to the system realizes the method
Steps flow chart, that is, the embodiment in the system can be regarded as realizing the preference of the method, and it will not be described here.
One skilled in the art will appreciate that in addition to realizing system provided by the invention in a manner of pure computer readable program code
And its other than each device, completely can by by method and step carry out programming in logic come so that system provided by the invention and its
Each device is in the form of logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc.
To realize identical function.So system provided by the invention and its every device are considered a kind of hardware component, and it is right
The device for realizing various functions for including in it can also be considered as the structure in hardware component；It can also will be for realizing each
The device of kind function is considered as either the software module of implementation method can be the structure in hardware component again.
The description that specific embodiments of the present invention are carried out above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make various deformations or amendments within the scope of the claims, this not shadow
Ring the substantive content of the present invention.
Claims (10)
1. a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, which is characterized in that including：
S1：Original high dynamic range video is read, standard dynamic range image is sheared and be converted into, with high dynamic range images group
At the data set for having supervision, as subsequent training dataset and validation data set；
S2：It establishes the generation connected with great-jump-forward based on convolutional neural networks and fights network；
S3：Network is fought to the generation, the target loss function integrated by one-dimensional characteristic and Gradient Features is established, using described
Training dataset constantly training optimization obtains the generation confrontation network model that can complete negative tone mapping；
S4：The validation data set, which is input to, can complete the generation confrontation network model of negative tone mapping, and mapping obtains high dynamic
State range image.
2. according to claim 1 based on the high dynamic range negative tone mapping method for generating confrontation network, feature exists
In：In the S1, when establishing data set, single frames high dynamic range out is extracted existing HDR videos by scene by HDRTools
Image, and piecemeal cuts into the image of low resolution for network training.
3. according to claim 2 based on the high dynamic range negative tone mapping method for generating confrontation network, feature exists
In：In the S1, the high dynamic range images of several 512 × 512 resolution ratio are intercepted out from the video of 4K resolution ratio as number
According to the high dynamic range images of concentration, Reinhard tone-mapping algorithms are reused, convert high dynamic range images to standard
Dynamic image forms one-to-one data set for training and verifying.
4. a kind of based on the high dynamic range negative tone mapping side for generating confrontation network according to claim 1-3 any one of them
Method, it is characterised in that：The data set includes multigroup standard dynamic range and the corresponding image of high dynamic range for training mould
Type, it is in addition also multigroup for verifying, wherein：
High dynamic range images are the exr formatted files of 10 bit quantizations, and using BT.2020 standard color gamuts, maximum brightness is
1000 nits；
Standard dynamic range image is the png formatted files of 8 bit quantizations, uses BT.709 standard color gamuts, maximum brightness 100
Nit；
In order to easy to use in the training described in S3, when getImage, is stored in using vector in h5 files.
5. it is according to claim 1 a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, it is special
Sign is：In the S2, generates confrontation network and be made of generator and arbiter, standard dynamic range image is inputted and is generated
Device, the high dynamic range images output predicted；Arbiter be used for differentiate input arbiter image be generate or it is former
Raw high dynamic range images.
6. it is according to claim 5 a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, it is special
Sign is：The generator is made of 9 layers of convolutional neural networks, and every layer has batch normalization operation, wherein i-th layer and the
N-i layers have great-jump-forward connection, n 9；Arbiter is made of 5 layers of convolutional neural networks and 2 layers of fully-connected network.
7. it is according to claim 1 a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, it is special
Sign is：In the S3, target loss function is a Minimax Problems：
Wherein：L, H indicates standard dynamic range input and high dynamic range output respectively, and G, D indicate generator and differentiation respectively
Device,Indicate the content loss function of generator,Indicate that letter is lost in the confrontation of generator and arbiter
Number, λ is hyper parameter, controls the proportion of two kinds of loss functions.
8. it is according to claim 7 a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, it is special
Sign is：It is describedIt is written as：
α indicates the weight of one-dimensional characteristic and Gradient Features, dx、dyThe gradient in transverse and longitudinal direction in image is indicated respectively；
It is expressed as：
Wherein：E indicates expectation, (L, H)~pdataIt indicates to sample out data group (L, H) from training data concentration, G (L) indicates to generate
High dynamic range images, | | | |FIndicate the F- norms of the tensor, dx、dyIndicate every in horizontal and vertical direction in image respectively
The differential value of a Color Channel.
9. it is according to claim 7 a kind of based on the high dynamic range negative tone mapping method for generating confrontation network, it is special
Sign is：It is describedIt is obtained by the principle of GAN networks, the computational item and generator and arbiter are all related, tool
Body form is：
Wherein：H~pdataWith L~pdataIt indicates to sample out high dynamic range images and standard dynamic from training data concentration respectively
Range image.
10. it is a kind of based on the high dynamic range negative tone mapped system for generating confrontation network, including：Processor and memory,
It is characterized in that：Have program stored therein instruction in the memory, and the processor is for transferring described program instruction with perform claim
It is required that the high dynamic range negative tone mapping method based on generation confrontation network described in any one of 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810299749.9A CN108681991A (en) | 2018-04-04 | 2018-04-04 | Based on the high dynamic range negative tone mapping method and system for generating confrontation network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810299749.9A CN108681991A (en) | 2018-04-04 | 2018-04-04 | Based on the high dynamic range negative tone mapping method and system for generating confrontation network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108681991A true CN108681991A (en) | 2018-10-19 |
Family
ID=63800834
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810299749.9A Pending CN108681991A (en) | 2018-04-04 | 2018-04-04 | Based on the high dynamic range negative tone mapping method and system for generating confrontation network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108681991A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109618094A (en) * | 2018-12-14 | 2019-04-12 | 深圳市华星光电半导体显示技术有限公司 | Image processing method and image processing system |
| CN109712203A (en) * | 2018-12-29 | 2019-05-03 | 福建帝视信息科技有限公司 | A kind of image rendering methods based on from attention generation confrontation network |
| CN110113353A (en) * | 2019-05-20 | 2019-08-09 | 桂林电子科技大学 | A kind of intrusion detection method based on CVAE-GAN |
| CN110163808A (en) * | 2019-03-28 | 2019-08-23 | 西安电子科技大学 | A kind of single frames high dynamic imaging method based on convolutional neural networks |
| CN110197463A (en) * | 2019-04-25 | 2019-09-03 | 深圳大学 | High dynamic range image tone mapping method and its system based on deep learning |
| CN110225260A (en) * | 2019-05-24 | 2019-09-10 | 宁波大学 | A kind of three-dimensional high dynamic range imaging method based on generation confrontation network |
| CN110493242A (en) * | 2019-08-27 | 2019-11-22 | 上海网达软件股份有限公司 | Method, apparatus and storage medium based on the improved image enhancement of WGAN-GP and U-net |
| CN110610194A (en) * | 2019-08-13 | 2019-12-24 | 清华大学 | Data enhancement method for small data video classification task |
| CN110717868A (en) * | 2019-09-06 | 2020-01-21 | 上海交通大学 | Video high dynamic range inverse tone mapping model construction and mapping method and device |
| CN110728633A (en) * | 2019-09-06 | 2020-01-24 | 上海交通大学 | Multi-exposure high-dynamic-range inverse tone mapping model construction method and device |
| CN111709900A (en) * | 2019-10-21 | 2020-09-25 | 上海大学 | High dynamic range image reconstruction method based on global feature guidance |
| CN111757172A (en) * | 2019-03-29 | 2020-10-09 | Tcl集团股份有限公司 | HDR video acquisition method, HDR video acquisition device and terminal equipment |
| CN111754589A (en) * | 2019-03-26 | 2020-10-09 | 北京沃东天骏信息技术有限公司 | Color matching method and device, computer equipment and storage medium |
| CN111784598A (en) * | 2020-06-18 | 2020-10-16 | Oppo（重庆）智能科技有限公司 | Method for training tone mapping model, tone mapping method and electronic equipment |
| CN112102166A (en) * | 2020-08-26 | 2020-12-18 | 上海交通大学 | Method and device for combining super-resolution, color gamut expansion and inverse tone mapping |
| CN113228660A (en) * | 2018-12-18 | 2021-08-06 | 杜比实验室特许公司 | Machine learning based dynamic synthesis in enhanced standard dynamic range video (SDR +) |
| WO2023010749A1 (en) * | 2021-08-02 | 2023-02-09 | 中国科学院深圳先进技术研究院 | Hdr video conversion method and apparatus, and device and computer storage medium |
| CN117474816A (en) * | 2023-12-26 | 2024-01-30 | 中国科学院宁波材料技术与工程研究所 | High dynamic range image tone mapping method, system and readable storage medium |
| US12003746B2 (en) | 2020-02-19 | 2024-06-04 | Dolby Laboratories Licensing Corporation | Joint forward and backward neural network optimization in image processing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103843058A (en) * | 2011-09-27 | 2014-06-04 | 皇家飞利浦有限公司 | Apparatus and method for dynamic range transforming of images |
| US20150170389A1 (en) * | 2013-12-13 | 2015-06-18 | Konica Minolta Laboratory U.S.A., Inc. | Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification |
| CN107633223A (en) * | 2017-09-15 | 2018-01-26 | 深圳市唯特视科技有限公司 | A kind of video human attribute recognition approach based on deep layer confrontation network |
| US20180084176A1 (en) * | 2016-09-22 | 2018-03-22 | Pinnacle Imaging Corporation | High dynamic range imaging |
-
2018
- 2018-04-04 CN CN201810299749.9A patent/CN108681991A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103843058A (en) * | 2011-09-27 | 2014-06-04 | 皇家飞利浦有限公司 | Apparatus and method for dynamic range transforming of images |
| CN107103588A (en) * | 2011-09-27 | 2017-08-29 | 皇家飞利浦有限公司 | Apparatus and method for the dynamic range transform of image |
| US20150170389A1 (en) * | 2013-12-13 | 2015-06-18 | Konica Minolta Laboratory U.S.A., Inc. | Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification |
| US20180084176A1 (en) * | 2016-09-22 | 2018-03-22 | Pinnacle Imaging Corporation | High dynamic range imaging |
| CN107633223A (en) * | 2017-09-15 | 2018-01-26 | 深圳市唯特视科技有限公司 | A kind of video human attribute recognition approach based on deep layer confrontation network |
Non-Patent Citations (1)
| Title |
|---|
| SHIYU NING ETC: "LEARNING AN INVERSE TONE MAPPING NETWORK WITH A GENERATIVE ADVERSARIAL REGULARIZER", 《ICASSP 2018》 * |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109618094A (en) * | 2018-12-14 | 2019-04-12 | 深圳市华星光电半导体显示技术有限公司 | Image processing method and image processing system |
| US12086969B2 (en) | 2018-12-18 | 2024-09-10 | Dolby Laboratories Licensing Corporation | Machine learning based dynamic composing in enhanced standard dynamic range video (SDR+) |
| CN113228660A (en) * | 2018-12-18 | 2021-08-06 | 杜比实验室特许公司 | Machine learning based dynamic synthesis in enhanced standard dynamic range video (SDR +) |
| CN113228660B (en) * | 2018-12-18 | 2023-12-12 | 杜比实验室特许公司 | Machine learning based dynamic synthesis in enhanced standard dynamic range video (SDR+) |
| CN109712203A (en) * | 2018-12-29 | 2019-05-03 | 福建帝视信息科技有限公司 | A kind of image rendering methods based on from attention generation confrontation network |
| CN111754589A (en) * | 2019-03-26 | 2020-10-09 | 北京沃东天骏信息技术有限公司 | Color matching method and device, computer equipment and storage medium |
| CN111754589B (en) * | 2019-03-26 | 2024-03-01 | 北京沃东天骏信息技术有限公司 | Color matching method, device, computer equipment and storage medium |
| CN110163808A (en) * | 2019-03-28 | 2019-08-23 | 西安电子科技大学 | A kind of single frames high dynamic imaging method based on convolutional neural networks |
| CN110163808B (en) * | 2019-03-28 | 2022-06-10 | 西安电子科技大学 | Single-frame high-dynamic imaging method based on convolutional neural network |
| CN111757172A (en) * | 2019-03-29 | 2020-10-09 | Tcl集团股份有限公司 | HDR video acquisition method, HDR video acquisition device and terminal equipment |
| CN110197463B (en) * | 2019-04-25 | 2023-01-03 | 深圳大学 | High dynamic range image tone mapping method and system based on deep learning |
| CN110197463A (en) * | 2019-04-25 | 2019-09-03 | 深圳大学 | High dynamic range image tone mapping method and its system based on deep learning |
| CN110113353A (en) * | 2019-05-20 | 2019-08-09 | 桂林电子科技大学 | A kind of intrusion detection method based on CVAE-GAN |
| CN110225260A (en) * | 2019-05-24 | 2019-09-10 | 宁波大学 | A kind of three-dimensional high dynamic range imaging method based on generation confrontation network |
| CN110610194A (en) * | 2019-08-13 | 2019-12-24 | 清华大学 | Data enhancement method for small data video classification task |
| CN110493242B (en) * | 2019-08-27 | 2022-02-11 | 上海网达软件股份有限公司 | Method, device and storage medium for improving image enhancement based on WGAN-GP and U-net |
| CN110493242A (en) * | 2019-08-27 | 2019-11-22 | 上海网达软件股份有限公司 | Method, apparatus and storage medium based on the improved image enhancement of WGAN-GP and U-net |
| CN110728633A (en) * | 2019-09-06 | 2020-01-24 | 上海交通大学 | Multi-exposure high-dynamic-range inverse tone mapping model construction method and device |
| CN110728633B (en) * | 2019-09-06 | 2022-08-02 | 上海交通大学 | Multi-exposure high-dynamic-range inverse tone mapping model construction method and device |
| CN110717868A (en) * | 2019-09-06 | 2020-01-21 | 上海交通大学 | Video high dynamic range inverse tone mapping model construction and mapping method and device |
| CN111709900A (en) * | 2019-10-21 | 2020-09-25 | 上海大学 | High dynamic range image reconstruction method based on global feature guidance |
| US12003746B2 (en) | 2020-02-19 | 2024-06-04 | Dolby Laboratories Licensing Corporation | Joint forward and backward neural network optimization in image processing |
| CN111784598A (en) * | 2020-06-18 | 2020-10-16 | Oppo（重庆）智能科技有限公司 | Method for training tone mapping model, tone mapping method and electronic equipment |
| CN112102166A (en) * | 2020-08-26 | 2020-12-18 | 上海交通大学 | Method and device for combining super-resolution, color gamut expansion and inverse tone mapping |
| CN112102166B (en) * | 2020-08-26 | 2023-12-01 | 上海交通大学 | Combined super-resolution, color gamut expansion and inverse tone mapping method and equipment |
| WO2023010749A1 (en) * | 2021-08-02 | 2023-02-09 | 中国科学院深圳先进技术研究院 | Hdr video conversion method and apparatus, and device and computer storage medium |
| CN117474816A (en) * | 2023-12-26 | 2024-01-30 | 中国科学院宁波材料技术与工程研究所 | High dynamic range image tone mapping method, system and readable storage medium |
| CN117474816B (en) * | 2023-12-26 | 2024-03-12 | 中国科学院宁波材料技术与工程研究所 | High dynamic range image tone mapping method, system and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108681991A (en) | Based on the high dynamic range negative tone mapping method and system for generating confrontation network | |
| Ying et al. | From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality | |
| CN109559276B (en) | Image super-resolution reconstruction method based on quality evaluation and feature statistics | |
| CN110363716B (en) | High-quality reconstruction method for generating confrontation network composite degraded image based on conditions | |
| CN110728633B (en) | Multi-exposure high-dynamic-range inverse tone mapping model construction method and device | |
| CN109671023A (en) | A kind of secondary method for reconstructing of face image super-resolution | |
| CN111798400A (en) | Non-reference low-illumination image enhancement method and system based on generation countermeasure network | |
| CN110197468A (en) | A kind of single image Super-resolution Reconstruction algorithm based on multiple dimensioned residual error learning network | |
| CN110717868B (en) | Video high dynamic range inverse tone mapping model construction and mapping method and device | |
| CN109903223A (en) | A kind of image super-resolution method based on dense connection network and production confrontation network | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN108495135A (en) | A kind of fast encoding method of screen content Video coding | |
| CN112614070B (en) | defogNet-based single image defogging method | |
| CN106327443A (en) | Night image enhancement method based on improved genetic algorithm | |
| CN110717953A (en) | Black-white picture coloring method and system based on CNN-LSTM combined model | |
| CN110120034B (en) | Image quality evaluation method related to visual perception | |
| CN117252936A (en) | Infrared image colorization method and system adapting to multiple training strategies | |
| CN111882516B (en) | Image quality evaluation method based on visual saliency and deep neural network | |
| CN107018410A (en) | A kind of non-reference picture quality appraisement method based on pre- attention mechanism and spatial dependence | |
| CN117788873B (en) | Engineering building BIM data security management method | |
| Wu et al. | Remote sensing image colorization based on multiscale SEnet GAN | |
| Oza et al. | Automatic image colorization using ensemble of deep convolutional neural networks | |
| CN118096922A (en) | Method for generating map based on style migration and remote sensing image | |
| CN115953330B (en) | Texture optimization method, device, equipment and storage medium for virtual scene image | |
| CN112508847A (en) | Image quality evaluation method based on depth feature and structure weighted LBP feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181019 |