CN107155110A - A kind of picture compression method based on super-resolution technique - Google Patents
A kind of picture compression method based on super-resolution technique Download PDFInfo
- Publication number
- CN107155110A CN107155110A CN201710446896.XA CN201710446896A CN107155110A CN 107155110 A CN107155110 A CN 107155110A CN 201710446896 A CN201710446896 A CN 201710446896A CN 107155110 A CN107155110 A CN 107155110A
- Authority
- CN
- China
- Prior art keywords
- picture
- layer
- characteristic
- low
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006835 compression Effects 0.000 title claims abstract description 69
- 238000007906 compression Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000009467 reduction Effects 0.000 claims abstract description 19
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 16
- 230000004927 fusion Effects 0.000 claims abstract description 14
- 239000000284 extract Substances 0.000 claims abstract description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 45
- 238000000576 coating method Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 7
- 239000011248 coating agent Substances 0.000 claims description 4
- 241000208340 Araliaceae Species 0.000 claims 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims 1
- 235000003140 Panax quinquefolius Nutrition 0.000 claims 1
- 235000008434 ginseng Nutrition 0.000 claims 1
- 238000003475 lamination Methods 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 3
- 230000008929 regeneration Effects 0.000 abstract description 2
- 238000011069 regeneration method Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 18
- 230000008569 process Effects 0.000 description 8
- 230000003247 decreasing effect Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 5
- 230000001537 neural effect Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000013480 data collection Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 101100365548 Caenorhabditis elegans set-14 gene Proteins 0.000 description 2
- 230000001595 contractor effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention discloses a kind of picture compression method based on super-resolution technique, and it comprises the following steps：Step 1：The user of mobile Internet platform photo to be uploaded is pressed into scalingsLow clear picture is obtained after progress is down-sampled, then by low clear picture compression and is uploaded；Step 2：Mobile Internet platform is stored and distributed to the low clear picture of compression；Step 3：Mobile Internet platform user downloads the low clear picture of compression by different receiving terminals, and receiving terminal carries out corresponding decoding to the low clear picture for downloading compression and obtains low clear picture；Step 4：Decoded low clear picture is reconstructed into high definition picture using the reduction of depth convolutional neural networks model to browse for user；Depth convolutional neural networks model includes low-level image feature and extracts the stage, extracts high-level characteristic stage, Feature-level fusion stage, characteristic layer dimensionality reduction stage and deconvolution stage and phase of regeneration using dense network module.The present invention is faster uploaded, downloaded and browsing pictures；The carrying cost and bandwidth cost of saving at least 50%；Do not change the original code/decode format of picture, can each seamless compatible terminal.
Description
Technical field
The present invention relates to Computer Image Processing field, more particularly to a kind of picture compression side based on super-resolution technique
Method.
Background technology
In recent years, with the fast development of various mobile Internets, substantial amounts of picture is had daily in transmission over networks, and
It is downloaded and checks in different terminals, how efficiently stores and these pictures turn into one urgently with low bandwidth cost transport
The problem of solution.In the mobile device early stage of development, because mobile phone pixel is not high, the photographic resolution taken pictures is smaller, shared by photo
Internal memory and upload network data traffic it is seldom.Improved constantly with the resolution ratio of mobile phone camera, the quality of photo has
Significantly lifted.However, the high-quality photo of these high-resolution can bring Railway Project：(1) first, these high-quality photographs
Sector-meeting takes substantial amounts of memory space.For example, Samsung GALAXY J7 camera pixels have reached 13,000,000, the photo of shooting
More than 10M can be reached.More high-quality photos are stored such as how smaller space, are one the problem of need solution.Together
When, for the social activities such as wechat, microblogging and today's tops, media platform, user uploads mass picture to the clothes of platform daily
Business device, also brings huge carrying cost to platform.(2) if in addition, mobile phone to be taken to the photo come not by certain
Processing be uploaded directly into network, because original photo data volume is very big, uploading speed can be very slow, while user is in browsing pictures
When, because picture data amount is larger, it is necessary to more download times, brings bad upload to user and browse photo
Experience.(3) except influence carrying cost and picture transfer speed, each platform also will pay huge for transmission mass picture
Bandwidth cost.For example, by December, 2016, wechat active users every month reach 8.89 hundred million, the average good friend's quantity of user
Up to 194 people, the number of pictures downloaded and watched by wechat platform daily has hundreds of hundred million to arrive several hundred billion, and platform transmits these seas
Amount photo needs to pay huge bandwidth cost.Therefore, in order to reducing huge carrying cost and the band that mass picture is brought
Wide cost, and upload and the access speed of photo are improved, it is most effective approach that efficiently compression is carried out to picture.
The purpose of picture compression is the redundancy entered to picture source data by certain rule in line translation, reduction image data
Information, picture is represented to try one's best few bit number, so as to more efficiently store and transmit picture.Meanwhile, compressed
Picture be required to be recovered to the quality of original image well, it is reached the requirement of certain applications.It is main at present
The picture compression algorithm of stream has jpeg algorithm, JPEG2000 algorithms, picture compression based on wavelet transformation etc..Meanwhile, each grand duke
Department also releases the picture compression algorithm of oneself, the JPEG XR forms of such as Microsoft, Google WebP forms, the TPG lattice of Tengxun
Formula.These algorithms can further be effectively compressed storage 30%-40% in the case of holding definition identical with JPEG, from
And reduce carrying cost and bandwidth cost.However, because various terminals (browser, mobile phone etc.) are compatible to these picture formats
Property bad, the picture format or JPEG of current main flow, these efficient picture compression algorithms only used in major intra-companies.
Therefore, in order to improve the versatility of compression algorithm, it is necessary to study in the case where not changing picture format, how one is entered to picture
Step compression has higher commercial value.
The content of the invention
It is an object of the invention to overcome the deficiencies of the prior art and provide a kind of picture compression based on super-resolution technique
Method.Picture super-resolution technique is dexterously utilized, on the premise of original picture format and encoding and decoding are not changed, is reached efficiently
The purpose of compressed picture.First, by reducing the resolution ratio of picture in source, the size of picture is reduced, so as to reach compression figure
The purpose of piece, secondly, is transmitted or is stored on the internet source, then, in receiving terminal to the picture after resolution decreasing
The picture of compression is reduced into high score in different terminals using the super-resolution technique based on newest depth convolutional neural networks
Resolution picture, such terminal user's viewing is still original high-quality picture.Tested on three sets of public data collection, we
Reconstruction effect reached level advanced in the world.Can be not only various social activities by such picture processing and transmission strategy
Platform saves a large amount of carrying costs and bandwidth cost, reaches the purpose of picture compression, meanwhile, flow, drop can be also saved to user
The stand-by period that low user uploads and Loaded Image, lift Consumer's Experience.
The technical solution adopted by the present invention is：
A kind of picture compression method based on super-resolution technique, it comprises the following steps：
Step 1：By the user of mobile Internet platform photo to be uploaded by scaling s carry out it is down-sampled after obtain low
Clear picture, down-sampled algorithm uses bicubic interpolation algorithm；Again by original picture compression coded system such as JPEG, WebP etc.
By low clear picture compression and upload；The data volume that so user uploads can be reduced further at least than original compression coding mode
50%, flow is reduced needed for uploading, and uploading speed is accelerated.
Step 2：Mobile Internet platform is stored and distributed to the low clear picture of compression；Due to the data volume quilt of picture
Further compression at least 50%, carrying cost and bandwidth cost needed for such social platform also can be reduced further at least
50%.
Step 3：Mobile Internet platform user downloads the low clear picture of compression, receiving terminal pair by different receiving terminals
Low clear corresponding decoding such as JPEG, the WebP of picture progress for downloading compression obtains low clear picture.
Step 4：Decoded low clear picture is reconstructed into high definition picture using the reduction of depth convolutional neural networks model to supply
User browses；
The depth convolutional neural networks model include following module stage, be respectively low-level image feature extract the stage,
High-level characteristic stage, Feature-level fusion stage, characteristic layer dimensionality reduction stage, deconvolution stage and again are extracted using dense network module
Build the stage, the depth convolutional neural networks model specifically includes following sub-step：
The low resolution subgraph that step 4.1 is inputted to network model carries out convolution algorithm and activation primitive computing, obtains
Including marginal information low-level image feature, its calculation formula is：
Fl(x)=max (W1*X+B1,0) (1)
Wherein W1And B1It is first layer convolution masterplate parameter and offset parameter, F respectively1For resulting low-level image feature；
Step 4.2 obtains high-level characteristic using the study of multiple dense network modules；
Further, each dense network module includes 8 convolutional layers and 8 active coatings, and wherein activation primitive is rule
Integralization linear unit function, the convolution kernel size of all convolutional layers is 3*3；The feature that each layer of active coating is obtained can be with superposition
Mode is added inside later layer, and controls using feature increment rate k the characteristic pattern port number that each layer is obtained；In intensive net
In network module, n-th layer output has k*n characteristic pattern.Feature growth rate k, which is taken in 16, each dense network module, 8 layers, each
The characteristic pattern port number of dense network module output is 128.
Step 4.3 merges low-level image feature and high-level characteristic, forms abundant characteristic set, so as to being high resolution graphics
The reconstruction of picture provides more information.Low-level image feature obtained by step 4.1 contains many edges and shape of image
Information, and the more abstract and texture informations of high-level characteristic phenogram picture obtained by dense network module in step 4.2.
Therefore, reduction high resolution graphics can be given with reference to the high-level characteristic obtained by the low-level image feature and step 4.2 obtained by step 4.1
High-frequency content as in provides more abundant information.
Low-level image feature and high-level characteristic are all to obtain last characteristic set by great-jump-forward connection directly superposition, specifically may be used
To be expressed as：
Fn+1=Hn+1([F1,F2,…,Fn]) (2)
Wherein Fn+1For the characteristic set of fusion, F1For the 1st layer of obtained low-level image feature, F2To FnFor dense network module institute
Obtained high-level characteristic；H is the fusion function of different characteristic, and n is the number of plies of convolutional network.
In the Fusion Features stage, F is connected to great-jump-forward before having due to all characteristic layersn+1, so in backpropagation
When gradient information can solve the problems, such as the gradient disappearance that network depth increase is brought directly from top-level propagation to bottom.
Step 4.4 reduces the port number of characteristic layer, improves the arithmetic speed of convolutional neural networks model.
After the Fusion Features of step 4.3 are completed, network model can obtain 128*n feature, when n is larger, feature
Port number will be very big, so add the computation complexity of network model training stage and test phase, reduce network
The operation efficiency of model, and model parameter also can accordingly increase.
In order to reduce the convolutional layer progress characteristic layer passage that 1*1 is employed in the computation complexity of network model, the present invention
Number dimensionality reduction, and the parameter amount of model is decreased simultaneously.Further, after 1*1 convolutional layer dimensionality reductions, obtained by the present invention
Characteristic layer port number is 256.
Step 4.5 obtains the feature of high resolution space using the layer sample operator in study that deconvolute.In step 4.4
Resulting feature is all that, in low-resolution spatial, image size is d*d, so having to that these features are carried out to adopt
Sample, obtains the feature of high resolution space, could carry out the reconstruction of high-definition picture.In the super-resolution based on deep learning
Algorithm SRCNN[1]And VDSR[2]In, up-sampling operator directly uses bicubic interpolation, so not only causes deep learning model
Simple interpolation algorithm is relied on, and the amount of calculation of convolutional Neural model can be increased, because convolution process is in the laggard of interpolation
OK.The present invention can not only learn the operator up-sampled using the layer that deconvolutes, and improve the reconstruction performance of image, while
Carried out so that convolution operation is transferred to low-resolution spatial from high resolution space, accelerate the convolution algorithm of neural network model
Process.
Step 4.6 reconstructs high-definition picture using resulting feature of deconvoluting.By the image of reconstruction with it is original
High-definition picture is compared, the reconstruction loss of computation model and gradient, and is arrived along the back-propagation gradient of network model
Among each layer of network, the parameter in network is updated, is solved after successive ignition and obtains final network model parameter W and B.
The present invention innovatively combines dense network module, merged bottom and high-level characteristic and introduces the layer that deconvolutes
Sample operator, not only increases the reconstruction effect of high definition picture in study, but also accelerates the arithmetic speed of process of reconstruction.
During the network optimization, the picture of reconstruction is compared with original high-resolution pictures, the reconstruction of computation model loss and
Gradient, and along the back-propagation gradient of network model to each layer of network among, update network in parameter, successive ignition solve
Obtain final network model parameter W and B.
Because the picture transmitted is low resolution, data volume has been further compressed at least 50%, and user can be with
At faster speed download pictures and save flow consumption.Meanwhile, utilize depth convolutional network resulting in step 3 of the present invention
Model is rebuild to the picture of download, is reduced the high definition picture being reconstructed into and is checked for user in different terminals, does not influence user
High definition viewing experience.
The present invention uses above technical scheme, by the super-resolution rebuilding technology of innovation, can reach preferable picture pressure
Contracting effect, and propose it first with the picture compression and transmission of mobile Internet, not only lifting user uploads and browsed
The experience of picture, and at least 50% carrying cost and bandwidth cost can be further saved for each social platform.The present invention
A kind of picture compression method based on super-resolution technique proposed, with advantages below：
(1) picture compression method proposed by the invention, can not only reach the purpose for being effectively compressed picture, and be not required to
Change the original form of picture and code encoding/decoding mode, can combined with the picture format of main flow (including JPEG, WebP and TPG
Deng form), compatibility is fine, so as to be used well in each terminal.For a user, upload and download is browsed
Speed faster, while flow can be saved., can be with by using the technology of the present invention for mobile Internet platform
On the basis of existing picture compression encoding and decoding (including JPEG, WebP with TPG etc.) further save at least 50% storage into
Originally with bandwidth cost.
(2) super-resolution algorithms for picture reconstruction proposed by the invention are based on a kind of newest depth convolution god
Through network model, and dense network module is introduced in a model, not only reduce the parameter of model so that the information in network
Backpropagation is constantly present shorter path, optimizes flowing of the information on profound network, effectively solves the instruction of depth network
Practice problem.Meanwhile, low-level image feature and high-level characteristic are combined, the information for learning to obtain in network is made full use of, to high-resolution
The reconstruction and reduction of rate picture provide more abundant multi-level information.
(3) picture super-resolution algorithms of the present invention introduce the layer sample operator in study that deconvolute, and not only carry
High reconstruction effect, and accelerate the operational performance of network.And in the super-resolution algorithms SRCNN [1] based on deep learning
In VDSR [2], up-sampling operator directly uses bicubic interpolation, so not only causes deep learning model is relied on simple
Interpolation algorithm, and the amount of calculation of convolutional Neural model can be increased, because convolution process is carried out after interpolation.The present invention's
Innovation can not only learn the operator up-sampled using the layer that deconvolutes, and the reconstruction performance of picture be improved, while also making
Obtain convolution operation and be transferred to low-resolution spatial progress from high resolution space, accelerate the convolution algorithm mistake of neural network model
Journey.
Brief description of the drawings
The present invention is described in further details below in conjunction with the drawings and specific embodiments；
Fig. 1 is a kind of flow chart of picture compression method based on super-resolution technique of the present invention；
Fig. 2 is a kind of depth convolutional Neural model structure of picture compression method based on super-resolution technique of the present invention
Figure；
Fig. 3 is the original image before the present invention is transmitted to resolution decreasing；
Fig. 4 is the picture reduced after the present invention is transmitted to resolution decreasing；
Fig. 5 is that the picture and original image reduced after the present invention is transmitted to resolution decreasing is big in the storage of different quality coefficient
Small contrast；
Fig. 6 is that the reconstruction effect of img096 pictures is compared figure to the present invention on Urban100 data sets with prior art；
Fig. 7 is that reconstruction effect of the present invention with prior art on Set14 data sets is compared figure；
Fig. 8 is that reconstruction effect of the present invention with prior art on B100 data sets is compared figure；
Fig. 9 is that the reconstruction effect of img099 pictures is compared figure to the present invention on Urban100 data sets with prior art.
Embodiment
As shown in one of Fig. 1-9, the invention discloses a kind of picture compression method based on super-resolution technique, it includes
Following steps：
Step 1：The user of mobile Internet platform photo as shown in Figure 3 to be uploaded is dropped by scaling s
Low clear picture is obtained after sampling, down-sampled algorithm uses bicubic interpolation algorithm；Original picture compression coded system is pressed again
Such as JPEG, WebP are by low clear picture compression and upload；The data volume that so user uploads is than original compression coding mode meeting
At least 50% is further reduced, flow is reduced needed for uploading, uploading speed is accelerated.
Step 2：Mobile Internet platform is stored and distributed to the low clear picture of compression；Due to the data volume quilt of picture
Further compression at least 50%, carrying cost and bandwidth cost needed for such social platform also can be reduced further at least
50%.
Step 3：Mobile Internet platform user downloads the low clear picture of compression, receiving terminal pair by different receiving terminals
Low clear corresponding decoding such as JPEG, the WebP of picture progress for downloading compression obtains low clear picture.
Step 4：Decoded low clear picture is reconstructed into high definition picture using the reduction of depth convolutional neural networks model to supply
User browses；High definition picture is as shown in Figure 4.
As shown in Fig. 2 the depth convolutional neural networks model includes following module stage, it is low-level image feature respectively
The extraction stage, utilize dense network module extract the high-level characteristic stage, the Feature-level fusion stage, the characteristic layer dimensionality reduction stage, warp
Product stage and phase of regeneration, the depth convolutional neural networks model specifically include following sub-step：
The low resolution subgraph that step 4.1 is inputted to network model carries out convolution algorithm and activation primitive computing, obtains
Including marginal information low-level image feature, its calculation formula is：
Fl(x)=max (W1*X+B1,0) (1)
Wherein W1And B1It is first layer convolution masterplate parameter and offset parameter, F respectively1For resulting low-level image feature；
Step 4.2 obtains high-level characteristic using the study of multiple dense network modules；
Further, each dense network module includes 8 convolutional layers and 8 active coatings, and wherein activation primitive is rule
Integralization linear unit function, the convolution kernel size of all convolutional layers is 3*3；The feature that each layer of active coating is obtained can be with superposition
Mode is added inside later layer, and controls using feature increment rate k the characteristic pattern port number that each layer is obtained；In intensive net
In network module, n-th layer output has k*n characteristic pattern.Feature growth rate k, which is taken in 16, each dense network module, 8 layers, each
The characteristic pattern port number of dense network module output is 128.
Step 4.3 merges low-level image feature and high-level characteristic, forms abundant characteristic set, so as to being high resolution graphics
The reconstruction of picture provides more information.Low-level image feature obtained by step 4.1 contains many edges and shape of image
Information, and the more abstract and texture informations of high-level characteristic phenogram picture obtained by dense network module in step 4.2.
Therefore, reduction high resolution graphics can be given with reference to the high-level characteristic obtained by the low-level image feature and step 4.2 obtained by step 4.1
High-frequency content as in provides more abundant information.
Low-level image feature and high-level characteristic are all to obtain last characteristic set by great-jump-forward connection directly superposition, specifically may be used
To be expressed as：
Fn+1=Hn+1([F1,F2,…,Fn]) (2)
Wherein Fn+1For the characteristic set of fusion, F1For the 1st layer of obtained low-level image feature, F2To FnFor dense network module institute
Obtained high-level characteristic；H is the fusion function of different characteristic, and n is the number of plies of convolutional network.
In the Fusion Features stage, F is connected to great-jump-forward before having due to all characteristic layersn+1, so in backpropagation
When gradient information can solve the problems, such as the gradient disappearance that network depth increase is brought directly from top-level propagation to bottom.
Step 4.4 reduces the port number of characteristic layer, improves the arithmetic speed of convolutional neural networks model.
After the Fusion Features of step 4.3 are completed, network model can obtain 128*n feature, when n is larger, feature
Port number will be very big, so add the computation complexity of network model training stage and test phase, reduce network
The operation efficiency of model, and model parameter also can accordingly increase.
In order to reduce the convolutional layer progress characteristic layer passage that 1*1 is employed in the computation complexity of network model, the present invention
Number dimensionality reduction, and the parameter amount of model is decreased simultaneously.Further, after 1*1 convolutional layer dimensionality reductions, obtained by the present invention
Characteristic layer port number is 256.
Step 4.5 obtains the feature of high resolution space using the layer sample operator in study that deconvolute.In step 4.4
Resulting feature is all that, in low-resolution spatial, image size is d*d, so having to that these features are carried out to adopt
Sample, obtains the feature of high resolution space, could carry out the reconstruction of high-definition picture.In the super-resolution based on deep learning
Algorithm SRCNN[1]And VDSR[2]In, up-sampling operator directly uses bicubic interpolation, so not only causes deep learning model
Simple interpolation algorithm is relied on, and the amount of calculation of convolutional Neural model can be increased, because convolution process is in the laggard of interpolation
OK.The present invention can not only learn the operator up-sampled using the layer that deconvolutes, and improve the reconstruction performance of image, while
Carried out so that convolution operation is transferred to low-resolution spatial from high resolution space, accelerate the convolution algorithm of neural network model
Process.
Step 4.6 reconstructs high-definition picture using resulting feature of deconvoluting.By the image of reconstruction with it is original
High-definition picture is compared, the reconstruction loss of computation model and gradient, and is arrived along the back-propagation gradient of network model
Among each layer of network, the parameter in network is updated, is solved after successive ignition and obtains final network model parameter W and B.
The present invention innovatively combines dense network module, merged bottom and high-level characteristic and introduces the layer that deconvolutes
Sample operator, not only increases the reconstruction effect of high definition picture in study, but also accelerates the arithmetic speed of process of reconstruction.
During the network optimization, the picture of reconstruction is compared with original high-resolution pictures, the reconstruction of computation model loss and
Gradient, and along the back-propagation gradient of network model to each layer of network among, update network in parameter, successive ignition solve
Obtain final network model parameter W and B.
Because the picture transmitted is low resolution, data volume has been further compressed at least 50%, and user can be with
At faster speed download pictures and save flow consumption.Meanwhile, utilize depth convolutional network resulting in step 3 of the present invention
Model is rebuild to the picture of download, is reduced the high definition picture being reconstructed into and is checked for user in different terminals, does not influence user
High definition viewing experience.
In order to verify the compression performance of algorithm proposed by the invention, Android and iOS of the present invention from social platform wechat
Client on downloaded 2270 pictures altogether, and be compressed using the algorithm of the present invention.Before compression, 2270 pictures
406M memory space is taken altogether.After our technology, the storage size of 2270 pictures is changed into 111M, and picture has altogether
The 27.4% of original size is compressed into, required memory space reduces 72.6%.However, as shown in Fig. 2 the figure that user is browsed
Piece, the super-resolution algorithms by the present invention, which are rebuild, to reduce, and is visually not different with original image, reaches that good user is clear
Look at effect.In addition, in order to JPEG compression algorithm comparison, the present invention have collected 1427 cell phone pictures.Cell phone pictures are not from
Same mobile phone model, including Android phone and iPhone.As shown in figure 5, in the case of different compression quality coefficients, this
Inventive method can also further be effectively compressed storage size 70%-72% on the basis of JPEG compression.It is emphasized that
The inventive method is not limited only to JPEG compression, (includes WebP compressions and the TPG pressures of Tengxun of Google in other picture compression methods
Contracting) on the basis of, inventive algorithm still can further compress at least 50%.
In order to verify that super-resolution algorithms proposed by the invention transmit the reconstruction performance of picture after to resolution decreasing, this
Invention is tested using disclosed four sets of data collection in the world, respectively including Set5, Set14, B100 and Urban100.Than
In relatively testing, scaling s takes 4.The reconstruction effect that the present invention is obtained is inserted with the Technical comparing delivered at present, including bicubic
Value method (Bicubic interpolation), the Aplus based on dictionary learning[3]Algorithm, based on 3 layers of convolutional network model
SRCNN[1]Algorithm, the VDSR based on 20 layers of convolutional neural networks[2]Algorithm and the DRCN that technology is supervised based on depth[4]Algorithm.This
Invention is using Y-PSNR (PSNR:Peak Signal to Noise Ratio) and structural similarity (SSIM:
Structural Similarity Index) weigh the reconstruction performances of high-resolution pictures.From table 1 it follows that this hair
The bright super-resolution algorithms SRCNN more classical than in the prior art PSNR values be respectively increased on four sets of data collection 1.44dB,
1.54dB、1.01dB、0.63dB.Meanwhile, tested using four all pictures of sets of data collection, than what is delivered in the world at present
0.51dB and 0.56dB has also been respectively increased in the PSNR values for rebuilding the best VDSR algorithms of effect and DRCN algorithms.Fig. 6-9 is this
Invent the reconstruction effect with prior art on different pieces of information collection and compared figure, it can be seen that high-resolution pictures of the invention are rebuild
Effect visually has than the reconstruction effect of prior art to be obviously improved.As can be seen here, the present invention is compared with other prior art phases
Than the reconstruction effect of high-resolution pictures has significant raising, has also been reached in the reconstruction performance of picture super-resolution new
International standard.
The PSNR values and SSIM values of the present invention of table 1 and prior art
The present invention uses above technical scheme, by the super-resolution rebuilding technology of innovation, can reach preferable picture pressure
Contracting effect, and propose it first with the picture compression and transmission of mobile Internet, not only lifting user uploads and browsed
The experience of picture, and at least 50% carrying cost and bandwidth cost can be further saved for each social platform.The present invention
A kind of picture compression method based on super-resolution technique proposed, with advantages below：
(1) picture compression method proposed by the invention, can not only reach the purpose for being effectively compressed picture, and be not required to
Change the original form of picture and code encoding/decoding mode, can combined with the picture format of main flow (including JPEG, WebP and TPG
Deng form), compatibility is fine, so as to be used well in each terminal.For a user, upload and download is browsed
Speed faster, while flow can be saved., can be with by using the technology of the present invention for mobile Internet platform
On the basis of existing picture compression encoding and decoding (including JPEG, WebP with TPG etc.) further save at least 50% storage into
Originally with bandwidth cost.
(2) super-resolution algorithms for picture reconstruction proposed by the invention are based on a kind of newest depth convolution god
Through network model, and dense network module is introduced in a model, not only reduce the parameter of model so that the information in network
Backpropagation is constantly present shorter path, optimizes flowing of the information on profound network, effectively solves the instruction of depth network
Practice problem.Meanwhile, low-level image feature and high-level characteristic are combined, the information for learning to obtain in network is made full use of, to high-resolution
The reconstruction and reduction of rate picture provide more abundant multi-level information.
(3) picture super-resolution algorithms of the present invention introduce the layer sample operator in study that deconvolute, and not only carry
High reconstruction effect, and accelerate the operational performance of network.And in the super-resolution algorithms SRCNN [1] based on deep learning
In VDSR [2], up-sampling operator directly uses bicubic interpolation, so not only causes deep learning model is relied on simple
Interpolation algorithm, and the amount of calculation of convolutional Neural model can be increased, because convolution process is carried out after interpolation.The present invention's
Innovation can not only learn the operator up-sampled using the layer that deconvolutes, and the reconstruction performance of picture be improved, while also making
Obtain convolution operation and be transferred to low-resolution spatial progress from high resolution space, accelerate the convolution algorithm mistake of neural network model
Journey.
The bibliography being related in the present invention is as follows：
[1]C.Dong,C.C.Loy,K.He,and X.Tang.Image super-resolution using deep
convolutional networks.IEEE transactions on pattern analysis and machine
intelligence,38(2):295–307,2016.
[2]J.Kim,J.Kwon Lee,and K.Mu Lee.Accurate image superresolution using
very deep convolutional networks.In IEEE Conference on Computer Vision and
Pattern Recognition,pages 1646–1654,2016.
[3]R.Timofte,V.De Smet,and L.Van Gool.A+:Adjusted anchored
neighborhood regression for fast super-resolution.In Asian Conference on
Computer Vision,pages 111–126.Springer, 2014.
[4]J.Kim,J.Kwon Lee,and K.Mu Lee.Deeply-recursive convolutional
network for image super-resolution.In IEEE Conference on Computer Vision and
Pattern Recognition,pages 1637– 1645,2016。
Claims (8)
1. a kind of picture compression method based on super-resolution technique, it is characterised in that：It comprises the following steps：
Step 1：By the user of mobile Internet platform photo to be uploaded by scaling s carry out it is down-sampled after obtain low clear figure
Piece, then by low clear picture compression and upload；
Step 2：Mobile Internet platform is stored and distributed to the low clear picture of compression；
Step 3：Mobile Internet platform user downloads the low clear picture of compression by different receiving terminals, and receiving terminal is to downloading
The low clear picture of compression carries out corresponding decoding and obtains low clear picture；
Step 4：Decoded low clear picture is reconstructed into high definition picture using the reduction of depth convolutional neural networks model and supplies user
Browse；The depth convolutional neural networks model includes following module stage, is that low-level image feature extracts the stage, utilized respectively
Dense network module extracts high-level characteristic stage, Feature-level fusion stage, characteristic layer dimensionality reduction stage, deconvolution stage and rebuilds rank
Section, the depth convolutional neural networks model specifically includes following sub-step：
The low resolution subgraph that step 4.1 is inputted to network model carries out convolution algorithm and activation primitive computing, including
Marginal information low-level image feature, its calculation formula is：
Fl(x)=max (W1*X+B1,0) (1)
Wherein W1And B1It is first layer convolution masterplate parameter and offset parameter, F respectively1For resulting low-level image feature；
Step 4.2 obtains high-level characteristic using the study of multiple dense network modules；Each dense network module includes multiple volumes
Lamination and multiple active coatings, wherein activation primitive are Regularization linear unit function, and the convolution kernel size of all convolutional layers is 3*
3；The feature that each layer of active coating is obtained can be added in the way of superposition inside later layer, and is controlled using feature increment rate k
Make the characteristic pattern port number that each layer is obtained；In dense network module, n-th layer output has k*n characteristic pattern；
Step 4.3 merges low-level image feature and high-level characteristic, forms abundant characteristic set：
Low-level image feature and high-level characteristic are all to obtain last characteristic set by great-jump-forward connection directly superposition, specific to represent
For：
Fn+1=Hn+1([F1,F2,…,Fn]) (2)
Wherein Fn+1For the characteristic set of fusion, F1For the 1st layer of obtained low-level image feature, F2To FnObtained by dense network module
High-level characteristic；H is the fusion function of different characteristic, and n is the number of plies of convolutional network；
Step 4.4 reduces the port number of characteristic layer, improves the arithmetic speed of convolutional neural networks model；
Step 4.5 obtains the feature of high resolution space using the layer sample operator in study that deconvolute；
Step 4.6 reconstructs high-definition picture using the feature of high resolution space obtained by deconvoluting；By the image of reconstruction
It is compared with original high-definition picture, the reconstruction loss of computation model and gradient, and along the reverse biography of network model
Gradient is broadcast among each layer of network, the parameter in network is updated, is solved after successive ignition and obtains final network model ginseng
Number W and B.
2. a kind of picture compression method based on super-resolution technique according to claim 1, it is characterised in that：The step
Down-sampled algorithm uses bicubic interpolation algorithm in rapid 1.
3. a kind of picture compression method based on super-resolution technique according to claim 1, it is characterised in that：The step
It is compressed in rapid 1 during picture compression using conventional picture compression coded system, conventional picture compression coded system includes
JPEG, PNG and WebP.
4. a kind of picture compression method based on super-resolution technique according to claim 1, it is characterised in that：The step
Each dense network module includes multiple convolutional layers and multiple active coatings in rapid 4.2, and wherein activation primitive is that Regularization is linear
Unit function；The feature that each layer of active coating is obtained can be added in the way of superposition inside later layer, and is increased using feature
Rate k controls the characteristic pattern port number that each layer is obtained；In dense network module, n-th layer output has k*n characteristic pattern.
5. a kind of picture compression method based on super-resolution technique according to claim 4, it is characterised in that：Each
Dense network module includes 8 convolutional layers and 8 active coatings, and the convolution kernel size of all convolutional layers is 3*3.
6. a kind of picture compression method based on super-resolution technique according to claim 5, it is characterised in that：The spy
Levying growth rate k and taking has 8 layers in 16, each dense network module, the characteristic pattern port number of each dense network module output is 128
It is individual.
7. a kind of picture compression method based on super-resolution technique according to claim 1, it is characterised in that：The step
The convolutional layer that 1*1 is employed in rapid 4.4 carries out characteristic layer port number dimensionality reduction.
8. a kind of picture compression method based on super-resolution technique according to claim 1, it is characterised in that：The step
In rapid 4.4 after 1*1 convolutional layer dimensionality reductions, resulting characteristic layer port number is 256.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710446896.XA CN107155110A (en) | 2017-06-14 | 2017-06-14 | A kind of picture compression method based on super-resolution technique |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710446896.XA CN107155110A (en) | 2017-06-14 | 2017-06-14 | A kind of picture compression method based on super-resolution technique |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107155110A true CN107155110A (en) | 2017-09-12 |
Family
ID=59796254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710446896.XA Pending CN107155110A (en) | 2017-06-14 | 2017-06-14 | A kind of picture compression method based on super-resolution technique |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107155110A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108022212A (en) * | 2017-11-24 | 2018-05-11 | 腾讯科技（深圳）有限公司 | High-resolution pictures generation method, generating means and storage medium |
| CN109325491A (en) * | 2018-08-16 | 2019-02-12 | 腾讯科技（深圳）有限公司 | Identification code recognition methods, device, computer equipment and storage medium |
| CN109583576A (en) * | 2018-12-17 | 2019-04-05 | 上海联影智能医疗科技有限公司 | A kind of medical image processing devices and method |
| CN109829855A (en) * | 2019-01-23 | 2019-05-31 | 南京航空航天大学 | A kind of super resolution ratio reconstruction method based on fusion multi-level features figure |
| CN109949223A (en) * | 2019-02-25 | 2019-06-28 | 天津大学 | Image super-resolution reconstructing method based on the dense connection of deconvolution |
| CN110189282A (en) * | 2019-05-09 | 2019-08-30 | 西北工业大学 | Based on intensive and jump connection depth convolutional network multispectral and panchromatic image fusion method |
| CN110312139A (en) * | 2019-06-18 | 2019-10-08 | 深圳前海达闼云端智能科技有限公司 | The method and apparatus of image transmitting, storage medium |
| WO2019237659A1 (en) * | 2018-06-11 | 2019-12-19 | Suzhou Aqueti Technology Co., Ltd. | Blind compressive sampling method, apparatus and imaging system |
| CN110728726A (en) * | 2019-10-24 | 2020-01-24 | 湖南大学 | Image compression method based on user interaction and deep neural network |
| WO2020057492A1 (en) * | 2018-09-19 | 2020-03-26 | 北京金山办公软件股份有限公司 | Image compression and decompression method and apparatus, electronic device, and storage medium |
| CN111050315A (en) * | 2019-11-27 | 2020-04-21 | 北京邮电大学 | Wireless transmitter identification method based on multi-core two-way network |
| CN111581408A (en) * | 2020-04-27 | 2020-08-25 | 北京电信易通信息技术股份有限公司 | Super-resolution-based picture storage method and system |
| CN113658230A (en) * | 2020-05-12 | 2021-11-16 | 武汉Tcl集团工业研究院有限公司 | Optical flow estimation method, terminal and storage medium |
| CN114071021A (en) * | 2022-01-14 | 2022-02-18 | 苏州乐佰图信息技术有限公司 | Processing method, device and system for improving resolution of image formed by camera |
| US11341734B2 (en) | 2018-12-17 | 2022-05-24 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for image segmentation |
| US11526545B2 (en) | 2020-02-10 | 2022-12-13 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method of processing cover page image of multimedia information, client and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295139A (en) * | 2016-07-29 | 2017-01-04 | 姹ゅ钩 | A kind of tongue body autodiagnosis health cloud service system based on degree of depth convolutional neural networks |
| CN106791927A (en) * | 2016-12-23 | 2017-05-31 | 福建帝视信息科技有限公司 | A kind of video source modeling and transmission method based on deep learning |
-
2017
- 2017-06-14 CN CN201710446896.XA patent/CN107155110A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295139A (en) * | 2016-07-29 | 2017-01-04 | 姹ゅ钩 | A kind of tongue body autodiagnosis health cloud service system based on degree of depth convolutional neural networks |
| CN106791927A (en) * | 2016-12-23 | 2017-05-31 | 福建帝视信息科技有限公司 | A kind of video source modeling and transmission method based on deep learning |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11373275B2 (en) | 2017-11-24 | 2022-06-28 | Tencent Technology (Shenzhen) Company Limited | Method for generating high-resolution picture, computer device, and storage medium |
| CN108022212A (en) * | 2017-11-24 | 2018-05-11 | 腾讯科技（深圳）有限公司 | High-resolution pictures generation method, generating means and storage medium |
| CN112470472B (en) * | 2018-06-11 | 2023-03-24 | 无锡安科迪智能技术有限公司 | Blind compression sampling method and device and imaging system |
| CN112470472A (en) * | 2018-06-11 | 2021-03-09 | 无锡安科迪智能技术有限公司 | Blind compression sampling method and device and imaging system |
| WO2019237659A1 (en) * | 2018-06-11 | 2019-12-19 | Suzhou Aqueti Technology Co., Ltd. | Blind compressive sampling method, apparatus and imaging system |
| CN109325491A (en) * | 2018-08-16 | 2019-02-12 | 腾讯科技（深圳）有限公司 | Identification code recognition methods, device, computer equipment and storage medium |
| US11494577B2 (en) | 2018-08-16 | 2022-11-08 | Tencent Technology (Shenzhen) Company Limited | Method, apparatus, and storage medium for identifying identification code |
| WO2020057492A1 (en) * | 2018-09-19 | 2020-03-26 | 北京金山办公软件股份有限公司 | Image compression and decompression method and apparatus, electronic device, and storage medium |
| CN109583576B (en) * | 2018-12-17 | 2020-11-06 | 上海联影智能医疗科技有限公司 | Medical image processing device and method |
| US11836925B2 (en) | 2018-12-17 | 2023-12-05 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for image segmentation |
| CN109583576A (en) * | 2018-12-17 | 2019-04-05 | 上海联影智能医疗科技有限公司 | A kind of medical image processing devices and method |
| US11341734B2 (en) | 2018-12-17 | 2022-05-24 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for image segmentation |
| CN109829855B (en) * | 2019-01-23 | 2023-07-25 | 南京航空航天大学 | Super-resolution reconstruction method based on fusion of multi-level feature images |
| CN109829855A (en) * | 2019-01-23 | 2019-05-31 | 南京航空航天大学 | A kind of super resolution ratio reconstruction method based on fusion multi-level features figure |
| CN109949223B (en) * | 2019-02-25 | 2023-06-20 | 天津大学 | Image super-resolution reconstruction method based on deconvolution dense connection |
| CN109949223A (en) * | 2019-02-25 | 2019-06-28 | 天津大学 | Image super-resolution reconstructing method based on the dense connection of deconvolution |
| CN110189282A (en) * | 2019-05-09 | 2019-08-30 | 西北工业大学 | Based on intensive and jump connection depth convolutional network multispectral and panchromatic image fusion method |
| CN110312139A (en) * | 2019-06-18 | 2019-10-08 | 深圳前海达闼云端智能科技有限公司 | The method and apparatus of image transmitting, storage medium |
| CN110728726B (en) * | 2019-10-24 | 2022-09-23 | 湖南大学 | Image compression method based on user interaction and deep neural network |
| CN110728726A (en) * | 2019-10-24 | 2020-01-24 | 湖南大学 | Image compression method based on user interaction and deep neural network |
| CN111050315A (en) * | 2019-11-27 | 2020-04-21 | 北京邮电大学 | Wireless transmitter identification method based on multi-core two-way network |
| US11526545B2 (en) | 2020-02-10 | 2022-12-13 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method of processing cover page image of multimedia information, client and electronic device |
| CN111581408A (en) * | 2020-04-27 | 2020-08-25 | 北京电信易通信息技术股份有限公司 | Super-resolution-based picture storage method and system |
| CN113658230A (en) * | 2020-05-12 | 2021-11-16 | 武汉Tcl集团工业研究院有限公司 | Optical flow estimation method, terminal and storage medium |
| CN113658230B (en) * | 2020-05-12 | 2024-05-28 | 武汉Tcl集团工业研究院有限公司 | Optical flow estimation method, terminal and storage medium |
| CN114071021A (en) * | 2022-01-14 | 2022-02-18 | 苏州乐佰图信息技术有限公司 | Processing method, device and system for improving resolution of image formed by camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107155110A (en) | A kind of picture compression method based on super-resolution technique | |
| CN109118432B (en) | Image super-resolution reconstruction method based on rapid cyclic convolution network | |
| CN109886871B (en) | Image super-resolution method based on channel attention mechanism and multi-layer feature fusion | |
| CN107463989B (en) | A kind of image based on deep learning goes compression artefacts method | |
| CN110163801B (en) | Image super-resolution and coloring method, system and electronic equipment | |
| CN111242846B (en) | Fine-grained scale image super-resolution method based on non-local enhancement network | |
| CN103167284B (en) | A kind of video stream transmission method and system based on picture super-resolution | |
| CN111861961A (en) | Multi-scale residual error fusion model for single image super-resolution and restoration method thereof | |
| CN109741260A (en) | A kind of efficient super-resolution method based on depth back projection network | |
| CN112017116B (en) | Image super-resolution reconstruction network based on asymmetric convolution and construction method thereof | |
| CN111640060A (en) | Single image super-resolution reconstruction method based on deep learning and multi-scale residual dense module | |
| Xu et al. | Joint demosaicing and super-resolution (JDSR): Network design and perceptual optimization | |
| Guan et al. | Srdgan: learning the noise prior for super resolution with dual generative adversarial networks | |
| CN115018708A (en) | Airborne remote sensing image super-resolution reconstruction method based on multi-scale feature fusion | |
| CN117422614B (en) | Single-frame image super-resolution method and device based on hybrid feature interaction transducer | |
| CN109615576B (en) | Single-frame image super-resolution reconstruction method based on cascade regression basis learning | |
| CN116958534A (en) | Image processing method, training method of image processing model and related device | |
| CN115984111A (en) | Image super-resolution method and device based on knowledge distillation compression model | |
| CN114359039A (en) | Knowledge distillation-based image super-resolution method | |
| CN114170082A (en) | Video playing method, image processing method, model training method, device and electronic equipment | |
| CN117237259B (en) | Compressed video quality enhancement method and device based on multi-mode fusion | |
| Hamis et al. | Image compression at very low bitrate based on deep learned super-resolution | |
| CN117455804A (en) | Image deblurring method based on attention mechanism residual Fourier transform network | |
| CN116309078A (en) | Single image super-resolution method based on multi-angle knowledge distillation | |
| CN112070676B (en) | Picture super-resolution reconstruction method of double-channel multi-perception convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170912 |