Anime4K is a set of open-source, high-quality real-time anime upscaling/denoising algorithms that can be implemented in any programming language.
The simplicity and speed of Anime4K allows the user to watch upscaled anime in real time, as we believe in preserving original content and promoting freedom of choice for all anime fans. Re-encoding anime into 4K should be avoided as it is non-reversible, potentially damages original content by introducing artifacts, takes up to O(n2) more disk space and more importantly, does so without any meaningful decrease in entropy (lost information is lost).
Disclaimer: All art assets used are for demonstration and educational purposes. All rights are reserved to their original owners. If you (as a person or a company) own the art and do not wish it to be associated with this project, please contact us at anime4k.upscale@gmail.com and we will gladly take it down.
Anime4K is optimized for native 1080p anime encoded with h.264, h.265 or VC-1.
Even if it might work, it is not optimized for downscaled 720p, 480p or standard definition anime (eg. DVDs). Older anime (especially pre-digital era production) have artifacts that are very difficult to remove, such as bad deinterlacing, camera blur during production, severe ringing, film grain, older MPEG compression artifacts, etc.
This is also not replacement for SRGANs, as they perform much better on low-resolution images or images with lots of degradation (albeit not in real time).
What Anime4K does provide is a way to upscale, in real time, 1080p anime for 4K screens while providing a similar effect to SRGANs and being much better than waifu2x (See comparisons).
Currently, research is being done on better real-time upscaling for lower resolution or older content.
Results from the experimental SRGAN shaders for 360p -> 4K: (zoom in to view details)
The images are sorted by algorithm speed, bicubic being the fastest. FSRCNNX and Anime4K are real-time while waifu2x and Real-ESRGAN are not.
We introduce a line reconstruction algorithm that aims to tackle the distribution shift problem seen in 1080p anime. In the wild anime exhibit a surprising amount of variance caused by low quality compositing due to budget and time constraints that traditional super-resolution algorithms cannot handle. GANs can implicitly encode this distribution shift but are slow to use and hard to train. Our algorithm explicitly corrects this distribution shift and allows traditional "MSE" SR algorithms to work with a wide variety of anime.
Source: https://fancaps.net/anime/picture.php?/14728493 | Mode: B
Source: https://fancaps.net/anime/picture.php?/13365760 | Mode: A
Performance numbers are obtained using a Vega64 GPU and are tested using UL shader variants. The fast version is for M variants.
Note that CUDA accelerated SRGANs/Waifu2x using tensor cores can be much faster and close to realtime (~80ms), but their large size severely hampers non-CUDA implementations.