More Web Proxy on the site http://driver.im/

research-article

Visually Lossless Content and Motion Adaptive Shading in Games

Authors:

Emmett Kilgariff,

Matthew Johnson,

Henrik RydgårdAuthors Info & Claims

Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 2, Issue 1

Article No.: 6, Pages 1 - 19

https://doi.org/10.1145/3320287

Published: 03 June 2019 Publication History

Abstract

We present a technique that adaptively adjusts the shading rate during rendering based on the scene content and motion. Our goal is to improve performance with no loss in perceived quality. We determine per-screen-tile shading rate by testing an error estimate against a perceptually-corrected just-noticeable difference threshold. Our design features an effective and efficient error estimate using spatial and frequency analysis of half and quarter rate shading. We also study the effect of motion in reducing perceived error, a consequence of display-persistence and/or motion blur effects. Our implementation uses the computed per-tile shading rate with variable rate shading (a recent GPU feature) to lower shading cost. We demonstrate the quality and performance of our technique on two high-end game engines and shipped games.

Supplementary Material

yang (yang.zip)

Supplemental movie, appendix, image and software files for, Gradient Estimation for Realtime Adaptive Temporal Filtering

Download
62.14 MB

References

[1]

Kurt Akeley. 1993. Reality engine graphics. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM, 109--116.

Digital Library

[2]

Tomas Akenine-Moller, Eric Haines, and Naty Hoffman. 2018. Real-time rendering. AK Peters/CRC Press.

Digital Library

[3]

Magnus Andersson, Jon Hasselgren, Robert Toth, and Tomas Akenine-Möiler. 2014. Adaptive texture space shading for stochastic rendering. Computer Graphics Forum 33, 2 (2014), 341--350.

Digital Library

[4]

Swaroop Bhonde. 2018. Turing Variable Rate Shading in VRWorks. https://devblogs.nvidia.com/turing-variable-rate-shading-vrworks/. (2018).

[5]

Mark R Bolin and Gary W Meyer. 1998. A perceptually based adaptive sampling algorithm. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. ACM, 299--309.

Digital Library

[6]

Huw Bowles, Kenny Mitchell, Robert W Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative image warping. In Computer graphics forum, Vol. 31. Wiley Online Library, 237--246.

[7]

Christopher A Burns, Kayvon Fatahalian, and William R Mark. 2010. A lazy object-space shading architecture with decoupled sampling. In Proceedings of the Conference on High Performance Graphics. Eurographics Association, 19--28.

Digital Library

[8]

George Casella and Roger L Berger. 2002. Statistical inference. Vol. 2. Duxbury Pacific Grove, CA.

[9]

Stanley H Chan and Truong Q Nguyen. 2011. LCD motion blur: modeling, analysis, and algorithm. IEEE Transactions on Image Processing 20, 8 (2011), 2352--2365.

Digital Library

[10]

Petrik Clarberg, Robert Toth, Jon Hasselgren, Jim Nilsson, and Tomas Akenine-Möller. 2014. AMFS: adaptive multi-frequency shading for future graphics processors. ACM Transactions on Graphics (TOG) 33, 4 (2014), 141.

Digital Library

[11]

Petrik Clarberg, Robert Toth, and Jacob Munkberg. 2013. A sort-based deferred shading architecture for decoupled sampling. ACM Transactions on Graphics (TOG) 32, 4 (2013), 141.

Digital Library

[12]

Jean-Philippe Farrugia and Bernard Péroche. 2004. A progressive rendering algorithm using an adaptive perceptually based image metric. In Computer Graphics Forum, Vol. 23. Wiley Online Library, 605--614.

[13]

David J Field. 1987. Relations between the statistics of natural images and the response properties of cortical cells. Josa a 4, 12 (1987), 2379--2394.

[14]

Yong He, Yan Gu, and Kayvon Fatahalian. 2014. Extending the graphics pipeline with adaptive, multi-rate shading. ACM Transactions on Graphics (TOG) 33, 4 (2014), 142.

Digital Library

[15]

Karl E Hillesland and JC Yang. 2016. Texel shading. In Proceedings of the 37th Annual Conference of the European Association for Computer Graphics: Short Papers. Eurographics Association, 73--76.

Digital Library

[16]

Gábor Liktor and Carsten Dachsbacher. 2012. Decoupled deferred shading for hardware rasterization. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 143--150.

Digital Library

[17]

Justin Luitjens. 2016. Faster Parallel Reductions on Kepler. https://devblogs.nvidia.com/faster-parallel-reductions-kepler. (2016).

[18]

Don P Mitchell. 1987. Generating antialiased images at low sampling densities. In ACM SIGGRAPH Computer Graphics, Vol. 21. ACM, 65--72.

Digital Library

[19]

Diego Nehab, Pedro V. Sander, Jason D. Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-Time Shading with Reverse Reprojection Caching. In ACM SIGGRAPH/Eurographics Symposium on Graphics Hardware. 25--35.

Digital Library

[20]

NVIDIA. 2015. VRWorks - Multi-Res Shading. https://developer.nvidia.com/vrworks/graphics/multiresshading. (2015).

[21]

NVIDIA. 2016. VRWorks - Lens-Matched Shading. https://developer.nvidia.com/vrworks/graphics/lensmatchedshading. (2016).

[22]

NVIDIA. 2018. NVIDIA Turing GPU Architecture Whitepaper. https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf. (2018).

[23]

Alan V. Oppenheim and Ronald W. Schafer. 2009. Discrete-Time Signal Processing (3rd ed.). Prentice Hall Press, Upper Saddle River, NJ, USA.

Digital Library

[24]

Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG) 35, 6 (2016), 179.

Digital Library

[25]

Olivier Penacchio and Arnold J Wilkins. 2015. Visual discomfort and the spatial distribution of Fourier energy. Vision research 108 (2015), 1--7.

[26]

Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, and Frédo Durand. 2011. Decoupled sampling for graphics pipelines. ACM Transactions on Graphics (TOG) 30, 3 (2011), 17.

Digital Library

[27]

Jeremy Shopf. 2009. Mixed resolution rendering. In Game Developers Conference.

[28]

Michael Stengel, Steve Grogorick, Martin Eisemann, and Marcus Magnor. 2016. Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 129--139.

Digital Library

[29]

Ee-Leng Tan and Woon-Seng Gan. 2015. Computational Models for Just-Noticeable Differences. In Perceptual Image Coding with Discrete Cosine Transform. Springer, 3--19.

[30]

TestUFO.com. 2017. Blur Busters UFO Motion Tests. https://www.testufo.com/. (2017).

[31]

Robert Toth, Jim Nilsson, and Tomas Akenine-Möller. 2016. Comparison of projection methods for rendering virtual reality. In Proceedings of High Performance Graphics. Eurographics Association, 163--171.

Digital Library

[32]

Karthik Vaidyanathan, Marco Salvi, Robert Toth, Tim Foley, Tomas Akenine-Möller, Jim Nilsson, Jacob Munkberg, Jon Hasselgren, Masamichi Sugihara, Petrik Clarberg, et al. 2014. Coarse pixel shading. In Proceedings of High Performance Graphics. Eurographics Association, 9--18.

Digital Library

[33]

Karthik Vaidyanathan, Robert Toth, Marco Salvi, Solomon Boulos, and Aaron Lefohn. 2012. Adaptive image space shading for motion and defocus blur. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics conference on High-Performance Graphics. Eurographics Association, 13--21.

Digital Library

[34]

Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612.

Digital Library

[35]

Andrew B Watson, Albert J Ahumada, and Joyce E Farrell. 1986. Window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays. JOSA A 3, 3 (1986), 300--307.

[36]

Kai Xiao, Gabor Liktor, and Karthik Vaidyanathan. 2018. Coarse pixel shading with temporal supersampling. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 1.

Digital Library

[37]

Lei Yang, Pedro V. Sander, and Jason Lawrence. 2008. Geometry-Aware Framebuffer Level of Detail. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering 2008) 27, 4 (2008), 1183--1188.

Digital Library

[38]

Lei Yang, Yu-Chiu Tse, Pedro V Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L Wilkins. 2011. Image-based bidirectional scene reprojection. In ACM Transactions on Graphics (TOG), Vol. 30. ACM, 150.

Digital Library

[39]

Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 667--681.

Digital Library

Cited By

Li YNumerow LThomaszewski BCoros S(2024)Differentiable Geodesic Distance for Intrinsic Minimization on Triangle MeshesACM Transactions on Graphics10.1145/365812243:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658122
Deng YWang MKong XXiong SXian ZZhu B(2022)A moving eulerian-lagrangian particle method for thin film and foam simulationACM Transactions on Graphics10.1145/3528223.353017441:4(1-17)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530174
Cardoso JKerbl BYang LUralsky YWimmer M(2022)Training and Predicting Visual Error for Real-Time ApplicationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35226255:1(1-17)Online publication date: 4-May-2022
https://dl.acm.org/doi/10.1145/3522625
Show More Cited By

Index Terms

Visually Lossless Content and Motion Adaptive Shading in Games
1. Computing methodologies
  1. Computer graphics
    1. Rendering

Recommendations

Improved Specular Highlights With Adaptive Shading
CGI '96: Proceedings of the 1996 Conference on Computer Graphics International

Gouraud shading and Phong shading are widely used interpolation methods to render a polygon mesh of a curved surface. When an illumination equation has a specular reflection term Phong shading produces more realistic results than Gouraud shading. The ...
Micropolygon ray tracing with defocus and motion blur
SIGGRAPH '10: ACM SIGGRAPH 2010 papers

We present a micropolygon ray tracing algorithm that is capable of efficiently rendering high quality defocus and motion blur effects. A key component of our algorithm is a BVH (bounding volume hierarchy) based on 4D hyper-trapezoids that project into ...
Towards multi-perspective rasterization

We present a novel framework for real-time multi-perspective rendering. While most existing approaches are based on ray-tracing, we present an alternative approach by emulating multi-perspective rasterization on the classical perspective graphics ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques

Proceedings of the ACM on Computer Graphics and Interactive Techniques Volume 2, Issue 1

May 2019

132 pages

EISSN:2577-6193

DOI:10.1145/3339245

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2019

Published in PACMCGIT Volume 2, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
576
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)9

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YNumerow LThomaszewski BCoros S(2024)Differentiable Geodesic Distance for Intrinsic Minimization on Triangle MeshesACM Transactions on Graphics10.1145/365812243:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658122
Deng YWang MKong XXiong SXian ZZhu B(2022)A moving eulerian-lagrangian particle method for thin film and foam simulationACM Transactions on Graphics10.1145/3528223.353017441:4(1-17)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530174
Cardoso JKerbl BYang LUralsky YWimmer M(2022)Training and Predicting Visual Error for Real-Time ApplicationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35226255:1(1-17)Online publication date: 4-May-2022
https://dl.acm.org/doi/10.1145/3522625
Jindal AWolski KMyszkowski KMantiuk R(2021)Perceptual model for adaptive local shading and refresh rateACM Transactions on Graphics10.1145/3478513.348051440:6(1-18)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480514
Mueller JNeff TVoglreiter PSteinberger MSchmalstieg D(2021)Temporally Adaptive Shading Reuse for Real-Time Rendering and Virtual RealityACM Transactions on Graphics10.1145/344679040:2(1-14)Online publication date: 27-Apr-2021
https://dl.acm.org/doi/10.1145/3446790

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents