One of the basic difficulties with interactive walkthroughs is the high quality rendering of object surfaces with non‐diffuse light scattering characteristics. Since full ray tracing at interactive rates is usually impossible, we render a precomputed global illumination solution using graphics hardware and use remaining computational power to correct the appearance of non‐diffuse objects on‐the‐fly. The question arises, how to obtain the best image quality as perceived by a human observer within a limited amount of time for each frame. We address this problem by enforcing corrective computation for those non‐diffuse objects that are selected using a computational model of visual attention. We consider both the saliency‐ and task‐driven selection of those objects and benefit from the fact that shading artifacts of “unattended” objects are likely to remain unnoticed. We use a hierarchical image‐space sampling scheme to control ray tracing and splat the generated point samples. The resulting image converges progressively to a ray traced solution if the viewing parameters remain unchanged. Moreover, we use a sample cache to enhance visual appearance if the time budget for correction has been too low for some frame. We check the validity of the cached samples using a novel criterion suited for non‐diffuse surfaces and reproject valid samples into the current view.