Our approach automatically splits input images into layers motivated by light transport, such as (a): occlusion, albedo, irradiance and specular, or (b): the six major spatial light directions, which can then be manipulated independently using off-the-shelf photo manipulation software and composed back to an improved image. For (a) shadows were made deeper, albedo hue changed, saturation of irradiance increased and the specular was blurred for a more glossy material. For (b) the front lighting was made weaker and light from the left had been tinted red.

Abstract

Photographers routinely compose multiple manipulated photos of the same scene (layers) into a single image, which is better than any individual photo could be alone. Similarly, 3D artists set up rendering systems to produce layered images to contain only individual aspects of the light transport, which are composed into the final result in post-production. Regrettably, both approaches either take considerable time to capture, or remain limited to synthetic scenes. In this paper, we suggest a system to allow decomposing a single image into a plausible shading decomposition (PSD) that approximates effects such as shadow, diffuse illumination, albedo, and specular shading. This decomposition can then be manipulated in any off-the-shelf image manipulation software and recomposited back. We perform such a decomposition by learning a convolutional neural network trained using synthetic data. We demonstrate the effectiveness of our decomposition on synthetic (i.e., rendered) and real data (i.e., photographs), and use them for common photo manipulation, which are nearly impossible to perform otherwise from single images.