Denoising

 Code so Far

Here is the code for this section

Denoising

Last time I implemented a path tracer that used importance sampling for optimization. This week, to wrap up this project I added a denoiser.

When it comes to denoising there is a couple approaches. They are described here but they are filtering technique, machine learning techniques, and sampling techniques. Filtering techniques are cheap but blur the image. Machine learning uses autoencoders to reconstruct images from noisy images. Sampling techniques use spatial and temporal data to denoise the image.

Because, I am not denoising the image per frame I wouldn't be able to take advantage of spatial temporal solutions like Nvidia's real time denoiser (NRD). NRD would have been a better solution than machine learning however due to its performance.

For my program I opted for a machine learning encoder. There were multiple to choose from the most popular were Nvidia's Optix, and Intel's Open Image Denoise (OIDN). I ended up deciding on using OIDN, because of its relative ease of integration into the project.

Integrating OIDN

When integrating with OIDN there were a couple gotchas.

When you want to denoise an image you need to pass in a input and output buffer for the image. The buffer in almost of OIDN's examples is allocated and maintained by OIDN. Using this method, in order to get the UAV output image to OIDN it needs to be copied from the GPU to the CPU to feed the image data to OIDN. This can potentially be a slow process. As of version 2 of OIDN you can supply it a Window's handle to a shared buffer. This can be accomplished by making the committed resource of the UAV shared, and retrieve a Window's handle to the buffer using ID3D12Device::CreateSharedHandle.

I did this an soon realized I had another problem. OIDN only supports input and output formats of r32g32b32a32 float, whereas my UAV output image is in the form of r8b8g8a8 which the back buffer requires. I thus needed to convert from 8 bit to floats, send to OIDN, then convert back to 8 bit color. This might have been easy to to convert to 8bit to float, because I could have just written to two buffers in the raygen shader. The problem is converting OIDN's output image format to 8 bit color values. The right way to have done this would be to create a compute shader whose sole job is to do the image format conversion but this would have required me to create an entirely new pipeline, compute shaders, and root signatures. I opted out of doing all this by doing the conversion on the CPU side and having OIDN manage its own buffers. This incurred approximately 20ms that could be improved upon if I had more time.

In order for me to invoke OIDN on the UAV output, the UAV output image had to be completely written to and the data needed to on a buffer that the CPU could access. With the old code my UAV buffer was living on the default buffer. I created a new readback buffer to copy the UAV data to. After copying to the readback buffer I needed to wait until the GPU finished the commands through a fence. I then was able to to use OIDN by copying the data from the readback buffer to OIDN's input image buffer. After that, I copy the output image of OIDN to a upload buffer and copy the upload buffer directly to the back buffer.

This was a lot, but the results are substantial. See the output of using the denoiser below using 500 samples per pixel.

The downside to using this denoiser it that is slow. with a 800x600 image the denoiser is taking ~ 60ms, not including the copying between image formats. If I could further improve this I would switch to NRD.

That is it!

Although my path tracer doesn't account for materials to generate the BSDFs, the importance sampling is very limited, and the denoising is slow, the code can be expanded to improve these things with minimal rewrite. I also learned a lot knowing very little about Dx12. In this project I built both a rasterizer and ray tracer and the ability to switch between the two. I also learned a little about PBR, and denoising tools.