Denoising

 Code so Far

Here is the code for this section

Denoising

Last time I implemented a path tracer that used importance sampling for optimization. This week, to wrap up this project I added a denoiser.


When it comes to denoising there is a couple approaches. They are described here but they are filtering technique, machine learning techniques, and sampling techniques. Filtering techniques are cheap but blur the image. Machine learning uses autoencoders to reconstruct images from noisy images. Sampling techniques use spatial and temporal data to denoise the image.

Because, I am not denoising the image per frame I wouldn't be able to take advantage of spatial temporal solutions like Nvidia's real time denoiser (NRD). NRD would have been a better solution than machine learning however due to its performance.

For my program I opted for a machine learning encoder. There were multiple to choose from the most popular were Nvidia's Optix, and Intel's Open Image Denoise (OIDN). I ended up deciding on using OIDN, because of its relative ease of integration into the project.

Integrating OIDN

When integrating with OIDN there were a couple gotchas.

When you want to denoise an image you need to pass in a input and output buffer for the image. The buffer in almost of OIDN's examples is allocated and maintained by OIDN. Using this method, in order to get the UAV output image to OIDN it needs to be copied from the GPU to the CPU to feed the image data to OIDN. This can potentially be a slow process. As of version 2 of OIDN you can supply it a Window's handle to a shared buffer. This can be accomplished by making the committed resource of the UAV shared, and retrieve a Window's handle to the buffer using ID3D12Device::CreateSharedHandle.

I did this an soon realized I had another problem. OIDN only supports input and output formats of r32g32b32a32 float, whereas my UAV output image is in the form of r8b8g8a8 which the back buffer requires. I thus needed to convert from 8 bit to floats, send to OIDN, then convert back to 8 bit color. This might have been easy to to convert to 8bit to float, because I could have just written to two buffers in the raygen shader. The problem is converting OIDN's output image format to 8 bit color values. The right way to have done this would be to create a compute shader whose sole job is to do the image format conversion but this would have required me to create an entirely new pipeline, compute shaders, and root signatures. I opted out of doing all this by doing the conversion on the CPU side and having OIDN manage its own buffers. This incurred approximately 20ms that could be improved upon if I had more time.

In order for me to invoke OIDN on the UAV output, the UAV output image had to be completely written to and the data needed to on a buffer that the CPU could access. With the old code my UAV buffer was living on the default buffer. I created a new readback buffer to copy the UAV data to. After copying to the readback buffer I needed to wait until the GPU finished the commands through a fence. I then was able to to use OIDN by copying the data from the readback buffer to OIDN's input image buffer. After that, I copy the output image of OIDN to a upload buffer and copy the upload buffer directly to the back buffer.

This was a lot, but the results are substantial. See the output of using the denoiser below using 500 samples per pixel.




The downside to using this denoiser it that is slow. with a 800x600 image the denoiser is taking ~ 60ms, not including the copying between image formats. If I could further improve this I would switch to NRD.

That is it!

Although my path tracer doesn't account for materials to generate the BSDFs, the importance sampling is very limited, and the denoising is slow, the code can be expanded to improve these things with minimal rewrite. I also learned a lot knowing very little about Dx12. In this project I built both a rasterizer and ray tracer and the ability to switch between the two. I also learned a little about PBR, and denoising tools. 



 

Comments

Post a Comment

Popular posts from this blog

Enabling Raytracing - Part 2

Image
Code so Far Here is the code  for this section: Enabling Raytracing - Part 2 This week I added the shader binding table for raytracing. The shader binding table is where the data is bound to root parameters. This is contrary to rasterization where the parameters are bound through command list calls such as SetGraphicsRootConstantBufferView . One that was done I updated the render loop to support raytracing. This included transitioning the output buffer to unordered access, setting the pipeline state, and calling the DispatchRays method using the  D3D12_DISPATCH_RAYS_DESC which contains info about how many rays to generate, and the shader binding table. For my demo, one ray is generated for each pixel. After calling DispatchRays , our output image is written to. I then copy the image to the render target. The shaders were also updated to actually generate rays, and output the color of the cube. The RayGen shader was updated to call the TraceRay. The ClosestHit shader was ...
Read more

DXR Demo - Introduction

Welcome. I am a programming student at the Florida Interactive Entertainment Academy (FIEA) and this blog's aim is to chronicle my ten-week solo programming project during my final semester. The project at hand involves the implementation of raytracing, harnessing the power of DirectX 12 and DXR. DirectX 12, the latest DirectX API as of its introduction on July 28th, 2015, alongside Windows 10, allows for advanced low-level programming. It streamlines graphics programming by reducing driver overhead and enhances efficiency in multi-threading tasks. This level of control, however, comes with an increased responsibility for the graphics programmer to thoroughly understand and manage the complexities involved in getting a program operational. As a first-time user of DirectX 12, I anticipate a challenging yet rewarding journey ahead. In this blog, I aim to document the developmental milestones, challenges, and technical insights that arise during the creation process. Expect to see dee...
Read more

Simple Lighting

Image
Code so Far Here is the  code  for this section Rendering a Scene Now that I can render a scene, and have the raytracing pipeline setup, I want to make the raytracing scene look better. In rasterization there is diffuse light, specular lighting, etc., but how do you do proper shading for a ray tracer? To answer this question I picked up the book  Physically Based Rendering, Fourth Edition , by Matt Phar et al. This book goes in depth in the theory and implementation of how to achieve photorealistic renderers using raytracing. A critical understanding of how light actually works is necessary (at least at a macro level) in order to replicate its behavior. When a light source strikes an object it's light can either be reflected, refracted, or scattered in any direction due to subsurface scattering. The probability of each direction can be described by a probability distribution called bidirection scattering distribution function (BSDF).  Surface scattering example (Matt...
Read more