Lighting / LookDev / Compositing – Dominik Benz

Still Life – CG Integration

This project was originally created during my advanced compositing course at PIXL VISN, and since I've always wanted to do a proper CG integration, this was the perfect opportunity to do so. For the mood of this integration, I wanted to do some sort of still life scene with wine, cheese, grapes, etc. But since I like to challenge myself, I decided to make the integration a little more difficult by integrating a refractive object, and to make things even more difficult, I wanted to integrate two objects that would also appear as real objects in the footage.

References

The references I gathered for this project were mainly to get an idea of the possible layout and objects I could place in the scene. This part definitely helped me create a warm and appealing scene. All the other references I used later to make the integration as realistic as possible, I had to shoot myself to recreate the appropriate lighting and shading of the objects in my scene. I also took additional references for the environment from different angles, which helped me align everything in 3D. In addition to the references I was taking, I always had the glass right next to me while I was working on the project. So I could always look at it from different angles and see how the surface interacted with the light.

Process

After I had everything more or less planned out, it was time to shoot my footage. With my layout references in mind, I started experimenting with different layouts until I was happy with the look of the scene. Next, I shot some reference plates with real objects as placeholders to get a feel for the camera movement and look of the final shot. Then I replaced the real objects with tracking markers and started shooting some takes for the final plate. It took some time to get it the way I wanted it, since the camera movement was completely manual. To get the best possible result, I also had to shoot a lens grid for the lens I was using and create a 360° HDRI of my scene.

Modeling

The two models were pretty simple, so I quickly modeled them myself using some references I took.

Texturing/shading

After the models were ready, I started texturing them. For the glass model, I projected some surface imperfections in Substance Painter and blended them together with some masks to get some breakup. Later during the shading of the glass, I also experimented with a little more breakup by adding minor displacements. But after comparing the result to my references, I found that this was not necessary because the displacement added some details that were not visible in the reference. So I kept the shading of the glass fairly simple.

Texturing the grape was a bit more complex, but not too complicated either. I started by creating some quick procedural masks in Substance Painter and exported them to Mari. In Mari, I used these masks and some other procedurals to mix some colors that I picked from my references. In addition, I hand painted some areas to bring out details that I couldn't quite get with the procedural masks. I only created the base color in Mari, because I was able to do the rest of the shading directly in Arnold with some procedural noise and ISO masks I extracted from Mari. Doing the shading itself was a bit tricky, getting just the right amount of subsurface scattering in while still preserving all the texture detail. I tried to get as close as possible in shading and later on adjusted it in Compositing so that it pretty much matched the plate perfectly.

Lighting

The lighting aspect of a CG integration is pretty straightforward because there's not a lot of room for creativity and the only goal is to match the lighting one-to-one. As a starting point for the lighting, I used the HDRI I created on set, but before I could use it properly, I had to color correct it a bit as the light colors didn't match the light colors in my footage. After I adjusted the colors, I tried to match the light directions of the HDRI to the reference. Since I didn't shoot a grey and chrome ball like I should have, I was fortunate to have a strong reflecting object in my scene that I could match the light direction to. In addition to the HDRI, I created some supporting lights to get all the reflections I saw in the references.

Since the glass is a refractive object, I had to project my plate as a background to get the correct refraction in the glass, and it also helped to get some better reflections of the environment. But that was also where a lot of problems came in. First, I had to figure out how to project the undistorted footage with the overscan so that I could later redistort it in Nuke. So the problem was that when I exported the footage with the overscan, Maya simply pushed the overscan inside the frame while projecting. To fix this problem, I decided to transform the overscan inside the frame before exporting to Maya, and then push everything back out in Maya with the inverse transform. This was a pretty messy way to solve this problem, but it worked.

The next problem, which I already anticipated would be a bit problematic, was adjusting the defocus without double defocusing the refracted background. At first, I thought that separating the glass from the background using Light Path Expression (LPE) would be enough. So I made a small script that created all the LPEs and AOVs for each light group separately, which also allowed me to adjust the shading and lighting of my objects later in compositing. But when I tried this approach, I realized that this wouldn't work because there would be some sharp edges in the background while the glass would be blurry and out of focus. Thus, I needed a different solution. The idea was to simply freeze the last image of my plate, where the background was as sharp as it could be, and use that as the background for the area where the glass was. That way I could defocus the refracted background again to match the defocus of the glass.

Compositing

Compositing was by far the most complicated and time-consuming part of this project, and probably the biggest composition I've ever done so far. First, I had to prepare my footage for tracking, paintout, and projection. This included undistorting the footage using the lens grid and degraining it. As soon as I had everything prepared, I started tracking the camera movement using the Nuke Camera Tracker. Because of the rack focus in my plate, it was a bit difficult to get a good camera track, but after many tries and experimenting with different settings, I finally managed to get a good track. Once the tracking was done, I was able to export the camera to Maya and start painting out the tracking markers. For this part, I used Photoshop as it is more convenient for me to use than Nuke. I used the frames where I painted out the markers and projected them back onto the original plate to create a cleanplate. Whenever I was done with an important part of my composition, I always pre-rendered that part to keep my composition more lightweight. For the shadows of the objects, I rendered some shadow passes with Arnold and adjusted them with some rotos so that the shadows only appeared where they were supposed to be.

Using all the LPEs and AOVs that were rendered, I had full control over the shading and lighting of my objects. This was really useful for adjusting many aspects of the shading and lighting without having to re-render everything.

Once I was happy with the look of my objects, I added some more minor lens effects like lightwrap, diffuse, halation, etc. to make them blend more seamlessly into the plate. For the halation, I also tried adding a small color change that was visible in the original plate when the focus changed for some of the highlights.

One of the many problems I encountered during the compositing phase of this project was matching the rack focus to the plate. Fortunately, I found a tutorial by Ben McEwan that showed how to automatically match the rack focus. His method required me to track one of the bokehs in the background and remap the tracked values for the defocus nodes. This worked really well and was much easier than animating the defocus manually. But there was one last problem I needed to solve for a proper defocus. I noticed that the bottle behind the broken glass was more defocused than the actual bottle in the plate, so I needed to defocus the bottle behind the glass independently of the fabric at the very back. To accomplish this, I simply used two different defocus nodes for the broken background and blended them using a keymix and a roto as a mask. 

To make everything a bit more appealing, I made some minor color corrections and other adjustments at the end.

Final Words

I really learned on this project that planning ahead is key for creating a good CG Integration, and even if you plan enough, there will still be some challenges that you can't foresee. But if you plan ahead, you can at least minimize the number of problems that occur. Also, I realized how important it is to have on set references because otherwise you have no comparison of what to aim for. The last thing I learned on this project is how important it is to get good feedback, sometimes you're so focused on your work that you don't even notice certain mistakes. In those cases, it's really helpful to have an outside perspective that can point out mistakes that you missed yourself. At this point I would like to thank my Compositing instructor Marcel Pichert, who always gave me great feedback and pushed me to improve this project.

Software used

Fantasy Environment

The Fantasy Environment project was my first demo reel project that I worked on at PIXL VISN. For this project we had to create some kind of fantasy environment, which limited the possibilities a bit. But I was happy to find a cool concept from Lok Du that I really liked.

References

For this project, I already had a good concept where everything was clear and there wasn't much room for speculation. Still, I gathered some more references for different parts of the environment to get a better understanding of what some of these objects should look like. I also decided early on to change the concept a bit. Since I didn't want to spend too much time modeling a character, so I decided to replace him with a tent next to the campfire.

Environment creation

I started this project by trying to line up the camera and blocking out some of the environment. But I quickly realized that the concept's perspective probably wasn't entirely accurate, so I tried my best to get a similar layout to the concept anyway. After I was satisfied with the blockout of the scene, I moved on to modeling the statue, since it was the hero asset of the environment. For modeling the statue I began by using a base mesh which I then adjusted inside Zbrush to match the statue in the concept. After I was done modeling the statue, I quickly modeled the wood logs for the camp and tent and created the cloth in Marvelous Designer. This was also the first time I used Houdini for some of the subtle cloth simulations, and I was really impressed with how quick and easy it was to create a cloth simulation in Houdini. Most of the other props in the camp area were some Megascan assets that I downloaded since they weren't really specific to the concept.

For the scattering I mostly used MASH, but since I feel like it really lacks some good scattering features, I also explored the scattering tools in Houdini a bit and ended up using Houdini to scatter some of the plants and the moss on the broken house. That worked pretty well so I am probably going to use it a lot more in the future. To transfer the scattered objects between Houdini and Maya I used Arnold Standins, which turned out to be a really efficient way to work. The majority of the plants I scatter were also Megascan assets, but for the trees I bought a tree from the SpeedTree library and adjusted its settings a bit to create multiple variations of it.

Since I couldn’t really find a good campfire stock footage or VDB I decided to create a small campfire simulation myself using Maya Fluids. The good thing about this is that Maya Fluids are really well integrated with Arnold, so I didn't have to do much to make it look good in the render.

Shading

The only shader that was a bit more complex was the one for the statue, since it was the hero asset in the scene. I began by layering several concrete textures on top of each other and used different noises to break up the surface of the statue. Once I was satisfied with the breakup in the concrete texture, I layered some moss on top of everything. Again, I used Utility Nodes to make the moss appear mostly in areas near the ground and in areas exposed to a lot of moisture and combined them with some noises. For the displacement, I first created some surface details like cracks and other rock details in Zbrush and later on added some larger and smaller noises in shading to get a really rough looking statue like the ones I had in my references.

For the plants in my scene, I only had to make some minor shading adjustments to give them more detail. Other than that, I didn't have to add too much detail in most of the shaders, since the majority of the objects were far away from the camera and the scene was very dark overall.

Lighting

The most interesting part of this project for me was probably the lighting, because it was the first time I really tried to recreate the lighting of a concept art in 3D. I started by lighting the area around the campfire and added several bounce lights for all the surrounding objects. For some lights, I had to use light blockers and light linking to make them affect only certain objects. The point I struggled with the most was the reflection of the campfire on the water, but after some experimentation I concluded that it was probably not physically possible to achieve such a reflection in lighting.

After I was finished with the main area around the camp fire I added some fill lights and actually two Skydomes one as indirect light source for the environment and another one just to be able to brighten everything up in comp later on if needed.

To create the final render, I set up four render layers for the background, foreground, atmosphere, and one for the smoke footage around the campfire. This gave me the flexibility to reduce or adjust the render settings for each of the layers to make everything more efficient and reduce render times. For the atmosphere, I had three different light sources, two for the atmosphere in the distance and one for the atmosphere around the fire. I assigned each of them one of the primary colors so I could separate them in compositing and adjust them individually as needed.

Compositing

I'm always impressed with how effective compositing can be in shaping the final look of a project. For this project, compositing played a big role in achieving a look similar to my concept. It's really handy to be able to make small changes to your render and get a little closer to the final result with each step.

First, I needed to create a background for my render, and since the background in the concept was very specific, I decided to create the whole background from scratch in Nuke. To do this, I used some ramps to create a gradient with colors I picked from the concept, and multiplied that gradient with some noises to break it up a bit. Next, I used several noises to create different types of stars, and mixed them together using rotos. It was a pretty simple and efficient way to create a custom night sky. In order to make the shot a bit more dynamic and lively, I added some birds in the background.

Then I split the beauty render into the different light groups and adjusted them a bit to match the mood of the concept and tweak the overall lighting of the shot. Since the main light source of my scene was the campfire, I wanted to add a little flicker to the lights to recreate the feel of a campfire. To do this, I used the Curve tool to analyze the average intensity of my simulated fire and then remapped the resulting curve to values between 0.6 and 1. The remapped curve was then used to control the gain for all the lights I created for the campfire. To add a little more of a fantasy world feel, I also added some floating particles near the camera. Finally, I added some camera and lens effects like lens distortion, vignette, grain, etc. and I was happy with the final result.

Final Words

Since this was my first demo project and I wasn't completely satisfied with the first result, I thought I should just drop the project and not include it in my demo reel. But after receiving some feedback, I decided to revisit it and try to improve it a bit more. Now I'm very happy with the result, and I've learned that sometimes it's better to let a project sit for a while and come back to it later with a fresh mind. This was also the first project where I tried to use a bit of Houdini to create the environment, and I'm really impressed with how useful Houdini can be for that. So I'll probably try to incorporate Houdini into my workflow a bit more when creating environments.

Software used

Thank you for taking the time to read the detailed breakdown of my projects. I hope you were able to get a better understanding of my process for creating these projects. If you have any feedback, feel free to leave your thoughts in the comments.

For any further questions about these projects or just questions in general, feel free to contact me on LinkedIn, Artstation or via email: [email protected]