KAIBUTSU - Julian Frank

How did I come up with the initial idea?

I originally had two ideas for the Midterm Project 300. The first idea was very technical and significantly more complex to execute, while I was already familiar with the pipeline and workflow for the other. I had to decide and weigh which concept I had more motivation for and which was realistically achievable as an individual.

Given that we only have a week for the end-/midterms, I thought it was better to set aside the complex idea for now. Another decisive factor was which concept I was more excited and motivated about.

When I recently watched the series "Shogun," some shots stood out to me. They stayed in my mind for weeks afterward and became one of the main inspirations for my project.

I quickly realized how I needed to approach the project to achieve a similar result with my own artistic touch.

How important was the use of reference material overall?

For me, references are an absolute must for any 3D project. They form the foundation for the accuracy, authenticity, and detail of the work. Using reference material ensures that models, textures, and the Lookdev appear realistic and convincing.

References serve as a visual guide for me. Specifically, for lighting, I analyzed several shots from films to understand better how light behaves, how colors interact, and what color contrasts are created. The same applies to the behavior of materials and textures, allowing me to replicate them as accurately as possible.

In my opinion, you learn a lot by examining your own references closely. You develop a better sense of color, contrast, light, composition, and textures. This process gradually shaped the look and feel for my Midterm 300 project.

In summary, references are indispensable!

how did I approach the project?

From the beginning, I knew that this time I wanted to create a single shot. This approach allows me to focus on more details and dedicate more attention to them.

Step 1: Pencil and Paper

Since I also enjoy working analog, I collect my initial ideas on paper. It usually looks very chaotic and  incomprehensible to outsiders, but it helps me organize my thoughts. From this process,  a rough concept  was created.


Step 2: Gathering References & Look and Feel

As previously mentioned, references play a crucial role in my work. I often watch "beautiful shots of movies" videos to find more inspiration for color contrasts, lighting, composition, and so on, to enhance my idea.

Step 3: Shot Planning & Time Management

I already had a rough idea of my camera movement, composition, and characters in mind.

Now the questions arise:

How do I plan my 3D file? Is it realistic to complete in a week? How large should the scene/environment be?

I quickly created this very bad and abstract drawing of my scene planning.
Once the next step is clear, it should be more understandable.

Time management plays a crucial role in my project. It was important for me to dedicate at least one day to the breakdown, as it is also very important. It's always a shame when you can't allocate enough time for compositing and the breakdown at the end.


Step 3: Shot Planning

Now it's time to jump into the program and create a rough layout. After my research, two aspects were certain. First, I knew which characters I would use, and second, that it would be a slow-motion shot.

Therefore, it was clear that I needed to time remap my mocap/self-animated data. This was quite straightforward using the Time Editor. After I was initially satisfied with the rough draft, I rendered several playblasts to check the timing of the animation.

Step 4: Set Dressing & Render Layers

Next, I filled the scene with scattered vegetation and objects to enhance the look and feel. I created the vegetation using Megascans and trees made in Speedtree.

To be as efficient and flexible as possible in post-production, I always work with render layers. Each layer contains vegetation and animations.


STEP 5: TEXTURES & SHADING

Next, I set up all the textures. The foreground ground needed to look as detailed as possible. Therefore, I blended two ground materials and their displacement maps to achieve a more interesting texture.

The setup is very basic, but also very efficient. Essentially, I adopted the motto "keep it simple" for my midterm to achieve the result as quickly as possible.

The texture served its purpose, although I could have improved it further. Given my strict time management, I had to focus on the next step to complete the project.

Mutant texture


I retextured the mutants in Substance Painter. It was important to me that they appeared distinct and not monotonous. Therefore, I designed two different textures for them.



STEP 6: LIGHTING & AOVs PER LIGHT


To create atmosphere, we need lights. I already knew in the back of my mind that I couldn't go overboard with light sources because each additional light source adds to the render time.

It was very important for me to have full control in Nuke.
So, I rendered not only the light groups as AOVs but also the AOVs PER LIGHT.

Disadvantage: Limited use of number of lights in Maya

Advantage: Full control over lights in Nuke :)

Light list in the scene:

1 Directional Light

2 Area Lights

1 Spot Light

Gobos:

On the left and right sides of the scene, there is a red area light each, which has an animated flame gobo assigned with color in the shader. This allows me to fake simulate a flame in the light simply and effectively.

STEP 7: RENDERING

The look and feel slowly came together, allowing me to start rendering the individual render layers. My goal was to start as early as possible.

A challenge for me was optimizing the scene, working with instances, LOD distances, and LOD texture sets. The "Sprite Node" in Redshift also helped me significantly reduce the render times of scattered elements.

I used Redshift as the preferred render engine because it renders via GPU. While the scene was rendering, I could seamlessly script in Nuke and continue working on the breakdown. Once again, a very effective workflow. At this point, I was shocked myself that nothing had gone wrong yet. I was surprised that my PC hasn't exploded yet.


STEP 8: STYLIZED LOOK

For this project, I wanted to try something new, unsure if it would work or not. If it didn't work, I had an alternative solution in mind.

The Plan:

The shot was intended to end with a transition into an abstract 
look, resembling a painting or etching

Train of Thought:

I asked myself how it would look if I assigned all materials in a new file a 
stylized shader and composited it over the rendered footage in Nuke.

Testing Phase:

Before changing all materials in the scene, I conducted rendering 
tests on a single object first.

First Image: Raw Render
Second Image: Stylized Shader
Third Image:  both combined

After the experiments, it became clear to me that the results were not satisfactory. Nevertheless, it was very interesting to have tried it out. However, I didn't want to give up on the idea of the stylized look.

Alternative solution:

I chose a simpler, more traditional solution by creating several matte paintings in Photoshop and matching them in Nuke using Cards and the Position AOV. These matte paintings will then be blended in at the end with an animated mask.

From the beginning, I was aware of this alternative solution, so I also decided to create at least 5 render layers to integrate the matte paintings on different levels.

STEP 8: NUKE SCRIBT

The Nuke script is structured top to bottom. Naturally, everything 
 must be very tidy and follow to a naming convention for easy navigation.

each render layer had positional AOVs, allowing me to accurately 
place the cards based on the point cloud.

STEP 9: ANIMATIC/SOUND Design

I like to prepare everything while it's rendering, not just the Nuke script but also my Premiere Pro file. Inside, there's usually a playblast or a test render, where I work on the sound design. This part was also very experimental as I considered which musical direction to take.

I asked myself whether I wanted it to be completely epic or if I should incorporate an oriental touch. I ultimately decided on a blend of both.

Premier Pro Insight

STEP 10: BREAKDOWN

Since I was working on many things simultaneously, I wanted to visualize this in my breakdown. I built a 3D space with flying monitors there. In the Breakdown I reused the fire as an element to improve the connection between shot and breakdown phase. My PC was also on fire at the time.

Additionally, I wanted to create transitions so that I didn't have to show my references, etc., with a harsh cut. Everything should flow smoothly, maintaining a coherent narrative.

Final Shot Compositing:


I also like to draw lines through my image for compositing to consciously guide the viewer's eye. I've chosen the golden ratio, which often appears in Renaissance and Baroque paintings (see reference).

My challenges on this Project

Challenge 1: OPTIMIZATION

One challenge for me was optimizing the scene and continually reducing render times. Whether it was through instances, LOD assets, LOD textures, LOD tessellation, LOD vegetation, etc. Although I've had mixed experiences with large-scale environments and their render times in the past, it worked quite well here.

Challenge 2: NIGHT LIGHTING

Additionally, I had a full moon lighting mood. I didn't have much time to research light spreading and interaction at night. Therefore, I also found it challenging to create a night scene that wasn't underexposed.

Challenge 3: STYLIZED LOOK

One challenge that failed was to generate a complete stylized look. I was aware of other ways -paint normals etc.- to create an abstract look, but I completely ruled them out because it wouldn't have been possible in a week. However, the alternative solution worked very well.

Challenge 4: TIME MANAGEMENT

Another challenge was planning everything in time. I always set the actual deadline one to two days earlier so that I don't have to rush everything at the last minute. It's important for me to invest enough time in creating a convincing breakdown, as it's also crucial for such work.

Challenge 5: BREAKDOWN CONCEPT

Talking about the Breakdown it was important for me personally to develop a breakdown that reflects my project. Developing suitable transition concepts and the layout of the screens in the 3D space was another challenge.

Challenge 6: Navigating Post-Production Effects Without Overdoing It

Since I had many layers and ample room in post-production to add effects, it was also a challenge not to overdo it, whether with flames, explosions, dust, etc.

What did I learn from this project?

Time managment ist key.

I've learned that sometimes it's better to develop one singel shot in such a short time. Additionally, I've realized that it's crucial to create a proper schedule and, if something doesn't work out, to flexibly search for new solutions. If I was ahead of schedule, I would set myself additional tasks that could enhance my project even further. (for example the stylized shader experiments)

Working alone un such a project is very challenging.  Furthermore, everything went as planned, and it was very satisfying to have complete control in Nuke over my lights.

Thank you for taking the time to read my breakdown up to this point. I gladly accept criticism and suggestions for improvement, as they help me a lot.

There are certainly better and quicker solutions to problems and workflows. However I would have to act relatively quickly here, as the project only had seven days.

have a nice one :)