A procedural city inspired by Liguria, Italy. Done procedurally in Houdini, and combined with live action footage. Responsible for concept and execution of all aspects of the project except for the live action footage, which was filmed by Matt Warren.
Hidden City is one of my most recent, exciting, and ambitious projects. The intent of the project was to do a deep dive into all aspects of the VFX pipeline, and complete a difficult shot from start to finish. The shot was done as a project for an independent study at my university, so I had to explore many aspects of the pipeline on my own. A large part of this project happening is owed to Barbra Meier who was my mentor throughout this process.
I started by coming up with a pretty simple idea for a shot: I wanted to film a shot that pans over a live action mug, and then "drops" into the mug to reveal a city nestled inside of it. I designed the shot to have an element of surprise to it, and was personally very excited at the idea of exploring the contrasting scales of the mug and the world inside. As with any story, this shot has a larger backstory that is hidden from the viewer, to make sure that the narrative in the shot is cohesive. The idea is to explore what happens to a mug of tea that one starts drinking, and then forgets about? Perhaps if left out long enough, a world can develop inside of it.
After coming up with the idea for the shot, I worked on gathering reference. I realized early on that the city that I was imagining in my head looked a lot like Liguria, Italy, so I spent some time looking at images of the region to familiarize myself with the landscape and look of Liguria before diving in. Some of the references I used can are included bellow (1st image is taken from Artem Chebokha's ArtStation):
The next step was to obtain the live action footage, and do a camera track. I ended up using Blender's camera tracker, mostly since it was what I was most familiar with. Unfortunately, I found that most of the detail in the shot rested in the single plane of the newspaper, so I adjusted the motion track by hand as well.
With a completed camera track, it was time to dive into the asset creation stage. I broke down the project into 5 core parts: Cliffs, Trees, Houses, Docks, Ocean. All of these elements were done in Houdini and rendered with Mantra.
I started with the cliffs. I measured the cup I was using to film the footage, and re-created an approximation of its inside with a cylinder. I then decided to use the base of the cylinder to weight-paint elevation. This method balanced control and proceduralism in a very organic way. The painted weights were used to generate a rough blocked out mountain that was then smoothed by conversion to VDB and back to polygons. The final steps included adding shade variation, and geometric displacement with various noise functions. One aspect of the cliffs that I am very proud of is the base shader for the grey rocks. In looking at rocks up close, I noticed that when a rock is grey, it is commonly actually made up of tiny specks of all different colors. We interpret the overall color as grey based on a sort of averaged brightness. For this reason I used a square of generated RGB noise as the base texture on all of the rocks.
Next came the docks. I knew ahead of time that I wanted to re-use the tool that I used for the docks to create the stairs and bridges that connected the houses. For this reason, I designed a wrangle that would generate a dock/house structure from a curve. If the curve was flat, the result would look like docks. If the curve had any change in height, the result would look like stairs. This scheme ended up working pretty well. A crucial part in the dock generation was the decision process of where the railings supports were placed. The solution I came up with used a maxDist value, that would put a support on a straight segment of docks every maxDist units. If the angle between two curve segments was greater than some threshold, the maxDist would be overridden and a support would be placed at the point of the dock where a large change was detected. This ended up giving railings that had controllable/even supports, and that had supports in parts where the stairs took aggressive turns.
Next came the houses. In order to generate the houses on the cliff, I once again, turned to weight painting. I flattened parts of the cliff where I wanted houses by revisiting the elevation weight paint. I then used weight painting to select organic shapes on the cliff in order to separate out geometry where I wanted to generate houses. A huge speed-up to this process included generating the houses on a ploy-reduced version of the mountain geometry. On that geometry I ran a looping algorithm that would iteratively add cubes to the area, only if the cubes were not intersecting with a pre-existing cube. This gave me a set of locations for making houses, along with the dimensions that each house should be. In other words, I generated the bounding boxes of the houses first, in order to speed up intersection calculations. The extra add-ons to the houses such as balconies, roof types, and railings, were selected randomly and with weighted distributions. The thing that really made the houses look like the reference came with my discovery that the houses in Liguria are not randomly colored, but are actually made from a smaller set of colors. I wrote a weighted coloring scheme that imitated the mauve, orange clay, and turquoise, house colors found in Liguria.
The ocean and trees were made using pretty standard methods. The ocean was made predominantly using the ocean shelf tool, and procedural shading in mantra to highlight the crests of the waves. I decided to keep the ocean a tea-like color to remind the viewer that all of the action takes place in a mug that was filled with tea. For the trees I made a base tree model using a scatter/copy to points method. I then made a couple of color variations on the leaves, to make all of the leaves slightly different shades of green, and then made a slightly yellower and a slightly browner tree. I then instanced the trees on weight painted areas of the mountain. A very crucial part to this process was using proxy geometry in the instancing step. The result was a set-up that allowed me to develop the look of the trees in the viewport without suffering from speed setbacks. Overall the trees are, in my opinion, one of the weakest looking parts of the project. While they look decent in far-away shots, they still tend to look more like moss rather than large trees.
The final portion of the project focused mainly on rendering, compositing, and color grading, and was done in Adobe After Effects. I discovered a rendering option in Mantra that allowed me to create a cryptomatte linked to geometry nodes in Houdini. As a result I had separate mattes for the trees, houses, docks, ocean, mountain, and cup insides that I was able to access. This gave me a lot of control on the color grading, and allowed me to continue tweaking the color and look of the final shot in the post-processing stage. A difficult obstacle I had to figure out when planning this shot ended up being the reflections of the 3D assets one the side of the live action mug. I ended up rendering a reflective band that opened up. This allowed for reflections in the render, and did not block the camera view into the mug once the camera was looking directly at the city.
While there are still many things I would want to improve in this project, I am quite satisfied with the results. I was very pleased with the process of recreating a real-world location. A particularly pleasant part of the project was when I was adding in the stairs that connected the whole city, and found myself imagining I am an inhabitant of the city examining the rock face, and deciding where the best place for a staircase would be. It is in these moments where I became a character in my own shot, that I had the most fun. I also learned an irreplaceable amount of information about planning shots properly, amending production mistakes, thinking ahead about compositing roadblocks, and color theory.