Overpainting

Photoshop File, Folder and Layer Structure and Nomenclature

I recognised from the outset that this project would in all probability end up comprising of potentially dozens of files, each containing a section of the overall matte painting. With this in mind I felt it was important to standardise the files as much as possible so that I became extremely familiar with how they were structured, how each layer and folder was named and what each folder would contain.

The image below shows the layers panel for Image Set D, which is projected from the camera in the hierarchical set derived from frame 140.

Each file has a root folder named with the image set letter (in this case D) followed by the projection frame number (in this case 140). The base alpha from the render is applied to the root folder as a layer matte. When projecting, it is possible to simply reference the alpha from the channels panel. However, this approach was taken in response to the likelihood that the masks will need to be modified, tested, repainted etc. when managing termination points and transition blends between overlapping projections in the hierarchical layer set.

The comparative image pair (below) shows an example of this in use. Note the base alpha (left) derived from the render from 3D incorporates the full extent of the buildings. However, when blending this projection with others in the set, modification of the alpha holds out areas of the overlapping texture to smooth the transition at natural termination points.

Adding the alpha as a layer mask allows the alpha to be modified directly over the image, which is more accurate and a quicker and easier process.

Referring back to the Photoshop layer set, note that the file is tailed off with a pure black base layer which serves no purpose in the actual projection of the image but provides a backdrop which I find easier to work with than a transparent base when overpainting.

In between are two subfolders. One holds all the layers used for overpainting and the other holds all the arbitrary output variables (AOV’s), rendered from the 3D software.

AOV’s

The AOV’s are essentially the components that make up a beauty render but split into separate layers. This allows different post processing operations to be undertaken in Photoshop later and without having to revisit the 3D application.

For this project, I set the 3D application to render five beauty AOV’s; these being diffuse, specular, shadow, reflections and ambient occlusion. The same passes were captured for all image sets and from each projection camera.

To reinstate the beauty image from these individual passes, a standard compositing setup has to be deployed. Again, this was deployed uniformly across all the image sets. Firstly, the AOV’s need to be placed in a fixed layer order from top to bottom, and secondly specific blend operations need to be applied to certain passes. Colour passes (i.e. specular and reflection) are assigned the additive blend mode.

The image below (left) shows the reflection pass and (right) shows the assignment of the Additive blend operation. This effectively makes pixels commensurately transparent the greater the black colour values so only the light detail is retained layered over the other passes in the stack.

Subtractive passes such as the shadow and ambient occlusion work in the opposite direction. Here we need to retain the dark values and make the pixels commensurately more transparent the greater the white values.

The image below (left) shows the shadow pass and (right) shows the assignment of the Multiply blend operation. This effectively blends the dark detail over the other passes in the stack and makes the pixels commensurately more transparent the greater the white values so only the dark detail is retained.

The 3D application is also used to serve the project by rendering separate alphas for all the key elements of the buildings in the image set. These are stored in the Channels panel for use in the overpainting process.

There is certainly a time investment associated with configuration of the initial render settings in the 3D application and then adding these into the Photoshop file and naming them for easy access. Moreover, it is inevitable that only a few of the alphas will need to be used on an image-by-image basis. However, at the point of creating the file, it is impossible to know which alphas will be deployed during overpainting or compositing so necessary to build the whole set.

The benefit of this approach is that the alphas are generated procedurally and therefore 100% accurate to the object based on the camera view. The primary gains are therefore a greater level of accuracy and also saving on the time that otherwise would be spent manually painting these alphas.

Light and Reflections

One of the key objectives for this project was to create a complex system of light, reflection and shadow that varied through the camera move and thereby creating inconsistencies in the tonal ranges of the projection layers. This was primarily to establish whether a hierarchical layered projection setup could handle these variances. A second positive consequence of this was that the scene would have lighting that was physically accurate and more natural whilst reducing the amount of manual overpainting.

This began with the primary 3D geometry with base textures applied.

A sky dome was added to the scene, which is an infinite sphere capable of receiving a shader or material. I applied a high definition sky texture to a shader and rotated the sphere until the dominant light source was coming from the upper right direction relative to the shot camera.

Whilst this gives a nice directional light that bounces off the reflective surfaces such as the glass and metallic objects (below), this does not provide reflections that are truly indicative of the environment. We would expect other scene elements, such as adjacent buildings, to be seen in the reflections.

To achieve this, it is necessary to create a high dynamic range file that gives a 360° representation of the scene.

For this I added a sphere into the centre of the scene and applied a highly reflective texture.

This was then framed to fill the viewport and rendered in HDR format, which is a 32-bit high dynamic range image.

This is taken into Photoshop and the sphere is isolated from the peripheries. Then the sphere is unwrapped to create the LatLong image

This image is then used to replace the original sky texture assigned to the sky dome. The result is that the light, reflections and shadow is a true 360° representation of the scene and leaves only one final step, which is to rotate the sky dome object until the light direction and reflections align and then make the skydome invisible to the camera so that only its influence is seen.

To boost the illumination, an Area light is added to the scene and aligned to match the dominant light source from the HDRI, which is from the upper right side of screen from the perspective of the shot camera.

Overpainting Treatments

An example of overpainting on image set D. A sourced grime texture is positioned over the stone frontage, scaled and rotated and distorted to align. The texture is then blended over, using a multiply operation and opacity reduction. Finally, a layer mask is added to the texture and transparency painted in to blend the transition.

A green copper patina texture is laid over the pipe using exactly the same approach. The pipes alpha is copied from the channels panel and added to the texture as a layer mask. This is then modified to create non-uniform areas of coverage. This method was applied to a number of pipes elements across the scene.

I used a common approach to weathering to the roof tiles on several image sets, seen below on the rightmost building of image set E. This involved placing a stain texture over the roof area and transforming to match the perspective. Then a multiply blend operation to retain just the dark detail followed by the roof alpha, drawn from the channels panel and added as a layer mask.

Additional colour corrections were then applied, again using the roof alpha to holdout any overlapping stain texture.

The image (below) shows some more complex overpainting techniques. Note that a copper patina has been added over the pipes and restricted to specific areas with a layer mask. Note also how a Curves adjustment is added and clipped to the patina texture so it only influences that part of the image.

This principle is also applied to individual AOV passes throughout the matte painting. Note how a copy of the reflection pass has been brought into the overpainting and has two colour corrections clipped to it to brighten and desaturate the pipes and this has been constrained to just the leftmost building using a layer mask with the alpha sourced from the channels panel.

The image (below) shows the application of a sign to the leftmost building on image set A. The image was sourced, placed over the pipe backdrop, transformed and warped to follow the curvature. A colour correction is then clipped to the element and used to bring the sign into the correct tonal range.

For the Graffiti element, an image was sourced, layered over and transformed. Then a pure black layer mask was added to completely occlude the graffiti, which was then painted to reveal the required section of the element. Finally a Multiply blend mode with small opacity reduction was used to complete the integration.

A number of adjustments were applied to the lamps on image set C. The image (below) shows the application colour corrections to both the body of the lamps and the illumination.

The comparative image pair (below) shows the before and after effect. Note how the body of the lamps have been darkened and given more contrast and the glass element has been lifted to imply they are lit.

A similar effect was applied to the wall mounted lamps on the rightmost building of image set D.

A flat white layer is placed over the entire image. A pure black layer mask is added to fully hold out the white and then simple white brush used to open the mask to reveal small areas of white where the bulbs would be.

The glows are applied onto a new transparent layer by painting areas of white with a larger soft brush and then reducing the opacity of the layer until the glow influence on the wall looks correct. In this case, a value of 50% was used as the glow influence would be minimal during daytime.

As this matte painting depicts a daytime scene, it is unlikely thar external lights would be turned on but I wanted to test the plausibility of this type of illumination when applied as part of a hierarchical layered projection.

A steampunk bicycle was incorporated into the scene to provide an additional, albeit subtle, object of interest. This was sourced on 30^th May 2022 from 3D Trader https://www.cgtrader.com/3d-models/vehicle/bicycle/bicycle-steampunk. The model is titled ‘bicycle steampunk’ and created by aleks.pogonin.

To generate a shadow render, to capture both the contact shadow with the ground and shadows on the wall, the model was placed against the building object and shadow catcher material added to both the building and ground objects.

This shadow pass was generated from frame 140 of the shot camera motion path and added to the composite on the composite building on image set D, also at frame 140.

The texture was projected onto the bike using two cameras, one was the default projection camera from frame 115 and the other was a new camera that was broken from the motion path and placed to provide an orthographic ‘profile’ view in order to prevent smearing on the side perspective.

For compositing purposes alphas were generated from 3D for the wheels, frame, bars and saddle and used for colour correction.

The image pair (below) shows application on the orthographic projection texture. Note how colour corrections are used to darken the default tyres and desaturate the copper texture on the frame. Also note how the alphas are drawn from the channels panel and applied as layer masks so as to hold out the corrections to the desired parts of the image. 

The ground object comprises of pavement, curbs, street, manhole covers and drainage grates. The image(below) shows all these elements from the perspective of the render taken from the third projection camera.

The AOV’s follow the standard format in terms of passes, blend operations and nomenclature.

Each element was rendered with a subdivided alpha, which are stored in the Channels panel of the Photoshop file

These are then used as layer masks to hold out any colour corrections or other post-processing operations

There are many more overpainting operations applied across the image sets, but these tend to use the same techniques so do not need further discussion.

Gyerman, T. Build a Fantasy World; An Expert Guide. 3D World. Issue 285, 2022, pp.48-53