At the outset of this project, it seems appropriate to offer some kind of indication as to what can be assumed as ‘common knowledge’ within the domain of camera projected digital environments.
To do this, I have performed a search and basic analysis of relevant learning materials, within the subject area, and that are widely available within the public domain.
Digital Tutors (2013) Creating a 3D Scene with a 2D Image in NUKEX
A video tutorial series presented by instructor Laura Hawk. The image (below) shows the matte paining used throughout the tutorials
This learning series demonstrates the multiplaning (or 2.5D compositing) technique. In this case the software used is Nuke but there are many similar resources, which show application of the technique using other compositing software (i.e. Fusion, Shake, After Effects etc.)
This technique involved sections of an image to be separated, based on their distance to camera, and applied directly to the UV’s of a flat geometric surface rather than projected. See below.
In this tutorial, the layers are separated using Roto techniques directly inside the compositing software (see image below), although it is more common, and more practical, for the layers (and alpha channels) to be separated using image manipulation software (i.e. Photoshop) and imported to the compositing software as a layered file.
The advantage of the 2.5D compositing technique is speed, given that is provides plausible parallax, especially on distant elements, with no significant work required to prepare the geometry for projection.
However the technique has clear limitations but these are not discussed by the instructor, even though there are obvious visual flaws in the finished sequence.
On this render of the finished sequence, we can see the hard edges between layers. However, what is more significant is that parallax is limited to whole layers and their distance in z space to the shot camera. The limitation is apparent on the pier as the upright parts of the structure should parallax in relation to each other parts but, because the whole pier is projected onto a single flat surface, all these elements move in unison.
D.B Mattingly (2011) Digital Matte Painting
Book released by Wiley Publishing Inc. which provides the broad topic of matte painting but does include a chapter on camera projection using Maya as the primary tool.
Here we can see the use of simple low-resolution geometry modelled over the matte painting
This shows the application of a projection camera and the use of a Shader to project the matte painting onto the geometry.
Lynda.com (2014) Digital Matte Painting Essentials
A tutorial series by David Mattingly providing all the tutorial components from his book in screencast format. This includes the chapter on camera projection described above but does not expand on any of the content provided in this chapter of the book.
Both the book and the audio/visual tutorials show the use of simple geometry as the basis for camera projection, which addresses the fundamental principle of parallax, which can only be partially exploited using the Multiplaning (2.5D compositing) technique. However the scenario presented in the tutorial involves an extremely gentle (forward push) animation of the shot camera which does not reveal any doubling or smearing of textures that would occur if the camera move were to reveal areas of geometry that are not represented by texture in the matte painting and therefore not visible to the projection camera. Neither the book nor the video tutorials address this issue, where and why it might occur and how to address it. I would therefore conclude by arguing that the tutorial series provides no more than a primer for the camera projection technique
The Gnomon Workshop (2012) 3D Matte Painting and Camera Mapping.
A video tutorial series presented by renowned Matte Painter Chris Stoski. This shows one of the concept art pieces used in the tutorial.
The content is similar to that provided by Mattingly. This shows geometry modelled over the main building.
The instructor does show use of overscanning by changing the focal length of the projection camera to increase the range of projection. However there is no discussion on the limitations and implications of using this technique.
The Gnomon Workshop (2011) Camera Projection Techniques in Maya
A video tutorial series presented by instructor Darin Hilton. This shows the concept art which represents the matte paining used throughout the tutorials
Again the content covers much of the ground as presented by Mattingly.
However, what is significant about this resource is that the instructor (Darin Hilton) applies a more adventurous animation to the shot camera with a translation along the x and z axis and a rotation. Consequently this reveals a camera projection artefact, called texture smearing, seen in the image (below) and was not found in any other resource during my search.
The tutor demonstrates a more advanced technique to deal with this. He uses a layered Shader and second projection camera to patch the offending area of the matte painting.
Whilst this does not, by any means, cover the extent of challenges emerging from camera projected matte painting, it represents to most advanced learning resource found during the search.
Digital Tutors (2014) Animating a Mountain Scene Matte Painting in Photoshop and NUKE
A video tutorial presented by Ed Lopez. The mountain scene (below) is provided as a layered Photoshop file and acts as the plate for the tutorial set.
The content offers nothing beyond that presented by Mattingly. However it does shows how extremely simple geometry, in specific circumstances, can yield sufficient parallax to sell the shot. See below.
FXPHD (2016) The Castle Project
Again this series of video tutorials shows the basics of camera projection but without revealing limitations in the technique. The castle scene (below) is provided as a layered Photoshop file and acts as the plate for the tutorial set.
This shows the creation of simple geometry for camera projection.
What we also see in this tutorial is additional geometry to extend the footprint of the original castle. We also see work on the UV’s to retexture the model (see below).
This allows the artist to use lights in 3D to match the lighting and shadows to the scene. See below.
Conclusion
In concluding my investigations of learning materials on camera projection, available within the public domain.
I have found a range of materials showing approaches to the multiplaning (2.5D Compositing) technique using various VFX software. In all cases the scenario in the tutorial was created in such a way as to show the plausible parallax effects but without exposing the obvious limitation in the technique. Some authors hinted at the limitations of the technique but did not show this or suggest how any problems may be overcome.
I have found a wide range of authors that provide an introduction to the concept and application, albeit at a basic level. However, in all instances, animation of the shot camera was limited to a gentle translation along the z axis and therefore unlikely to expose the doubling and smearing of textures which would occur if the movement of the shot camera was to reveal areas of geometry not initially visible in the matte painting.
Peter Holmes PhD Journal 