Ever since the first recorded example, produced by Norman Dawn for California Missions, released in 1907, matte paintings have been an intrinsic and essential part of filmmaking. In the early days, matte paintings were produced by placing a sheet of glass between the camera and the scene to be filmed. The artist would then paint an image on the glass that blocked and replaced that part of the scene (Mattingly, 2011, p.16. Rickitt, 2010, p.244). This self-portrait sketch by Norman Dawn (Figure 1) showing his tented workstation with Debrie camera and the glass shot used to restore the old dilapidated ‘Mission La Soledad’ in Monterey, California.
The limitations of this approach are perhaps obvious, from the simple logistical challenges associated with large and heavy panes of glass on a film set, and the practicalities of the artist painting in real time, which required them to work extremely quickly in order to reduce the time in which the artwork was exposed to the elements and, given that the that principal photography could only commence once the painting was complete, minimise delays in the production schedule. In 1911 Dawn pioneered an alternative method of combining live action and painted elements known as ‘original negative matte painting’. This involved exposing only the live-action area of the frame while on location, leaving a portion of the image unexposed for the painted elements to be added later. (Rickitt, 2007, p.246) Similar to the basic glass shot, a sheet of glass is placed in front of the camera on location but, rather than paint replacement scenery directly on to the glass, the appropriate areas of the glass are painted black, a process that could be accomplished much more quickly. This blacked out area, which became known as the matte, remained unexposed thereby allowing the matte painting to be created later and added during the photo-chemical process. Figure 2 shows this method used in the construction of the iconic ‘Xanadu’ from Citizen Kane (1941) with Mario Larringa’s matte painting of the edifice under construction. (Cotta-Vaz & Barron, 2004, p.16)
This principle was further adapted so the ‘blocking’ of sections of the film was achieved by attaching pieces of black card to the front of the camera lens and later, through innovations in photo-chemical processing, painted directly onto the film stock. Both methods removed the requirement for painting onto glass during principal photography. From these early days of film, through to the current day, Dawn’s fundamental ‘matting’ principle has remained unchanged despite the evolving tools and techniques used in the process, notably the emergence of the Optical Printer, a mechanical device that allowed multiple reels of film to be combined and re-projected. Then the emergence of digital technologies in the late 1980’s, which provided much quicker and more flexible approaches to matte creation and the compositing of many more layers of artwork than could ever have been achieved by the Optical Printer. Rickitt (2007), Mattingley (2011) and Cotta-Vaz & Barron (2004) all provide excellent chronological summaries of when the various tools and technologies emerged, and how they shaped the workflows and techniques deployed by matte painters across the generations. Consequently, there is no need to elaborate further other than to re-emphasise that the principles founded by the early pioneers in film are as relevant today as they were at the beginning of the film era.
What is of direct relevance to this study is that, as cameras became smaller and more manoeuvrable, directors increasingly looked to incorporate camera movement into their shots. This presented problems on shots featuring matte paintings because, if the camera moved, then the area targeted by the matte would change relative position and/or shape over the duration of the shot. Innovations in photo-chemical processes, most notably the Williams process (also called the black-backing travelling matte process) patented in 1918 by one-time Keystone Kops cameraman Frank Williams provided a way, albeit far from perfect, in which a matte could move around screen, or change position relative to movement of the camera. This was superseded later by processes that are now ubiquitously known as Chromakey (greenscreen) and Rotoscoping, where the matte is painstakingly redrawn on every frame. The second problem was that the actual artwork in the matte painting could not return any parallax shifts or changes in perspective as these would occur naturally over the course of the shot in response to the camera movement. In simple terms, the painting is a flat two-dimensional representation of an environment (or part of) and therefore incapable of behaving as if it was three-dimensional and/or existing as part of a wider 3D space.
Matte Painting in the Digital Era
Garrett Fry is an experienced matte painter and digital environment artist with IMDB credits on titles such as The Curious Case of Benjamin Button (2008), Rise of the Planet of the Apes (2011), The Hunger Games (2012) and The Great Gatsby (2013). In an audio-visual webinar entitled “How to break into VFX”, published in 2017, he described a shift in the expectations of film directors over recent years in terms of how to deliver environments to screen:
Up to just a few years ago, matte paintings were delivered, either from a static camera or gentle dolly-in or dolly-out movements. However, in recent years directors want more dynamic representations involving increasingly expansive camera angles and movement.
Indeed, since the emergence of digital technologies in filmmaking, questions have been asked about the on-going viability of the matte painting workflow and technique in its traditional sense. In an article for the SIGGRAPH Conference in July 1998, Craig Barron argued that “…many are beginning to question whether this technique is still a viable option in the digital age. Like many other filmmaking techniques, traditional matte painting is being completely redefined by computer-generated imagery. Traditional matte painting is quickly becoming a lost art, eclipsed by digital backlots and virtual sets.”
The abiding point would appear to be that, as the capability of the technology reaches the point where we are able to create fully realistic CGI environments entirely from inside 3D software, then would this invariably lead to the demise of matte painting in the same way that the emergence of digital tools led to the demise of the optical printer? At the time of Barron’s article, 3D software was incapable of delivering totally realistic images in all cases, even with the most talented operators, but in 2021, the software has reached a level sophistication whereby, in the hands of good modellers, texture and lighting artists, such outcomes are entirely achievable.
The question would therefore be why does matte painting continue to be as relevant and popular as ever in the creation of digital environments for film and television? To some extent this is answered by Dylan Cole, one of the prominent contemporary matte painters with IMDB credits on titles such as Avatar (2009), Maleficent (2014), Tron Legacy (2010), Golden Compass (2007) and the Lord of the Rings Trilogy (2001 – 2003). In an interview to the 2D Artist Magazine in 2012, he describes three basic kinds of matte painting:
There is the straight 2D painting that is composited with live action. This is the simplest kind because it just involves a 2D painting created in Photoshop. The next kind is camera projection or 2.5D. This consists of projecting a 2D painting onto 3D geometry that approximates the features in the painting. You are then able to do a slight camera move. If you reveal areas of the area that are stretching, you then just apply a patch with an alpha channel. You can do quite large camera moves in this fashion and still retain the control and ease of working in 2D. The third is full 3D which I am not even sure you can call matte painting. I guess where it is a matte painter doing the full 3D with modelling, texturing and lighting. (Tilbury, 2012, p.9)
The key words that emerge are ‘control’ and ‘ease’. Cole acknowledges that there is a point where even a 2.5D approach will not yield the desired results and would trigger a full 3D solution but, for most shots, he implies that this is not the case and therein lies the benefit. A full 3D approach is slow and expensive in terms of human resources and would represent overkill for a one-off shot. This would nevertheless still be a logical choice if the environment was going to feature in many shots, from different angles, perspectives, and fields of view, or indeed subjected to expansive camera movements. However, a matte painter can produce a totally realistic environment much more quickly and less expensively to the production so, where the shot is to be seen just once or twice, or from primarily he same angle or field of view, and with relatively simple camera movement, a matte painting would remain as the logical choice.
The Technical Problem
The primary technical issue is perhaps obvious. A matte painting, as a piece of artwork, is a flat two-dimensional representation of an environment. This is a perfectly acceptable format for presenting to an audience as a static shot because normal depth cues such as scale, position, occlusion etc. can be incorporated into the painting. The problem arises when the matte painting is required to be seen from a moving camera. By this I mean a camera that, over the duration of the shot, changes position or rotates on any (or all) of the horizontal (X), vertical (Y) or depth (Z) axes. Limitations are exposed in the artwork because all elements become three-dimensional. Flat artwork cannot present how the perspectives of objects in the scene distort relative to changes in the position and angle of the camera over the duration of the shot and how parallax causes objects, set at different positions in depth within the scene, to move further apart, closer together, or even overlap each other.
The earliest incarnations of what has come to be termed the 2.5D approach, described previously by Cole, was born in animation, initially to create the effect of traveling towards or across the landscape. By simply using a telephoto lens to zoom into the image, “…it was possible to provide the illusion of moving forwards. Rocks, trees, and bushes will appear to grow larger, but so will mountains, clouds, the sun, and everything else on the two-dimensional painting. The illusion of travelling forward in the landscape could not be accomplished realistically using this method.” Disney 1957, p.10) In their study of motion parallax as a determinant of perceived depth, Gibson et al describe ‘apparent motions’ of stationary objects which arise during locomotion. (1959, p.40)
The issue was, and remains, that zooming has a magnifying effect on the whole image and is therefore unable to access true depth information. The limitations of the artwork, as essentially a two-dimensional representation of an environment, are exposed as incapable, at least in its default state, of presenting as three-dimensional by delivering parallax over the duration of the shot. To overcome this limitation, the ‘Multiplane Camera’ was developed by the Disney Corporation during the 1930’s (Figure 3). This was, trialled on the movie short ‘The Old Mill’ and popularised in 1937 during the creation of ‘Snow White and the Seven Dwarves’.
The term is misleading in the sense that it involves a ‘process’ rather than use of a single piece of bespoke equipment such as a camera. It works by splitting a field of view into different planes or levels. The artwork is separated (i.e. foreground, midground, background) and placed on different planes relative to the camera. If the camera is then moved (dollied) closer to, or further away, from the composite artwork, the layers will translate across the camera’s field of view (Rogers, 1979, p.125) and therefore change position relative to each other. The same principle can be used to achieve a plausible forwards or backwards-motion shot, for example the view of a landscape from the perspective of a moving car or train. The setup of the artwork is identical, but the camera is tracked in a left-to-right (or vice versa) direction. Objects (or layers of artwork) that are closer to the camera appear to move more quickly than those further away such as distant hills or the sky. This approach was used by matte painter Syd Dutton in Batman and Robin (1997) for an establishing shot that featured a push-in camera move, and also in the release of Anna and the King (1999) in which created a horizontal tracking shot depicting a view of 19th century Bangkok. (Cotta-Vas & Barron 2004, p.248-249)
By placing artwork at different distances relative to the shot camera, the Multiplane approach is able to deliver plausible parallax shifts on both dollying and tracking shots. However, the obvious limitation remained that each layer was a flat two-dimensional representation of part of the scene and could not therefore respond to shifts in perspective that would change the shape of the element or perhaps reveal or hide parts of an object that would be visible/hidden from some angles but not from others.
Digital tools allow the principles of the Multiplane camera effect, to be applied with greater speed, flexibility, and efficiency. In its simplest of configurations, sections of a digital matte painting are applied onto planar geometric surfaces, each set at different positions in depth relative to a moving shot camera. As this method involves applying a two-dimensional artwork on to a flat surface, the limitations around perspective distortion still exist. However, applying the artwork to more complex geometric shapes such as cubes, cylinders and spheres, or indeed geometry that is modelled to follow the contours of the artwork, such as the undulations in a terrain, would appear to address some of the problems.
The issue of parallax was never adequately resolved until the late 1990’s when the larger VFX studios, primarily Digital Domain and Industrial Light and Magic (ILM) began using digital three-dimensional software such as Soft-Image, 3DS Max and Maya to build representations of sets and applying sections of matte paintings to the surfaces using texture mapping. Figure 4 shows a digitally created façade, in wireframe view, used in The Truman Show (1998) and how this extended the existing film set when the matte painted area was mapped to the surface of the geometry.
This era of innovation coincided with the emergence of new software that was able to analyse a piece of footage, interpret and recreate a digital version of the camera, including any movement and optical changes undertaken during filming. This data could be applied to the 3D environment and therefore ensuring that, once the sequence was rendered from 3D, the movement of the CG sets would align perfectly with the live footage when composited over at a later stage in the post-production workflow. (Seymour, 2004)
Whilst Cole’s asserts that “…you can do quite large camera moves” using 2.5D, there is a clear implication that there is a point at which this approach must be relinquished in favour of a full 3D workflow. What is less clear to establish is the threshold at which the 2.5D approach becomes impractical and a full 3D solution is required.
Peter Holmes PhD Journal 