Mark Ruff deploys unique Avatar Factory technology on Mortal Kombat (2021) and other dynamic projects – by Tanya Thomson
Specialising in the creation of three-dimensional avatars and cyber-scanning for the feature film and broadcast industry, The Avatar Factory uses over one-hundred and forty high-resolution 24-megapixel cameras, alongside an additional six 50-megapixel DSLR cameras. It uses ‘photogrammetry’ to capture photo-realistic three-dimensional models in an instant.
It’s a one-of-a-kind mobile rig developed and owned by Australian and long-time ACS member, Mark Ruff, who in 2002 received the ACS Ross Wood Award for Innovation and Advancement of Cinematography and also the ACS Miller Award for Technical Achievement in 20??.
Capture is instant and takes just twenty seconds to download a take. A check of the images takes just a few minutes more, and talent is consumed by just a few minutes of their time.
Ruff has been working with multiple cameras since 1999. His first build in 1999 utilised sixty analogue cameras to explore ‘bullet time’ effects. “It was a proud moment,” says Ruff, who displayed his system for the first time to the ACS at Lemac in 2001. “All I could do was fire the shutter,” Ruff goes on to say. “As impressive as this sound was, with a 10-millisecond delay between cameras, the results took about a week.”
Then came version two. Digital cameras. Turnaround was just one-hour compared with the one-week from Ruff’s first attempt. And his version three build, using second-generation digital technology, just fifteen-seconds versus one-hour. Ruff was looking ahead at the business model of three-dimensional avatars, body scanning and digital doubles and he believed there was a future in this style of image making. “And there was,” he says.
When Ruff was commissioned to work on Alex Proyas’ Gods of Egypt in 2015, the visual effects department inquired if he could provide digital doubles. “Give me a couple of weeks,” he replied. But that ‘couple of weeks’ turned out to be nine months of research and development. “Back then there were few resources, so, it was like reinventing the wheel.”
How is it done? Multiple cameras capture a subject, then specially designed software detects common features across all the images. Each feature becomes a ‘vertex point’ and multiple vertex points make a triangle. A final capture with around one-hundred-and-fifty cameras creates a three-dimensional mesh ‘mesh’ as high as 100 million triangles. The photographic component is then laid over the mesh. The result is a photo-realistic, three-dimensional model. In the world of 3D, you are no longer working with a ‘burnt in’ two-dimensional image. One of Ruff’s avatars can then be re-lit and animated.
Last year, Ruff was able to achieve very flat lighting for Mortal Kombat (cinematography by Germain McMicking ACS) as well as another big-budget film recently shot on the Gold Coast. For Mortal Kombat, a 6x6x3-meter white cube, where light was bounced off all the walls, was incorporated with pleasant results. “Flatter than flat,” says Ruff. Enter ‘cross polarisation’, technique used by still photographers to eliminate specular highlights off the brush strokes of oil paintings on canvas.
Most cinematographers will understand the concept of ‘polarisation’ whereby a filter is used to reduce the phases of light. This helps in darkening blue skies, reducing reflections off windscreens of cars or simply making driving more pleasurable with polarised glasses. ‘Cross polarisation’ goes one step further. The light source is polarised, and the subject is polarised.
The result is that only the ‘diffuse’ or reflected light from the subject enters the camera. Ideally, the incidental light that strikes the subject is invisible to the camera. Important when you are covering a subject over 360-degrees.
“It looks a bit odd since the light illuminating the subject from behind is near black,” says Ruff. “It is used in photogrammetry to eliminate the specular highlights of skin at the pore level and even out the reflective properties of fabrics. Since the incident light is mostly invisible, there is minimal flare.”
“When a three-dimensional person or object is photographed, you do not want anything such as light to get in the way,” explains Ruff. “I know, it sounds absurd, light, getting in the way of image making. But a correct texture holds the information of the subject, and it appears black until you put light on it. This is just like the real world; a subject with no light appears black. If the texture is contaminated by light that originally lit the subject, then the texture is compromised. A highlight or reflection may exist where you do not want a highlight.”
Marvel films, and the like, use a three-dimensional avatar to preform seemingly unreal stunts. The avatar must match the original as close as possible since we may see both in the same or consecutive shot.“Three-dimensional avatars are not ‘born’ to replace actors, rather to extend a performance,” says Ruff.
Whilst working on Mortal Kombat, Ruff had to relocate the set up to be closer to production. At that time there were eighty-four cameras. “It was an effort to rig and de-rig, involving a half day to strike and a further day to rig, but, we did it,” says Ruff. When it came to filming on the Gold Coast there were more than one-hundred-and-forty cameras.
Even with a third assistant it was just not feasible to set up such a complex environment for one day. It was then that Ruff knew his only solution was to have a mobile set up. Enter a single-axle fifteen-tonne truck almost eleven meters long. Inside the truck is divided in half, with computer workstations and a booth with all the cameras setup to capture a subject.
“The total time imposed to talent is less than five minutes, which includes instruction on how to pose, taking the exposure and a check of the images,” says Ruff. On a recent feature film, the team took just 75 seconds from instruction to check and clear. “We were able to achieve up to 120 scans in around 180 minutes in a single session. Exhausting!”
Ruff’s Avatar Factory now extends to his entire family with Kate Ruff responsible for data logging and wrangling, Chloe Ruff greeting and instructing talent and Amy Ruff slates and documents talent. By the time of publication, Ruff hopes to have version three underway which includes pattern projection.
Mark Ruff is a proud member of the ACS.
Tanya Thompson is a contributor to Australian Cinematographer Magazine.