GROUP PROJECT

The Assignment

As Lead Compositor for the project it will be my responsibility to manage this side of the pipeline as well as foresee any issues that could arise.

My plan to tackle this project is to break each area down into its core elements and work on it bit by bit day by day while watching and absorbing as much information about composting as I can.

Key things on my mind at this stage are:

-Cards or Projection Mapping

-Adding Motion to the Environment

-Tying the foreground, midground and background together using assets

-How to eliminate Keying issues using costume and planning

-Expanding my knowledge and skillset

My Goals

I spent the summer learning and absorbing information about compositing that I intend to show and push to its extremes. In particular I want to expand on my use of AOVs rendered out of a software like Maya

My Goals for this project are:

- Showcase AOV Compositing Skills

- Utilise Intermediate workflows such as using an objects Normal to relight and adjust

- Further develop colour correction skills to seamlessly integrate a greenscreen plate

- Look into using expressions and custom scripts

Inspiration

Titanfall 2:

The Imposing design of Titanfall 2's Titans and key art for the game helped aid both the concept of the shot and the design of the robot for me.

Fallout:

The Fallout TV show used matte paintings projections and models to bring the wasteland to life I intend to follow their compositing strategy to help make my wasteland feel real and lived in

Compositing Academy:
A huge influence over the summer for me I followed lot of his tutorials to help me grasp A the basic skills such as clean plating and B further skills such as ST maps and utilising render layers at an intermediate level

Hugo Guerra:

Hugo's videos on Deep Compositing inspired me to use and learn how to cleanly deep composite effects into my scene, I am planning on using Embergen to create these effects too.

Bladerunner 2049:

Similar to fallout I was looking at how Framestore made their Trash Mesa and found similar strategies of photoscanning, cards and models.

Star Wars: The Force Awakens:

Taking inspiration from the Tie Fighter sunrise shot I aim to make an accurate sky though the details on how ILM made it remain scares I plan to photobash the sky together

Titanfall 2 Key Art

Fallout TV Shot Screenshot

Star Wars the Force Awakens Screenshot

Bladerunner 2049 Screenshot

Pre Production

Testing Ideas

Pur Ref File containing all the images used for scrap mouintian

I first wanted to explore the methods of Matte Painting, I decided to test creating the background mountains using either cards or projection mapping.

To create the scrap mountain I collected images of scrap piles from the internet and used Nukes roto node and luma keyer to remove the sky from each image, I then merged the images down a pipeline to create the shape of the mountain

Video by me showing Projection Mapping tests

I then used grade nodes to colour correct each image to match each other and finally made sure that the image was the correct resolution in a reformat node, I then tested both Projection mapping the image onto some geometry and creating a card with the mountain on and looked at the two shots

Both videos are shot to show the extreme issues with both methods the camera in the final shot will not show such harsh angles, The main issue with projection mapping in this test the shape doesn't hold I loose a lot of the sharp edges of scrap as the camera flies upwards, with the card the shape holds however looks really 2D and flat due to the camera movement, I will need to do further testes once we get some proper footage tracked and ready to go

Lastly I experimented with animating AOV layers, here I animated the light at the top of the pylon to gently pulse on and off using a merge node and the APGlow node from the Nuke Survival Kit

Video by me showing card testing

Video showing animating emission AOVs from maya

Further Testing: Nuke and Live Plates

Using test footage shot I did a rough key and a basic grade to match the rendered Pre-Vis plate to the test footage, things I noticed from doing this was that Ryan was peaking over the top of the greenscreen and that where they are looking is not where the robot it for our next test shoot I advised that we make sure that the actors are inside the greenscreen and that we make sure the actors are framed up properly in camera so they look at the robot in the final comp

Video showing test keying and tracking with live plate

Here we used a dummy stand in for the test, we were looking to recreate a sunrise outside, we filmed in the middle of the day to get this look i did a day to night grade dialled it back then graded the plate to pull out the oranges from the light we set up I then added light to the areas that we missed in the shoot and keyframed a mask to flow up the body in time with the sunrise. to help this test I also composited in a background element and did a similar method to create a fake sunrise

Video showing test sunrise with nuke effects helping

Working with ACES

How does ACES work?

"ACES was established by the Academy of Motion Picture Arts and Sciences to create consistent, color-accurate, and high-quality motion picture images."- (2024) Academy of Motion Picture Arts

ACES widens the range of colours in an image making sure that the highlights don't peak and that shadows aren't blacked out meaning scenes with bright lights or dark shadows still retain details and data reducing clipping thought the shot

Benefits of using ACES

-Wider dynamic range makes CG renders feel more photorealistic

-Reduced data loss from clipping

-Colourspace works thought pipeline

- No real downsides

Managing Assets

The academy has information on working in the ACES colour space in Maya and Nuke it is important to make sure everything is textured and rendered in ACES to reduce friction in the final comp

For images that are imported with ought the ACES colour space I will need to edit their input transform to match the rest of the pipeline

Project Settings for ACES workflow

EXR Input Transform Settings

EXR Write out Settings

Final MXF Comp

The final comp will be done in DaVinci Resolve where I will apply the LUTs our colour grader will send over and do the final export with shot 1 and 2

On the other hand I was also looking into exporting the MXFs out of Nuke and using Premier Pro for the final edit, I am more confidant in my ability to transform the aces colour space into SRGB for viewing and render it from Nuke as well I have more experience with Premier Pro however I am aware that I need to be able to produce an image sequence that an editor could drag into a software an use therefore I am planning on using Davinci

MXF Final Export Settings

Planning The Environment Structure

The Original Deisgn

When i was put in charge of designing the background environment I drew inspiration from Cyberpunk 2077 and Bladerunner 2049 this concept shows scrap hills and mountains pushing far up into the right side of the scene, I knew that the robot would be on the left this would give the robot a nice shilouette

Then we lined the shot up in Maya to see how it lines up cinematically, we wanted the right side of the frame to be tall and dark as the robot arrives on the left this will give a focal point as well as making the characters seem trapped

Original Concept for Enviroment

Pre Vis Concept (85mm Lense)

Developing the Topography

Next up I drew out the Topography of the landscape along with numbering key areas and features, the darker the shading the higher the land, the powerlines are meant to point towards where the robot will be appearing using leading lines as soon as we get the live plate my first task will be to track the scene using 3DE and fSpy and make rough geo to send out to the rest of my group.

Points of Interest:

1. Foreground the scene will take place here

2. 1st Pylon

3. The sulfur lakes - these will be puffing out a soft smoke

4. Crates in the distance

5. Crate house - Not too sure if this will be seen at all yet

6. 2nd Pylon

7. Scrap Mountian

8. 3rd Pylon

Topography of the enviroment

Production

Live Shoot #1

After shooting the live plate I took some time to review each piece of footage we filmed in high detail I found that shot 1 had the most issues:

- Shot is too grainy
- Shot has Horizontal Banding

- Shot needed to be moved a lot to match up with frame
Will 100% need a re shoot, given its the most important shot

Rough Key for Shot 1

Shot 2 Issues:

- Not Stable Enough

Shot 3 Issues:

- Actors not all behind greenscreen

- Lense dirty

Overall this was the shot with the least amount of issues

Rough Key for Shot 2

Pre Meeting thoughts

My plan to pitch during our group meeting is:

- Start filming at 4
- Shot 1 don't film in the dark
- Use tripod tilt for shot 2
- Shot 3 make sure the lens is clean

This should alleviate a lot of the issues with these plates.

I also think the lack of leadership in the shoot made us waste time this lead to people becoming tired and sloppy mistakes.

Rough Key for Shot 3

Trouble Shooting

After a meeting with my group we decided to use the test footage we filmed, this footage is usable because there are no real direct shadows, to make this shot blend well with sh002 I will grade both to meet at a middle ground, this shot will be darker and sh002 will be brighter

To tackle shot two's stability issue firstly I used Davinci Resolve's Stabilise feature to smooth out the bulk of the movements. Unfortunately due to the camera's movement (that you can't see in the shot) the point cloud generated from every 3D tracking software was constantly wrong. To cheat this issue I stabilized the shot completely in Nuke, then pulled a key after a de-spill and edge blur I plugged the still footage into a card in 3D and used Nukes camera to re create the smooth movement of the original plate then finally I exported the camera to create the rough Geo in Maya

We have decided to not use sh003 as it would take too long to manually roto Ryan instead if we decide that the project needs a final shot we will make it completely CG

sh001 still

Nuke Script for fake camrea movement

After finalising all the shots I created rough foreground geo in Maya, This took a long time as I couldn't make a point cloud and I struggled to get fspy to work. I used Google Earth to get the measurements of how far the camera is from the action and made sure that everything was lined up properly, the people, floor, and horizon line with this done I uploaded the Maya project to the share point so the rest of my group could work with it

Mays veiwport for blocked out envroment

Post Production

Background Matte Painting

The back ground sky is designed to look like a sunrise, to keep busy while I wait for the live plate footage I've decided that I will make 3 skies one for blue hour, one for twilight and one for sunrise. Hopefully the wide range will mean I can have lost of flexibility to match the live plate.

Pur Ref Containing different Sunsets

This sunrise is based off twilight, so there are lots of purples and blues and oranges, this was made by using photoshop to combine a bunch of images together then scaled the image up to 8k for use in nuke

First Matte Paiting

This painting is based off a normal sunrise focusing on a pastel blue sky with dark orange on the horizon line I also included the sun in this sky as we would see it and finally I scaled the image to 8k

Second Matte Painting

The final matte painting uses deep blues to replicate Blue Hour and again I then upscaled the image to 8k resolution

Third Matte Painting

With all three paintings completed I am ready to see which ones fit the plate best then I can go back and make any further edits I feel to be necessary

PostVis

Combining all the matte painting and cards I made I produced the PostVis for our project, though it doesn't have the robot in it this is a good proof of concept for how I will composite the project together

Keying The Footage

Shot 1

Quite Interestingly due to the framing we wanted I had to deal with some interesting issues with keying, due to using the test footage the greenscreen wasn't completely flush against the frame leaving lots of folds and lines in it. When it came to keying this I had to push the gain of the key light quite high to pull a key without artifacts for the left side I didn't have to worry about it as Louis never goes near the more difficult area. I tackles the issue with the top of the greenscreen by manually rotoscoping Ryan's hat during the shoot we were aware that Ryan would be too tall for the screen hence why we made him ware a hat to make my life easier

Images Highlighing poor areas of the greenscreen

Ryan's Poncho required an extra level of work as the tassles were small, which led to a lot of green spill. To key them effectively, I decided to create a roto around each group and key the left and right sides individually to try and bring as many of them out as possible.

Rotos around Ryan's tassles

I used a luma key to get the soft edges of Louis hair then when I added it to the plus merge pipeline any holes in his face were filled in

Louis' hair key

sh001 live plate alpha

Shot 2

There weren't many issues with the keying of this shot, given the stabilization it reduced the amount of roto work I had to do Ryan's Tassles again needed their own separate key same as Louis' hair though with my experience from the first shot this was all done quickly and easily

sh002 live plate alpha

Developing The Background

Developing The Background, Shot 1

Screenshot of Mountains

When creating the landscape I used 15k images I created in Photoshop by editing together lots of images of junkyards to create this scene, I ran back and forth with my group to get their opinions and settled on this design

Screenshot of Initial comp

In a similar vein I created this background sky that I then graded down slightly to act as a background, I made sure to use lightwrap on the far off edges to create a soft bloom and bleed the light into the mountain

Creating the Depth Grade script

Screenshot of Initial comp with colour haze script

To get this result I made a depth-grade script following along with the images on the right, I used the scanline render to create hold-out mattes to create a depth pass. Creating this hub here allowed me add to it later when I bring in more assets.

After the scanline render I shuffled the depth layer into the alpha and had to really crunch the grade as the depth data was kicked out at 0.00001, to get the haze at the back instead of the front I used an Invert Node and a soft blur. I then had one last grade in the end that I used to control how hazy the scene would be pushing the alpha up and down.

To create the haze I used two constant nodes picked to be the sunlight at the brightest and the sunlight median marked by a radial node that I placed over the sun this helped create a more realistic haze effect then merged the colour into the scene with the screen operation masked by the depth channel we altered before.

Compositing the Pylons

Screenshot with pylons composited inside

Screenshot of pylon compositing script

The Pylons were rendered out of Maya with AOVs included to get this effect I graded up the specularity of all the pylons to help give them a level of visibility, I wanted to add some bloom to the highlights so I crunched the pass-down to remove all the middle highlights leaving me with the parts that I want then used the AP glow gizmo, that creates an effective glow quickly. I then lastly added holdouts to the depth pass scene to apply the Colour Haze correctly in line with the rest of the scene.

Adding elements

Screenshot of assets composited in

Too add separation between the layers of scrap I added 2D elements to help add to the comp, I used smoke effects on cards and graded them to be yellow, previously we discussed that there would be sulfur planes around the shot that churn out smoke. I also started looking at assets I could include to add life to the still-shot, I added a flock of birds in the distance and began working on a particle simulation to look like flies

Example smoke asset

Example birds

To create the flies I made a particle simulation where the particles would bounce around quickly inside of a sphere upon practice this created a passable fly effect, to make sure I had plenty of frames I rendered out 2000 frames

Example Flies

Flies Nuke Script

Adding Foreground Elements, Shot 1

I made all the assets in this section using Maya, Embergen, and Nukes Particle System

For the floor, I wanted to blend three renders from Maya together with the live plate to create a final look. To go about this I took three renders from Maya and merged them over top of each other using a noise map inside a 3D scene this gives me a nice blend of all three plates, then with the live plate, I made sure the areas were the actors walk are visible as well as bringing back the middle of the road to make the scene feel more run down. I then did a similar technique of using a noise texture to merge as the final addition I grabbed Z depth data from one of the plates to help dive the Z defocus node

Nuke Script with Veiwpart render (top left)

Up next I added the Maya scene as a deep EXR using a deep composing workflow will save me lots of time in the long run as I won't have to worry about importing the robot plate, masking, and layering correctly. this replicates industry standard practice and allows me to place 2D elements inside the scene and have the robot walk through and mask correctly

After merging in I used a Luma key to pull out highlights and lastly a soft light wrap to help blend with the background

Nuke Script with Veiwpart render (top right)

Lastly, I placed the background Embergen smoke into the shot and masked out Louis and Ryan. after merging with the live plate, I placed and defocused the final foreground elements. I then rendered a separate smoke pass in Embergen with a holdout matte where they were standing and placed that in front of the live plate to give the illusion that the smoke was warping around them

Its important to note that I will need to re-render the Embergen scene with the robot once I get the final files

Nuke Script with Veiwpart render (top left)

Progress Render of Shot 1

Adding Foreground Elements, Shot 2

Copying the workflow of shot 1 I rendered out 3 layers of floor, we don't see Louis and Ryan's feet in this shot so don't need to add any road in

Nuke Script with Veiwpart render (top left)

I then brought the foreground in graded it to match shot 1, as well as hue corrected the sunlight to match the pink environment.

Nuke Script with Veiwpart render (top right)

Copying the method I used for shot 1 I simulated and composited smoke to flow through the shot. The major change involved animated the colour of the smoke to change with the sunrise

Progress Render of Shot 1 and Sh002

Cleaning up and Adding Elements

Sh001 had edits made to the scene adding scrap in the middle of the road adding bloom to the highlights of the pylon and the car as well as grading the background mountains down to make them feel more far away. I also had to fix the "sparking" issue with the feet by working on the rotoscope on the feet as well as manually painting out some of the issue frames

Changes made to Sh002 involve adding god rays to the background, converting the scene to deep data, and grading the car down lightly. Across both shots, I animated the sky layer using a displacement map and the Idistort node to slowly move the clouds across the shot and warp them slightly.

Version 1 of the Project ready to be send off for feedback