Virtual Production and Environments in VFX

Unreal Engine recently published an article explaining how Studios were more and more inclined to use Virtual Production at the beginning of a project, during the shoot, for the previz or postviz, to shape and affirm directors visions and let them see their film before having to wait for the VFX to be delivered months later.
It’s not completely new, but it seems to really have taken a lot of popularity over the last few years, thanks to the incredible quality we can get, real time, through software like Unreal Engine or Unity and it’s clearly showing signs that it will become central in the next decade.
That’s understandable : being able to test shots straight away, by combining acting, real time motion capture, real time fully lit environment with reflection, spec, volumetric, animation, totally redefine the way directors are working today on blockbusters.

I think we can assume that our traditional VFX Pipeline starts to be a bit outdated…
Most of the time spent in Environment today consists in building multiple versions of the same assets, changing the layout dozen of time, tweaking the lighting and rendering the same environment over and over again. So yes, thanks to software like Clarisse or Houdini, it’s really quick to see a preview of your render, but the final one is still going to take a lot of time, which makes the turn around for addressing notes relatively long.
Imagining if we didn’t have to actually wait for any render to be done ?

First, I love the idea of the director shooting his film like they used to do 60 years ago, by shooting actors in an environment. We don’t care if the actor is in a mocap costume on a green screen, the director is actually shooting a troll in a Jungle, and he sees the troll, in the jungle.  The new ability for the director to visualise his scene will definitely help him to pick the best angle, define the best lighting for his shot, focus on the action, the acting and even if some changes need to happen later, they will be instantaneous (almost).
It’s like being back to the fundamentals of cinematography : telling a story.

Then, I think it will be much more interesting for Environment artists to work that way, as we can hope that most of the time will be spent at the beginning of the production, to create a massive real time setup which the director will use as a Set.
Instead of waiting for the plate to be shot, and the artist to build and fill the green screen, we will build everything before the actual shoot, and that should slightly minimise the amount of changes and notes coming after. The work will not be easier, but probably more time will be allocated to create a nice environment, and less will be used to tweak an existing one. I see in this technique a way to refocus on what we are supposed to do well.


Matte-painting is a different story. Traditionally, the matte painter creates something based on a locked plate and a locked lighting. He will pick the colours, the blacks levels, the amount of haze etc… and will do his magic depending on a relatively locked camera (in the idea, not the position). If any big change happens, then the whole work will need to be redone, which happens sometimes today already.
It means that the use of matte-painting in a real time production will have to be redefine. I am absolutely confident that matte-painting will still be used heavily in the future, but maybe we will have to think about a new pipeline, a new workflow, to be able to accommodate the flexibility of a real time pipeline.

Anyway, I’m really looking forward to seeing how the work will evolve in the next 5 years, especially in Environment and DMP, how much prior the shoot VFX Companies or Virtual Production companies will be involved and how schedule and bid days will move from Post Production to Pre Production.
That’s the fun part of our industry, we always have to adapt and redefine ourselves !