Virtual Production and Environments in VFX

Unreal Engine recently published an article explaining how Studios were more and more inclined to use Virtual Production at the beginning of a project, during the shoot, for the previz or postviz, to shape and affirm directors visions and let them see their film before having to wait for the VFX to be delivered months later.
It’s not completely new, but it seems to really have taken a lot of popularity over the last few years, thanks to the incredible quality we can get, real time, through software like Unreal Engine or Unity and it’s clearly showing signs that it will become central in the next decade.
That’s understandable : being able to test shots straight away, by combining acting, real time motion capture, real time fully lit environment with reflection, spec, volumetric, animation, totally redefine the way directors are working today on blockbusters.

I think we can assume that our traditional VFX Pipeline starts to be a bit outdated…
Most of the time spent in Environment today consists in building multiple versions of the same assets, changing the layout dozen of time, tweaking the lighting and rendering the same environment over and over again. So yes, thanks to software like Clarisse or Houdini, it’s really quick to see a preview of your render, but the final one is still going to take a lot of time, which makes the turn around for addressing notes relatively long.
Imagining if we didn’t have to actually wait for any render to be done ?

First, I love the idea of the director shooting his film like they used to do 60 years ago, by shooting actors in an environment. We don’t care if the actor is in a mocap costume on a green screen, the director is actually shooting a troll in a Jungle, and he sees the troll, in the jungle.  The new ability for the director to visualise his scene will definitely help him to pick the best angle, define the best lighting for his shot, focus on the action, the acting and even if some changes need to happen later, they will be instantaneous (almost).
It’s like being back to the fundamentals of cinematography : telling a story.

Then, I think it will be much more interesting for Environment artists to work that way, as we can hope that most of the time will be spent at the beginning of the production, to create a massive real time setup which the director will use as a Set.
Instead of waiting for the plate to be shot, and the artist to build and fill the green screen, we will build everything before the actual shoot, and that should slightly minimise the amount of changes and notes coming after. The work will not be easier, but probably more time will be allocated to create a nice environment, and less will be used to tweak an existing one. I see in this technique a way to refocus on what we are supposed to do well.


Matte-painting is a different story. Traditionally, the matte painter creates something based on a locked plate and a locked lighting. He will pick the colours, the blacks levels, the amount of haze etc… and will do his magic depending on a relatively locked camera (in the idea, not the position). If any big change happens, then the whole work will need to be redone, which happens sometimes today already.
It means that the use of matte-painting in a real time production will have to be redefine. I am absolutely confident that matte-painting will still be used heavily in the future, but maybe we will have to think about a new pipeline, a new workflow, to be able to accommodate the flexibility of a real time pipeline.

Anyway, I’m really looking forward to seeing how the work will evolve in the next 5 years, especially in Environment and DMP, how much prior the shoot VFX Companies or Virtual Production companies will be involved and how schedule and bid days will move from Post Production to Pre Production.
That’s the fun part of our industry, we always have to adapt and redefine ourselves !

5 thoughts on “Virtual Production and Environments in VFX”

  1. Interesting article Ludo, thanks for this. In my opinion, concerning matte painting, we’ll have to consider environment will be probably full CG even when they were easily “simple” 2D before; we’ll probably have to paint less and do more environment work, and this is actually what’s happening already if you think about it. So having experience in shading and lighting will be extremely valuable. Just my 2 cents though.
    In any case, as you say, we have to constantly adapt and reinvent and, if you think about it, it’s luck.

    1. Yes, I make a distinction between Matte Painting and Environments here. With Matte Painting, I consider DMP, 2.5D, paintover, these sort of things.
      So yes, environment wise, realtime is interesting and promising where DMP wise, it will be more challenging to reinvent ourselves.

  2. hi ludo, is been 2 years since you post this article, and you are right.
    environment artist with the real time engines combine with a tradicional 3d softwares and megascans have a solid pipeline. but for DMP looks like they can still work in diferent productions. even if you can get in virtual producction with out a expencive led walls you can use a tracker and another accesories with tradicional green screen to do the job. but lets faced, there is a lot of production that will use tradicional pipelines for a few years more.
    what do you think now?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.