The film and TV industry has a history of innovating quickly and delivering new consumer experiences. One of the latest shifts in the industry is toward the use of virtual production in the development of high-concept film and TV. Popularized by shows such as The Mandalorian and The Lion King, virtual production is slowly becoming the new standard for creating realistic visual effects and computer-generated live-action productions.
Virtual production is a new kind of on-set production that utilizes emerging technologies such as real-time rendering, virtual reality, and LED wall technology to create virtual sets in real-world spaces. This method allows actors, cinematographers, and other creatives to interact with virtual sets, combining the production and post-production processes into a single environment.
Virtual production is bringing new challenges and opportunities for 3D artists. 3D environments now have to flexibly build for on-set changes during production based on director feedback. Furthermore, rendering workflows and render farms are being rethought as real-time rendering takes over for on-set rendering. In this article, we'll be looking at how innovations in virtual production are set to revolutionize the film and tv industries and how the technology is changing how visual effects artists build and render 3D environments.
Virtual production has taken many forms, but at its core, it involves the integration of digital scenes with real-world sets. This can include anything from motion tracking to cameras synced to virtual worlds to fully 360-rendered environments using LED technology.
Most virtual productions involve some motion tracking. Most commonly, cameras are tracked with virtual reality trackers, such as HTC Vive or purpose-built film trackers. Using these trackers allows filmmakers to use real cameras in real sets while streaming camera position, rotation, and settings back to a render farm for the virtual portions of the set. Traditional actor and prop motion tracking methods are also being used, streaming actor movements directly into game engines for previs rendering.
Virtual sets are also playing a major role in virtual production. Displayed on high-resolution LED walls and projection systems, virtual set extensions show real-time CG environments as backgrounds, allowing the camera's on-set to capture a complete image without blue or green screens. The major advantage of this kind of system is how the virtual sets can immerse actors into the virtual environments as they act, leading to better performances, lighting, and overall image quality. Additionally, this method can vastly reduce time spent on rotoscope work and other tedious VFX tasks.
Just like most major productions, the virtual production process begins with previs. Pre-visualization sets the expectations and needs of the production environment. Unlike traditional productions, previs also sets the tasks of the virtual effects team for what they need to produce before filming begins. Concept art, previs animations, and storyboards help the VFX team know what sets they need to develop.
After previs comes virtual scouting, where the film production team reviews the built virtual environments for shot locations and other production needs. Digital asset management becomes a significant tool in this step, as environments are rebuilt and modified to the needs of the production. Strong digital asset management allows virtual sets to be a practical production solution. Having the ability to easily change renderings on the fly maximizes the creativity of the production crew and saves time on busy production schedules.
Next is techvis; this step is a technology dress rehearsal, in which systems are tested, scenes are blocked, and everything is adjusted for the final production. This step is crucial for the complicated systems that virtual productions need to maintain.
During production, the virtual sets, game engine, trackers, LED walls, cameras, and render farm service contribute to the final on-camera image. While arguably more complex than most productions, virtual productions offer one major advantage: a significant portion of production staff can work remotely, building, managing, and modifying sets. Tools like cloud render farms and real-time collaboration platforms like the Omniverse mean that very few technical staff need to be on-site during production, vastly reducing production costs.
As you may have noticed, virtual productions rely heavily on real-time rendering using game engines like Unreal Engine 5 and Unity. Real-time rendering makes virtual production feasible at a large scale, beating out greenscreen technology and other traditional VFX methods. Yet, the rise of real-time rendering has brought massive changes to how render farms are used for productions. Rendering huge 3D scenes composed of dozens of 1k, 2k, or 4k panels in real-time is no easy feat. Render engines and render farm technologies are becoming more optimized than ever to help with the growing demand for faster turnaround times.
Studios have used two different methods for solving the rendering problem for virtual production. The first is to use on-site render nodes to deliver renders with low latency directly to production displays and LED walls. This is the preferred method for large productions or action-heavy scenes where latency is essential. However, it can be more expensive than other methods due to the additional on-site needs.
The other method is to use an offsite rendering service, like an online render farm, to generate images that are streamed back down to the set through a gigabit or ten-gigabit ethernet connection. Cloud solutions like this are still on the cutting edge as internet speeds and file encoding processes are catching up. Still, smaller productions with less hefty requirements are starting to use remote rendering as a cost-effective way to enable virtual productions. As technologies advance, this method will likely gain more traction as a viable method for studios.
Many studios have moved to virtual production workflows because of how cost-effective the method can be. Even though technology costs are high, they are still cheaper than building live-action sets or moving production outside of the studio to remote locations. So as costs for rendering and set development decrease, it's easy to see how virtual production could become a new norm in the Film and TV industries.
Virtual production methods are advancing at a rapid pace. In the coming years, it’s expected that there will be great improvements in every part of the production process. AI tools combined with motion tracking could reshape the industry and redefine creative boundaries.
Cloud render farm tools are also set to change. As virtual production needs increase, render farms will need to tailor their offerings for flexible real-time rendering nodes that have low enough render latency to be used on set. Render latency is a huge impediment to many virtual production scenarios, and will likely take sufficient work to overcome for all scenarios.
Cost is also a major factor in how virtual production will be implemented across the industry. Traditional VFX techniques, like greenscreen, are far more accessible for small studios than virtual production technology. LED walls, motion trackers, and render servers can be expensive. So unless costs can come down, many studios may not have the resources to develop virtual sets. However, online render farms and equipment rental houses may provide a solution to these problems for some productions.
Despite its challenges, virtual production is transforming the VFX and rendering industry by increasing efficiency and cost-effectiveness compared to traditional methods. The rise of virtual production is also presenting new opportunities for artists and filmmakers to be creative and develop new skills. Film sets are more interdisciplinary than over, as directors, set designers, and artists are working on learning how to use virtual technologies to execute their visions.
As virtual production becomes more prevalent, the demand for high-quality, real-time rendering is also increasing, leading to a greater need for render farms and render technologies to support these workflows. Technical crews and render providers are evolving to meet these needs by making tools more flexible than ever through custom render farm services and tailored render node development.
.jpg)
