It's almost redundant to say that we have witnessed impressive leaps of progress in computer graphics in the past decade, with the upward trend becoming less and less gradual with every new year. Today, in the age of content creation, we're seeing exciting applications of Artificial Intelligence even in niche aspects of 3d production that promise to shorten the pipeline to the point where an entire animated short can be produced by a few individuals, if not a single person.
In this podcast episode, DJ and Amiel sit down with Kevin He and Ergin Dervisoglu of Deepmotion.com to talk about Animate 3D - a markerless, trackless cloud-based motion capture solution.
Animate 3d allows users to add motion capture animation to their 3d characters with nothing more than video footage, eliminating the need for costly hardware and a studio. Animate 3D is browser-based and only requires an internet connection.
Kevin and Ergin lead their team alongside industry veterans like themselves, with decades of experience from companies like Blizzard, Pixar, Disney, ROBLOX, Microsoft, Crystal Dynamics, and Ubisoft.
According to Kevin, despite the rapid advancements we've seen in 3d rendering in general, 3d animation seemed to be the least addressed aspect of production. 3d character animation is one of the more specialized and time-consuming processes that heavily impact an animated film's final result. The skill required to create convincing character movement and expression is attained through years of learning and practice, sometimes excluding other general skills such as modelling, texturing, and lighting.
Kevin recalls his time in Blizzard as part of the development team for World of Warcraft. Each asset and animation clip was handmade by a group of professional 3d artists and designers. He notes that the industry at large has since adopted more procedural methods and motion capture to speed up production. However, for the small independent studio or the lone content creator, motion capture equipment can prove very costly, with the suits alone costing upwards of $1000 on average.
Kevin and his team conceived Animate 3d to meet this need. They decided to focus on introducing procedural approaches to animation with the help of AI and physics simulation to produce lifelike character animation, initially starting with a physics engine that would allow the simulation of weight and emotion.
They turned to AI to further enhance the diversity of motion and authenticity. They developed what they call their Motion Brain, which one could think of as being comprised of two hemispheres. The Perceptive Motion Brain, which parses a subject's movement from video footage and attempts to mimic that movement. And the Generative Motion Brain, which creates keyframes on a 3d character and interpolates the movement and can generate new motion on top of what is obtained from a two-dimensional video.
Kevin hopes Animate 3d will cater to animators, game developers and content creation in general. The software lowers the barrier of entry for creators from all media to adopt 3d animation. Game devs and industry animators might use it to more quickly and cost-efficiently create intricate character sequences.
Another considerable advantage is while traditional animation takes very long, this technology can also bring character animation to the concept stage of production, allowing for a faster and maybe more intuitive way to plan out a character-driven short.
Ergin believes Animate 3D opens up 3d production to a broad audience but highlights that it comes with a set of tools designed to help animation specialists as well. He also says the team continues to close the quality gap between the casual and more advanced users of Animate 3D.
Because Animate 3D leverages deep learning, the more footage it becomes trained on, the better it will become at analyzing a more comprehensive range of movement. On top of that, the team behind DeepMotion has been keeping an ear out for feature requests.
At the time of this cast, the company has introduced better tracking of hands and fingers, foot-locking (properly evaluating foot position on a ground surface) and facial expression tracking. It continues to improve the quality of life in general for users, including enhancing the core processing of video footage and hopefully getting better tracks even out of relatively rough video clips. One significant new feature is Animate 3D's capability to interpret half body footage, making it easier to get a clear read on facial movement when recording from a smartphone, for example.
When asked about what the introduction of AI to CG production might mean for artists working in the industry, Kevin explains that AI is good at removing technical complexity in the 3d pipeline. The quality of results from AI improves every year, but AI is limited based on what material is provided to it - material done by the human hand.
The evolution of AI based results is also dependent on how humans progress and raise the bar on delivering the said result as well. This means people won't likely lose their jobs but instead be given more opportunities to iterate faster on technical processes and focus their energy on more rewarding tasks. He also mentions that he doesn't believe AI will replace handmade content but can help in creating derivative work.
Animate 3D, along with other services from DeepMotion, are currently all accessible on the cloud. When asked about why Animate 3D is offered as a cloud service, Kevin and Ergin cite these advantages:
- no software download or installation needed
- because of the relatively early stage of AI, the frequent updates make it easier for users to work in the cloud
- no hardware requirements to use Animate 3D/DeepMotion
However, they understand that some might prefer a software-based solution and are also looking into how they might accommodate this.
DeepMotion is at the vanguard of a new breed of third-party services that seek to positively disrupt existing industry conventions to democratize 3d content creation for artists and storytellers of any background - and they're just getting started.
Listen to the podcast for the full conversation, and get started using Animate 3D with this Tutorial Article (uses Animate 3D and Blender but steps can be replicated for other DCCs)!
