I’ve recently started experimenting with “Style Transfer”, a new way to get painted animation implemented over actual video footage. The technique resembles the end goals of Rotoscoping where you draw over your video frame-by-frame.
Rotoscoping, in its essence, is typically a slow form of stylizing a video. I have done rotoscope for a few projects here or there looking for a painterly or drawn feel to existing video work. The problem is that it takes longer than people anticipate.
A lot of the attempts at implementing “art filters” over footage, like through After Effects, has been haphazard at best… giving people “certain” kind of stylization to their footage, but not looks that they expect or want.
“Style Transfer” promises to get whatever style you draw or paint in, and implement it onto your video in its entirety. It certainly has its (largely logical) limitations, but it so far seems incredibly promising as a possible replacement for rotoscope.
The video above took roughly a couple hours to do, running it through Style Transfer, and then doing some work in After Effects to give it a bit more “oomph”. If this was done using Rotoscoping technique, I could only imagine it taking upwards of tens of hours… for a little gag.
Like all new technologies in animation, there is always a worry that the “older version” of an animation technique becomes obsolete.
Industry-wise, you have seen a large replacement of paper-drawn animation with painted cels to digital means. “Puppet Animation” that utilizes 2D character rigs have long replaced its older brother of cutouts, for instance.
And for Style Transfer, I can definitely see that this becomes the way to implement “rotoscope” faster. And for most people’s use-cases is great as a complete replacement.
The technology though doesn’t mean that it solves every single niche case of rotoscoping, just like Digital Puppet Animation doesn’t solve every single niche case for cutouts.
But for the lot of what’s needed, I believe it would be the future.