This 4K video was recorded to a cassette tape quite a few years ago - and yet for the most part
it just looks like a meh phone recording, or with a little tweaking, it looks pretty solid.
That’s the power of AI at work.
VHS tapes are a fun medium. Different lengths of time available based on quality - very
similar to video compression today - no unskippable ads,
it remembers where you left off, and forever, and held everything from intimate family
memories to bootleg music videos and TV recordings to full commercial releases.
But as much fascination as I may hold for this medium, it’s quite fragile and the tapes you
may have stored in a box are expiring fast, if they haven’t already. I’ve been working
on my VHS archiving guide I promised, but in the meantime I got distracted searching for
the best way to upscale and preserve these captures in a more modern way.
This asks the question: can Artificial Intelligence help us preserve VHS captures?
I’m EposVox, the stream professor and I’m a VHS hoarder. Movies and TV shows from my childhood,
but what I’m really interested in are tapes of game previews and demos that were sent to press,
storefronts, or included with magazines. If you have any, please email me.
There’s a lot of considerations when it comes to the hardware side of capturing tapes, which we’ll
cover in a future video, but regardless you end up with a 480i analog-degraded image. It doesn’t look
great blown up on a modern TV. What if we could fix that? Or at least give it a little life boost?
Upscaling is only part of the battle:
deinterlacing in a pleasing way makes a huge difference, as well.
Today’s video focuses on comparing 3 ways of upscaling VHS captures:
basic upscaling in a video editor if you just ingested the footage at native resolution,
normal bicubic upscaling with the QTGMC deinterlacing algorithm for super smooth motion,
using DaVinci Resolve’s GPU-accelerated “Super Scale” tool with their new “Neural Network”
AI deinterlacing algorithm, and lastly, a full AI solution using Topaz Labs Video Enhance AI.
Topaz Labs makes a full suite of AI tools for upscaling and working with photos and videos.
I’ve been playing around with Video Enhance AI
since last summer and wanted to see how it handles interlaced content.
Well, thankfully they added a neural model for this, called Dione. This is designed for
interlaced content with two different versions based on capturing cleaner DV tape sources
or older analog tapes such as VHS and 8mm. We’ll mainly be looking at the TV option, including the
secondary “robust” algorithm - but I also ran a DV tape capture through the DV model, too.
The results here are quite interesting. You're going to see better results all around with
cleaner and higher-quality tapes, but these are what most people will be capturing: Home videos
from the 80s and 90s from VHS, 8mm transfers, and so on. Things that aren't super high quality.
For this, we're taking native 720x480 or 720x486 interlaced captures and scaling them by more
than a factor of 4 to 2160p high for 4K. Plus, deinterlacing. That's a lot to push this footage.
Overall, I was honestly shocked to see that Resolve's "Super Scale" amounted to nothing
more than just scaling up the original footage on the timeline. I didn't have high hopes,
but I expected... something. It's so much slower and seems to be doing something,
but realistically the results aren't much different than just scaling the footage
up normally. For this, I had "Sharpness" and "Noise Reduction" set to Medium, the default,
but realistically not much would change with the other settings here.
It didn't matter if I was scaling animation, a higher-quality tape, or an old 8mm transfer,
Resolve Super Scale looked marginally sharper than a native scale, but that's it.
I still had to test it, and even setting "Sharpness" and
"Noise Reduction" to High.... didn't make a big enough difference. There's s SLIGHT,
SUBTLE change compared to Medium, but still not competitive with the other options.
StaxRip's Bicubic scaling with QTGMC deinterlacing actually provided really compelling results
in the animation tests. There were some situations where the perceived aliasing or "jaggies" on lines
were almost entirely removed with this scale, with noise reduction competitive with the AI upscaling.
The Dione AI upscaling and deinterlacing provided a cleaner image on the animation
footage, but did not handle line artifacts as well. I'm surprised.
***[Brilliant Ad]***
If you want to develop your own AI-based tools like this one,
you'll have to learn to code, and learn about neural networks. Where do you start?
If you're interested in learning computer science, programming, or other STEM topics,
you could watch more educational videos on YouTube - you're already here,
cool. But in order to learn these kinds of things, you have to actually DO it.
Brilliant is a website and app built off of this very principle:
You learn best while doing and solving in real-time. Jump right into solving problems
and be coached bit-by-bit until you've learned a new STEM subject before you've even realized it.
Pick a course you're interested in, get started, and don't worry about memorizing long formulas or
fact sheets. That's not how you learn a skill. Working through problems, reading explanations
for mistakes or places you get stuck on, and learning at your own pace - that's education.
Brilliant has something for everybody - whether you want to start at the basics of math, science,
and computer science, or dive right into cutting-edge topics like Neural
Networks - the stuff building tools like this. If you'd like to join me and a community of 8
million learners and educators today, click the link in the video description or head to
brilliant.org/eposvox - the first 200 who do so will get 20% off the annual Premium subscription!
***[end Brilliant ad]***
Switching to live action footage, though, and even other animation
clips, and generally speaking the Dione AI upscaling provides smoother results. Less noise,
clearer shapes and colors, and so on. It's not magic, you're not getting detail that was never
present, and the more degraded your tape is, the less you have to work with, but this does
look significantly better blown up to a 4K TV than any of the other options, in most cases.
I will say that with the 8mm car tape, I found the "Robust" version of the Dione AI to look the best,
the standard one produced a bit of ghosting - but this
was the only sample I found that result to matter.
Using the Dione DV AI on my widescreen miniDV tape produced remarkable results... Like
WOW. Noise is mostly gone, colors are back a bit, sharpness is far improved.
This is beyond impressive, especially compared to all of the other options. Add a bit of a color
grade and this isn't terrible-looking footage for what it is, considering the terrible lighting.
Letting curiosity get the best of me, I also decided to test the Dione DV AI on a couple HDV
clips I had from some experimenting back in 2017. Something with better light and less baked-in
noise. So here we're taking 1440x1080i footage and deinterlacing while also quadrupling up to 4K.
Honestly, the AI upscaling doesn't do anything magical here, either. Deinterlacing is done
quite smoothly, but the results via StaxRip and QTGMC are mostly identical.
Also, Video Enhance AI kept crashing on the MPEG capture for one of these clips.
In fact, there were quite a few issues that I found with Video Enhance AI for myself,
personally. On top of it really not liking my MPEG HDV files - which a simple copy to AVI via FFMPEG
fixed - you have no control over the aspect ratio. By default if you scale a 4:3 image,
it will crop to fill frame, but you have an option to disable that. Cool. However,
after that, you have no control over how to handle the aspect ratio. So one of my HDV clips
was detected properly in 16:9 via a wider pixel aspect ratio since the footage was 1440x1080.
But the second clip, same resolution and pixel aspect ratio, was detected as 4:3, so I had to
stretch it out to fill the proper ratio after the fact. Plus, the actual 4:3 scaled results
here came out a tad wider than my normal scaling results in Resolve, though closer to StaxRip.
I'd honestly lean on suggesting Video Enhance AI might be more correct here, but it's still weird.
Lastly, framerate detection was... all over the place. Since I'm working with interlaced clips
specifically here, that's understandable, but even between two different PCs with the same
version of the app would have different results between 29.97fps and 59.94fps.
And then you have the MPEG clip, which I copied to an AVI container, which it decided was 120FPS
and exported as such for some reason?! Give me framerate controls, please.
Those were quite a few complaints, but really - I've been playing around with this software, and
even using it in videos without saying, for nearly a year now and it's been a blast. You can do some
wacky stuff with Video Enhance AI and it has models for computer graphics, real-life footage,
faster stuff, slower stuff, lots of details to play around with. If you're a Digital Foundry
patron, they've been using it for upscaling old lossless game reveals and such that they have.
There's also a trade-off, where Video Enhance AI takes exponentially longer than normal solutions,
requiring a decently-high-end graphics card and measuring render times in seconds-per-frame
rather than frames-per-second, but doing hard work takes time, I suppose.
To answer the main question, though: Can AI help preserve VHS captures? I'd say yes.
The natural-looking noise removal, the extra enhancement of detail and sharpness, without
doing anything too wonky, along with near-perfect deinterlacing most of the time means that my old
family videos - or even just fun, commercial tapes - can look better than ever blown up on
the big screen. Sure, you're technically losing "accuracy" from how the tape actually looked - but
in many of these cases, that's not accurate to how it was originally recorded anyway either.
Thanks for coming along on this journey with me. Links
to StaxRip and Video Enhance AI will be linked in the video description,
if you're interested. Let me know what fun you've gotten into with
your tape upscaling in our Discord server at Discord.gg/eposvox. Remember: Be kind, rewind.