I can upscale pictures with great success. Using AI it can fill in the gaps and make
a convincing result. But what about videos? Surely they’re just lots of pictures, put
together. Wouldn’t it be great to be able to convert all your old home videos and fraps
recordings to 4K? I’ve actually been experimenting with that
since last August. And I think I might have gone a little crazy. What started off as one
or two projects has ballooned to hundreds. Occasionally I’ll put them all in a folder
somewhere, only to immediately start on something new. Now I’ve got so many upscaled videos
to show you that I don’t even know where to begin.
What I learned from all this is that upscaled videos aren’t as convincing as upscaled
pictures. When motion is a factor, it amplifies any of the bugs and imperfections of upscaling.
So while the upscaled video might look sharper, or more detailed, it’s also so distracting
to look at that you might be better off watching the smoother, blurrier original instead.
For video upscaling to work, you’d somehow have to invent an AI that could intelligently
carry details across frames to reduce the spindly, shimmery nature of upscaled videos.
And that’s exactly what the makers of Gigapixel AI have done, with the creatively-named ‘Gigapixel
AI for Video’. It was kind of separate from the main site in a secret beta-stage for a
while, but now you have to buy it, or stick to a free 30 day trial.
Alternatively, you can use this site to upload up to 60 seconds of footage for free, and
it’ll give you a side-by-side comparison before and after.
OR you can upload a full video and get it upscaled up to 16x the resolution. You pay
for this PER PIXEL! Plus 2 dollars. Since I still have a 30 day trial, I can tell
you a thing or two about it and how it performs. For a start: you’ll want a powerful PC.
I have a Geforce 2080 TI. And while that kind of power is useful for gaming, I think I’ve
actually been using it more for AI upscaling. It can upscale low-resolution videos almost
in realtime, but upscaling HD to 4K still takes about an hour a minute. So you can only
imagine how long it would take on less powerful hardware.
But any kind of automated video upscaling is better than having to manually upscale
thousands of pictures. I suggest you make the most of the free 60 second upscaling they
offer to test different resolutions to see how the results will look before you invest
the time or money into upscaling the whole thing.
I’ve been using this AI video upscaling as an opportunity to return to some of my
older videos and to see how much better I can make them look. This is a bit of a trip
down memory lane for me, and I’ll be releasing these over the coming days in new, higher-resolution
glory! In this video I’ll quickly talk about each
of them, the challenges involved with upscaling them, and what I hoped to get out of it. I’ll
also be sharing some tricks I used to improve them in other ways, which might help you if
you’re looking to restore some of your own videos.
GOING LOW IN CS:GO – FRIENDS. This is personally my favourite Going Low
in CS:GO video that I’ve made. No narration is required to tell a tale of pity and…
romance? You’ll see for yourself when you watch it. This video was already in full HD,
60 FPS. I chose to upscale this one to 4K. My reason for choosing this one is for the
intro sequence. I was only recording in 1080p at the time, and this scene required a LOT
of zooming, to the point where the end result looked far blurrier than I’d have liked.
And that’s a shame! In fact it was this video that drove me to make future ones in
4K. Never could I have imagined that the day would come where I’d be able to bring the
details back to this video. But here we are. The key to good upscaling with videos is the
same as with pictures. You ideally want the resolution you’re upscaling from to be as
small and as crisp as possible. So because the intro to this video was so zoomed in,
I actually upscaled it several times at lower resolutions, decided on which looked best
for each scene and pieced those together back into a full video.
The result of this upscaling isn’t perfect. Everything looks a bit painted in, and the
chain link fences are still a mess. But I feel I’ve done as well as I can, given what
I had to work with. GOING LOW IN CS:GO – CRAZY KNIFE DUDE
And the same applies to this video, which was the first to be uploaded in 1080p, 60
fps. I upscaled this one to 4K as well. The Inferno map has brighter colours and more
contrast, which I think results in a better quality upscale even though the video itself
is older, recorded way back in 2014. You can tell because although it’s a gold nova game,
everybody by today’s standards would be low silver. Fascinating to see how things
have changed, and how harshly older videos can be judged by today’s players.
In 2012, before Shadowplay was even a thing, I filmed this trailer for de_elysium, a map
for CS:GO that I made just months after the game’s release. This was captured at 1080p,
30 FPS. Now this is a dilemma- do I keep it at 30 FPS to retain maximum sharpness to show
off the upscaling, or do I use frame-rate conversion software to up it to 60 with nice
motion blur? (Which I think will make the video look more modern). Right now I’m thinking
I’ll upload both and will let you decide for yourselves which is best. But before any
framerate conversion takes place, the upscaling must come first. It always should. If you
do the framerate conversion first then it adds artefacts to all 60 frames, rather than
to 30 which are then smoothly transitioned between. So actually, I think framerate smoothing
being applied after upscaling will help to mask the upscaling effects and shimmer! I
didn’t record this with anti-aliasing on. I don’t know why. I knew what I was doing
in 2012. I even had ambient occlusion forced on in the Nvidia drivers! So while the result
is nice and sharp, all of the jaggy edges are upscaled to look… horrible and sharp.
This presented me with a dilemma! Do I upscale the 1080p original and remain aliased, or
squish the original down to 540p, which will make edges look smoother at the loss of some
surface detail? I chose to use both- the 1080p upscale with moving images to reduce shimmer,
and 540p for the still images to cut down on aliasing. Will any one notice? Probably
not, but that’s 90% of my life in a nutshell. The one new scene I added was just after the
bomb explosion. I always regretted not having some awesome far-out shot here so used this
project as an opportunity to add one. Plus, it’ll let you compare the rest of the video
to a native 4K, 60 FPS shot. See if you can spot the difference.
Almost a decade ago I juggled up in the Lake District. Some of it was captured at 50 FPS,
others bits, 50 FPS interlaced. And this presents another upscaling challenge!
Interlaced videos only capture half the video’s lines at once. Good software can try to join
it together again, but the result is never as good as if it had been recorded properly
in the first place. Going slightly off-topic, some time last year I upscaled all of the
episodes of Peep Show and some of these suffer badly from interlacing. I discovered that
by resizing the image to half the vertical resolution, and then upscaling that (and then
squishing it back to normal shape again) it preserved a LOT more detail than halving both
dimensions, AND removed interlacing pixilation. From all this work, I am proud owner of the
finest quality Peep Show episodes in the world! …and you can see non-copyrighted results
in this upscaled juggling video. Although the Pet Shop Boys song is still copyrighted.
It’s from the era! Now for something a little different.
In 2011, an intro for Atomic Amnesia was made. The show was going to be about naked old men
floating around in space. I wrote a song for it, my friend did some character art and we
came up with some pretty amazing episodes that we wanted to make… then life got in
the way. And then Rick and Morty came along and used such similar ideas it would have
made it look like we were copying it. Whatever. Here is the mock-up I made for the start of
the first episode. It seemed a bit desperate to upscale perfectly
good video clips, so instead I just upscaled the still images to bring the whole thing
up to about the same level of quality. I just saw this upscaling project as the best excuse
I’ll ever have to release this video so, after 9 years, I can finally get the project
files off my hard-drive. If I was ever to return to it I’d start it again from scratch
any way.
Now for something else a little different. In 2008 I have what is one of the final things
I did with my Counter-Strike Source clan. We decided to make the GREATEST machinima
ever. If you’d like to know, the story behind Final Styricum Shower is that they discover
a brilliant planet full of a liquid called Styricum that people really like to shower
in, so they build a huge cable going from that planet to Earth’s and it results in
world peace because everybody’s too busy rubbing Styricum all over each other’s bodies.
But for some reason, the pipe is cut! What will happen?! I’m… not sure how the machinima
was eventually going to explain this… but that’s fine cos we only filmed this intro
sequence any way. You know how old black and white footage makes
stuff look older than it actually was? I feel the same about Counter-Strike Source videos.
They were normally recorded in TERRIBLE quality, and uploaded to youtube in 360p. But for whatever
reason I did this one right by capturing it in full 720p!
…but unfortunately, I rendered it at the wrong framerate of 25, hence all these horrible
ghostly outlines around anything moving. But I have a solution to that! How do you get
rid of horrible motion blur? YOU ADD EVEN MORE! I upped the framerate to 200 frames
a second! …and then I crushed it back down to 50 again, with resampling enabled. The
result: higher framerate, and better motion blur. I’m not saying this is the ideal solution.
It’s probably far too much blur to be acceptable. But I’m proud this botched solution, where
I fought resampling with yet more resampling. So there.
Framerate conversion will result in a weird ‘morph’ when the camera suddenly changes
angle- so I had to cut out those mutant frames and stretch the clips either side. This means
the action will slow down just before and after a cut. I tried to stretch the scene
with the least action to make it less noticeable, and even at points for cinematic effect- but
it’s amazing how your brain filters the slow-down out. I guess it’s a bit like how
you don’t see yourself blinking. With all this work done, I then upscaled again
to 4K. Remember how I said you should always upscale first before upping the framerate?
Yeah, didn’t take long for me to ignore my own advice, did it. But in this case I
did it to save time- that framerate conversion took over 4 hours at 1080p- I just couldn’t
be bothered with the time that 4K would have taken.
Now back to 2007. Back then everybody on the messaging software, xfire, was addicted to
Solitaire. A dark time indeed. In a cry for help, I made this video. I’ll warn you now-
there’s only so much you can do with something of such limited quality. My room was dark.
My camera, BAD. And when I compared the upscaled result with the original, I was horrified
to discover that it looked no better, other than for the colour and contrast adjustments
that I had made to it. But with the upscaling, to get it to work effectively I had to crush
the video down to such a small resolution that it no longer had much information to
work with! I was close to giving up with this video completely, but by happy chance discovered
that my upscaled my face looked terrifying- though this is only when using the picture
upscaling software. And I had to upscale each frame of the video individually. But I’m
proud of this video because it’s an example of using upscaling to achieve something OTHER
than better quality- I’m using it here to make it look trippy and sinister. Every time
my face was on show, I manually adjusted the resolution to get the software to upscale
my face in the most unflattering way possible. Suddenly, this video has a new purpose to
exist, and releasing it as a new stand-alone video is justified!
Now back even further, to Morrowind. I’m not sure when exactly I made this video. The
Bloodmoon expansion I used came out in 2003 and I was heavily inspired by The Two Towers,
so I was probably 12. I hope I was. But I might have been 13. Or 14. Or 16 I don’t
know. Any way, this is an absolute worst-case example for video upscaling. Low resolution,
pixelated, dark, 10 FPS! It’s like upscaling on HARD mode. I think, per-pixel, I have obsessed
more over this video than any other. It’s not even a good video! I just see it as a
challenge to restore to the best quality I can. I have run every frame through a jpeg-artefact-remover.
I have tried blending several different upscales together. I’ve tried anything that could
possibly have helped- and more besides. But after all that, I eventually decided just
to run it through the upscaler and to be done with it. In the future this video should be
used to benchmark how good video restoration software is. It’s got everything!
It’s got snowy scenes- which gigapixel fills in with lots of snowflakes which weren’t
really there. It’s got dark scenes, where it has about
a dozen blocky squares to make a picture from. It’s got high contrast outdoor scenes, but
without enough detail to generate believable textures from.
And it’s all at 10 FPS. Here’s the problem with framerate enhancing. It’s most needed
on low framerate videos, but it’s low framerate videos that it performs worst with. 25 FPS
is bad enough, but the jumps between each frame of a 10 fps video are so BIG that it
has no choice but to do weird transformations that look utterly nauseating, to the point
where, although I see it as an overall improvement, I know already that uploading it to Youtube
at 60 fps will result in dozens of comments from people saying it looks weird and that
I should have kept it at 10 fps. And now I’ve said this, it’ll lead to dozens of comments
from people ironically saying it looks weird and that I should have kept it at 10 fps.
I know you so well. I was even thinking of making it 60 FPS in the centre of the screen
and having it fade to the 10 FPS original around the sides, which is where the framerate
conversion bugs are normally the most obvious. But the contrast between a smooth 60 in the
centre and a juddery 10 around the outside was equally distracting.
This video is hell to upscale. But that’s also its appeal.
I’d be better off just loading up Morrowind and re-recording this entire project. But
that would be cheating! And that’s it! That’s as far back as my
videos go. But in secret, another upscale was made. No, not Lord of the Rings! I’m
talking about the true Lord of the Rings. A decade ago I worked with the Fart Master
on a movie, but production stopped after some disagreements about the plot. Because at some
point you need something other than farts. But I’ve been keeping tabs on this legend.
He had a video scheduled to be released every month, spanning centuries into the future!
Clickbait channels had their fun with him. They showcased his works and laughed at him.
Here’s Chocolate Rain dude appreciating his craft!
But while others mocked him, to me he was a misunderstood genius, ahead of his time.
Unfortunately, in a bid to make Youtube more child-friendly last year they sent him a hundred
strikes within a matter of seconds, wiping him from the site.
Which is a shame, because he managed a 2 and a half minute-long fart. The longest on record-
more than twice as long as Mr Methane’s best! So, out of respect for this legend…
this rainman of the gaseous elements, I hand upscaled each and every one of this video’s
frames to 8K.
And I think this video perfectly encapsulates the beauty of upscaling. It’s to respect
the past. To acknowledge videos which deserve better than whatever format they find themselves
bound by. It is for timeless content that you want future generations to appreciate,
long after those who recorded it are gone from this world. It’s truly remarkable,
and a testament to mankind’s drive to better themselves. It takes my breath away. Thank
you for your service, AI upscaling.