Upscaling is suddenly everywhere! AMD has just released FSR- their own smart
upscaling solution. Nvidia has DLSS, and then we have tools like Waifu2x and Gigapixel,
for everything else. All of these things use different techniques to try and make an image
look higher quality than it did before. SO WHAT IF WE UPSCALE AN UPSCALE?!?! Can we
FSR a DLSS, or gigapixel a waifu? Can we just keep adding more and more detail to everything,
forever? Let’s begin by repeating the same upscaling
technique several times. Because this should make it clear what its limitations are.
Here’s a lovely bunch of foxgloves, in high resolution just so we know what they’re
MEANT to look like. Now, let’s crush this image down to a super
low resolution. And now let’s upscale it.
And again! And again.
Not a bad result. In fact, given the circumstances, I think it’s a really good result. But what
we see here isn’t reality! THIS is how the original looks in comparison. I’ll jump
between them a few times so you can see the differences.
Upscaling gives the impression of adding more detail, but the more it adjusts the image,
the further it deviates from ‘reality’. In other words, upscaling makes a better first
impression, but it holds up worse with scrutiny. So in this sense, the limitations of upscaling
depends on how many artifacts you can tolerate. Interestingly, because the backdrop was blurred,
the upscaling has tried to refocus it, so the result is something that’s less blurry
than the original! …but that doesn’t mean it’s better.
The upscaled plants look convincing at first, but if you zoom in and compare them with how
they’re meant to be, you can see the upscaler got their leaf patterns all wrong.
And the fox gloves have become watercolours of their former selves.
And that’s it- upscaling an image gradually distances itself from the original. It becomes
a watercolour of its former self. Because it’s smoothing edges that have already been
smoothed, or transforming groups of pixels into larger blobs of pixels. After a certain
point, the upscaling no longer upscales the images, but merely its own upscaling.
But, if you need to see the original to notice its limitations, does it even matter?
Given upscaling’s strengths, it’s no wonder that games are so keen to use it. Games have
been fighting a war against pixellation and blockiness since time began. Anti-aliasing
has been the conventional solution. But this comes with a framerate penalty, and has become
less effective as game engines have become more complex. Upscaling, on the other hand,
can now be easily added to games, and even lets them run faster than if it’s NOT being
used! Win-win. DLSS, I have already covered many times. It
is Nvidia’s AI-powered upscaling solution, using sub-pixel details, motion vectors and
lots of clever trickery to construct a detailed and smooth looking image. I still think it’s
the most effective form of upscaling that games can use, but it seems like Nvidia needs
to be involved to get it enabled in games, plus you need an Nvidia graphics card to run
it. FSR, one the other hand, is something new
that AMD has just released. It doesn’t appear to rely on ‘AI’ or anything like that-
think of it more as a ‘smart resizer’ that relies on a resizing algorithm that keeps
details crisp, and pixellation to a minimum. I DON’T think the visuals are as good as
DLSS, but the one advantage FSR has is how easy it is to integrate into games. It only
takes minutes to do, and as seen in GTA 5 recently, doesn’t even require input from
the game devs or from AMD or Nvidia. So I expect FSR to be seen everywhere, pretty quickly.
And inevitably, people will soon be adding it to games with DLSS already and they’ll
be upscaling upscaling! HOWEVER, until then I’ll just experiment
with upscaling DLSS and FSR with gigapixel, which I find produces GOOD upscaling results.
It’s not without its problems- it isn’t the best for faces, it does require manual
tweaking for the best results, and it’s SLOW. It can take a few seconds to upscale
a single picture. There’s no way you could use this in real-time for something like gaming!
But it’ll still be interesting to see if I can extract more information out of already-upscaled
images from using it.
So I’ll start with AMD’s FSR, because it’s new, exciting, and everybody can use
it. We’ll try upscaling NATIVE 720p, then FSR Quality, and FSR Performance. Unfortunately,
I couldn’t test FSR’s best quality mode, being Ultra Quality, because it messed with
the shadows in this game. I even tried restarting it and everything. But the point is: CAN upscaling
an already-upscaled image improve it? First of all, upscaling the native image.
This is 720p, this is it upscaled to 5K. 720p, 5K. 720p, 5k. The difference isn’t night-and-day,
it just looks a little bit sharper. If we zoom in on the result, the difference is more
pronounced and you can see the upscaling has made it less blurry. The lines of grass look
smoother and less pixelated. And the grass texture here is made sharper, but more like
a water-colour. The gems look nicer in my opinion. But the robot looks more like a cartoon
than the original art style may suggest. I believe that upscaling HAS improved this image,
but has sacrificed the art style slightly in the process.
Now let’s upscale an upscaled image. This is FSR in quality mode. And this is it upscaled
to 5K. Again, not a HUGE difference, but zooming in we can see what’s happened here. It’s
been made less blurry. And I’m not sure if things look more detailed, or if they just
display more artifacts than before. I’ll be honest with you: I think, when the base
resolution is this low, FSR Quality mode has already deviated far enough from the original
image that it’s already beginning to look artificial even before a second upscaling.
And Performance mode is even more extreme. It already looks like a 16-bit watercolour
even before we upscale further.
This may have seemed like a waste of time, but we’ve learned something important from
this: if you want to upscale an upscaled image, that first upscale needs to have produced
an acceptable quality image in the first place! So let’s jump to a more sensible resolution
like 1440p, which FSR Ultra Quality mode seems to work fine at. Great- this is probably how
people would be using it in practice any way. And this looks perfectly fine! I’ll reserve
my thoughts about FSR for another video, but at this setting you’d be hard-pressed to
tell it isn’t native resolution. Let’s upscale this to 5K and see the result!
The result is SUBTLE, but when I blindly compared both, I consistently preferred the one that
had been doubly upscaled, simply because it had better clarity and apparent texture definition.
The unfortunate conclusion here, though, is that double upscaling performs best when more
of the leg-work is done by having a higher base resolution in the first place. Sure,
you can expect extra upscaling to deliver better clarity and sharper edges, but beyond
a certain point it begins to betray the original art style. Don’t expect a miracle, or for
it to be able to conjure new details out of thin air.
Now let’s try DLSS. Does this squeeze every last pixel out of its upscale, or does it
leave enough for a second pass to have some value?
This is DLSS Quality mode, at 720p. I’m not sure what the base resolution is with
DLSS because they’re always a bit cryptic about this, and it varies slightly title-to-title,
but I’m assuming the dimensions here are about 2/3 of native.
And this is it, upscaled to 5K. And I’m going to admit, I’m really impressed with
this. Gigapixel has managed to extract a lot more clarity and detail from the image. I
feel like this winter setting is ideal for upscaling, as it comprises of lots of thin
details and high contrast areas. But regardless of how it did it, I got a really good result
using gigapixel to upscale DLSS Quality mode. It’s effectively making 36 pixels out of
every 1 that was originally rendered! If I compare it with a native 4K image, it’s
obviously nowhere close to that. Not by a long-shot. I can try sharpening the upscaled
image to further boost the clarity, but I don’t think it’s worth it as this really
highlights the limitations of the upscale. Having upscaled a lot of things with gigapixel,
I know another problem with this upscaling- it won’t be temporally stable. The details
will flicker and shimmer in motion, which can be super distracting. So here’s an alternative
upscale, which more greatly suppresses motion shimmering by hiding the smaller, less ‘stable’
details. Again, it’s a trade-off- this upscale will probably look better in motion, but it
looks worse in still images, since now the finer details, like tree branches and distant
cross-bracing on the bridge, have been reduced to a fine blur, and the art style has been
made to be more like a water-colour. It’s all a compromise. You don’t get something
for nothing. And while smoother, this upscale no longer has as much detail to it as DLSS
Quality did before the second upscale.
And now let’s try DLSS Super Performance mode. This is an extreme setting, better suited
for 4K or 8K where each pixel is less noticeable, but I enjoy using it at lower resolutions
because it reveals the secrets behind DLSS’s upscaling. Again, I’m not entirely sure
on the base resolution here- it’s 1/3 of native at higher resolutions, so it MIGHT
also be that at lower ones, or maybe not. Hence the questionmark. All I know is that
it’s trying to upscale from a lower resolution than Quality setting is. Let’s see if I
can upscale THIS upscale! I… can’t complain about this result, if
I’m honest. It appears better and less pixelated than simply resizing the DLSS’d image to
5K, and while DLSS ultra performance has a lot of visible checkerboarding, the second
upscale from gigapixel seems to do a good job of translating them into straight lines
or more organic looking details. I doubt this image would be at all temporally stable- even
Ultra performance mode has noticeable shimmer to it, and that’s before the second upscale!
But screw it, let’s upscale a few seconds of gameplay and see how bad it is.
Don’t expect upscaling a second time to reduce any flickering or problems with the
first upscale- it’s only going to make things like that worse. Yes, the result is sharper,
but other than that, many of gigapixel’s ‘improvements’ to the image simply don’t
hold up in motion. And this is just when standing still!
And let’s end with looking and walking around. To REALLY test the stability of upscaled upscale,
from such a low resolution! In conclusion: Want to upscale an upscale?
Sure, you can do that. But don’t expect to get something for free. It will ultimately
be up to you whether you deem it worth having extra sharpness at the expense of detail,
or higher framerates, but ones which look partially watercoloured. And for some people,
it will be worth that! And for others, running on slower hardware, they might not have so
much choice in the matter. And I still question whether there’s any benefit to multiple
upscales, or if it’s best just to choose one method and to go all the way with that
instead. Those fox gloves, I didn’t need to upscale them 3 times- I could have just
done it all in one go. So that’s why it’s silly to suggest upscaling an upscale.
And YET, it sounds like there should be value to FSR’ing a DLSS, or giga-pixelling a waifu,
but perhaps there isn’t if each upscaler has been designed to go as far as it can before
the illusion starts to break down. For good upscaling, each pixel fed into it
needs to be worth something. And any kind of upscaling process devalues each pixel.
It replaces actual detail with artificial, generated filler. And at some point, there’s
no use in going further. But it’s been fun to do. And who knows-
I expect FSR to get better as time goes on. Maybe somebody will produce better results
by combining 2 different upscaling methods together in a game of some kind. Maybe one
day, somebody will produce a gigapixel image of an FSR of a DLSS of a Waifu… that will
blow me away.