This image you’re looking at was rendered completely on a TI84 calculator.
No tricks, no “I rendered it on my PC and copied to the calculator.”
This is the real deal, a raytracing engine that runs on a calculator that supports textures,
reflection, HDR, gamma-correct rendering, dithering, and global illumination.
Now, most people familiar with my older catalog of videos probably know that a while back,
I already wrote a raytracing program on a graphing calculator. But that one had a few
limitations that I’d like to address. Mainly, my first raytracer wasn’t all that fast. It
took almost 6 hours to render this image here, which works out to about .0000485 FPS,
and less than 5 pixels per second. To be fair, raytracing is not something that can easily be
pulled off in realtime. Each pixel involves a whole boatload of computations to determine what
is visible and how it should be illuminated. Even now, you need a beefy GPU to pull it off. But,
even for the cheap little CPU in these calculators, 6 hours for this is just pitiful.
The reason everything is so slow is because TI-BASIC, the built-in language I used,
is interpreted. In short, the processor in the calculator doesn’t run my program,
it runs a program that runs my program. This makes sense for a calculator, since the whole
process is sandboxed, and a lot harder to crash, but for my code, it means every math operation I
do involves more work than it really needs to, and this overhead adds up. TI-BASIC also limits
you to 15 colors over a fraction of the full screen, and probably the biggest killer for
a project like this: TI-BASIC only lets you use single letter variable names. There are only 26
places to store values in the program, and the code is practically unreadable because of it.
This time, instead of TiBASIC, I’ll be writing this project in C++
using the toolchain provided by Matt Waltz and the other contributors to the CE Programming project.
C is really nice compared to BASIC. It’s compiled so the programs run directly on the CPU, code is
a lot easier to document and reuse, and I can poke around in memory basically anywhere I want,
including writing to every pixel. Best of all, I can write and test code right on my
computer through emulation, which has the added bonus of being faster than the real hardware.
After taking a look at some of the sample code in the toolchain repository, I figured the best
way to get started would be to rewrite the original version of the program, so I more or
less ported it over. Here’s a fun fact, did you know that these calculators have a 32,768 color
320x240 display? That’s on par with the Super Nintendo, and a whole lot better than the TiBASIC
graphical specs. Even this recreation of the old program already looks a whole lot better than the
original. As for speed? Well, the old version took just under 6 hours to draw this image.
The new version takes about 3 minutes. That’s nearly 400 pixels per second or an 80x speedup.
It’s a great start, but this video would be pretty short if I just stopped here. We can go faster.
At its core, raytracing is really just a bunch of math to model how light travels through a scene,
and to do all that rendering math in my code, I’m using floating point numbers,
which are like scientific notation but for binary numbers. Floats are a lot better than integers for
general purpose computation since they can work on really big numbers and really small numbers,
but the tradeoff is that generally math operations on floating point numbers are a lot
slower than integers, and on the Z80 processor inside the TI84 this is definitely the case.
There aren’t any hardware optimizations for floats like you would find on any
modern processor or GPU, which means that even for something as simple as adding two
numbers together: integers take 2 CPU cycles, and floats take a few thousand.
One of the reasons floats are so expensive is because you have to account for the fact that the
decimal in two floating point numbers might not be in the same place. Before we can add two numbers,
we have to make sure all their digits are lined up properly, but if we can be sure that the decimals
are always in the same spot, we can treat these numbers like integers. This approach is called
Fixed Point, since instead of floating around, the decimal point is always in the same place. A great
example is US Dollars. As long as we assume you can’t have a fraction of a cent, you can write
out any amount of money as an integer number of cents rather than a fractional number of dollars.
Fun fact time again! I actually lied about the processor that these calculators use. I said
that they use the Zilog Z80, which is an 8-bit processor from all the way back in 1974. Well,
that’s partially true. Up to and including the Plus C Silver Edition, everything in the TI84 line
was using a groovy 70s-era architecture, which, fair enough, for what they do these calculators
don’t need to be all that powerful. But, the model I’m using here, the 2015 skinny-legend TI84 Plus
CE finally gave the CPU an update. Now, it has a Zilog eZ80 under the hood. A whole lot faster,
plus a better instruction set AND 24-bit wide registers. Yes, you heard me right, 24-bit,
not something more typical like 16, 32 or 64, this thing is 24-bit. Your guess is as good as mine.
Anyways, I can make my fixed point numbers have 24 bits, 12 below the decimal, and 12 above.
This can represent numbers as large as 2047.999, and as small as .0002, not perfectly precise,
but good enough for graphics. Plus, these fixed point numbers are way fast. I was able
to render one test scene in less than a quarter of the time! An improvement
to be sure, but still, not good enough… Besides speed, one thing I really wanted to
improve in this raytracer was the complexity of the scenes it draws. The first one had
a reflective sphere, a tile floor, and a single shadow, all centered in the view.
Simple scenes like that can offer some mathematical shortcuts, but this time I
want to make things more complicated. So let's add a few walls, a ceiling, and another sphere.
The whole idea for ray-tracing is that we take some sight ray, and figure out what objects in the
scene it intersects with. Depending on the answer, we shade that pixel in the output a certain way.
But, if we have an efficient way to find out if a ray intersects something in the scene,
we can use this information to figure out other ways light is traveling.
For example, if there’s something in the way of a ray pointing from a surface to a light source,
we know that that point will be in shadow. There are two other properties of light that we’ll
also want to model here. The first is Lambert’s Law: light intensity on an object falls off the
more a surface points away from the light source. And the second is the inverse-square law, light
intensity falls off the further away you get from the source. The two of these have to do with the
angular size of the surface from the perspective of the light source. If we assume the lamp gives
out equal energy in all directions, the bigger the solid angle our surface takes up, the more
light it’s gonna reflect. Turn the surface away from the lamp, and the solid angle gets smaller.
Move the object twice as far away, and now it takes 4 times as much area to take up the same
angular space. In both cases, the surface will get less light, and appear dimmer.
When you put these two effects together with the shadows, things already look a whole lot better.
But, computer renders have this awkward tendency to look a little too clean,
and you can’t get much cleaner than a room with two perfect spheres and solid-color planes, so,
let’s look into texturing. All we have to do is to take an existing picture, and to project it
down onto an object in the scene when rendering. In our case, whenever we’re rendering a pixel on
the floor, we’ll want to use the color from a pixel in some image like these hardwood planks.
Since the floor is an axis-aligned plane, that task is pretty simple:
just take the x and z coordinates where we hit, use those to pick a pixel in the
texture, and then interpolate between any neighboring pixels to hide any jagged edges.
But here’s something else that doesn’t look too great. If you look closely at any surface,
you’ll see bands of color rather than a smooth gradient. As unnecessarily good as the display
hardware in these calculators is, we are still bound by 15-bit color. In the red, green, and
blue channels there are only 32 different levels of brightness to choose from, and so in areas
where color is changing slowly, the transition between the two closest colors stands out.
Thankfully, dithering is an easy fix to this problem. By mixing together a few pixels of each
color along a transition, we can get the illusion of a bigger palette at the expense of a little
pixel noise. There’s no shortage of dithering algorithms to choose from depending on if you want
to prioritize speed, image quality, or noise, but here, I’m doing arguably the simplest technique.
Taking whatever error there is from the last pixel, and just adding it onto the next one in
the row. Even though there’s not much to it, this technique helps a lot to smooth out the colors.
While we’re on the subject of color, I gotta confess, my renderer still is a little bit
entirely wrong. My code all operates on the assumption that light intensity in the renderer
scales linearly with the RGB value I set the pixels to. But in the real world, that’s not the
case. There’s a great MinutePhysics video talking about why this is, but long story short, standard
RGB values are grouped so that darker colors are very close together in terms of brightness,
but brighter colors are more spread out. If we’re assuming a linear mapping from actual
intensity to sRGB, things are going to look darker than they should. There is a conversion
function between the two, but, well, that looks hard. So instead, I added it as a lookup table.
The nice thing about the lookup table, is that I can use it to add Tone Mapping for HDR
at no extra cost, by adjusting the values according to this function.
Now the renderer preserves detail in areas that would normally appear too dim,
or so bright that they look washed out. Accurate color representation goes a long
way towards photorealism, but there’s still one glaring issue with this image. Anything
in shadow isn’t just dark, it’s pitch black, and in the real world, you don’t see this very often.
Even if something isn’t being lit directly by the sun, light can bounce off of something
else into the shadow. This is called global illumination, as opposed to direct illumination,
and it shows up all over the place. Under trees, along sidewalks, inside a room on a sunny day.
But compared to direct illumination, global illumination is a lot harder to simulate. Instead
of just one or two light sources, the illumination an object receives comes in from all directions.
Professional renderers generally use random sampling, since if you take the
average of enough light samples, you’ll eventually get close to the right answer,
but, since I’m not in the mood to wait a week for the render, I’m gonna use a different technique
called radiosity. The radiosity algorithm computes a lightmap for each object in the scene, which
is like a texture that keeps track of how much light is hitting the object at a couple points.
First, we get the direct illumination for every patch, and then we simulate light
bouncing around the scene by adding up the light contributed to each patch by all the others.
Repeat that enough times to account for the light paths with multiple bounces,
and we’ll eventually converge on a more accurate representation of how the scene is lit.
I still render direct illumination individually for every pixel, since the resolution of the
lightmaps make shadows look a bit chunky. But, I can also turn off the direct illumination
completely and render the scene with just light that’s reflected off the walls. One effect I
think helps with photorealism is color bleeding. You can see in the global image that the red wall
contributes a bit of red light to everything else, and so does the blue wall and the wood floor.
When we look in the fully rendered image, this means that the spheres have a bit of a colored
tint on the sides. And this is something we can also see in the real world. If a brightly colored
object like this box reflects a lot of light, things around it will have a similar color.
It’s a subtle effect, but once you know about it, you’ll see it everywhere…
For one last cherry on top, it wouldn’t be a raytracing demo without a mirror ball,
so I threw together some code to reflect a sight ray off of the one sphere, and then
render the scene from that direction. This is one of the things raytracing gives almost for free.
We already have all the code necessary to render sight rays from the camera,
all we have to do now is change the direction of the ray.
And, there you have it. A raytraced scene with reflection, global illumination, HDR, gamma
correct rendering, point lights and textures, all still on a TI84 calculator, in case you forgot.
For fun, I tried my best to recreate the scene in Blender and did a quick render on my computer to
compare. For the most part the calculator isn’t that far off. I noticed in some areas of the
image, the Blender illumination is a bit different from the calculator’s, notice how the mirror ball
is dimmer here, and the back wall has a bit more red reflecting from the side. It’s not perfect,
but remember this is still night and day compared to the graphics this calculator normally draws.
On my computer, this render took a couple minutes to perform using a modern multicore
processor, but, how long does it take for the calculator? Get ready for the quickness...
14 minutes. Granted, the calculator is performing a simpler algorithm than Blender, but time-wise,
we’re in the same ballpark. The original version took 6 hours for a lot less. This version runs at
roughly 90 pixels per second which is almost 20 times faster. Altogether we’ve got better
results in less time. I’d count that as a win. As always, there are still ways it could be
improved. When writing some of the more complex fixed point functions, I got a little lazy
and just used the floating point library instead of writing it myself in assembly.
I mean, I wrote my own fast multiplication and squaring routines in ASM since those get used
pretty often, but for something like square roots, I only use it a few times per pixel, so garbage
like this is good enough for me. There are a few redundant calculations my renderer performs which
could probably be stored and reused, mostly during radiosity. Cutting out all this work
would make things faster, but, there are memory limits to consider, and what I have works for now.
It’d be really cool to add support for triangle meshes to render more complex shapes, but that
comes at the cost of way more compute time per pixel. Other forms of lighting like directional
light and surface lights would also make for some interesting renders, as would refractive
materials like glass. I might get around to it in the future and give this video a sequel (no…
three-quel? idk) so uh, subscribe or whatever. But, if you’re the impatient type, the source
code is all going up on Github, and you’re more than welcome to add these or other features.
I’ve already spent enough time on this as is. So, in the interest of getting this video out,
I’m gonna have to be happy with what I’ve written. Ti84 calculators can now do raytracing faster,
and with more photorealistic results. I don’t know about you, but I for one am excited that
we’re now one step closer to real-time raytraced Minecraft on a calculator...