This is the music video to Toxic being played entirely off a 3.5” 1.44MB floppy. Seriously. You
can just take this disk, pop it into a computer running Windows, run the program, and watch the
ever-iconic music video get played out right in your terminal. Really. But, I pretend to hear you
ask, how does one play a video from a floppy disk. Well, even if it is old, a floppy is still a
storage device. All you need to do is take a video, and copy it onto the disk. Well,
so long as that video is smaller than 1.44MB. And therein lies the problem. Floppy disks are small,
and video is, not. My videos now generally render at around 20kbps, which means that a floppy
couldn’t even hold a second of HD video, let alone three minutes. That’s at 60FPS 1080p though,
and as you might expect, lower quality implies lower bitrate implies a smaller file size.
When it comes to video encoding, there are a bunch of dials to control resolution,
framerate, audio sample rate, the number of audio channels, what codec to use, and so on.
Turn these knobs all down, and we get a video that easily fits on a floppy, but it looks like this.
Redditor RichB93 from the LGR subreddit used these encoding settings, along with
a better disk formatting scheme, and was able to encode an entire five minute episode of LGR
down to a file small enough to be played off of a floppy disk. Clint even made a Blerb about it,
good stuff. Turns out, that’s all you need to fit a video onto a floppy.
Except, well, I dunno, that feels too easy in my opinion. The quality isn’t bad,
but it’s all dependent on modern video codecs like H.265, and if the computer you stick it in
doesn’t have the hardware or software to decode that, the video file is as good as useless.
I thought maybe I could try to include a playback program on the disk, but most video players like
VLC here are already way too big to fit on a floppy, even before we throw the video file in.
So, how do we actually get a video onto a floppy that we can playback on almost any computer? We’re
going to want something compact, efficient to decode, and portable. And immediately my thoughts
went to text. We can take every frame of the video and turn it into ASCII art, and that’s perfect.
Text is relatively easy to compress, almost every computer has some type of terminal output,
and as a rule, using a terminal-based interface over a graphical one automatically increases your
hacker cred by at least a few points. Converting video into text isn’t that
new of a concept. In fact, there’s a really cool demo from 1997 called BB,
which plays a procedurally generated animation using only text in your terminal. Neat stuff, but
by far the coolest thing about the BB project, is that they released their image to text conversion
tools under the name aalib. That means that converting a video into ASCII art is as simple
as loading each frame in from the mp4, and pushing them all through aalib. But, something spoke to
me. Maybe it was the naive programmer’s desire to “build everything all by myself,” maybe it was
just me skipping ahead a bit and realizing that a more complex character set would kill any attempts
at compression we might need to use. Whatever the case, I decided not to use aalib, and instead,
make the video to text conversion all by myself. So… How does one make a video to ASCII converter?
Well, it’s important to remember that a video is just a sequence of frames. And each frame is a set
of three monochrome images. And each image is a 2D grid of intensity values, one for each pixel.
In ASCII art, for our purposes anyways, each character is essentially one pixel,
so our first step is to scale down each frame to the resolution the text output is going to have.
Then, for each pixel, we need to decide what character we’re going to use to represent that
color, and here is where things get interesting. You’ve probably already noticed from some of the
ASCII art examples I’ve shown, some characters like spaces or commas work well at representing
dark areas, since the character doesn’t have a lot of white pixels, and similarly, hashes and
at signs are good for bright colors since they’re mostly filled in with white pixels. But what about
everything else? There are a lot of colors between black and white. Well, rule #1, for now, I’m not
going to be dealing with RGB color. Don’t get me wrong, I could totally do it, but color makes
things more complicated, it’ll take up more space on the floppy, and it makes my program much harder
to port between Linux and Windows, so for now, we’re just gonna change every frame to grayscale.
Greyscale ends up being nice to work in anyhow. Each color exists on a brightness scale from zero
to one, and we can pick which character to use based on which range it falls in.
To get the ranges, I made a program to print out all the characters to the terminal, took
their average brightness, plotted them out, and selected seven characters whose brightnesses were
pretty evenly spaced apart. And, when we run the video through this process, this is what we get.
It’s at least recognizable, but it does come with a slight problem. We only have seven brightness
levels here, which means that we lose detail between the levels, especially on the darkest
one. But this is a problem dithering can easily fix. If pixels alternate between two brightness
levels in a checkerboard type pattern, we can give the illusion of more colors than we have,
since our eyes blend the colors together. I’m using Floyd-Steinberg dithering here, which
distributes any error accumulated when picking the color of a pixel into its neighbors. It’s a
simple algorithm, but this step single handedly makes our text render of the video frames look
a lot better, especially in the darker frames. As it turns out, a scaling step, a greyscale step,
and a dithering step are all you need to convert a frame of video into a block of ASCII text,
and those steps are all easy enough to work into a short python script.
Before I move onto the next section, I should probably answer the question I’m sure a number
of you are wondering… “Why toxic?” Originally the idea was just to use ANY music video,
since I wanted to use something recognizable that was under 4 minutes. Originally, I was
thinking of using the Rickroll video, or Take on Me, but those videos don’t have a lot of contrast.
When you try to convert them to text, they come out looking mostly gray, and it’s hard to make out
much in terms of details. On the complete opposite side of this problem is the Bad Apple video, which
barring some anti-aliasing is entirely monochrome. Pretty boring for this project, in my opinion,
plus just about every possible port of this has been made by programmers far more talented than
me. Which brings us to the music video for Toxic. Turns out, this video is perfect for this kind of
text rendering. Almost all the scenes have strong contrast between bright lights and dark shadows,
and there are lots of closeups on peoples’ faces to show off the detail I’m able to preserve.
Plus, I just like the song, ok? Free Britney. I did tease color video to text a minute ago,
so I should probably talk about that a bit. First off, how is this even possible? Most terminals,
at least in linux-land, support a set of formatting codes called the ANSI escape codes.
These are a sequence of bytes that appear in the output stream. When the terminal sees them,
it goes “oh, this must be formatting information” and it follows the encoded
command to move the cursor, or switch the text color, or start writing in bold text.
When it comes to setting colors, you get to change the foreground color and the background color, and
depending on how advanced your terminal is, you have either 8, 16, or 256 colors to choose from.
The cool thing about the 256 color set, is that the standard palette includes a set of RGB colors
with 6 brightness levels per channel. Generating color video is as simple as printing a bunch of
spaces, and formatting the background to whatever terminal color is the closest to our pixel.
The result is recognizable albeit a bit pixelated. But, if we’re able to blend black to white using
text, we can blend between the colors we’re given to get an even better color depth. I wrote a brute
force program to try every foreground, background, and character combination in the ANSI palette,
and determine the best combination for each RGB color from 0-255. In a sense,
I generated a much more advanced 3D version of the grayscale bounds we talked about before.
The result being, a table you can give any RGB color, and it tells you the exact foreground,
background and character combination to match that color the closest in an ANSI terminal.
And, it works. I’m able to draw absolutely beautiful color images in my terminal, still using
nothing but colored text. The only drawback for video, is that the Windows terminal isn’t great
at parsing these ANSI codes, and with multiple formatting sequences per character, it’s no longer
able to keep up with our framerate. But that’s something I can fix through the magic of editing.
Alright, that’s been enough tangents, let’s get down to the real business of this video.
How do we get this text encoding of Toxic onto a Floppy disk? If your answer was drag and drop
the text onto the floppy, you are unfortunately incorrect. While the source video is 18MB, the
text encoding of Toxic, rendered at 160 columns like I’ve been showing, comes out to just about
45MB. That’s about 30 floppies worth of data. So, first things first. Let’s drop our resolution down
by half in each dimension. I needed to do this anyways to match the width of the old 80 column
terminal in Windows, but that step alone gets us to 11MB, which is now only eight-ish floppies,
but still more than one. To bring this down further, we’ll need some form of compression.
I want this thing to be as simple as inserting the floppy and clicking the program, so ZIP
archives aren’t gonna cut it. Instead, I tried a few different methods for compressing images
with small color palettes, like the classic Run Length Encoding. The idea is in most drawn images,
you’ll find long horizontal strips of the exact same color, so, what’s easier to write down? 50
individual blue pixels, or one blue pixel times 50. In our case, since we only have 7 characters,
there are really only 3 bits of significant data per byte, and we can use the spare 5 bits to
contain the run length for each character. In the optimal case, one byte can stand for 32 pixels.
In practice, it’s not even close. The dithering patterns kill the RLE here,
since on average, each run only covers two or three characters before we need to start another.
That didn’t work, so here’s another approach more suited for text: Huffman Coding. In this scheme,
we build up a decision tree to decode bits from the compressed output.
Most importantly though, different characters can have different encoding lengths and we can exploit
this to give the most common characters the shortest encodings. It’s a brilliant technique,
but, in the case where letters are used fairly evenly, the results are only as
good as the 3-bit encoding we started with. One thing I started to play around with was
constructing a Markov matrix of the transitions between characters as we print out a line.
Basically, what are the probabilities of one character following another in our image?
As it turns out for each character, almost always, it’s highly likely that we stay on
that character or go to a character of similar brightness due to dithering.
Given the previous character we can rank each choice of what comes next based on its likelihood,
and I ended up encoding the image this way. Now, instead of one representing brightness level one,
it means, pick the most likely next character. Two, means pick the second most likely step and
so on. In this encoding, most of the pixels turn into ones or twos, and since one is way more
common than the other numbers, it encodes nicely on a Huffman tree as a single bit.
Theoretically, this algorithm can produce an 8x compression rate, and in practice it gets about
5x. This method gets us all the way down to 2MB, and the egotist in me would like to point out,
that’s actually better than ZIP compression. 2MB is still too big for a formatted floppy, but
I feel like I’ve worked enough here, so I’m gonna cut one last corner and switch the playback frame
rate from 30 FPS to 15. I’m sure there are going to be a bunch of 240FPS pro gamers in the comments
saying otherwise, but honestly, when rendering to text, I can barely tell the difference, and
it cuts our data in half. With that, the entire 18MB video rendered in text, can be compressed
all the way down to just 1.1MB of data. Now we just have to get it off of the floppy.
This step just involves a C program to extract and print our compressed text. Writing it was
mostly straightforward, though getting it to run well on older hardware required a few fixes. To
improve speed, I needed to cut my program down to just one printf call per frame. The older terminal
also had a hard time scrolling all the text my program put out, so now instead of starting a new
line when drawing a frame, I just put the cursor back at the top left and draw over the last one.
With those beautification steps in place, let’s see it. Here I have my TC1100 tablet running
Windows XP and the Toxic floppy. All we have to do is pop it in, and run the program. It takes
a little time to load the program in, but once it’s done, we see the Toxic video. Pretty slick,
if I do say so myself. There’s no sound, since I was focusing mostly on the video for this project,
but I’m sure with the right bitrate, and maybe some more compression on the whole disk,
you could probably squeeze in some audio data to get the full Toxic experience.
The cool thing is, since this is a 32 bit executable, it should be able to run on Windows
versions as far back as Windows 95. Testing in a virtual machine, I can confirm that,
yes it definitely still runs on Windows 95. But, the output does have a hard time
keeping up. Even so, that is still definitely ASCII Spears on the screen, so I’ll take it.
My original goal with this project was to be able to fit a reasonable length video,
and the software to play it back all on a single floppy disk, and that’s exactly what I have in my
hand here. All it took was converting the video into a stream of ASCII characters,
then figuring out a way to compress all that text. I compiled it here for Windows, but this video
should be displayable on just about anything with an 80-column terminal. If anyone wants to make a
port to some really obscure hardware, or make some improvements to my program, I’d love to see it.
Like always, I’ll have a link in the description to the GitHub repo with the code necessary to
textify and floppify any video you want, so be sure to check that out. Anyways, there now exists
a floppy disk port of the Toxic video. I’m not sure who was asking for this, but you're welcome.