A lone mysterious text teases your curiosity. You’re in a dark room, let’s say in a
museum basement. No, even better, somewhere more adventurous: a cave, yeah. Your excitement
fades to baffled wonder as your brain starts to hum with the tedium of cracking this text,
a tedium that calls not for the brazen adventuring of an Indiana Jones but the careful testing
of a Turing or Champollion. You don’t know the language of this text. You don’t even
know what words it encodes. You can’t even start to pronounce its symbols. You’re not
even sure this is writing.
At this moment of confusion, if your ear is free, fellow language traveler and budding
decipherer, let me hold out my hand and offer a morsel of hope. What if I told you that
you can know what these symbols are without ever needing to know how to read them?
And what if I also told you that all you needed to do this was to know how to count?
First, pluck out the basic writing types you met in Thoth’s Pill. What, you haven’t
seen that? No way! Come on, let’s go watch it first and then come back here and pluck
out the basic types of writing types you met in Thoth's Pill: consonant and vowel symbols,
syllable symbols and word symbols. Now let’s start counting.
Take a string in English (just a bit of written text) and count the number of distinct graphemes
you find in it.
Ok, but longer is better here. More chances for all the signs in the writing system to
show up, including those less common ones: your q’s, your x’s, your z’s. So let’s
count the signs in all of Shakespeare’s Sonnets. I'm counting everything but X.
So keep taking more and more bigger and bigger samples in English, and you’ll end up counting
the 26 letters of our alphabet.
Do the same for Russian, and I hear you’ll count 33. Try it with Hawai’ian, and you’ll
only find 13.
Now reach for Japanese texts written entirely in syllables. You’ll count 46 hiragana.
Cherokee has 85 syllables. Wow, now here’s a jump up to Hittite with over 500 symbols,
Maya with more than 800 and Chinese with thousands upon thousands.
So we end up with alphabets on one end, syllabaries in the middle and logographic or logophonetic
scripts on the far side.
But now you’re facing a new script. Totally unknown to you. And you count the symbols.
It has hundreds. What kind of script do you think it is? What if it has 20? What if it
has 50 or 60?
We’re using this simple algorithm to make an educated guess about an unknown script.
This thing's referenced conceptually in papers about decipherment, but I don’t know what
it’s called. Robinson points us back to Archibald Sayce, a man who loved himself some
Assyrian. So do we call it the Sayce test? The Archibald heuristic? Pulling a Sayce?
Yeah, pulling a Sayce. Pulling a Sayce does come with challenges and drawbacks, too.
First, there’s the problem that you still can’t read the text. True, but before you
take all of the wind out of my sails, please celebrate what this handy tool
can do for you!
Second, scripts can emphasize things, but they’re not really purely phonetic or pure
logographs. I mean, think about the mix of letters, numbers, punctuation and even ideographs
you find all around you, and that's just in English.
Perhaps the biggest problem is how to identify distinct symbols. If you were trying to decipher
the Latin alphabet, could you tell that all of these are the “same" symbol? How? What
about capital letters and lowercase letters? Are accent marks distinct letters? How do
we count ligatures, linked, scripted together, mashed-together characters? Or the component
consonant-vowel pieces of a single Indic syllable? Or the sound plus meaning pieces inside of
a single Chinese character?
We have explanations for these, and maybe they’re good enough for the scripts we already
know, but what about trying to read an undeciphered script like Rongorongo? Are these strange
people-looking things two flipped variants of a single sign
or are they two different signs?
True, this little algorithm doesn’t pull us out of the dark. But it gives us a starting
point. We can identify script types, we can list symbols, we can then use those lists
to identify the symbols when they show up in other texts. With a smile on our face.
Because, even though this script is still unknown to us, we’ve devised a clever way
to reach a strange familiarity with it, a big step on our way to decipherment.
I took a little time to play around with and put together some simple code that runs this
test on any piece of text you give it. It counts the number of different symbols that
are in that text, then uses those cutoff numbers from earlier to guess what kind of
writing system this is.
So, let’s try it for English. The English alphabet. Here's the 26 English letters.
We'll pass it down through here and run this thing… it tells us that English is an alphabet!
Who'd have thought?
Same thing but with some random sentences. Still an alphabet.
Let’s go for Hiragana down here.
Run it. And... it’s syllables!
What about Hawaiian? If you too have ever shared the misfortune of leaving Hawai’i,
you may have heard this. And it’s in an alphabet.
Then there’s Kanji. Oh, yes. A whole list of them. And they’re word symbols.
Well, that’s super nerdy, but thanks for letting me play with language a bit here.
It’s kind of nice doing a more off the cuff video like this. That’s a thing, right?
Off the cuff? It sounded wrong for some reason. Well, leave comments and subscribe, and encourage
me to come back again! Ok, bye.