Now I'm going to define the parser, and I want it to have this signature.
I'm going to define a function pare, which takes a symbol,
saying I want to parse something like an expression.
It takes text that we're going to parse, and it takes the grammar
that defines that symbol and all the other symbols.
I want it to return--when we had regular expressions we looked at returning a set of remainders.
I'm going to have this return a single remainder.
The idea is that we want the author of the grammar to have some control over what's going on
and make this grammar be unambiguous so that there is a single result for each parse
rather than having to keep a set of results.
The idea is that for each symbol we're going to consider the alternatives in left-to-right order.
If we're asked to parse an expression, we'll say can we parse this alternative.
If we can, if we can parse in succession a term and then a plus or minus
then an expression and then we'll commit to that.
We'll say that's the result we're going to return.
That's the single remainder, and we're not going to even explore the other alternatives.
Because we have this left-to-right choice, that's why we decide to write the expressions in this order.
We don't write them in the opposite order because if we wrote the rule this way
and we were trying to parse the expression a + 3,
we'd be asked to parse this, we'd try to parse a term, we'd say, yes, a is a term,
and then we'd stop, and we'd say that's the end.
We wouldn't even consider parsing off the term plus the expression.
This is no good.
It's up to the author of the grammar to write rules in the correct order
so that the longest thing gets tried first.
Our regular expression functions are what is known as recognizers.
They told us--yes or no--is a string part of a language.
Here what we're doing is different. It's a parser.
It tells you--yes and no--is it part of the language, but then it also gives you
an internal structure, a tree structure of the parts of the expression.
Here it would be a + 3.
If we had m * x + b,
then that would parse into a structure that had m * x + b.
Here I said we were a remainder, but actually I want to return a two element tuple
of the tree followed by the remainder.
Here's what we're going for.
If I asked to parse the expression a * x with the grammar G,
I want to get back this tree structure. It looks kind of complicated.
All it's really saying is that we have an a in the first element, then the multiply sign,
and then an x in the third element, and there's no remainder.
We parse the whole string.
That's what it says, but there's all these intermediate parts here because that's the way the grammar is defined.
I should say here that this is a tree, remainder result.
We're going to use the convention that Fail corresponds to a value of None.
Let's think then about what we have to do to be able to parse a text.
I can think of four cases that we have to deal with.
One, we want to be able to parse an expression or a symbol like the expression keyword.
To do that we've got to be able to look that up in the grammar G.
We've also got to be able to parse a regular expression, plus or minus,
and we've seen how to do that before, so we reduce that to a solved problem.
We're going to have to be able to parse a tuple of alternatives.
Here one alternative or another alternative done in left-to-right order.
Finally, we're going to have to be able to parse a list of atoms representing a sequence--1, 2, 3.
Now, I'm going to tell you the plan for how we're going to implement this as code
within the function parse.
So this first case I'm going to handle with a subroutine called "parse_atom."
This is an atomic expression. We should be able to handle that.
The second part, the regular expression, is a type of atom so it occurs within the parse_atom routine.
We're going to define a variable called "tokenizer" to help us do that.
Then we're going to use the built-in re.match along with the tokenizer
to recognize those regular expressions.
Then for the alternatives, a tuple of alternatives is what you get when you look up
the symbol, like exp, in the grammar.
I could have had a separate function called parse_alternatives
but instead I just decided to have that be part of parse_atom,
because it was so simple it was just an immediate step
to go from the symbol to the list of alternatives.
Finally, to parse this sequence of atoms, I have a subroutine or function called "parse_sequence."