Humans spend much of their lives searching for meaning.
Coders spend much of their time searching through strings.
While my programming prohibits me from assisting you in your first search …
I am here to help you with the latter.
Regular Expressions - regex for short -
is one of the OG ways to search for patterns in strings.
Python appreciates their importance,
and has provided you with the ‘R-E’ module for all your searching needs.
From carets to dollar signs and everything in between,
your search is over,
because our regex expo begins …
now.
A regular expression is a cryptic-looking string that describes a search pattern.
To save time and oxygen, people say “regex” for short.
Here are a few regular expressions to set the stage.
If you feel a sense of confusion, then keep watching.
If you do not feel a sense of confusion, then keep watching…
at a faster speed.
Each of these regular expressions describes a way to search a string for a pattern.
Some parts of a regex, like the caret and dollar sign,
describe a position.
Other parts like the ‘\W’ and “a-t” in square brackets,
describe a set of characters.
And the plus sign, asterisk, and 9 in braces
describe a quantifier -
a way to specify the number of character occurrences.
Let us review some frequently used patterns in regular expressions.
The regular expression F-r-a-n
will look through strings for these 4 letters in this exact order.
So it would match both Francois and Nick Francesco.
A caret is used to match the start of a string.
So “^Fran” would only match those names that start with F-r-a-n.
This regex would only match Francois.
To match the end of a string, use a $ sign.
Using this, we can find a regex that will match all names that end with the letter ‘i’.
This regular expression would only match Sagun Khatri.
You can remember the difference between a caret and dollar sign by thinking of vegetable farmers.
They start with carrots …
something happens …
and then they profit!
You also need a way to describe sequences of letters, digits, and other characters.
To match a numerical digit, use \d.
This regex will scan each string and look for a single numerical digit.
In this example, it only matches a single name -
the only name containing a digit.
On the other hand, THIS regex would not match any of the names
because there is no name with 2 consecutive numerical digits.
To match a whitespace, use \s.
This regex will match any name containing a space.
It will only match these names.
The \w sequence will match any letter -
lower or upper case - any digit, or an underscore.
The w stands for “word” character.
This regex will match every name, because every name contains one of these characters.
But what about THIS regular expression?
This will match any name that begins with 4 consecutive word characters followed by a whitespace.
This regex will only match the two people with a 4 letter first name.
Another essential ingredient in regular expressions are quantifiers.
These are ways to specify the number of copies of an expression.
We will use these four quantifiers in today’s examples.
Do keep in mind there are many other quantifiers.
For example,
this regular expression would scan a string looking for two consecutive vowels.
There is only one name that fits this description.
The list of characters inside square brackets is a way to create a custom character set.
This will only match one of these 5 letters.
And the number inside braces is a quantifier that specifies the number of repetitions of the expression before it.
This regex searches for any names that are exactly 7 consecutive word characters -
no more, no less.
And what have we here?
If you did not include the caret and dollar sign, the regular expression would behave very differently.
This regex searches for any name that contains 7 consecutive word characters.
There are FOUR names that match this criteria.
Now start your Python consoles,
because the regular expression race is about to begin.
To use regular expressions with Python, import the ‘R-E’ module.
After hours of research we were unable to find what ‘R-E’ stands for.
To explore the ‘R-E’ module, we will first create a list of names.
There are several functions you can use to search a string for a pattern.
The first we will test is the aptly named - search function.
Let us search through our list of names to find people with a first and last name only.
We will skip people with 3 or more names, or people with a single name.
First we need a regular expression.
The string should start with a positive number of word characters….
have a space…
then end with a positive number of word characters.
We will loop through the names..
.. scan each name with our regex ..
.. and print those names with a match.
Run…
We see our regex does work.
But our regular expression is quite strict.
If for some reason Finn had two white spaces separating the first & last name,
and Ron had three..
.. and we ran once more ..
.. then the regex would no longer match these strings.
This is because we built our expression to require a single whitespace between the first and last name.
If instead we look for one or more whitespace characters..
And run..
We will catch such cases..
In our code, we are using the return value of the search function in a boolean context.
We are not making full use of the returned object.
We are only checking if it is None or an object that evaluates to True.
But the search function DOES return something interesting.
Let us see just how interesting.
We will run this once more,
but this time we will print the serialized object instead of the name.
Run..
We see that the search function returns a match object.
Match objects contain a wealth of information about the result of a regex string search.
I shall demonstrate..
This time,
we will search each name for a sequence of word characters that starts with a capital C.
Once again we iterate over our list of names.
We get a handle on the match object..
And if there is a match, we will display the details.
For each name, we will display the start and end values in the match object..
Run..
The start and end values are the indices that identify the part of the string where a match occurred.
For convenience, you can retrieve both indices with a single call to the span() method.
Run..
If you want to display the substring that matched the regex, you call the group() method.
If you would like to learn about new videos, code samples, or more Python goodies,
then use your browser to visit this URL.
This is the portal that leads you to a tastefully infrequent newsletter
by us,
from us,
to you,
on Python...
Let us explore the different ways you can access search results more deeply.
Here is a new collection of names.
One distinguishing example is a string that contains a non word character.
Suppose we want to scan each string to see if it matches a regex that is testing for a first name and last name.
Run..
This regex fulfills its purpose.
But what if we wanted to access the first and last names?
For this, we are going to use groups.
A group is a way to identify - and even name - a part of the regex pattern.
To create a group,
you put parentheses around a section of the regular expression.
This regex now has two groups.
Now we can call the group() method on the match object to access parts of the regex search.
If you do not pass a value or if you pass in 0..
Then the returned value is the part of the string where a match ocurred.
You can access the first group by passing in 1,
and you can access the second group by passing in 2.
Run..
As we just saw,
you can access each matching group of characters by its index, starting with 1.
For flexibility and readability, you can give each group a name.
To do this,
you start the sequence inside parentheses with ‘?P’.
Next,
write the name of the group inside angle brackets.
We will call the first group “fn”
and the last group “ln ”.
Whenever you have a match, you can now access each group by name.
Run..
That is satisfactory.
Notice that the final name in our list has avoided detection by all of our regular expressions.
Let us learn a few new tricks in order to remedy this.
The challenge is the exclamation mark.
This is not a word character, so the ‘\w’ character set does not match it.
We can, however, create our own custom character set.
To do this,
list the characters and character sets inside square brackets.
This expression will match any string that is a positive number of lowercase letters,
uppercase letters, or an exclamation mark.
Run..
Final name detected..
There are several other functions in the ‘R-E’ module that we should see.
For the most part they operate like the search function.
Example.
Often times a regex will match several parts of a string.
The search function just returns the first match.
But to find them all, use the oddly named findall function.
Let us scan each name for consecutive sequences of lower case letters.
And run..
Notice how the function returns a list of strings, not match objects.
If you would prefer access to each match object,
call the “find iter” function and loop over the iterator.
Run..
and done..
Before you are permitted to leave, there are a few other functions I want to quickly introduce.
First, the match function.
This will use the regex to test a string for a match from the beginning.
For example, consider these strings.
Suppose you want to determine which ones begin with HTTP or HTTPS.
Here is a regex that uses the question mark quantifier.
Remember,
this says the previous character can occur 0 or 1 time.
Also notice that we did NOT use the caret to indicate the start of a string.
We now search each string using this regex.
Run..
We see that only the first two strings matched the pattern.
The last string was there to confirm that while there is a match at the end,
the match function only searches from the beginning.
Next,
the fullmatch function is used to not just scan a string for a match,
but check to see if the entire string fits the regex.
The pattern we will use is far from ideal,
but it will check that the string starts with HTTP or HTTPS,
has a WWW, dot, a sequence of word characters, dot, then ends with ORG or COM.
Run..
The first two strings do fully match the regex.
There are numerous other functions, constants, and other features in Python’s regular expression module.
It is good practice to become comfortable with Python’s reference documentation.
The split function can be used to partition a string into pieces
using a regular expression to define the splitting points.
The sub function can perform a search-and-replace action.
And there is more,
much more.
Regular expressions are a well-established and widely implemented tool.
The regex pattern syntax may be intimidating,
but it is worth your time to learn it.
From grep in the early 70s
to the R-E module today,
regular expressions are here to stay.
That is all I have to say…
One more thing I must relay, even though it is quite cliché,
please subscribe.. without delay.