Hello,
This video is an advanced look at the topic of filename substitution using wildcards in the Linux shell.
Specifically I will show you how to distinguish between regular files and directories in wildcard matching
and how to match filenames recursively inside sub-directories.
I will also introduce the find command.
Shell filename expansion, also called globbing, is a great feature
that allows us to match multiple filenames using simple meta-characters such as ‘*’, ‘?’ and ‘[ ]’.
These meta-characters match both regular filenames as well as directory names
and we use them within a specific directory location.
Our ability to distinguish between regular files and directories is limited with shell globbing.
Let me illustrate that with an example...
Here, my present working directory has a mix of regular files and directories.
Now I will use an “a*” glob to print only filenames that start with a lower-case ‘a’.
As expected, both regular files as well as one directory were matched.
If I want to match only directories, I can add a ‘/’ at the end of the glob
and that works ok.
This technique can be useful in some situations.
Unfortunately, there is no simple way to do the opposite action,
to match only regular files, while excluding directories.
The second limitation of shell globbing stems from the fact that each wildcard pattern
normally applies to a specific location,
either present working directory or another location specified as a pathname.
Admittedly, this is what we want in most cases but it would be nice to have an option
to match across sub-directories recursively.
Let’s have a look at my filesystem structure again, this time in more detail.
Notice that there are filenames starting with ‘a’ inside ‘apples’ and ‘pears’ directories,
yet they were not matched earlier.
Current version of the Linux shell does not have a straightforward way
to wildcard-match filenames recursively, inside sub-directories.
If we want to match filenames recursively, we rely on the find command.
In a most basic scenario, we can use find command to search for a specific filename,
without wildcards, for example ‘ambrosia’...
This command has 4 tokens:
find command, start location to be searched, which in this case is my present working directory,
the search type and finally, the filename.
You can provide more than one search location in form of directory names or pathnames,
or you can skip that token if you are searching in your present working directory, which is the default.
Here is the command again, this time without an explicit search location...
Results are the same, but a shorter command is more elegant in my opinion.
The ‘-name’ option accepts expressions that contain wildcard meta-characters,
the same way as in the shell.
However, meta-characters used here are processed internally by the find command.
This is why it is very important that any find command search pattern that contains meta-characters is quoted,
to prevent the shell from expanding it before the find command runs.
Otherwise, the shell may expand the wildcard expression with file names in present directory,
effectively breaking it.
Here is an example which will search present directory for filenames that start with lower-case letter ‘c’.
Note the quotes around the wildcard expression.
Find results are shown as pathnames.
For matches in present working directory, find command prefixes their names with ‘./’
to make the results easier to process later if filenames found are unusual,
for example when they start with a ‘-’.
I will talk about processing of search results in a few moments.
It is worth noting that as with the Linux shell in general, find command matches are case-sensitive.
You can use the '-iname' option if you want the search to be case-insensitive.
This modified command also found the ‘Crispin’ filename, which starts with capital ‘C’.
By default find command provides matches including contents of sub-directories,
but we have control through the ‘-maxdepth’ option as to how deep the find command goes.
'-maxdepth' option needs to be placed before the '-name' option.
As an example, setting '-maxdepth' to 1 will exclude all sub-directories below the starting point.
As I mentioned earlier, we can provide one or more starting points as an additional parameter
after the command name if we want to search outside of current directory.
Here is a command which will search my ‘Documents’ and ‘Downloads’ directories for any ‘pdf’ files.
As it happens, ‘Documents’ and ‘Downloads’ have similar names that start with ‘Do’.
With that in mind, I can shorten the above command by using a ‘Do*’ wildcard
to match both of these names.
Unlike with the search pattern, which needs to be quoted to protect it from shell expansion,
this pattern must not be quoted so shell can expand it to matching directory names.
As I explained a few moments ago, standard wildcard matching can come back with a mix
of regular files and directories, while in specific cases we may want only regular files
or only directories.
The find command has a ‘-type’ option, which allows us to limit the search to specific file types.
For example, I can modify one of the previous searches to only look for directories...
Or only search for regular files...
Something like this cannot be easily done with plain wildcards alone,
but works very well with the find command.
Let’s have a look now at the search results...
The find command provides its output as a list of pathnames printed to the terminal.
This may be all that you want,
but sometimes you may need to perform an action on the matching files.
We have the option to tell the find command to run a specific command
for each of the matched filenames by adding the ‘-exec’ option, followed by the target command.
'-exec' command can be any valid shell command
and we use a pair of ‘{}’ as a placeholder for the matching filename.
For example, I will modify the previous command, and instruct the find command
to copy matching files into ‘/tmp’ directory.
The ‘{}’ placeholder will be dynamically replaced with the found filename.
The ‘;’ marks the end of the -exec command.
The ‘;’ character is a special character to the shell, so it needs to be quoted,
otherwise the shell will intercept it and break the find command.
The command provided is executed one time for each of the matches,
and the ‘{}’ can be used more than once in the exec command if needed.
For example, here is the output of an echo command.
This time the echo command was executed 5 times,
adding its default newline at the end of each line of output.
We have the option to execute the command once, with all matching filenames used within one command.
The only change in such case is replacing the ';' with a ‘+’ character.
The ‘+’ does not need to be quoted as it is not special to the shell.
With this syntax the ‘{}’ represent all matches, as a space-separated list,
and ‘{}’ here can be only use once if the ‘+’ character is used.
The second method may not be compatible with all commands,
as the matching file list must be placed at the end of command.
Or, in some cases, you may need to modify default command syntax.
For example, normal ‘cp’ command syntax will not work.
With the ‘+’ sign used, the curly braces must be at the end of the command.
You can overcome that by using the -t option available with ‘cp’
(and ‘mv’ command for that matter),
which allows us to specify the target directory before the sources.
The ‘-v’ verbose ‘cp’ option documents the operation one line at a time,
but you can take my word for it that the ‘cp’ command ran here only once,
resulting in a more efficient process.
I strongly suggest to have a multi-step approach with the find command if you are using the ‘-exec’ option,
at least until you become proficient.
Run a plain find command first, just to display the results
and only then add the ‘-exec’ option for a second run.
You will make fewer mistakes that way, especially with exec commands that make changes to the filesystem.
Please give this video thumbs up if you found it useful and subscribe to my channel
if you want more videos like that.
Thank you for watching!