My name is Kevin Eliceiri from the University of Wisconsin at Madison,
and I work on the ImageJ project.
This is the pre-processing part of our bioimage analysis.
And in this section of the... of the video series,
we're gonna be talking about the importance of understanding where your data came from.
Image analysis has the goal to do measurements and further analysis,
but you can't blindly just take your data as it comes in.
You really need to understand where your data came from
and what were the constraints of the acquisition.
Whether it's the limitations of the system
or even mistakes that may have been made as part of the acquisition.
And it's really important to understand these constraints.
And this could be anywhere from parameters you want to collect,
such as a really big field of view,
where you want to stitch and bring your data together in big fields of view,
and understanding what are the constraints and necessary options for stitching.
To issues like your microscope
may not always be properly uniformly illuminated.
Understanding issues of illumination correction
and understanding how your sample is lighted up.
Also, registration.
You may have different views, different cameras,
where you want to bring them in and correlate them.
Or modalities such as light sheet microscopy,
where you look at different perspectives.
You want to be able understand
how to stitch and register those together.
But the number one thing is to always keep in mind that
you need good data in, good data out.
The importance that you don't use the analysis to correct for failures in acquisition.
You want the best data set possible
before you analyze.
And the big goal of that is to actually have the best signal-to-noise.
If you want to find a signal of an object of interest,
such as later talks where we talk about segmentation,
you want to make sure that you have good signal-to-noise,
which means good data.
And so, we're gonna start in this series
about more basic methods, and then move to the more advanced.
So, one of the most common things you might do is cropping,
where you want to actually crop a field of view,
and actually see...
kind of get rid of the unnecessary background,
the elements that might be confusing, distracting to the eye,
but also distracting to the algorithm.
So, it's a simple concept.
You might want to move the edges of culture dish,
or as I showed here, just the unnecessary black background,
reducing the non-information areas.
And it's very common that we do this for colocalization,
as we talk about in other videos,
where you just want to see how
two things of interests colocalize,
or are in the same place.
And so, I think less background can help with that,
because otherwise more background could influence certain calculations such as colocalization.
And so sometimes you need to reduce the areas without the signal,
just because it's confusing to the scientist.
Another thing you may want to do to make it easy for the algorithm
is just invert your sample.
So, again, a very simple concept is illustrated here,
where you just want to actually invert it.
Maybe the algorithm prefers to have something...
white objects on black.
You can very easily invert it.
And it's very important to always do this in a context of knowing,
what are your pixel values?
What is your bit depth?
What are your samples parameters?
And really knowing what you're doing to your data,
keeping the original data when you make these kinds of changes.
Sometimes you also just want to improve the noise
or smooth the edges of objects.
Filtering is an incredibly powerful way to better...
have better contrast to differentiate your objects.
And there are many classic filters,
such as median filtering, getting rid of noise,
or Gaussian filtering, where you get rid of background abstraction.
And these filters can often be used together.
And they can be chained in a series to get the optimal contrast for your particular case.
And they're available in many tools, such as ImageJ.
And there are more specifics on filtering
in other videos in our series.
As well, many times you actually
want to do colorimetric analysis.
For example, one of the gold standards in all clinical care
is pathology, where you use dyes and stains,
and for example, as illustrated here,
you would actually have stains such as hematoxylin in blue
and eosin in magenta.
You have these colorimetric stains
that can tell apart a tumor from a nontumor,
but you want to unmix them
and be able to extract these colors.
So, both CellProfiler and ImageJ
have very easy to use tools
for extracting these colors for further analysis.
As well, a very common thing is just to have better views.
We live in a world of 3D,
but when we analyze data,
we actually want to understand the 3D aspects.
And with these 3D objects,
we want to go back and forth.
We have easy ways of showing them on the screen,
on video online, and even in prints.
And Z projections are one powerful way of condensing your data using projections.
And the classic is a maximum intensity projection,
which is really great for showing
what's going on in your data set in a condensed form,
that might help you, as shown here,
where I can actually condense all this data into this image here,
and I can analyze that data, for example for segmentation.
And it's a very powerful way of being able to see
what is happening.
In this case, if I just want to understand the objects.
Or if I want to do intensity,
I may want to sum all the signal, as shown here.
And so the power of different views,
as in the Z-projections,
is very important.
As well, you may want to have a three-dimensional view of your data set.
So, as shown on this screen,
you have data where you really want to look slice by slice,
and yet you want to have it three-dimensional.
You don't want it to be constricted just to a projection view.
You want to see the full 3D rendering.
And so this is something that you can do
in a tool called SciView, for example, in ImageJ,
where you can actually understand the 3D rendering.
And Loic Royer, our colleague in another video,
will be talking about 3D rendering in depth,
and I encourage you to look at that.
Another issue, as I mentioned early on,
is understanding the limitations of your system.
Ideally, you wanna understand your acquisition
so you can fix it when you acquire it.
A classic example is the bulb on a microscope
being not set up and aligned correctly.
But there may be times where you can't deal with it.
It's already been acquired,
or it's not your microscope,
or other issues of the illumination correction.
And this image here shows this,
that on the far image here
you are seeing issues of the uneven light.
We've actually... in panel B,
have shown it in more clarity.
And we can actually correct that
through post-processing so that...
or, pre-processing, so that we can actually correct that frame.
Another important thing for tracking and segmentation,
as my colleague Anne Carpenter will talk about in one of our other videos,
is the importance of increasing the SNR.
And this is the idea of trying to understand our resolving capability,
and being able to detect objects from each other.
So, as shown in this frame here,
we actually want to be able to clearly make out
each element, each cell,
and be able to tell one from the other.
And so, there are some very nice pre-processing techniques
you can do to increase that,
and isolate those signals,
so that you can do...
decrease the noise and increase the signal
so that you can do things like signal processing and segmentation
and so on on those images.
What happens if you have to align two or more images of the same scene?
For example, if you have sets of cells,
and they're migrating,
and you want to actually see from different microscopes or different cameras
what is going on in that scene.
This is a process we call registration.
And it's the idea of transforming different sets of image data
into one coordinate system.
And the concept is very simple.
You calculate a transformation function,
which is essentially the math
for modifying the space relationship to...
between pixels.
And the idea of having an input range
where you transform against a reference image.
And then that transformation function
takes into account the geometric distortions
-- angle, orientation, shifting, and distance --
as you can see in this pipeline, here, on the screen.
And the thing to keep in mind is there could be many sources
for why you might want to register and deal with...
issues of error, for example.
When you're sectioning tissue,
your knife blade may miss a cut.
You may actually miss a section.
And you may have to register,
when it's harder to see between different views,
even though it's the same sample.
Or in the case of light sheet,
where you actually have different views and you want to correlate them.
So, registration is a very important concept.
And as I mentioned, you know,
this general technique can be used in many ways.
Not only in light sheet and dealing with different views,
but also if you want bigger fields of views,
such as you want to actually analyze many different positions
and stitch them together.
And you can do this fairly automatically.
You can actually have points and features shared between the images.
And you can do it manually by hunting and pecking and finding these,
or you can use algorithms that can find them automatically.
So, for example, in ImageJ
we have the ability to computationally find,
as you see here in yellow, corresponding points and features in the images,
and use these to actually determine the best transformation function.
So, it can do it for you to register these images,
which is incredibly useful,
particularly for large data sets.
And in a very famous example,
as shown in this plugin from ImageJ called Unwarp,
you can actually use a type of elastic algorithm
to actually correct for deformations in the image,
and still register them,
as you see here in the Eiffel Tower image
that has been corrected and registered.
And this is an example of the registering of sequence images
I've talked about.
And there's... in general, the also...
the idea that not only can you do registration for a single field of view,
but what if I wanted a giant field of view.
So, a good example is if you went to the Grand Canyon
and wanted to take a picture of the Grand Canyon,
your favorite smartphone is capable of
actually automatically taking bunches of photos
and stitching them together.
And we want to do the same thing in imaging.
We want to actually have the high-resolution view
-- maybe part of the Grand Canyon --
but then, like Google Maps, be able to zoom out and see the entire thing.
And so, you can do that in microscopy too,
where a single picture can't capture it all,
so we take many pictures,
and then we can automatically stitch that together.
And that's a common application of registration and stitching.
And so, motorized stages allow you to do this,
where you can roam around,
and you can actually have coordinates,
so the stage can be... kept track of all the coordinates,
and you can computationally stitch all these coordinates together
with the help of algorithms.
And these stitching tools have gotten quite powerful,
where you can do it from whole bunches of samples,
or less samples,
and it can actually use the registration to deal with
issues of overlap and lack of overlap
in getting the best results
based on whatever data you acquired from your microscope.
And this is an example of what you can do here.
You can see all of the images that we've stitched together,
and the final results.
And as you can see here, you may not...
often need to correct for the brightness difference at the borders
using correction techniques that we very mentioned very briefly,
so that you get the best result, and the smoothest result,
so you don't see the little squares and the artifacts,
as you can see in particular in the first...
the middle one, there.
Another really important aspect of pre-processing is deconvolution,
understanding the noise and error in your system.
When you acquire fluorescent images in particular,
these are very digital views of your actual sample.
And you have to understand the limitations
and the error of your actual microscope
and the defining optics.
And it's important to understand the convolution of error
that can be introduced in your acquired images
by the fundamental hardware on your system.
And we often call this the optical blurring,
and you can see the image is convoluted.
Deconvolution is trying to correct that systematic error
-- this is due to the hardware on your scope --
to get rid of that blur.
And this can be very important, because it can often result
in why your microscope may look more noisy or have less contrast
than a colleague's microscope that seems to have similar capabilities.
You want to be able to see smaller features, for example,
and that can be prevented by problems due to convolution
or noise introduction.
And so what you can do in this kind of work
is understanding, for example, a basic bead,
and trying to understand,
what is the actual object you're trying to measure?
And trying to measure what's called the point spread function.
Basically, this mathematical measurement of,
what is your point spread of your sample?
Understanding, in Z, what is the error
produced by error of your optics, your actual glass, issues like optical oil,
or things being left on that might contaminate your view?
And we can correct for that.
We can understand this mathematical function,
and then use other algorithms,
together with the point spread function,
to correct for the inherent convolution of your system.
So, think of it was a way for correcting the error of your optics.
And we can correct the raw image
before we ever get to additional post-processing,
as we talk about in this video series.
So, the importance the deconvolution...
it can be used on fluorescence, confocal microscopy, and other...
and other types of data
to get the best possible quality data.
And so, with that, we're gonna end this part of the series.
But thank you for your attention,
and I hope you get a chance to watch the other videos in our image analysis series.