in the first video we covered how deep
learning can be used to solve practical
engineering problems
specifically problems where you're
trying to classify complex patterns
in data and we left off having briefly
introduced
the deep learning design workflow now in
this video
we're going to focus on the first step
in this workflow which is having access
to data
and i want to reiterate the caveat for
this series that i'm not
trying to explain everything you need to
know about deep learning i just want to
introduce some concepts and get you
thinking about the scenarios in which
deep learning might be the right choice
to solve your engineering problems and
part of making that decision
comes down to the type and amount of
data that you have access to
and to ground this conversation with
some realism we're going to use a
practical engineering example
which is waveform identification in
radar and communication
applications and that's a mouthful
but if you stick around i'll explain
what that means and how access to
data and deep learning can play an
important part in solving this problem
i'm brian and welcome to a matlab tech
talk
i'm going to cover two use cases for why
we may need to identify rf waveforms in
the first place
and these are for communication and
radar
we'll begin by describing the overall
problem you know what's
the need and then we'll talk about why
traditional methods for classification
could be difficult and why it makes
sense to use data and deep learning to
solve this problem
and then i want to end by talking about
how we can synthesize the training data
through simulation
rather than going out and collecting it
directly with field measurements
and keep in mind as we walk through this
very specific problem of classifying rf
waveforms
that the general idea of getting
something useful
out of confusing and messy time series
data
is a common engineering problem so even
if you're not into radar
or communications most of what we'll
cover is applicable to many other
engineering problems
all right with that out of the way let's
get back to communication
to communicate at a distance we need to
be able to send information from a
transmitter to a receiver
and this is done by choosing a carrier
frequency that is
modulated in some way that encodes the
information
for example we may use a digital
modulation process like binary phase
shift keying
or bpsk simply put if the transmitter is
sending a zero
then the phase of the carrier signal is
unaffected and when a 1 is sent
the phase is shifted by 180 degrees
as long as both the receiver and the
transmitter are expecting the same setup
that is the same carrier frequency
modulation scheme and waveform
parameters
then the information can be decoded
correctly at its destination
however it isn't always the case that
the receiver is aware of all of this
ahead of time
for example signal intelligence and
surveillance systems
might just be listening for signals any
signals that happen to be out there
and by detecting the modulation scheme
they could characterize and identify the
type of transmitter
that is sending it or in the case of 5g
and other wireless schemes
if there's interference and noise
swamping the signal
it's beneficial to understand and label
the source of the interference
so that it can know what frequency and
modulation scheme to switch to
in order to minimize this interference
and even if avoidance isn't the end goal
it's still helpful to understand if the
noise is coming from a specific
external signal so that at the very
least the operators know that it isn't a
hardware problem
they're seeing there is a similar case
for radar systems
with radar tracking and target
identification we send out radio waves
to probe the environment and listen for
their reflection
and a common way to do this is to pulse
the radar signal
alternating between transmitting and
listening
and the pulse itself can have different
waveforms like rectangular
or linear frequency modulation or barker
codes
just to name a few but instead of you
sending out the pulses and therefore
knowing what the reflected waveforms
will look like
let's say you wanted to detect radar
signatures before you yourself
are detected then it might be helpful to
have a system that can search across
say four gigahertz bandwidth and find
and classify
any of the known radar pulse waveforms
within an incoming signal
and then again use that to be able to
determine the type of radar that is
emitting
that signal this is what is meant by
modulation identification
or waveform identification we have this
need for
a function or a set of functions that
take in
raw iq signals from the antenna and
label the waveform
and its parameters so you can start to
see
how this is shaping up to be a deep
learning problem where we could learn
this classification model
using labeled waveform data
however i think a good rule of thumb is
to not start with deep learning or other
machine learning techniques when
traditional
rule-based methods will work so the
question is
can we rely on traditional methods to
classify waveforms
for example why not use our knowledge
and expertise in rf signals
to write some code or build hardware
that processes the incoming signal in
some way that makes
feature recognition more obvious and
then we pick out certain features within
the process data
and then from that write code that
determines the waveform
and it kind of feels like building a
system that can distinguish between
say these two waveforms is pretty
straightforward
they look quite unique to each other
unfortunately things can get pretty
complicated
in a hurry which can make pattern
recognition with traditional methods
difficult
or at the very least time consuming
to show you an example the left plot is
the time domain signal for a linear
frequency modulated pulse
with a random sweep bandwidth pulse
width pulse repetition frequency
and sweep direction so the whole thing
is randomized
and the right plot is the frequency
domain representation of it
the lfm waveform is modulating a carrier
signal
which is a higher frequency also
randomized somewhere around the 20
megahertz region
so this is what one version of an ideal
lfm waveform looks like
however there are many things that can
impair this signal weather and physical
obstacles can affect
different frequencies in different ways
and this can change the shape of the
waveform as it propagates through them
and there are hardware distortions from
the radio electronics
which cause white noise and other phase
and frequency offsets
which i'm modeling here by just adding
some white gaussian noise
there are also reflections off obstacles
near the antenna
which can lead to the signal interacting
with slightly out of phase versions of
itself
as well as many other sources of noise
and errors which can affect
the received signal so this is one
noisy lfm waveform but here is
another and another
and this is really the crux of our
problem
our waveform classifier needs to be able
to recognize
all of these as linear frequency
modulations
and more than that it also needs to
recognize other modulations that look
very similar to lfm and take up the same
frequency bandwidths
and are subjected to the same noise
sources
and errors therefore our solution space
that is the entire set of conditions and
scenarios under which our classification
algorithm needs to work is massive
and designing a classification model
that uses a rule-based approach that can
handle all of these variations
might not be easy but finding complex
patterns in large
messy and confusing data sets is exactly
the type of problem where deep learning
approaches can be beneficial
but to accomplish this you need access
to
training data as an oversimplification
of the deep learning problem
we could set up a network architecture
that will accomplish all of the data
processing
feature extraction and waveform
identification tasks
then if we have access to enough labeled
data
we could employ a deep learning
algorithm that will tune this network
to accurately classify waveforms in
unlabeled data
that's the goal but what does it mean to
have
enough labeled data and where does that
data come from
to answer that i want to start by saying
that no matter what method you choose to
design your classification algorithm
you need data even if you're building a
rule-based algorithm you have to
understand your system
and the signals that it will see well
enough to be able to write
those rules but the difference between
that set of data and what a deep
learning system needs
is mostly a question of quantity
when a person is designing an algorithm
they are
bringing along years of experience and
knowledge about the problem
which helps them to quickly dismiss
certain approaches
or ideas that are obviously not the
solution
or for example they understand what
white noise is and so they can more
quickly recognize it in a frequency plot
however unless you are starting with a
partially trained model
the classification network we're
designing with deep learning
has no experience or existing knowledge
to draw from
it doesn't know what is obvious
therefore it takes many more examples of
labeled data for the network to
understand
even basic concepts like rising edges in
a signal
let alone combining those edges into
more abstract concepts
like waveforms and noise
so in this way we're using more data to
offset the experience and knowledge that
humans would normally bring to the
problem
now what i'm talking about here is a
full end-to-end deep learning approach
where raw signals are fed into the
network and a label is applied to it
this requires the most data to train
since we aren't supplementing the
network with any human knowledge
however this isn't always the case
for example a person could use their
knowledge to pre-process the data
by say filtering it first or by
transforming it in a way
that makes some distinguishing features
more obvious
or by going so far as to extract the
obvious features themselves
before a machine learning algorithm
determines the classification
in this way we are using human knowledge
to shrink
the remaining classification problem
which is the part that still needs to be
learned by the machine
and therefore in general requires less
training data
now regardless of where your problem
fits on this curve
the bottom line is that you need some
amount of good labeled data
that covers the entire solution space
that the classification algorithm needs
to handle
which in our case the data needs to span
all of the modulation schemes
at many different carrier frequencies
noise conditions bandwidths
and so on so the next question is
how do you acquire this labeled data
and one way if you're lucky is to just
use an existing database
for example if you're working with
images you can just start from an image
database like
imagenet and then you can add to that
database with your own
labeled data to fill in any missing gaps
however
at the moment i think most engineering
problems are unique enough
that augmenting or extending an existing
database is just about as large of a
problem
as creating your own database from
scratch
so that leaves the other option which is
to collect your own
data this can be done by placing a
transmitter and receiver out in the
field
and sending various waveforms with
different parameters
while simultaneously adjusting the
environment that's
things like noise parameters and other
rf sources
you can imagine that this could be
fairly difficult and time consuming to
do
especially if you're trying to control
for
weather or different propagation
distances
in some cases like with autonomous
vehicles the field is still the best way
to
collect data because there are billions
of cars on the road
the approach is to place sensors on cars
that are driven by people in a vast
array of scenarios and conditions
then over time and millions of driven
miles
and countless hours of labeling a
database is built
up and this could also be the approach
we use for our waveform database
after all there are billions of
receivers in the world and it would be
nice if they recorded their received
signal
labeled it with the modulation scheme
they were designed for and then
saved it off in a global database
however in this particular case
there's a faster and cheaper way to
access labeled data
and that is by generating it through
simulation
as long as you understand the scope of
the solution space that you want to
solve for
you can then build a simulation that
takes all of it into
account for example we can list out the
modulation schemes we want to classify
and their specific parameters the types
of impairments we want to be able to
handle
the variations in the hardware and
anything else we deem important
and then using all of that we can build
a simulation that will generate
realistic
received signals across the entire
solution space as long as you trust your
simulation to represent the important
features and characteristics of the real
signals
then generating millions of test cases
is relatively quick and easy
and a benefit of synthesized data is
that the label
practically comes for free since you
need the label to generate the data in
the first place
now it's important to understand though
when simulating data makes sense
and when it doesn't like if you wanted
to build a network that could classify
words in audio signals then simulating
people saying words is probably much
harder than just
collecting a lot of real audio but for
this particular problem
where the physics are well understood it
makes sense to build a model and to
generate
the data and this is exactly what's
being demonstrated in this
matlab example and i encourage you to
check this out if you really want to
understand what's going on
but right now i just want to quickly
highlight a few things
this first section is using a
pre-trained network to recognize
11 different modulation types this is an
example of
hey if you can find a pre-trained
network that already does what you want
then you're done but if you can't you
need to train one yourself
so if we scroll past that the next
section is where waveform data is
generated
which can be used to train a new model
you can see here that it's generating 10
000 frames
for each modulation type so we're gonna
get a hundred and ten thousand signals
in just a few minutes and for each
signal it's adding
random amounts of white noise multi-path
fading and
hardware offsets so the idea is that
we're covering the entire expected
solution space and a little further down
it's plotting a few random signals so
you can see what they look like in the
time domain
and as a spectrogram now we can use this
simulated data to train the network
which took about an hour for this
particular case and ended up correctly
labeling about 95 percent of the
simulated
validation data which is nice except
who really cares how well this network
can label
simulated data i mean it was trained on
simulated data so of course it learned
to do a good job at recognizing it
but the real test is how well this
network can label real
rf data and in the last section this is
exactly what's done
a software defined radio is transmitting
various waveforms
and a receiver is recording the signal
and using the train network to label
that waveform and according to the
confusion matrix it does
a really good job about 99 overall
accuracy
which seems amazing but we have to
consider
that in this test the two radios were
stationary
and placed two feet from each other
which would limit the amount of
propagation noise and multi-path fading
and maybe other things so these were
relatively
clean signals the real test would be to
validate this network on hardware
that is put in a more realistic scenario
the bottom line though is that it is
possible and
sometimes preferred to train a network
using
simulated data and it will work in real
situations
as long as the simulation generates
signals that closely match the
conditions of the real system
or is more strenuous than the real
system
so it bounds it all right
that's where i want to leave this video
for now hopefully you have an idea of
how you would go about collecting
labeled data for the particular
engineering problem that you're trying
to solve
whether that means pulling from an
existing database collecting real data
yourself
or simulating data in the next video i
want to talk about how we can use
our engineering expertise to pre-process
the raw data
in some way which will help speed up
training
and simplify the network so if you don't
want to miss that or any other tech
talk video don't forget to subscribe to
this channel
also if you want to check out my channel
control system lectures i cover more
control theory topics there as well
thanks for watching and i'll see you
next time