So far in this series, we've talked
about so-called offline system identification methods.
That is identifying a model using data that was previously
collected from the system, and then
used all at once to estimate the model.
And with these methods, the amount
of time it takes to get to a solution
isn't critically important.
You could take your input and output data,
kick off the learning algorithm, and then go have some tea
and relax a little bit while it runs.
And whenever it finishes, you'll have an answer.
Online system identification basically
means that the algorithm is running during the operation
of the physical system.
And these algorithms estimate the parameters
in states of a model as new data is
measured and available in real-time or near real-time.
And the sort of relaxed approach that we
did with the offline methods doesn't really
work in these situations, because usually
computational time and resources play a much larger part.
So all of that fancy nonlinear regressors and nonlinear output
functions and optimization algorithms
that we did in the last video might produce a good answer,
but not necessarily one that can be completed
during real-time operation.
So in this video, we're going to talk about what online system
identification is; why it's a good option for real time
situations; and the general idea behind how it works.
So I hope you stick around for it.
I'm Brian, and welcome to a MATLAB Tech Talk.
Let's describe online system identification
using the example of a thermostat that
uses a thermodynamic model of a house
in order to best control the temperature.
And if the thermostat isn't provided a model,
it has to build one based on the available information.
For example, it could use the current inputs
into the system, which are the control inputs like command
and fan speed and compressor working fluid temperature, as
well as other inputs like the outside temperature.
And it can use the current output
of the system, which is the measured temperature
inside the house.
And it can use all of that to develop a thermodynamic model
over time.
It does that by updating the model parameters each time
step.
And here's one way to do that.
The algorithm could take the measured house temperature,
y, and compare it to the predicted house
temperature, y hat, to produce a prediction error.
The predicted temperature comes from its internal model, which
we'll say is a linear model in the following form.
Where if you remember from the second video,
psi is a vector of the linear aggressors, and theta hat
contains the estimated model parameters.
And at the beginning of operation,
theta is just initialized to some default values.
But if there's a difference between the measured
and predicted output, then the model parameters aren't correct
and they need to be updated.
And an online algorithm would adjust the model parameters
in some way to reduce that error.
And one way is simply to use a gradient method
where an adaptation gain K scales the error
and then adds it to the old parameters
to get a new set of estimated parameters.
And then in the next time step, a new set of measurements
come in, which are then compared to the new prediction
from the model.
And that error is then used to update the parameters again.
And it keeps doing this until it converges on the model.
This is a recursive infinite history algorithm,
which we're going to get into in a little bit what that means.
But for now, all I want to illustrate
is how, with online methods, the model
is developed over time using the data as it's available.
All right so, why would we want to solve system identification
online one data point at a time rather than offline
and just all at once?
Well, as you might imagine, there's lots of reasons.
And I'm not going to get into most of them in this video.
But the one reason that I do want to go into detail on
is that online system identification is useful
if the system changes over time, or the parameters of the model
are time varying, and you want the model
to constantly adapt to it or to recognize those changes.
With the temperature control example,
the house thermodynamics can change from weekday
to weekends based on whether people are home and opening
windows and doors, or throughout the year as the seasons change,
or when more energy-efficient insulation is installed,
and any number of things that would cause a change in the way
heat flows into and out of a house.
So for control purposes, by maintaining an accurate model
of the house as the parameters vary,
a temperature control system can be
developed that uses that model to more effectively maintain
the desired temperature.
This is a form of adaptive control.
And adaptive control and other methods
that use system ID, like model predictive control,
are absolutely awesome.
And I've left some links to some MATLAB examples
that you can play around with in the description of this video.
But perhaps my favorite use case for online system
identification is for fault detection,
because it just makes so much sense to me.
With a model that is adapting to changes in the system,
we can create fault tests that look
at the parameters of the model rather than
the measurements of the system to determine
if a fault has occurred.
And it can work like this.
Imagine we have a DC motor where we
can apply a voltage to the motor which changes
the angle of the motor shaft.
And we can wrap a control system around it
to control the angle and speed.
Now, over time, the lubricants and the bearings
might degrade, causing more friction
in the motor, which would then require
more current or a higher voltage to achieve the same spin rate.
The controller will compensate for this increased friction,
and the motor might still spin as commanded.
However, this might not be ideal,
and so we want to detect this high-friction condition
and trigger a fault response.
Now, instead of setting up a fault test that
looks at current draw and motor speed and motor acceleration
and then tries to come up with a threshold that indicates
a high-friction scenario, we could
approach this with online system identification.
We could estimate the parameters of the motor model itself,
which, if we set it up to do so, could
have a parameter for the motor viscous friction constant.
Now, instead of tracking current and trying
to figure out some complex fault condition,
we can just check if the friction constant is larger
than some maximum threshold.
And no matter what speed we're running the motor
or how it's accelerating, we have a direct estimate
of the parameter that we care about.
And I think that's pretty awesome
because estimating model parameters
rather than looking only at the measured data
could give you a lot more insight
into the properties of the system
that you care most about.
And we can't do this kind of real-time adaptive control
or real-time fault detection with offline system
identification.
Well, at least not for systems that
change quickly relative to how long it takes the offline
algorithms to run.
If your system is slow enough, like some chemical processes,
then collecting a bunch of data once a day
and then just running offline identification
is probably sufficient.
But for fast-changing systems, we need online algorithms.
All right, so with that behind us,
let's talk a little bit about what
these online algorithms are.
And I already mentioned that we can't really
approach them exactly the same way that we
did with the offline methods because of timing concerns.
And a lot of this comes down to the fact
that those offline models, like the nonlinear ARX,
are often non-convex and therefore require
slower optimization solvers that iterate
towards the solution over time.
So to get around this, we're going
to forego those more complex models and algorithms
and limit ourselves to models that
can be solved with just a few matrix operations
without having to iterate on the solution.
And we're going to look at two different ways
that we can use the online data to update these linear models--
finite-history algorithms and infinite-history algorithms.
Finite-history means that we're only
using part of the past data.
And, to me, these types of algorithms
are the most intuitive because they just seem to make so much
sense.
When a real-time system is running,
it's generating a continuous stream of data.
And it doesn't make sense to try to store all of that data
and use all of it to develop a model; that
would get overwhelming quickly.
So instead, we just store a fixed number of past data
points in a buffer.
And we would store both the input and output data
and anything that's needed to identify a model.
And when a new data point comes in,
it just pushes out the old one so that the window is always
the same length.
Now, at each time step, we use this entire buffer of data
to estimate a linear model.
And in this way, this is very similar to linear offline
methods because we're fitting a linear model
to a complete set of data all at once by minimizing a least
squares problem.
And it just so happens that this data is collected real-time
from the system; and we're limiting ourselves
to just the last in samples.
And one thing that makes finite-history algorithms
a good choice for rapidly-changing systems
is that we only ever look at and consider the latest
data when fitting a model.
We move that window of buffer data
forward in time and therefore only based the updated
model on the most recent data.
So if the system is in one state in the past,
and we estimate a model of the system,
then, later in the future, when the system has transitioned
to a new state, none of that past data
will influence the new model.
Now, a downside of finite-history algorithms
is that we need to store all of that data.
And when we perform the identification step,
we have to compute a model using all of that data.
And this can be potentially time-consuming and
resource-intensive if the window needs
to be really large to capture the essential dynamics
of the system.
On the other hand, recursive infinite-history algorithms
don't typically have that problem.
Infinite-history algorithms use, in some small part,
all of the past data to estimate the model.
And this gets us back to the equation
that we created for the smart thermostat
from earlier, where the new model parameters are
a function of the old parameters plus only
the latest set of measurements.
These methods are typically much faster to run and require
much less stored data because only the most recent data point
is used to update the result and not an entire buffer's
worth of data.
In this way every, past data point
has impacted the solution in some way,
since this is a recursive algorithm.
And therefore, the result is always a combination
of all of the past values--
hence, infinite-history.
Oh, and by the way, just as a side note,
this is very much analogous to finite impulse response filters
and infinite impulse response filters
in how they each use past data points.
So if you're familiar with them, then this concept
shouldn't be a surprise.
OK.
So we have this general form of the infinite-history recursive
algorithm, but how do we set it up?
That is, how do we choose the adaptation gain
so that the parameters are updated appropriately based
on the prediction error each time step?
And one way is to just set the gain equation
to a fixed value, Q, multiplied by the regression vector.
With this, each time step--
the error between the prediction and the measurement--
is scaled by a fixed amount proportional to the gradient
of the linear system.
And then that is added to the old parameters.
And over time, the parameters will converge on the solution
since it's descending the gradient, assuming
the gain isn't so large as to cause an instability, and not
too small as to not be able to keep up
with the time-varying parameters.
In this-- the linear regression case--
this method is also known as the least mean squares method.
And there are actually multiple flavors of this approach.
I just described the unnormalized gradient method,
but there's also a normalized method
that scales the adaptation gained by the square of the 2
norm of the gradient vector.
And if you're interested in learning more,
check out this document that I reference down below.
OK, if picking a fixed gain seems arbitrary and not
optimal enough for you, well, then we
can approach this in a more optimal way.
Consider the measurements y generated
by the actual system, which we're assuming
is linear, plus some measurement noise assumed
to be Gaussian white noise.
In this way, we can't trust the measurements 100%
because the real output of the system
is corrupted with some measurement noise.
In addition to that, the true parameters of the system
aren't static.
They vary over time.
And let's say that the parameters
of the system at time t is a function
of the parameters at time t minus 1 plus some random value.
Again, this random value is assumed
to be Gaussian, which means that the parameters are kind
of on a random walk over time.
All right, so now here's where it gets interesting.
When a new measurement comes in, we
want to know how much to update the old parameters.
What should this gain be?
Well, we have the measurement itself, which
might be corrupted with noise.
And we have a sense, at least statistically, of how much
the parameters could move in one time step.
So if the measured y is drastically
different than the predicted y, we
might be inclined to believe the measurement
and want a large Q value to correct the parameters.
However, if we know that there is a lot of measurement noise,
then maybe we don't want to believe the measurement as
much, and only move the parameters a little bit,
despite the error.
So a small Q value.
Yet if we know that the parameters in the system
can move quite a bit each sample time, then maybe
that large difference between the measurement
and the prediction can be trusted more.
And we want a Q value somewhere in the middle.
And this problem of trying to balance two uncertain elements
is starting to look a lot like what a Kalman filter does,
right?
In fact, we can actually just use a Kalman filter
to solve this problem.
A Kalman filter estimates the state
of the following standard format linear system, which
if we look closely, we can see that they exactly
match our measurement and parameter models if F of t
is set to the identity matrix and H of t
is the regression vector.
So now to find the gain Q of t, we
can use the Kalman filter to solve for the optimal gain,
where P is the prediction covariance,
R2 is the measurement covariance,
and R1 is the process covariance.
And note, that even though this is significantly more complex
than a fixed gain approach, it's still
something we can solve for using a relatively small number
of matrix operations, which is blazing fast compared
to the more iterative optimization methods
that we used in offline system identification.
This is why recursive approaches like this
are so well-suited for real-time operations.
And over time, one sample at a time,
this will converge on the optimal model parameters.
And by the way, if you want to learn more about the Kalman
filter, the assumptions under which it was derived
and all of that, check out the MATLAB Tech Talk series on them
that I've linked to in the description of this video.
All right, you might notice that this video was
heavy on the general concepts and light
on the specific mathematics or applications.
Plus, I didn't even cover many other popular online methods,
like adding a forgetting factor into the infinite history
algorithms so that old measurements are discounted
exponentially, leaving just the newest
measurements, influencing the optimal result more heavily.
Well, this is because there's always
too much to cover in short videos like this.
The goal here is to just hopefully give you
some baseline knowledge and motivation
to go and learn the details of the mathematics
and to go and try out an application yourself.
And to do that, I recommend checking out
all of the references below, especially
chapter 11 of Dr. Lennart Ljung's book
that covers recursive estimation methods
and the MATLAB and Simulink examples that demonstrate
each of these concepts.
Specifically, check out this Simulink example
that shows how you can use online recursive least squares
estimation to detect a change in an engine's inertia.
And if we jump over to Simulink, I
can show you some of the highlights of this example.
The engine is nonlinear.
And this portion of the model here
represents the true dynamics of the system.
And at about 300 seconds, the engine inertia
changes from a little over 7 to just over 2 kilograms
meter squared.
And this is what we want to detect.
This second portion is where the motor model is identified.
And this is done using just the throttle position and the motor
speed's input, which are used to make the progression
vector, which is then used to estimate
the parameters of a second order polynomial model.
And inside this block, you can see
that I'm using the Kalman filter to estimate the model
parameters.
The process noise covariance and the parameter covariance matrix
are both specified.
Now, this model is set up such that the first parameter, A1,
is proportional to the motor inertia.
And if we look at this scope, you
can see that it takes 100 seconds or so
for the filter to converge.
And then at 300 seconds, it becomes very obvious
that the inertia has changed.
And what's cool about this whole thing is, like I said,
the entire system identification algorithm
is just packaging the regressors, some really basic
matrix operations, and then some function
that determines if a parameter has surpassed a threshold.
So we could test these algorithms
in simulation like this and then generate
code that can run on relatively low-power microcontrollers.
And by the way, I used a Kalman filter here,
since that's what we talked about in the video.
But if you run the default example,
you'll see that it uses the forgetting factor algorithm,
which converges much faster--
in about 50 seconds or so initially
and then just about 15 seconds when the inertia changes.
And this is because it's discounting past measurements
and only looking at about the last 15 seconds
or so in this particular example.
So this is a good example to check out
if you'd like to see what we talked
about in this video in action.
All right, so that's where I'm going to leave this video.
And if you don't want to miss any future Tech Talk videos,
don't forget to subscribe to this channel.
And if you want to check out my channel, Control System
Lectures, I cover more control theory topics there as well.
Thanks for watching, and I'll see you next time.