hi everyone welcome back I'm Steve Brenton and I'm really excited today to tell you about dynamical
systems and this lecture on the anatomy of a dynamical system how you build them how you
analyze them how you understand them I absolutely love dynamical systems they are our models for the
evolving world around us so they describe the rich behavior of mixing fluids they're how we build
models that we would use to design rockets and land them we use dynamical systems to understand
the brain disease networks social networks you name it dynamical systems describe the evolving
world around us and this is really why I went into applied math and science and engineering in
the first place is to model these rich systems that change and that we interact with these
dynamical systems so this is going to be a video on the anatomy of a dynamical system what are the
challenges of modern dynamical systems to describe these really complex systems of interest and what
are some of the high-level objectives we have when we're working with dynamical systems so I hope you
enjoy this this is oftentimes kind of an intro piece of a longer lecture on how you would use
machine learning to learn these dynamical systems or to learn how to control these systems but I
think this can be kind of a standalone little piece on understanding what are the the integral
parts the bits and pieces that go together to make up a dynamical system so dynamical
systems are our models of reality they are how we describe the world around us especially how
things change and Co evolve in time so dynamical systems are fundamentally linked to time and how
systems change in time okay so this is our kind of simple model of a dynamical system it's a
system of coupled differential equations X is a vector that represents the state of our system
it is a vector that has the minimal number set of values that you need to describe your system
so for example if I have a pen then that pendulum is uniquely defined not just by its angle-theta
but by its angular rate theta dot so the state of that pendulum system is theta and theta dot
in a vector okay so that's what X is X is a vector of the state of your system if I'm looking at the
stock market the state of the system might be the value in all of the stocks and there might
be more information that needs to go into that state vector if I'm looking at the weather it
might be the weather condition on a one kilometer grid on planet Earth okay so the state vector X
can get very very large but it's a unique minimal description of the system you care about okay F is
what we call the dynamics it's a set of functions F 1 F 2 F 3 that describe the dynamics of state
1 state 2 state 3 so the time derivative of x1 will be given by the first row equation F 1 and
the time dynamics of x2 are given by F 2 and so on and so forth so F is a vector valued function
that tells me given a current state X how does that state change in the next time instant okay
so that is the dynamics it's a set of functions oftentimes this is given by like Newton's second
law or something like that and we sometimes say that this is a vector field because for every
point in space X this vector field essentially tells you how the derivative is pointing and
what vector Direction X dot is pointing at that point in space so it's a field of these Direction
vectors T is time and this is really important because in lots of dynamical systems the system
itself changes in time ok this is super important think about your brain state your brain state that
the dynamics the equations that would describe how your brain works are fundamentally different
when you're sitting here watching a video lecture versus when you're asleep different dynamics ok
so this is time varying the weather might change the climate might change maybe we are actively
changing the dynamics of a system okay the ocean dynamics the vector field of the ocean changes in
time good you are all of the variables that we have active control over so maybe we have some
variables we can manipulate to try to change the behavior of the system those are our control knobs
and those go in this vector U so in the case of the pendulum maybe I put that pendulum on a cart
and I can accelerate that cart I can drive that cart around and that would be my control and put
you to try to change the dynamics of my system beta are all the other parameters of the system
that I don't explicitly have control over but I might want to analyze and understand so in the
pendulum example this might be like the length of the pendulum arm or the mass of the pendulum and
things like that and so these beta parameters we want to understand the explicit dependence
of these dynamics on those beta parameters as I change those parameters I might get really big
changes in the dynamics called bifurcations and so these beta parameters are super important good
so that's our dynamical system now in addition we also have measurements of our systems so we
don't always have access to this full state X because maybe my system is exceedingly large
you can't measure all of the brain all of the neurons in your brain that would be you know
completely impractical so maybe you can measure a really really small subset or even some crude
measurement some proxy measurement like a EKG or EEG or something like that so you have these
measurements in Y and I now that we've kind of talked about what is the anatomy of a dynamical
system I want to walk you through one of the modern challenges so people have been writing
down dynamical systems and studying them and analyzing them for centuries right Newton's law
second law is a dynamical system that was kind of the beginning of our foray into dynamical systems
but in the modern era a lot has changed and there are some big challenges that were able to start
tackling now okay and so that's that's really what I want to talk about here so in modern dynamical
systems some of the biggest challenges probably the biggest one that goes unmentioned in a lot
of cases is that in most situations we actually don't know we don't know the dynamics we don't
have a model of our system so for mechanical systems like the pendulum yes you can derive
you know Newton's laws or the Lagrangian or the Hamiltonian and you can derive these equations of
motion from first principles that's like you crack your physics book and you derive this system from
scratch that's how we you know used to do things that's how we do things when we can but for modern
systems of interest we don't have access to these governing equations and F there is no
known master equation for how your brain works we don't know the F for a brain we don't know
the F for a disease Network we have models we have approximations but we can't agree on one governing
physical law for how a disease spreads or how the climate works ok so modern systems of interest F
is often unknown even in systems where you do know F like the Navy or stokes equations for a fluid
it might be far too complicated to use and so we would need to learn a reduced order model in terms
of the things we care about in terms of the less descriptive state X that captures the big things
I care about in my fluid the other really really big challenge in dynamical systems and I think
this this one is really maybe tied for number one is non-linearity so this function f in general for
systems of interest is often nonlinear and that means superposition of solutions doesn't hold if
my dynamics were linear if X dot equals a matrix a times X everything would be simple there would
be a canonical closed form solution you could just write down the solution and go home we would
all be out of business so non-linearity is what gives us job security and dynamical systems this
is the real challenge in many systems is that even small amounts of non-linearity make it so
there are not closed form solutions in general there are special classes of nonlinear systems
and special solutions that you can find but there is no generic one size fits all nonlinear theory
of nonlinear dynamical systems so that's really interesting and most systems we care about are
nonlinear other challenges are dimensionality of the state vector X so for many many systems this
state vector X might be exceedingly complex I think in your brain there are estimates that
you have a hundred billion neurons that's a hundred billion dimensional state vector just
to describe the state of all of those neurons uniquely describing the climate or the weather
is also very very massive problem if you want to get a really good description of weather on planet
earth you need a very high resolution and Earth's pretty big ok disease modeling if you want to
actually do a good job of this you really need to figure out like the network of how people interact
and that's you know there's billions of us and there's tons of connections in this network so
it's super complicated high dimensional dynamics in a lot of cases and that makes it really hard to
analyze these along with dimensionality is multi scale so these dynamical systems are not just big
but there's a range of scales from you know big scales all the way to really really small scales
so think about whether or or climate you know on the scale of the size of Earth there are things I
care about their seasonal changes you know big hurricanes but little teeny-tiny features also
might matter for the weather prediction three weeks out ok so you have this range of scales
you care about your brain also is fundamentally multi scale spatially there's regions and then
those regions are composed of you know individual neurons and things but there's also multi scale
in time you're listening to me talk about this right now you're hearing words I'm saying on
seconds you're connecting that to things I told you a few minutes ago but you also have
this lifetime of experience over months and years so you're extremely multi scale in time
dynamical systems are fundamentally chaotic in a lot of situations so not all nonlinear systems are
chaotic but many of them are and what that means is that you get a very sensitive dependence of
the outcome of these systems you get a very the future depends very strongly on small changes on
the initial condition and the parameters and the dynamics now so if you have your system and you
change your parameter even by a little bit you might get very very different phenomena or if
you change your initial condition a little bit you might go to very different places and that makes
it really hard to predict these systems so that's actually why weather is so hard to predict you
know we can't predict three weeks out because this is a super chaotic system and we can only measure
the system so accurately and we only can describe the equations so accurately and the parameter so
accurately so chaos is is still a huge problem latent variables hidden variables again this is
related to dimensionality if I have a big state vector chances are I'm not measuring everything
you can't measure all of the neurons in your brain you might only measure a few of them you
might only have a proxy measurement so latent variables hidden variables is a big deal they
might matter a lot and you can't measure them what do you do in those situations all real dynamical
systems have disturbances and noise and you have to consider this so your dynamical system is going
to be forced by external disturbances stochastic forcing and your measurements are going to have
noise and you can't always just assume that these are Gaussian white noise processes like what we do
in textbooks because that's the only thing that we can actually solve in the real world you're
probably going to have correlated structured biased noise and disturbance terms and
All of this gets wrapped up into this concept of uncertainty. There's a huge amount of uncertainty in every
single piece of this dynamical system. I have uncertainty in the model 'f'. I have uncertainty in
the parameters beta. I have uncertainty in what I'm measuring and how it's being measured. I might even
have uncertainty in the measurement time of when I take that measurement. I mean there's all kinds
of uncertainty and that all will conspire to make this a very challenging problem.
I have asterisks on the challenges because I want to point out that most, maybe not all, but most
of these problems can be recast as optimization problems. So optimally handling noise and
disturbances for future predictions or estimation. Optimally propagating uncertainty in the future...
And so on and so forth. Optimally reconstructing the full state from limited measurements 'y'.
And when I say that this is an optimization problem you should be thinking now "I can probably start
using tools from machine learning" because machine learning is just optimization based on data
it's building models using optimization and a wealth of data and so this is really nice
because modern dynamical systems kind of the anatomy of a dynamical system allows you to focus
on individual tasks and challenges with these emerging optimization techniques.
So now that we've understood kind of the anatomy of this dynamical system I want to talk a little bit
about how we would use it and there's so many uses I'm just gonna you know briefly
touch on some of them of course we might want to predict the future state of the system so I might
want to predict the weather a week from now when I think of future state prediction I think of you
know what if someone's lost at sea okay so we had some rough idea of their initial condition and we
want to integrate where we think they'll be two hours later okay so that's a future prediction
and the future prediction doesn't have to be deterministic it might be statistical I might
have to run an ensemble of initial conditions maybe I have uncertainty of where that person
was lost at sea or maybe this is the Gulf oil spill and I don't know exactly where the oil is
coming up so I have a Gaussian of uncertainty and I want to propagate that into a future estimate of
where the distribution of oil will be so that's a statistical or an ensemble prediction and I'll
point out all of these topics every term in this dynamical system every one of those challenges
every one of these uses there are dozens if not hundreds of people diligently working on these
problems these are not challenges these are whole careers okay design optimization that's another
big use of dynamical systems models what I might want to do is now that I have a model of my system
I might want to design betas to get a performance or an output that is desirable so think Formula
one cars or super yachts or rocket engines I want to design the parameters of my system to get some
desired high-level output like drag and lift and things like that okay and so that requires you to
be able to understand how your system depends on these parameters now if you do that optimization
in real time based on your measurements that would be called feedback control so if you can
actively manipulate some part of your system in real time those parameters become actuators and
this becomes a feedback control problem and that would be really satisfying and incredibly useful
in many many systems if we can actively control them to some engineering you know specifications
now most of the the first uses are very practical very engineering oriented but we also as humans
deeply care about understanding the world around us we don't just want to manipulate
it we don't just want to predict it we want to understand fundamentally how and why it works
okay and these dynamical systems give you a very kind of intuitive way of understanding
how the world evolves and what are the rules for how that evolves and so there's a couple of kind
of key words here that I think are really important interpretable and generalizable
so interpretable means that this model is not so complex that I can't communicate with you about
it we could if this model is not too complicated we can talk about it we can try to understand it
we can communicate and interpret this model generalizable is also really important if I
write down this model I want it to be useful in other scenarios that I might not have thought
of so being you know having some generality and how it how this model works even for parameters
I haven't tested that's really important and I always think of Newton's second law is kind of
the ultimate interpretable and generalizable dynamical systems model F equals MA it's super
simple so we can interpret and understand it it is very generalizable it works in so many scenarios
you know for different parameters for different state vectors and so on and it fundamentally has
changed the way that we understand with and interact with the world so we understand the
world better because of that interpretable model okay good so that's kind of the overview of the
anatomy of a dynamical system I strongly encourage you to start playing around with systems yourself
pick a system that you're interested in the brain or an epidemiological system or the climate pick
something interesting and start thinking to yourself what would be a state vector of
that system what would be the control inputs and the parameters what might be some rules for how
that system evolves in time what can I measure and start thinking about how those challenges
and these uses fit into your understanding of your dynamical system alright thank you