We are going to generalize what we talked about,
uh, last time, uh,
which will be all about generalizing and mathematically formalizing,
uh, graph neural networks.
The idea for today's lecture,
is to talk about deep graph encoders and mathematically formalize them,
and also show you the design space, the idea,
the diversity of what kind of design choices, uh,
do we have when we are making, uh,
these types of, uh, decisions,
uh, building these types of architectures, right?
So what we want is we wanna build a deep graph encoder that takes the graph, uh,
as the input, and then through a series of non-linear, uh, transformations,
through kind of this deep neural network,
comes up with a set of, uh,
predictions that can be at node level,
can be at the level of sub-graphs,
pairs of nodes, um, and so on.
And what we talked about last time,
is that the way we can define convolutional neural networks on top of graphs is
to- to think about the underlying network as the computation graph, right?
So the idea was when we discussed if I wanna make a prediction for a given,
uh, node in the network,
let's say this red node i,
then first I need to decide how to compose a computation graph,
um, and based on the network neighborhood around this node.
And then I can think of the- um,
of the computation graph as the structure of the graph neural- of the neural network,
where now messages' information gets passed, uh,
and aggregated from a neighbor to neighbor towards
to the center node so that the center node can make a prediction.
And we talked about how graph neural networks allow us to
learn how to propagate and transform, um,
information across the edges of
the underlying network to make a prediction and embedding,
uh, at a given node.
So the intuition was that nodes
aggregate information from their neighbors using neural networks,
so we said that every node in the network gets to define
its own multi-layer neural network structure.
This neural network structure depends on the neighbors and the,
uh, graph structure around the node of interest.
So, for example, node B here takes information from two- two other nodes,
uh, A and C because they are the neighbors of it,
uh, in the network.
And then, of course, the goal will be to learn, uh,
the transformations in- um, in- in this,
uh, in this neural network that- that would be parameterized and this way,
uh, our- our approach is going to work.
So the intuition is that network neighborhood defines a computation graph,
and that every node defines a computation graph based on its, uh, network neighborhood.
So every node in the graph essentially can get its own neural network architecture,
because these are now different kind of neural networks,
they have, uh, different shapes.
So now with this quick recap,
let's talk about how do we generally define
graph neural networks and what are the components of them,
and how do we mathematically formalize, uh, these components?
So first, in this general framework,
is that we have two, uh, aspects.
We have this notion of a message and we have a notion of aggregation.
And different architectures like GCN, GraphSAGE,
graph attention networks and so on and so forth,
what they differ is how they define this notion of aggregation,
and how they define this notion,
uh, of a message.
So that's the first important part,
is how do we define basically a single layer of
a graph neural network, which composed basically by taking the messages,
uh, from the children,
transforming them and aggregating them.
So that's the transformation and aggregation,
are the first two core, um, operations.
The second set of, uh,
operations is about how are we stacking
together multiple layers in a graph neural network, right?
So do we stack these layers sequentially?
Do we add skip connections and so on?
So that's the- that's the, uh,
uh, second part, uh, of the equation,
is how do we add this layer, uh,
connectivity when we combine Layer 1,
uh, with Layer 2.
Um, and then the- the last part that, eh,
- that is an important design- design decision
is how do we create the computation graph, right?
Do we say that the- the input graph equals the computation graph,
or do we do any kind of augmentation?
Maybe we wanna do some feature augmentation,
or we wanna do some graph structure manipulation.
Again, uh, in this lecture,
I'm just kind of giving you an overview of the areas where I will
be going to provide more detail and where we are going to go, uh, deeper.
So that's the- that's the third- the fourth area where this becomes,
um, very important, design decisions, uh, have to be made.
And then the last part is in terms of the learning.
You know, what kind of learn- what kind of objective function,
what kind of task are we going to use to
learn the parameters of our graph neural network,
right? So how do we train it?
Do we train it in a supervised, unsupervised objective?
Do we do it at the level of node prediction, edge prediction,
or entire graph, um, level prediction tasks?
So these are, essentially now gave you a kind of an overview of the parts, uh,
of the design- design space, uh,
for neural, graph neural network, uh, architectures.
So as I said, first is,
defining the layer, then it's defining connectivity between layers.
It's about, uh, uh, eh, layer connectivity.
It's about graph manipulation,
augmentation, feature augmentation, as well as,
uh, finally the learning objectives.
So these are the five, uh,
pieces we are going to talk about.