[MUSIC PLAYING]
MIKHAIL CHRESTKHA: Welcome to "Architecting
with Google Cloud."
Machine learning continues to be applied
to use cases across industries.
The airlines and transportation industry is no exception.
I'm thrilled today to chat with Ksenia Khlyustina, a machine
learning platform manager at FLYR Labs,
where her team leverages data and machine learning
to help airlines better forecast demand, set prices,
and optimize revenue.
I'm particularly excited to also have Ksenia give us a peek
into the core components of FLYR Labs as ML platform.
Welcome, Ksenia, and thanks for joining today.
Can you start off by telling us a bit about yourself?
KSENIA KHLYUSTINA: Sure.
Hello.
Yes, my name is Ksenia, and I've been with FLYR for three years.
So in general, I have about eight years
of experience in the industry in multiple domains.
But, however, I started working in machine
learning only three years ago.
MIKHAIL CHRESTKHA: And now, can you
tell us a little about your company, FLYR Labs?
KSENIA KHLYUSTINA: FLYR Labs is a company been bringing
intelligence to airlines.
So we are trying to use machine learning
and AI to be able to predict demand and optimize
the pricing for the airlines.
So we want to make sure that every flight, at takeoff,
is 100% full and bringing the maximum revenue to the airline.
MIKHAIL CHRESTKHA: Oh, wow, that's great.
And that field is especially particularly relevant
and important today for the airlines industry
in the midst of the pandemic and now how consumer behavior
changes really, really fast.
So now we are here to talk a little bit
about machine learning.
Could you discuss some of the top use cases ML is applied to?
KSENIA KHLYUSTINA: We basically have only one use case for now.
So we are taking all the historical data
from the airline.
We're taking their competitors' information.
We are taking information about future events
at the origin or destination of the flight
and putting this all into our models
and outputting the demand forecasts, which
are used to optimize the pricing and deliver this
to the airline that are lightly used in their production
systems.
MIKHAIL CHRESTKHA: And before we dive into that use case
and how you've architected a platform around it,
I think I wanted to dig a little bit deeper into your team
and how it fits within the broader FLYR Labs organization.
I think a lot of companies today,
they might only have one data science team.
They struggle to operationalize ML models.
Could you talk a little bit about your team
and the other teams at FLYR that are tackling this problem?
KSENIA KHLYUSTINA: Yeah, so FLYR is mostly
built around the design.
So of course, we have more than one data science team.
We have a data science organization,
which my team is a part of.
So this data science organization
consists of three teams--
data science team itself, machine learning platform,
and machine learning infrastructure team.
Besides this organization, there is also
validation team, which is kind of participating
in machine learning, data science, as well.
But they're a little bit apart.
But I will talk about them as well.
So data science team is a team responsible for basically
building a model itself.
They are pure data scientists, trying not to touch any MLOps.
Then machine learning platform is
a team that builds the platform, as you can understand
from the name, it builds a platform around these models
and machine learning.
So this is the platform that is used
to run inference and production environment and not
only production environment on daily basis,
and deliver and basically use the models on the production
system.
And then there is a machine learning infrastructure team,
which is focusing on empowering data science team
and machine learning platform team to do their job.
So they are mostly focusing on tools around all of this.
So that's how it looks.
And then validation team is a team
that is responsible for testing the models, let's say.
They are trying to see how the models are performing
in real life or over many days of being used in production.
So they are trying to validate how good our models basically
are.
MIKHAIL CHRESTKHA: Great.
Yeah, thanks for sharing.
I think it's great to see that you're really at a maturity
level where you have different teams focused
on different parts of the lifecycle
and really optimizing the time to get models into production.
And you really have great delegation
of duties across teams, here.
So we are here, again, to talk about architecting
on Google Cloud.
So maybe you could give us a little bit
of the story around why you initially chose Google
Cloud a number of years ago.
KSENIA KHLYUSTINA: We made this decision around six years ago,
when we were still a small startup.
And one of the main reasons for it
was Google Bootcamp, Google startup empowering program,
where Google gave us a credit and we
were able to use Google Cloud Platform basically
for free for a while.
This was quite a big thing for us
because we didn't have much money at the time, let's say.
But that was not the only reason.
The other reason was that Google had the great support
for Kubernetes, probably the best at the time on the market.
And we really wanted to be able to use
Kubernetes in our solution.
So this is reason number two.
Why are we staying with Google?
What do we like about this?
It's still the great Kubernetes support and BigQuery,
which BigQuery is--
basically, our whole company runs on BigQuery.
So we are tightly bound to this.
And so far it's been working great for us.
MIKHAIL CHRESTKHA: Yeah, thanks.
Yeah, great to hear that story.
I'm glad that FLYR Labs is still on GCP, kind
of growing with your business.
And it feels like, really, that big bet on BigQuery
and Kubernetes has paid off and is really part
of your core product offering.
So now let's get into the ML platform itself.
In addition to BigQuery and Kubernetes,
what are some of the other GCP services that
make up your ML platform today?
KSENIA KHLYUSTINA: Sure.
We're using Dataflow.
We're using Vertex AI.
We are using Kubeflow and TensorFlow.
MIKHAIL CHRESTKHA: OK, great.
So you seem to have really adopted some of the industry
standards and really focused on open source, which
I think is a big core component of Google Cloud.
As well, one of our main objectives
is really to support the community,
especially in the open-source space.
So now, can you maybe talk about the main components of your ML
architecture at a higher level, to start?
KSENIA KHLYUSTINA: Sure.
Three main components are our three pipelines--
training dataset generation pipeline,
model training pipeline, and daily inference pipeline.
MIKHAIL CHRESTKHA: Great, thanks.
Yeah, I think breaking it down like that really makes
things a little bit clearer.
I think sometimes architecting a broad ML
system can feel intimidating or complex for teams.
But focusing on how data flows through the system
with your three-pipeline design really makes
it much more understandable.
So can you start off with the training dataset generation
pipeline?
KSENIA KHLYUSTINA: Of course.
Training dataset generation demands the most resources.
So how it works-- it uses Kubeflow as an orchestrator.
It pulls data from BigQuery.
It's usually a few years of airline historical data.
And then it runs on Dataflow using beam programming model.
As an output, it generates TFRecords,
that we later put in Google Cloud Storage
and register in our model metadata store.
So this pipeline is very demanding.
That's why we run it on Google TPU.
And even though TPU is very fast,
it still takes hours to generate such a training dataset.
MIKHAIL CHRESTKHA: Yeah, I think it's somewhat common knowledge
that data preparation is still the biggest
task within a machine learning pipeline.
And it's interesting to kind of hear your story resonate
with that, that it is the most demanding and
resource-intensive part of the entire pipeline.
So now, once you have the raw data processed into TFRecords,
what happens then?
Can you walk us through the training pipeline?
KSENIA KHLYUSTINA: Yes.
Then there is another pipeline that also uses
Kubeflow as an orchestrator.
And then it takes TFRecords generated by previous pipeline.
It takes TFRecords from Google Cloud Storage,
and it takes model source code, which is just a Python package.
So it runs in Vertex AI, and as the output
it generates a trained model in the format of TensorFlow
SavedModel.
We put it in Google Cloud Storage as well,
and register in model metadata storage.
So model metadata storage, I already mentioned it.
Let me tell you about this.
This is our internal tool that we use for,
as you can guess from the name, storing model metadata.
So it has models with parameters.
And the parameters usually include model type,
model purpose, model ID, for which client
this model is being registered, and on which environment
we want to use this model.
So it can be a production model, integration model,
stage-in-development model, and so on.
And then in this model metadata storage,
we also keep history of our dataset.
So it also has versions of data set and their ID.
So later, you can just reuse models
and reuse data sets by just looking at the model metadata
storage.
MIKHAIL CHRESTKHA: And thanks for sharing that.
I think model registries and model metadata storage
is really a interesting topic today.
I think when you start with your first model or your first five
models, just storing those models in a blob storage
is sufficient.
But really, when you scale to tens,
hundreds of models, really developing
a specific workflow around managing models
becomes very important in the ML lifecycle.
So now that you have models, of course
the next important step is deploying those models
to run predictions.
Can you now walk us through the final pipeline around serving?
KSENIA KHLYUSTINA: Yes.
So this is our pipeline that we call daily inference.
And this is actually the pipeline
that my team is focusing on.
So this pipeline is listening to clients' data.
So it's listening to Pub/Subs from BigQuery,
where the client's data is being dropped.
After we collect all the necessary data,
the pipeline is triggered.
And as an orchestration, we use here Argo, not Kubeflow
as in two previous pipelines.
So using Argo Workflow, the pipeline
is being triggered and running in Dataflow as well,
using beam programming model.
Model inference is pulling model files from GCP,
using model metadata storage, and data from BigQuery.
So after the predictor is executed,
we have predictions in form of data in BigQuery.
And then using this data in BigQuery,
we can pack these predictions in whatever
form our customer needs.
So for some customers, we deliver it through the API,
and for some customers we deliver in the form
of text files on SFTP.
So on this prediction in BigQuery,
we also build some metrics which are later empowering our UI,
and where analysts from our airline can go
and check what's going on with any specific flight
and make some decisions if they want to adjust model outputs
or not, and if they want to adjust prices.
MIKHAIL CHRESTKHA: One follow-up question-- a lot of enterprises
are looking to future proof their platforms for scale.
So can you talk about maybe how you've designed this platform
for the scale you expect as you onboard new customers, more
models?
KSENIA KHLYUSTINA: Yes.
We are pretty ambitious, and we want to serve more airlines.
So, of course, scaling and onboarding
new customers is a big challenge for us.
But how are we solving this?
We don't want to be a professional services company.
We want to have one generic solution that fits all.
Of course, it can be configurable,
but it's still one solution.
And then when we have every new customer,
product team-- or our core engineering team--
don't need to do anything.
We already have the solution.
We already have all the pipelines.
So we only need to hire a customer-success team, which
needs to take that data from the customer,
transform it into our canonical data model, and then,
using the pipelines that are already created,
start training model on this specific customer data,
and then run this model on our prediction pipeline.
So, basically, they're empowered to do everything by themselves,
and core team can focus on other features.
That's how we onboard new customers.
It's how we scale in the amount of the airlines we can onboard.
But also, this is not the only challenge with scale.
And we, of course, want to broaden our services.
And we don't want to only predict
demand for economy cabin, for instance.
We also want to go farther and predict premium.
We also want to predict demand on luggage and the [INAUDIBLE]
services you can add to your tickets,
and then go to cargo, and so on.
So this, of course, will result in multiple models.
So how we will scale this?
We designed the pipeline in the way that--
where we're just able to add as many models as we
need and run them in parallel.
So the only thing that we will be
facing during the challenge we'll be facing is resources.
But, of course, we can always buy them from Google.
MIKHAIL CHRESTKHA: Great.
Thanks.
So we all know that ML platforms are a fast evolving space.
So curious what future enhancements you have planned.
KSENIA KHLYUSTINA: Yeah, we have a few.
So first of all, we want to add feature store to our tooling.
So this is something that would allow
us to version our features to just better manage them.
And then also, something that would let us not generate them
on daily basis but only append.
Because historical data from the airlines, it doesn't change.
We only get a new portion of data every day
and for yesterday.
But, yeah, we don't need to regenerate the entire history
to be able to run new inference.
So this is something we've been working on.
And apart from that, we also want
to be able to do dynamic pricing.
And for that we will need to add some online modelling
to our inference pipeline.
So we want to be able to receive a request from customer
and then be able to respond immediately
with an optimal price.
MIKHAIL CHRESTKHA: Great.
Yeah, thank you-- really exciting
roadmap, both from a new business capability
standpoint as well as the technology capabilities
of the ML platform.
So thank you, Ksenia, for joining us today and sharing
what FLYR Labs does and how you've architected an ML
platform on GCP.
KSENIA KHLYUSTINA: Thank you so much.
MIKHAIL CHRESTKHA: Yeah, really had a great conversation.
It's especially important that everyone learn from each other,
as MLOps and ML platforms are still a fairly new field.
I personally learned a lot, and I
know many others will as well.
Everyone, if you like this content,
please like and subscribe to this channel.
[MUSIC PLAYING]