Welcome to 'This is My Architecture.'
I'm Lana from AWS, and today we're talking to Frank from Dow Jones.
Welcome, Frank. - Thanks for having me.
Awesome having you here. Would you mind telling us
a little bit more about Dow Jones and the work you do there
with The Wall Street Journal? Sure.
So I'm a distinguished software engineer at Dow Jones.
Dow Jones is a global publisher of news and financial information.
And it produces publications in products
like The Wall Street Journal, Barron's Magazine, and Market Watch, and Factiva.
I would imagine that it's really important to make sure that you have
your newsrooms and your publishers have the information
they need at the right time.
So would you mind telling us a little bit more
about NewsGrid and the platform that you've built on AWS?
Sure. So the application that we built is called NewsGrid.
And the users that are running either in mobile or on desktop
access the application from all around the world.
We have news bureaus in countries globally.
So what we do is in order to provide disaster recovery,
the users will first access Route 53.
The application will determine where we're live,
whether it's in our primary region, like Virginia,
or in a backup region, like Oregon.
After we have resolved that, then the user gets redirected
to our load balancer,
and the load balancer then will enforce authentication
using Cognito and Okta as a identity provider.
Once the load balancer has ensured that the user has a session,
the request is forwarded to one of our availability zones running EC2s.
And the EC2s serve the purpose of providing the application
to the client, or to the browser,
and also proxying to some internal services on our VPC.
That makes sense. So I'm assuming you're using
a couple of availability zones here.
Yes. Yes.
So the availability zones give us the
redundancy and the ability to recover in case one of them is not available.
- Sounds great. So now I'm seeing that we're moving on from EC2s.
So we have some business logic and some processes that are running here.
And this is the fun part-- AppSync. - Yes.
So would you mind telling us a little bit more of how you're using AppSync
as a part of the solution? - Right.
So AppSync is core to our application.
The EC2s forward requests to AppSync,
whether they are mutations or queries.
And AppSync is the API that we provide.
We used AppSync instead of REST API because,
like I mentioned before, our users are located globally.
They're not always in an office with great bandwidth.
So we want to make sure that the data that we're sending to the client,
we can shape that payload and make it as small as possible.
AppSync provides a way for us to also
reach out to multiple different types of data sources.
So, whether it's DynamoDB, which is our primary data source,
or Elasticsearch, which we leverage for doing
searching, sorting, and advanced filtering,
or Lambda, which provides some offload of more complicated--
more complicated jobs and tasks that the client,
we want to move away from the client onto the server.
And I'm also seeing here that we're using Elasticsearch and DynamoDB.
So would you mind telling me a little bit more of
the different ways you're using these data stores?
Sure. So like I mentioned before, DynamoDB is our primary data source.
But we replicate our data from DynamoDB
through streams using Lambdas,
which then send the data to Elasticsearch to keep it synchronized
with DynamoDB, so that we have the same view of data when we search
that we do when we get or put an item.
- So I would imagine that would perform a much more
perform and querying functionality for your users.
Yes, and that was a critical component.
Initially, we didn't have search in our application.
And then when it was determined that we needed it,
we were able to stand up an Elasticsearch cluster pretty quickly.
- Sounds great. So AppSync has a lot of very interesting features,
like subscriptions. Have you utilized that as a part of this architecture?
Yes, absolutely. And that's critical because our news department
wants to make sure that they have the most up-to-date data
about articles that they're writing in the NewsGrid application
for The Wall Street Journal. So AppSync will actually send us
updates to all the stories, and plans, and subscriptions
that the users require in the browser,
and it uses the MQTT protocol to do that.
And what kind of clients do you usually have that users leverage?
So our clients are both mobile and desktop. Yeah.
And so someone could be in Afghanistan
writing an article and use their phone,
or they could be sitting in an office or a bureau somewhere else,
like in London or Australia.
That makes sense. Another question I had is that
you mentioned that you have multiple regions
as a part of your DR and HA approach.
So how are you replicating this data between multiple regions
and replicating this infrastructure?
Right. So this is a critical application we want to make sure that it
can recover from disaster.
So we have this entire infrastructure,
deployed both in Virginia and Oregon.
We replicate our data using DynamoDB global tables.
The data that's sent from DynamoDB in Virginia,
where we're potentially primary, will get sent to Oregon.
Once it's received, a stream will do the same sort of replication,
or export of the data, from DynamoDB through a Lambda
and into the Elasticsearch cluster in the replicated region.
That's great. So largely, we have, like, an async environment
with just the stateless data being replicated by the global tables.
- That's right. Yeah. - Sounds great.
Well, Frank, thank you so much for coming today
and telling us a little bit more of how
The Wall Street Journal leverages AWS Services for their newsrooms.
- My pleasure. Thanks very much.
- And thank you for watching 'This is My Architecture.'