Welcome to This is My Architecture. I'm Matt from AWS.
Today I'm joined by Abhishek from Vizio. Thanks for joining.
- Thanks, Matt. - So tell us about Vizio.
Vizio is one of the largest TV and sound bar makers in America.
And our focus is on making smart TVs and bringing a better experience for our customers.
Great. So this TVs are not the TVs that our parents had.
They're smart like you said.
They have lot of data, a lot of telemetry so,
I see we have a TV over here.
Tell us about how you're using that TV data with AWS.
So TVs do generate a lot of data for us,
and that data could be logs, right? - Yeah.
And we do post a lot of performance study metrics.
- Sure. - And whatever is happening on the TV.
And then there is also some user data.
We do wanna know how our application is being used to bring even better
experience for our users. - Yeah, sure, so you can do better things
for customers with this data. So the raw data doesn't tell you much;
you have to process it, ingest it.
So, I see you have Kinesis. Is that kind of your front door to the cloud?
It is true. That Kinesis Data Streams are our front door and everything
is being piped through Kinesis.
And we do run a HR Kinesis byte plan
which we have not yet needed to scale it up.
OK. So these TVs are talking directly to Kinesis to ingest or to send
their data up to the cloud? - Yeah.
And you could but you haven't had to scale the Kinesis advantage of removing shards?
- Not yet. - OK. Great. So data gets to Kinesis
from the TVs, different streams, I imagine, different types of data, logs performance user.
Do these have different characteristics or are they in different formats?
They are all different formats and different columns, if you will.
And the best part here is that we are not sticking to any particular schema yet.
So we are free to send anything we want, in any format,
and they we will pipe it through Firehose.
Kinesis Firehose takes the data as is and dumps it into Redshift.
So you're using Redshift as a data warehouse. So you collect the data,
it goes through Kinesis, you use Firehose to ingest it into Redshift.
I see S3 now here. Does that mean you're using Redshift Spectrum
or how how are you using S3?
S3 is just our data store and it's cheap, so we just keep it there forever,
for future use. We do not necessarily import all the columns we send out here to Redshift.
OK. So the raw data finds its long term home in S3,
and then a subset of that gets ingested into the data warehouse.
- That is correct. - OK. Have you always used Redshift
or have you experimented with different types of databases? - We did start experimenting with AWS Aurora.
- Yeah. - And we figured out that,
billions of roles, I know do not aggregate well in Aurora.
- and that was just not the right use case for it. - OK. Yeah, I mean, that's why we have
different types of data stores and databases in AWS, for different purposes.
And certainly, for the scale maybe you're talking, with all the TVs out there.
How much are you talking? What kind of scale are we talking about?
We are talking about millions of connect devices and billions of events per month.
OK, so billions of events per month that you want to aggregate.
It's definitely an analytics use case. It's a great case for Redshift.
So the data...I understand so far you persisted in S3 long term.
I guess you could reload it, if you ever need to replace the database or data warehouse.
That is true. And we did that when we experimented with Aurora.
- Yeah. - We had to quickly migrate to using Redshift,
and at that time we had everything in S3 and in the data streams,
we were able to just reload everything in Redshift without any data loss.
OK. Then how do you actually visualize the data? How do your users make use of it?
- For that we use Amazon Quicksight. - Mm-hmm.
And Quicksight is a great tool for visualizing aggregation
and we do use the in-built, in-memory database called SPICE.
Where we write Custom SQL or Custom Queries to download the data from Redshift
to the SPICE data sets, from where it is very snappy, you create, slice and dice
the data and create the reports you want.
Great! So you take subsets of data, using custom SQL, you put it into the in-memory SPICE database
for Quicksight so your users can visualize it quickly and generate the reports.
- Yes! Yeah. - OK. That's very cool.
I love the scale that you're operating at.
How long did it take you to build this whole thing?
So, we started this project with Amazon Professional Services Help
- Yeah. - and I think within a few weeks span,
we had the first prototype and then we were like 40, 50 viewers in the production.
So from 0 to users being able to use Quicksight and run reports on it,
this was like maybe 6 weeks.
It's great to hear you're having success with Professional Services and
like 6 weeks to production for an analytics where we're talking billions of messages.
- It's pretty amazing. - It is pretty amazing, yeah.
And Redshift, by the nature of it, it is a great way of
running large aggregation because it fetches columns
so the IO is very optimized and we could see the results right away in the through-put.
Absolutely. The benefits of a columnar database like Redshift
- That's true. Yeah. - for large scale analytics.
Very cool to see. Well, I like it's sort of an elegant solution
to a scale problem, to make use of all this data that our smart TVs
are able to produce these days.
Thanks for sharing with us. This was very interesting.
- Thanks, Matt. - And thanks for watching This is My Architecture.