[MUSIC PLAYING]
JASON DAVENPORT: Hello, and welcome to Next '22.
We hope you've enjoyed our sessions so far where we've
been covering all of the product innovations
that Google Cloud has to offer, as well as listening to what
some other developers and customers
are doing to unlock the business value on Google Cloud.
My name is Jason Davenport, and I'm a Developer Advocate here
at Google Cloud.
And I'm partnered today by Prasanna Selvaraj,
who is a Solution Architect with the Twitter development
platform.
Today, we're going to be talking about how you can leverage
omni-channel, using Twitter, for all of the good insights
and things that you can learn about customers,
and then leverage that into business insights
and activation for your team and across the stack.
Last, we'll talk about how all this comes
together using a new toolkit powered by Google Cloud.
PRASANNA SELVARAJ: Yeah.
Thank you, Jason, and thanks for having me here.
So by using Google Cloud, we built a toolkit purely
based on the serverless technologies of GCP.
The Twitter API Toolkit for Google Cloud
is a framework to ingest, process, and analyze
massive amounts of tweets, and lets developers
detect insights, patterns, and trends across Twitter.
So before I get into the design and architecture
of the toolkit, I would love to speak
about the value of Twitter.
And let's hear it out from Jason.
JASON DAVENPORT: Yeah.
Thanks, Prasanna.
So as I think about how we actually value a tweet,
there are so many things that go into that 280 characters
in really creating that value.
And if we think about 280 characters
being written by hundreds of millions of users, what that
really means is that we start to get to the world's largest
focus group that we can leverage if our company is big
or if our company is small.
So what does that really mean, and what's
in that data that's so valuable?
If we think about what a tweet actually entails,
these are reactions and experiences
that are unprompted, in the moment,
authentic, and emotional.
So let's break that apart.
First of all, unprompted and in the moment
really means that what we're getting from our customers
is their stream of conscious for how they feel about our brand,
or maybe topics related to our brand,
or other things that may just be how our customers are feeling
about things going on in the world
or sports or other topics.
Next, these are authentic and emotional.
What that really means is that, for our customers,
we get how customers are feeling at that point in time
without any filtering or years or months worth of tuning
on that.
And then we also get that actual sentiment associated with that,
not just maybe a logical view of the world,
but that truly emotional feel in that.
So let's talk about how we start to translate that
into business value.
First, what we can do is we can identify trends in real time
because Twitter, at its core, is a real-time platform.
Next, we can build brand affinity.
And what this really means is that we're
working to interact with customers and our brands
where they want to be met.
And so we can talk to customers in the way
that they want to be talked to.
And they can interact with us in the way they
want to be interacted with.
And we can bring this together in a way that
starts to create a virtuous cycle of engagement
for our organization.
And last, this helps us to identify
those moonshot opportunities for the big thing
that we can do next.
If you think of something as simple as a parking
garage, by analyzing tweets maybe of customers
or based on locality relation to the garage,
we could do things like understand maybe
if there are additional services that we could
be offering where our customers would then get even more value.
And we would only know that because of what they had posted
on Twitter at that point.
So what that really means is that now we
can start to think about our customer engagement
along each stage of the customer journey;
from awareness, so do my customers know me
and my brands; to consideration, would they actually
consider working with me; to conversion, which
means that they do want to work with my brand;
and finally with service, so how do they make sure
that they need help and that they're getting the right help
with the brand in that?
All right.
So with Twitter data, if we think
about the value of a tweet, now let's
talk about how Twitter data makes
this such a unique platform that we can leverage.
First off, all tweets are public,
which means that we have all this vast information that we
can start to decode and then understand key terms, trends,
and insights from.
Next, it's real-time and conversational.
So as users feel something, they're tweeting it.
And it's using conversational language
that we can better identify with and understand,
what do those words mean and how do those things actually trend.
Last, we have deep historical insight.
So because Twitter data goes so far back,
if we think about conversational search and trends,
we could understand where do trends actually
change in terms of how people talk?
And maybe things that were relevant in 2012
are no longer relevant today, and we have that huge history
in order to pull that together.
And then last, let's talk about some of the use cases.
So the first and easiest one is sales forecasting, right?
With sales forecasting, we could use tweets and Twitter impact
to understand, OK, maybe we should be
running different promotions.
What will a topic lead to in terms of more or fewer sales
for our organization?
We can also do similar things for New Product Introductions,
or NPI.
This essentially allows us to use Twitter as that focus group
and then understand how our customers want the best
product or service from our organization and tune that
to meet their interests.
Customer support is a very common one.
And here, what we're really focusing on
is how do we take that potential negative sentiment experience
and build it into a brand-building opportunity?
And then last but not least, how do we
pull this all together with marketing campaigns
and other data sources to really build that 360 degree
view of the customer?
All right, Prasanna, let me hand it off to you
here so we can get into the fun parts and the architecture
associated with it.
PRASANNA SELVARAJ: Yeah, absolutely.
So first of all, I want to second that, Jason,
in terms of what you said for the value of tweet.
So Twitter has long been considered a treasure trove
of information, analyzing tweets to understand
what's happening in the world, what are people talking
about right now, and how this information can
support business use cases.
So with millions of monetizable daily active users,
it's no wonder businesses large and small
consider Twitter to support their business intelligence.
But the language is complex.
The journey to words, transforming social media
conversations into actionable business insights,
often involves processing massive amounts of tweets
in terms of organizing, collecting, and sorting them.
Crucial to this process is Twitter API.
It's a set of programmatic endpoints
which lets developers to find, retrieve,
and engage with real time public conversations happening
on the platform.
JASON DAVENPORT: Nice.
So if I think about the broadness
and potential complexity of that,
what are some of the things that actually make it hard to get
insights from all that data?
PRASANNA SELVARAJ: Yeah, so there
is a high degree of friction when processing
massive amounts of tweets.
So a developer has to choose and evaluate
technologies, integrate them, set up data pipelines.
And this process can take anywhere between 60
to upwards of 100 hours.
And after investing that time in setting up the stack,
they still need to sort through the data
and see if what they are looking for actually exists.
So there is definitely a high degree of friction.
JASON DAVENPORT: Yeah.
I think, as you and I were talking
about how we make this better for developers, one of the core
things was how do we go from 60 hours to 60 minutes?
So how did we do that with Google Cloud?
PRASANNA SELVARAJ: Yeah, absolutely.
So with this friction, we always thought, indeed,
there has to be a better way.
So we chose the Google Cloud technology
and built a toolkit to reduce the friction into the data
consumption and also decrease the time to value, as you said,
from 60 hours to 60 minutes.
So the Twitter API toolkit for Google Cloud
is a framework to ingest, process, and analyze
higher volumes of tweets to help developers harness the power
of Twitter.
So in under 60 minutes, they can install the toolkit
and start exploring the tweets for insights.
So this toolkit can be used to detect
macro- and micro-level trends across domains, industries,
subindustries, and verticals.
So the toolkit reduces the engineering burden
for the developers, so developers
don't need to evaluate choose and integrate technologies.
They don't have to build data pipelines.
Rather, the toolkit automatically creates the cloud
infrastructure, automates the data pipeline process
to ingest tweets into the Google Cloud platform,
and offers visualization of trends
in an easy-to-use dashboard.
JASON DAVENPORT: Yeah, I think the core takeaway is 3D, right?
One, needs to be serverless, so I'm only
paying for what I use, but I can use it whenever I need it.
Second, we can obviously scale.
So to your billions of tweets, how do I actually
leverage that if I'm a small business organization?
And then third, what is that capability?
So how do I build on that with the other pieces of that?
PRASANNA SELVARAJ: Yeah, totally.
So scale, auto-provisioning of the infrastructure,
and big data analysis are paramount to this toolkit's
architecture and design.
So let us sift through this and talk to it in detail.
So the toolkit itself has five core components.
JASON DAVENPORT: OK.
PRASANNA SELVARAJ: So the crux of the toolkit is BigQuery,
so the rightmost lane in this swim diagram.
So all Twitter APIs that return tweets provide data
encoded using the JSON format.
The tweet objects has a complex nested structure.
So BigQuery being serverless and it's natural support
for multi-cardinality is a perfect solution
to translate the tweet object into its own schema, which
means developers don't have to translate the tweet object
structure into BigQuery schema.
The toolkit automatically does that with a single API call.
And plus, the toolkit also leverages the batch upload API
of BigQuery, which means it can just
find the tweets at a time, a very crucial thing
to handle scale in terms of millions and billions of tweets
per day.
And in terms of the compute layer,
the layer which is responsible to load the tweets
into BigQuery-- which is depicted in the purple middle
lane--
are comprised of a tweet streamer and tweet loader
service.
The tweet streamer service is responsible to listen
to the realtime filtered stream APIs
and temporarily pushes those tweets
into your Google Pub/Sub topic.
Whereas, the tweet loader service
triggered by a cloud scheduler pulls the tweets
from the Google Pub/Sub topic and ingests back
into the BigQuery.
So these two services are based on Node.js component.
And it could be deployed either on App Engine or Cloud Run.
Again, serverless being the pin here.
So sandwiched between the compute layer and the BigQuery
is the Google Pub/Sub topic, which
acts as a shock absorber for this whole architecture.
So, for example, when there is a sudden surge of tweets,
the Google Pub/Sub can handle the increased volume
with the help of the topic, and temporarily push the tweets
and lets the tweet loader work at its own pace
to ingest back into BigQuery.
And what brings life to this whole architecture
is the Google Data Studio visualization layer.
So Google Data Studio is the dashboard used in this toolkit
to detect insights and trends.
The Google Data Studio connects to BigQuery via SQL connectors.
So once the tweets are in BigQuery,
it's a matter of minutes when a developer can turn this
into a nice-looking dashboard.
JASON DAVENPORT: That's really cool.
So those are obviously a number of products.
How does the user get started with this?
PRASANNA SELVARAJ: Yeah.
So it's very simple.
If anyone has 60 minutes to spare,
they can sign up for a Twitter API essential access.
They can sign up for a Google Cloud, download the toolkit,
and start playing around.
JASON DAVENPORT: That's really cool.
So we've talked about how developers
can start to unlock this data.
Let's maybe shift now and talk about how
business users can get the most value out of these insights.
PRASANNA SELVARAJ: Totally.
Totally.
So let me illustrate a few dashboards that
can be built with the toolkit.
And before we get there, so what are the core capabilities
that a user would like to see from the dashboard,
specifically a business user?
So they would like to filter the tweets.
JASON DAVENPORT: Mhm.
PRASANNA SELVARAJ: They would like
to surface topics and trends.
And most importantly, they want to measure the business impact
and tie it to a higher value business metric
and see whether that kind of moves down or moves up
or basically holds the needle there.
So in the first example, illustrated in this slide,
is real time gaming trends on Twitter.
So the Twitter API allows the developer
to filter tweets based on a specific topic,
in this case, video games.
So on top of this, Google Data Studio can be used to slice
and dice the tweets based on metadata.
So speaking of tweet metadata, examples
could be hashtags, mentions, or topics of interest
based on tweet annotations.
So all of these metadata can be surfaced as insights.
So for example, the hashtags and mentions
are surfaced as pie charts.
Whereas, the topics of interest are overlaid on the top two
tables, and it also gives the users to kind of slice and dice
with additional filters such as language, country, time range.
So net-net what a user can get out of this
is, hey, what is the most popular game across the US?
Or for a given game, what is the most popular topic?
And finally, a user can drill down into this topic
and can get to the tweets which contributed to this trend.
JASON DAVENPORT: That's really exciting.
And as I think about the funnel of building information,
we've been able to go through now.
We've filtered our tweets to understand
the topics and the trends that work most for us.
How do we start to build metrics off of that?
PRASANNA SELVARAJ: Yeah, that's a very good question, Jason.
So to illustrate again the power of Google Cloud technologies,
and specifically Google Data Studio, here in this case,
the tweets are sliced and diced with Twitter metadata.
So, for example, this illustration
is about aggregating the conversations
around these sports teams over a time series.
So what this gives to a business user
is basically they can understand a surge
of a specific conversation and how
that has contributed to a share of voice of a specific esports
team.
And finally, as I said earlier, the user absolutely
has the ability to drill down and get to the tweet level
and understand what type of conversations contributed
to what kind of trend.
And another example could be--
you can absolutely bring in the non-tweet metadata.
So things like-- for example, in this illustration,
you could bring in an attribute like gaming genre or game
publisher.
And you can start to understand what type of game genre
was popular for a given month.
Was it the first person shooter games,
or was it action and simulation?
And similarly, for game publisher,
did a specific publisher games trend?
And you can start to understand for what reason they trended
and what were the conversations in Twitter that
contributed to the trend?
So net-net, you can intersect with tweet metadata.
And most importantly, you can bring
in additional attributes that could
be industry-specific or domain-specific.
And you can absolutely get a new dimension and color
that can give a different business
impact or a different picture altogether.
JASON DAVENPORT: Yeah.
I think, as you were talking, one
of the things that certainly resonated
with me was how do we really leverage all of the data
that we have?
So now that we've gotten our Twitter data,
we could get other marketing data, our sales data.
And now we've really created a way
to create that omni-channel view of our customer,
interact with them better, obviously help them do more
with us and help us to actually do more for them
at the end of the day.
So thank you, Prasanna, for walking us through that.
I think that was really helpful from our perspective.
PRASANNA SELVARAJ: Yeah, totally, Jason.
Yeah, my pleasure.
JASON DAVENPORT: All right.
So it's as easy as these three steps to get started,
which I think is fascinating and really great.
First, download the toolkit from GitHub.
Next, set up your Google Cloud project.
And last, deploy the toolkit to your project.
Once you've done that, you can start
doing all this different slicing and dicing and understanding
how Twitter can really work best with your organization.
With that, I'd like to thank Prasanna for his time today
and being able to share the booth with him.
And for everyone outside listening in,
thank you again for your participation,
and really hope you enjoy the next upcoming sessions here
at Google Cloud Next 2022.
Thanks again, and have a great day.
[MUSIC PLAYING]