DEBORA ELKIN: Hello, and welcome to this session.
My name is Debora Elkin, and I'm a technical solutions
consultant with Google Cloud.
I'll be presenting this session with Warren Vella.
He's an enterprise integration architect
with the Australian Energy Market Operator.
In this session, we will see how APIs
can help you unlock the power of your existing data
and how the Australian Energy Market Operator is
leveraging Apache to do so.
We will also see AEMO's APIs in action.
We all know that this is the age of connective experiences.
Your customers, or your customers' customers,
want to interact with applications and data
in different ways, in different channels,
on different platforms.
These interactions are constantly changing,
and everything's connected with everything else.
APIs are key enablers of digital experiences.
Think how pervasive Google Maps or how booking systems
have made our lives easier.
Of course, this data involved in all digital experiences,
whether it be available flights on a travel booking system
or your bank account details in online banking
or just about anything you can think of, really.
However, the data you need to build these digital experiences
is often buried deep within a company's systems of record.
It could be in data warehouse, data lakes, files
that could be scattered everywhere, disparate systems,
legacy systems have to connect to.
This data is most likely locked away, hard to get at,
and it takes a lot of time to surface it.
So somewhere, there is a system of record, holding data that
is hard to get at.
It can take months or years to make the changes required
to make that data available.
But customer or consumer's expectations are high.
They want the digital experience that
is underpinned by that data, and they want it quick.
This is where API management fits in.
It fills the gap between the systems of record
and the requirements of new digital experiences.
It helps organizations build a bridge between legacy systems
and mother applications.
And it can build that bridge in a secure, reliable,
and scalable way.
So let's talk about Apigee.
Apigee is Google Cloud's API management platform, not just
an API gateway, and gives you all the tools
you need for unlocking your data via APIs.
Apigee enables digital business acceleration
with a unified and complete platform, purpose-built
for the digital economy.
Apigee makes it easier to manage the entire digital volume
chain with API services, developer services,
and analytic services.
Let's start at the bottom with API services.
This is the API runtime and mediation engine.
API services lets you manage the connection between the API
consumers and the back-end service.
You have complete control over the request and response.
You can add security, transform requests and responses,
orchestrate calls across multiple systems
impose rate limits and quotas and prevent people
from using your services.
API analytics helps you measure, tune, and monitor
your API program.
It enables end-to-end visibility across the digital value chain
with developer analytics, operational and business
metrics, reports about app performance,
and custom reports that you may need.
All are necessary to monitor, measure, and manage
success of your API program.
Developer services enable a great developer and community
experience that accelerates API adoption, simplifies learning,
and increases the business value of APIs.
It provides a developer portal for your API consumers,
along with features for packaging, monetizing
your services and data.
When you're sharing your data, you
want to make sure you do this in a secure way.
When we talk about security, there
are many different aspects.
For instance, data should only reach the right audience.
This is where authentication and authorization come in.
In that respect, Apigee supports widely used standards,
such as OAuth and OpenID Connect.
You also don't want anyone snooping
around when the data is in transit.
This is where encryption plays a key role.
Apigee supports TLS, mutual TLS, and can
support different configuration of these protocols
when communicating with API clients and back-end services
that hold it back.
For example, you may configure one-way TLS on the client side,
but mutual TLS list for one of your back-ends
and one-way TLS for another.
Auditing is also very important.
You need to know who has access to what data, who
has made a change to the API that exposes that data.
Apigee auditing and logging capabilities have you covered.
Also, is this sensitive data?
No problem.
Use an Apigee extension to quickly redact data
with GCP data loss prevention services.
You could potentially think of making money out of your data
as well--
that is, if you wish to.
Of course, not all APIs should necessarily be monetized.
There are APIs that built consumer experience
or drive indirect value.
They should be free-- for instance,
think about API for ticket booking,
finding store locations, product availability, et cetera.
On the other hand, if your APIs help developers
build apps that generate revenue,
you may consider charging for those APIs--
for example, APIs that provide weather
data or messaging services.
Apigee API monetization offers multiple options
to set up charge plans.
You can configure fee-based plans, volume-based plans.
You can set up revenue sharing with developers,
freemium models, et cetera.
It also allows you to check monetization plan limits
during API execution.
In short, whether you want to build new digital experiences,
create a partner ecosystem, or apply artificial intelligence
or machine learning to your data, unlocking your data
is the key.
And APIs are the way to achieve this.
We will now see how the Australian Energy Market
Operator is unlocking its data and putting it
to effective work in an ecosystem.
I'll now hand over to Warren.
WARREN VELLA: Thank you, Deb.
Hello, my name is Warren Vella, and I
am the enterprise integration architect
at the Australian Energy Market Operator.
Today I'd like to talk through how AEMO has partnered
with Google and the Apigee platform
to unlock the power of our existing data through APIs.
We'll take a look at who AEMO is,
some of the unique challenges the energy industry is facing,
and how APIs have been instrumental
in our digital transformation.
So where does AEMO sit within the Australian energy market?
AEMO is responsible for the day-to-day operations
of the Australian energy market, which
is made up of multiple markets covering both gas
and electricity.
On one side, AEMO manages the physical named
grid, the transmission network, the large electricity
towers that you may have seen around the country.
On the other side, AEMO operates the financial energy markets
to enable energy flow all the way from generation
to the customer households via retailers.
As you can see, some of the actors in the energy markets
are typical, large-scale generators,
such as coal power plants and wind
farms, all the way to metadata providers, who
are responsible for sending the metadata from households
to AEMO.
Sometimes this is automated via smart meters,
and sometimes this is manually read.
One of the biggest shifts we are seeing in the energy market
is the active role of the customer, who is now
also playing a critical part of the grid,
especially as power is starting to flow two-way, as into houses
and now out of houses.
Let's take a look at some of the technology challenges
that the energy industry faces.
If we look deeper into the national electricity market,
which was built over 20 years ago,
the systems are largely are connected over either SCADA
operational systems over private networks or market data--
for example, dispatch data, which is an instruction sent
to a generator every five minutes--
or bids and offers into the market,
which is received from generators and retailers,
is still sent over to AEMO by batch-based systems
over private networks.
With the increase of DER penetration across Australia,
which is leading the world, we have a unique opportunity
to unlock the value in our data over modern protocols.
And so what can AEMO do about this?
We have a unique opportunity where
we are seeing an intersection between the energy industry
transformation and our digital transformation.
And as part of our digital transformation,
we are tapping into these opportunities
to enable new channels to really lower the barrier to entry,
promoting further competition in the market.
And as we digitize our systems, we
are looking to foster innovation within the energy market,
especially in the space of distributed energy.
More importantly, we are seeing the rise
of prosumers, who are looking to take control of their energy
profile and play an active part in this energy transformation.
And so working with Apigee has enabled
us to start this journey as we look to reach a wider audience.
Our new Apigee development portal arms us with the tools
to create a world-class developer experience,
accelerate API adoption, simplify learning,
and increase the business value of our APIs.
Developers are the key consumers of our APIs,
and we want to provide them with a great experience
and make it really easy for them to discover,
understand, test, and get going with our APIs,
from having an interactive sandbox to test
our APIs to letting developers manage their own keys
and access to the APIs, and not necessarily
in an open-for-all approach but with the right guardrails
that Apigee offers.
Our Apigee developer portal has all the tools
a developer needs to integrate with us.
Let's take a look at some of our large-scale programs
and opportunities to unlock the value of our data.
The energy industry is moving along
with many programs, centered around a common theme
in distributed energy resources.
Starting with the distributed energy resource register,
DER register allows registration of DER assets
sitting behind the meter--
for example, rooftop solar panels and household batteries.
APIs have enabled for this data to be registered
via the existing tools and mobile applications
that electricians and installers already used to.
The virtual power plant trial has
started to advance the integration of DER
into the national electricity market,
allowing consumers to group together
to become a virtual power plant.
APIs have enabled for the management of these VPPs,
especially in the transfer of telemetry data.
And in the space of five-minute settlements,
the market is moving to more granular data and settlement
down to five minutes.
APIs have enabled for meter data at five-minute resolution
to be sent across the country to aim as metadata management
database.
Next we'll take a look at the CDR.
The CDR is a national legislative program,
which is running across multiple industries,
with the energy industry next in line.
The CDR aims to give consumers better access to their own data
and also allows competition by using a data-driven, innovation
approach.
So let's take a look at how we are using the Apigee
platform to support the CDR as a use case.
Some of the key areas of why Apigee has been selected
was the choice of implementation options,
especially with the hybrid option giving us
even more flexibility on where to host our APIs,
whether that is in the Google Cloud platform
or in our private cloud.
Data sovereignty is especially important
as we operate national critical infrastructure,
and the flexibility of the platform
allows us to host the platform to many
of our strict obligations for the protection of our data.
And so the main focus of the Apigee platform
is to manage our externally facing APIs.
In the case of the CDP, this allows
us to expose data from our metadata management systems.
And like most organizations, we have
data spread over multiple cloud platforms and on-prem systems,
and Apigee allows us to unify the experience
when integrating with our APIs.
Apigee is truly enabling our digital transformation
as we renovate our internal systems from legacy integration
protocols to modern API ecosystems.
Now we'll take a look at a live demo
and explore how the Victorian government is
using our APIs to provide a comparison service
for Australian consumers.
Here we have an example of an external consumer.
The Victorian government energy comparison website,
who was consuming the AEMO's CDP APIs
to provide real-time metadata retrieval from AEMO's metadata
management systems.
And before the CDP APIs were available,
the end user needed to complete over 10 questions,
and in some cases, a PDF bill had to be uploaded.
Now that the API's available, the user only
needs to know their national metering
identifier, or their NMI, and the data
is retrieved live from our API.
I'll now hand it off to Deb to walk us through a live demo.
DEBORA ELKIN: The demo we're about to see
is from the point of view of the consumer wanting to understand
which energy retailer offers the best
rates for the particular electricity consumption
patterns.
I start looking around, and I find a public website
that has been set up by the government of Victoria,
Australia.
When this website was first launched,
the website would estimate the energy consumption pattern
based on a series of answers provided by the consumer.
This was the simplest option but still
required a lot of information to be provided.
The other option was for the consumer
to upload a PDF copy of their bill.
I will now start answering all the questions I am being asked.
After providing this information,
I also need to provide the details of my last electricity
bill.
At this point, I can finally see which
rates are the most convenient to my particular situation.
But I had to answer more than ten questions to get here.
And still, it is just an estimate.
Let us now see how APIs that expose data
can improve my user experience, as well as provide
more accurate results.
So let's start again.
This time I will choose a different method
for building my energy profile, quick and accurate.
This looks promising.
All I need to provide is a NMI.
This is a unique identifier for the smart meter installed
in my house.
I can find it in my electricity bill.
This option is now powered by AEMO.
What this means is that when I click on Get My Data,
this website uses an AEMO API to retrieve the actual energy
consumption for my house in the last 12 months.
This provides a very accurate picture
on how I consume energy.
Voila, I can now see the rates best suited to my household
without having to go through a lengthy questionnaire.
I, as a consumer, didn't even need
to see the raw data coming back from the API.
This useful experience is facilitated
by the API published by AEMO.
Looking at AEMO's API portal, a developer
can easily find the relevant APIs
and understand how to use them.
In this demo, we can appreciate the advantages
of unlocking data via APIs.
This is a public government website
that leverages information held by AEMO to provide
a much better user experience.
Not only are the suggestions more accurate to the consumer,
but also the consumer can see them in a couple of clicks.
Thank you for watching it.
WARREN VELLA: So what's next for AEMO and Apigee?
One of our main focus areas is to enable DER resources to be
integrated into the grid, which will increase API
calls to millions per day as we start to receive
more and more telemetry data.
And for the CDP API, we are looking to expand this service
and enable data service holders to also provide
data via the CVR gateway model.
We are always looking to reduce our participant onboarding time
and pivoting off our developer portal
to provide the right amount of information
and further reduce the time to first, "Hello world."
And of course, our API product portfolio
is expanding as we continue to transform our systems as APIs
become our primary form of integration
for our critical market systems.
Thank you for your time, and I'll now
hand it back to Deb for some closing remarks.
DEBORA ELKIN: Thank you, Warren.
Today we looked at how APIs can be used to securely expose
your data and how the Australian Energy Market
Operator is doing this.
We want to thank you for watching this session,
and if you'd like to continue the conversation,
we have set up some expert one-on-one sessions
over the next couple of weeks.
And of course, we will also be answering questions
you submit in the question and answer
section for this session.
Thank you.