Hi, I'm Adriaan and I am here with Charles from Accenture.
Today we're talking about knowledge management in the oil and gas industry.
Charles, can you tell me what problems Accenture has solved for customers in the oil and gas industry?
So clients tell us they are missing opportunities to buy raw oil
due to the lack of accurate or timely information and that costs them millions of dollars a year.
That sounds like an expensive problem.
How does technology help here?
So, the objective is to ingest documents
coming from multiple sources in the client's side.
It could be report of extraction, it could be process reporting,
it could be many different things via internal communications.
So we're extracting from the existing system of the client that information
and ingesting it into our system.
After that we need to create a sort of representation of knowledge to index documents
and after that a third step will be to retrieve the information.
So it's a three-step process.
It is an ingestion process, it is a knowledge representation process,
and it is a retrieval process.
Both the ingestion and the retrieval have common solution,
so we have been putting a lot of time and effort on specifically the knowledge representation aspect.
Okay, so how do their services that are here on the board help you with that?
So again, at the beginning we have documents coming in in multiple formats to our system.
So they go through our Lambda.
The Lambda will work as a sort of orchestrator between the different services
and create also some logic on top of it.
So first, we have been training ahead of having the system operational.
We have been training our SageMaker instances to do some tasks that are really specific to the oil and gas.
We have been training models specifically for our client programs.
So, it could be some document classification, it could be some key fact extractions,
it could be multiple things.
So we have been training the model first and then after the Lambda will be consuming.
So when the information is coming into the system, the Lambda will send the information, the raw data,
to the SageMaker to the model that has been trained
in order to extract again to some information,
type of information, key facts of the documents, things like that.
Then the Lambda receives the information from the SageMaker
and then will call the Comprehend as a component.
The Comprehend is really the natural language processing component of AWS.
This one goes at more of the sentence level to try to analyze the structure of a sentence.
So for example, it does some name recognition, or part of speech tagging, things like that.
And again, it's important because once that extracted the facts,
I want to understand the structure of my fact.
From there I can start saving my data into my knowledge graph
for future retrieval and for creating my representation.
Okay, what's the reason that you use a knowledge graph here with Amazon Neptune?
We use a knowledge graph for a few reasons, but the first one is obviously it's a database
so it helps us to store and to process data.
But on top of that, a fundamental difference between a knowledge graph and a traditional database
really a SQL database, is that it enables us to have a much more dynamic and flexible structure
where SQL structure has to be predetermined at the beginning of the project.
So it evolved through time with new information coming in.
And that's a fundamental aspect of why we picked a knowledge graph database.
Okay, so this flexible and evolving structure, can you perhaps illustrate it with an example?
Yes. We called the different type of oil - crude.
So, I ingested the first document and the document is about crude type A.
Crude A, from document I'm able identify that it has a density and a sulfur level.
Then, under the search index I ingest another document regarding a crude B.
This crude B has more information.
It has a density, it has a sulfur level, but this time it has a risk associated with it.
So, I can add to that relationship to my object.
That's one very important fact.
On top of that, I'm able to calculate to compute on top of my data some interesting facts.
In this context I'm able to see that.
While if my crude B has a very similar density and sulfur than my crude A,
but the crude B also has this, which is the risk associated with the processing of the crude,
it means that there is a very high chance, there's a probability that there is also
that risk is also associated with my crude A.
So that's one of those elements.
The dynamics and the flexibility of the database, plus the computation part.
That sounds really powerful.
Now, I noticed that you also have Elasticsearch here on the board.
If I have this powerful knowledge graph, what's the reason that I still need Elasticsearch?
So again, the Lambda, at the same time will trigger this process,
will also send the raw information
to the Elasticsearch to be indexed.
Why? Because the knowledge graph
represents a series or a set of concepts that are related with each other.
But potentially we would be missing information if we're not having another way to treat the information.
So here it's structured data.
So we go from unstructured to structure, but sometimes there is a piece of information
that is very important that is not meeting this structure.
So in this context, Elasticsearch makes sure that we are not missing this information.
So if the user types a free text query,
we are able to address even if it's not on our Neptune knowledge graph,
we are still able to address it through the Elasticsearch and to retrieve the information.
That's really interesting. Thank you for sharing this architecture.
And thank you for watching 'This is My Architecture'.
See you next episode.