Hello and welcome to the Exactpro channel.
If you want to know more about us please check out our website at exactpro.com
and subscribe to our channel so that you are not missing out on the upcoming videos.
At Exactpro we build software to test software,
so today we’ll cover yet another solution. In previous videos we’ve introduced ClearTH,
our testing tool for post trade. Now ClearTH is adapted for software testing
of Distributed Ledger Technology applications built using DAML, the open source smart contract
language created by Digital Asset.
DAML provides the capacity to run on top of multiple blockchains
which is a crucial step towards shaping industry standards for DLT
DAML easily allows parties to mutualize their processes
and secure a golden source of truth with built-in privacy.
As an open source and platform-independent smart contract language,
DAML enables the developers to write an application once
and deploy it anywhere.
Today we will have a look at ClearTH test
automation of the functional work flow of swaps
implemented in the IRS & CDS Swaps Lifecycle Module application
based on the Common Domain Model developed by the ISDA association.
Swap as a derivative with great flexibility is actually a very interesting and a challenging
case, as almost every change in the parameters will result in a different set of potential
events and cash payments. So it is necessary to make sure that all the corresponding events
are not only generated and can be accepted or applied, but that their creation is in accordance
with the specific derivative product that we aim to trade.
We decided that a model-based approach would
be the best to meet this challenge. The approach by Exactpro is an effort to build
a whole universe of test scenarios, as opposed to testing a certain limited number of scenarios,
and aims to cover all aspects of the system under test.
To study the Digital Asset’s Swaps application, the Exactpro team built a separate Excel model
describing first the type of the swap contract for example IRS, basis swap, equity swap, CDS
and the structure of the fields and possible variations,
and second the test scenarios specifics for example segregation of trades
or multiple contracts held by one member; time travel schedule; which events
are supposed to be there, etc.
This model is used as a template for test scenario generation.
Besides this template, there are spreadsheets containing Input data to be added into it:
a Configuration file with different variations for groups of parameters, and spreadsheets
with reference data about participant and their master agreements, accounts etc.
and market data for example Reuters feed for floating rate fixings.
Both template and the input data files are in a human-readable format.
Based on this, a programmed function within the model powered by a macros generates
a number of scenarios in the form of csv matrices,
which are the scripts that can be executed by ClearTH.
ClearTH takes the hundreds of scenarios generated within this approach,
groups them by the scenario steps,
executes concurrently against the application to see if there are any defects,
and produces the reports.
For the demo we have a scenario where ClearTH simulates a participant who connects
as a party and submits a trade. ClearTH script sets the ledger time to
the next fixing date and simulates floating rate fixings by the market data provider.
Then it “leaps into the future” and sets the ledger time to the next reset date
(payment date) and generates derived events.
And so on until the swap maturity. On different stages, ClearTH checks whether
the swap’s lifecycle is going correctly and whether the derived events are produced
in the system as expected.
So, let's see a short demo
To start, let’s have a look at the Excel model
mimicking the participant structure and the instrument structure of swaps.
This model also allows the user to create individual test cases, either manually or by generating
random scenarios using the swaps reference data avaiable.
Here, the user just produced ten scenarios
based on one model template and the input data with parameters combinations,
so ten trades will be uploaded into the system. During a real-life test run, there are hundreds
or even thousands of scenarios. Also, in contrast to what is shown in the demo, most of the
tests are run as a part of the CI/CD pipeline, rather than being manually uploaded into the system.
The test scenarios are uploaded to ClearTH,
we can see them under the Matrices tab.
In the Configuration and then Scheduler tabs
you can see so-called “steps” which correspond to certain sets of actions within a scenario.
The Connectivity section of ClearTH GUI helps
to manage the connections: we can see dealers, brokers and CCPs described by the matrices.
The scenarios themselves are keyword-driven scripts
that can be edited manually or,
as we already mentioned, generated using the Excel model or for example Python model.
We can now see them in a csv file - all 10 automated scripts generated from a template are in different
tabs as well as an auxiliary matrix which is created to govern processes affecting all test scenarios
in this case such a script describes an event of time-travel so that we could test certain
parts of the swaps lifecycle.
So, let’s launch both applications: the DAML-based Swaps application
the right section of the screen and ClearTH, which is to the left.
Now, the operational scheduler is starting.
It goes from trade upload through multiple stages, it contains 3 time shifts that help
navigate through the phases. During our test execution you can notice these
‘ask for continue’ pop-ups - this is an option and it allows to check scenario execution
between steps and is used only at the very beginning of the testing activities when it’s
very important to look at each step separately. In real-life testing this popups do not appear
and the testing is run independently. Then, the tool allows the user to view all
the messages and all the data frames accessed by the application, and it is possible to
check if there are any discrepancies between what we expected and what we actually received
right during the test execution. In this particular case, we faked a failure,
but, if a hundred trades were submitted, we could certainly have issues reported stating
discrepancies between what we expect and what we received.
Now, it starts to run derived events
the actual instructions for cash transfer, and it will report that there are discrepancies
between what we expected and what we received. The discrepancy that is reported here resulted
from the fact that we had introduced errors into the reference data for one of the swap,
so the derived event for it wasn’t generated and as there is no cash transfer even,
hence there are no parameters for that event to put into the comparison table.
And this is an example of everything going as expected
the user checks all the tags for a particular message.
And everything works very fine here.
And here you can see what happens when there is a discrepancy between the expected and
the actual result of the test - the tool reports the differences in particular data fields
and particular message tags. The tool provides raw reports in the form
of a table as well as in json format. Also, the reports can be aggregated providing the
summary and analysis of a number of executed scripts combined.
In this short demo we could see ClearTH automatically triggering different life cycle events,
such as cash flow generation, trade execution, novation, termination, and simulating
various potential actions of different parties involved in a business process, such as initiating
transactions or accepting trades. In addition to the simulation, the ClearTH
framework has components that perform checks of the transaction results,
including account balance totals, interest rates calculations,
and of whether the trade was proposed, accepted or applied.
The ClearTH test automation framework enables
users to monitor and control the nodes of the DLT-based post-trade system, as well as
re-execute the regression library covering complex multi-step business scenarios.
Thanks for watching and stay tuned for the upcoming videos.