So our next speakers are Daniel Koo and Bryan Robbins from FINRA.
They are also the inventors of a framework which they call MSL (“Missile”).
And they're going to tell us how MSL has helped them improve UI testing at FINRA.
>>Daniel Koo: Thanks for the introduction.
So my name is Daniel Koo.
We're here to talk about a tool that we open-sourced this year called MSL.
It stands for Mock Service Layer.
And, you know, we're very excited to come here and share about the tool.
And, hopefully, all of you guys will find this interesting and, you know, you guys will
use this when you guys go back.
I know we talked a lot about mocking and, you know, moving fast and hermetic testing.
And I think our talk will definitely resonate with those topics.
So we're very excited.
All right.
So what am I trying to say with this slide?
I think these are some of the trends that we are seeing in the industry today; right?
Things like agile, faster time to market, continuous delivery, having high quality.
All of these are great goals for software projects.
And, of course, these are goals that we had at FINRA, too.
But while trying to achieve these goals, we observed a few things.
First, we were observing that devs were actually moving faster, delivering faster, but tests
were sort of struggling, and they were becoming bottlenecks and moving slower.
So, you know, definitely there were challenges while trying to achieve these goals.
And FINRA had its own challenges, too.
So brief introduction on FINRA.
FINRA stands for Financial Industry Regulatory Authority.
We pretty much oversee and regulate the majority of the securities firms doing business in
the U.S.
We also monitor the market data.
More than 6 billion trades per day, more than 30 billion transactions per day, and more
than 10 terabytes of data is what we analyze on a day-to-day basis.
Our mission is to protect investors like us and promote market integrity.
So, you know, the nature of our business and our mission brought these unique challenges
to us.
First, the visibility aspect.
If we make a mistake and it happens in production, we could end up on the Wall Street Journal
the next day.
So we pay very close attention to the quality of our software.
And as I mentioned before, the enormous amount of data that we ingest every day and, you
know, the type of analysis that we do and the workflows that gets into these analysis
just makes our systems very complex in nature and in turn, it makes our testing that much
more challenging.
So this is when I started to think about, okay, I want to achieve those goals, move
fast, deliver fast to our customers while not compromising quality.
And that's how MSL was born.
So I'm going to hand it over to Bryan.
He's going to talk about how our automation approach has evolved throughout the years
and we'll dive into more details on MSL.
>>Bryan Robbins: Cool.
So this is another theme that we've already heard a few times at the conference.
This is our rendition of it.
It's basically to say that if you had come to us two and a half, three years ago within
our company and said, “how should I automate my test?” our primary recommendation would
be Selenium WebDriver.
Go and make functional, browser, full-stack automation for every feature and by the time
you're done you'll have great coverage and great -- it's a great approach to testing,
right?
But as we start to move faster, those things are flaky.
So I've got execution time on my Y axis.
You could put flakiness there.
You could put development effort there.
You could put a lot of different variables there.
And the more of these full stack UI tests, there's so many dependencies, it just becomes
intractable.
Again, that's a theme you've already heard.
So about a year and a half ago, one emphasis -- we started putting in place and swapping
out, so we'll replace some of these end-to-end tests, the ones that are trying to test back-end
features, we'll just use direct API tests for those.
This depends on having devs that design things properly, but most of our current frameworks,
right, they actually have MVC or something like that in place.
So API testing isn't that big of a deal.
MSL really comes in to the next level of this.
So last year at GTAC we heard about things like Karma and test runners for UI-specific
code, and could we take the same approach to isolation where we had been isolating the
back-end, could we isolate the front-end in the same way?
So that's how we view this sort of evolution at our company.
So how are we doing this with MSL?
It's basically a tool for what we call client-side integration testing.
That means to take your full client side code -- the JavaScript, the CSS, HTML and whatnot
-- and to locally deploy this using a Node.js server.
And this is going to let you, without having to build even in some cases, to start doing
some tests against your UI features.
All right?
So obviously we're removing a lot of dependencies when we do this, right?
So the MSL server at your app code, and it's basically going to use a test runner to open
up a test.
The test will have API calls back to the MSL server, and we'll show plenty of examples
of that later in the talk.
So basically the idea is a client-server kind of model where the server is doing some mocking
for you to aid you in setting up and executing tests.
Demo time.
>>Daniel Koo: It's demo time.
We're going to do a live demo because live demos are cool.
And we're going to show this with the Wi-Fi off, just to prove that we are doing hermetic
testing; right?
Okay.
So to install MSL server it's pretty simple.
It's up on npm so all you need to do is globally install MSL server.
If I can type.
Now, I'm not going to run this because it's already installed, so....
It’s already installed.
Alright.
So, now, the code that we are interested in is right here.
This has all the front-end code.
It has the JavaScripts, the HTMLs and CSS files that we try to test.
So I'm going to fire off the MSL server and we do it by this.
MSL, and we give the port number and there you go.
Now you have the MSL.
MSL has launched from here.
It's running on this directory, on this port.
So now let's open up the app.
Localhost.
Sample app.
And there you go.
So it's as simple as that.
We have the app up and running.
And what I'm going to do is I'm going to perform an interactive action which actually triggers
a REST call but it's going to fail because we have no REST services actually up and running.
So we see here we get a 404 and there's nothing coming back.
So I'm going to show you how we can mock this so we can actually test the front-end code.
All right.
So we're going to use Jasmine for running our tests.
So we've got the Jasmine library here.
And here is where we include our MSL client library.
So we've got the mockapi-browser.
So this is the JavaScript that allows you to talk back to the MSL server.
We also have the appcontainer-driver JavaScript which allows you to drive your interactive
scenario.
And of course, here is my Jasmine spec file.
So now I'll show you the spec file.
So we have two tests in here.
And the first thing we do before running any of the tests, what we do is we call this function
called openApp that's part of our appcontainer-driver which opens the app that's running on the
MSL server.
So we make sure we, you know, reset and always have the app running in the same state.
Next, going into the test.
So what we're going to do is we're going to stage the mocks before we do our interactions.
So we call this function setMockRespond, which basically tells the MSL server to intercept
all the URLs, all the requests that match this URL and respond with this JSON response,
and you can give the status code 200.
You can also add delay time but in our case we're not going to add any delays.
And then what we're going to do is we're going to use again our appcontainer-driver to get
the elements and perform some actions.
And we've implemented this using Jquery because everything is running inside a browser and
we didn't want to use Selenium or anything like that to drive the test from outside.
So what we do is we type, and we click, and then we're going to validate that the dropdown
that comes down, when we do a type ahead, is actually -- it contains the items that
we have mocked in the JSON response.
And then we're going to click on the second item, and we're going to see that it actually
selected the second item and the type ahead is populated with that.
The second test is going to test that the app fired off an XHR correctly.
So we do that by calling this function, setInterceptXHR, which again tells the MSL server go intercept
everything, all HTTP requests that matches this URL.
And then again, we do the interactions, and then we call this getInterceptedXHR method
which will retrieve back everything that's intercepted, and we're going to validate that
the URL was correct and it was a GET method.
All right.
So let's fire off the test.
It's going to be very fast so you guys have to watch carefully.
All right.
Tests done.
Two tests done.
And you guys saw the dropdown that came down; Right?
So that's what we have mocked.
So now Bryan is going to talk about how you can use this in Karma using our plug-in.
And before I hand it off, I'm going to turn off the server because our plug-in is actually
going to start up the MSL server by itself.
>>Bryan Robbins: Great.
So with the MSL stuff we've shown so far in hand, right, you take advantage of a framework
like Jasmine.
And what we need to add to that is some kind of runner.
Something to launch the browser, something to do reporting.
As it turns out there's some pretty smart runners out there.
We chose Karma and decided to implement a small plug-in with Karma that will start up
your MSL server and kill it when it's finished and those kind of things, and maybe some other
integration points.
So here's an example of how to hook it up.
You know, our plug-in is available on Karma's default repos; right?
So you can just add MSL as a framework.
And when you go through your files, there's no need to include any files other than just
your spec.
So you saw on our Jasmine spec, we had to add all of our specific files there.
You don't have to do that with the Karma plug-in.
And then there's a block for MSL specific configuration here.
So this is saying we want to run MSL on port 8002.
This is going to be behind Karma's own part which is usually something like 9876 or something
like that.
The Karma port needs to match the port in your test.
The MSL port is sort of hidden from all of that, and the way we accomplish that is via
Karma's proxy support.
So we're going to route all requests on the root URL to the mocking server; right?
So this is the way we hook this up.
We can foresee maybe extending some of the features of this plug-in to maybe set that
proxy for you automatically or do something cool, but not quite there yet.
So let’s quickly just run this.
>>> The browser is behind there, but it worked.
>>Bryan Robbins: Sure.
>>> Try running it -- there you go.
>>Bryan Robbins: There we go.
So again, Karma is running the same test but with jUnit reporting and all the nice features
Karma gives you.
You can also swap with other browsers and that sort of thing.
So one last thing.
We're out on github as we mentioned.
So we've got full docs, as well as the user -- you took off the Wi-Fi.
Yes, our github page is not a hermetic page.
>>> This is not part of the testing.
It's all right.
[ Laughter ] >>Bryan Robbins: Does that count?
So we're out on github.
We have full docs of all of our APIs there so we've written not just the JavaScript client
but also Java and Node.js and Python and Ruby.
Of course you can do something like Wiremock if you're totally Java all the time, but if
you need something that would maybe switch back and forth and leverage the same mocking
ideas, we do have the different clients.
And we need to give a shout-out to the other half of our team back in D.C.
Jake and Chien have put a lot of work into this stuff for us.
So that's the rest of our team.
And with that, we're done.
>>> Thanks.
[ Applause ]