We're going to look at an example of property based testing. [I'm] going to show quick check
Which is the tools [I] work with and the idea here is your test software
And that's very important to get software right?
But thinking out all of the cases that you want to test is not something that is very popular with software developers actually
And so [are] it's often not really done as thoroughly as you would like
And what I'm going to show you is a tool that we can use for generating tests so that instead of running?
Maybe you know a few dozen tests. You make up by hand
[we] can run hundreds of thousands or millions and get much more confidence [for] [the] software really behaves the way that we wanted to
Temple which is going to test that text message encoding, so this is actually a part of the 2g
[11th] Standard, so it's got old [but] then
[I] [think] we'll introduce text messaging that they were just making you for some spare bits that within the packages
And they wanted to fit as many characters they could into a text message
And there's only so much space so they realized that they only needed seven bits per character [from] so that [we] decided to use
But when you put those seven, [but] characters into the messages you have to fit into 8-bit byte
And that means that you can fit eight characters in 27 bites
So of course the standard says you must do that, and that means the way you have a message
[you] [want] to send you have to encode it by packing [a] [bit] closer together?
And when you receive it you have to decode it and here
I've got back some code that came from the company in France does that ankle Candy coating so let me just start by
Showing the code that we got so here it is. This is that line code?
and
This is the code for packing characters. You can see there's plenty of it
This is it comes from unpacking them again, and there's plenty of that
And we're not going to look in detail at this
The only thing I want to point out is that a lot of code and it's complex
So it's quite easy to imagine that they might be an error in there
So course we need test that
And down here we have some test code and what it does is it says well. Let's just take a sample message
Let's try
[backing] it
And then unpacking it and what it will call that tea and at the end of the day what we get after unpacking
must be the same as the message to be started off with
and then there's a little test suite here that says just try that for a number of different strings and with
Somebody thought about this little bit and they tried all left that message from 0 to 10, so yeah
Hopefully it works for these [strings] that will work in general what we can do now is just don't compile a program
[just] done and now I can run that test code [I] do so
Then we get [them] [out] [put] all of the [task] [past]. That's good it seems to work
But of course what always demonstrated is it [works] for these particular ten strings, so you might wonder?
if we give it another message, is there a risk that the code won't actually work as we expect, so
That gives us a chance to use a test generation tool
and when we use quick check
I go back to code here actually have what we call it quick check property that is the way that we
Write tests with property based testing we want something that's supposed to be true in general of our code here
[I'm] saying for all messages or msGS not
Just pretend that test which our list of [7th] values
So they're made of the 7th character
Then if I pack that message and one packet [I] should get the best
Off with that says the same thing [as] the test [I] had before but it says it in general
And that means that now I can use quick check to generate many tests [are] like and make sure that [coastal] works
[let] me go back to my test execution window here, and I will just use quick check
Call it and I'll give it that [property] that. I just showed an argument and we'll see what happened
well
After only a little more than a hundred test quick found an example in which the [prophet] doesn't hold and what you see here is
first of all a randomly generated message
It's a sequence of bytes which appears numbers here
You can think of it as a message, and that message was not packed and unpacked correctly
So after finding a random example like this quick goes on to simplify the [same] case as much as we can
And we end up with this message. Which is just eight zeros
So why does [quickcheck] say the problem occurs with eight zeros and not for example 70?
well
because if you have 70 still works
So this is the smallest example that doesn't work
and if I think which again
You see that it starts off finding a different random example, but when it simplifies the test case we get the same
80 [this] is one of the key things about this kind of property based testing you always get
The simplest case because you know it's the simplest that tells you a lot about what the problem might be
[okay], so it seems to be something to do with being a characters long. So maybe it works for all other [lengths]
What I can do is I can
investigate when the coattails now by changing my property
So if I go back to the property out here at [the] what it says for every list of seven that bike
Packing unpacking [gives] [us] [a] result now. I'm going to add identification. [I'm] going to say for every list of seven bit byte
Provided the length of the message is not [ate]
That should work, [so] my hypothesis is that the length of eight characters problem? Let's just test now
messages of other lands
[I] do that. [I] just recompile test code and rerun could check
Now [two] [thousand] [tests] passed. So maybe I was right. I'll just run a few more tests. Which I can do five
[growing] test let's say 10 seconds
[There] Are [0]
It still doesn't work
So now we find another example. [we're] packing and unpacking
Failed this is actually consists of 16 zeros
So maybe it's not just being of length [8] that matters may be any multiple of 8 is also the case that I fail
But let's investigate that [I] can go back to my property and I can change it to say that
provided the length
Is not a multiple of 8 that's what that means then packing and unpacking the message should give me the same result back
and if I
We can buy that and run the test again
Now quick check is
Discarding some tests [with] the crosses mean so running lost every [dollar] [is] a passing tests or by now for [Dr.]
10,000 passing [test] the cross is a test that
Where a multiple of 8 quick check is not found failing case after a hundred ninety-five thousand tests so that suggests that we're right
But as long as the lens isn't a multiple of 8 then
Code works
But I can also investigate the problem more closely [I] know
When the test is when the messages of multiple date then the constant of sales
but doesn't always [sale] let me go back to my property and I'll just
Take that out for truth and said we got nothing
Instead of generating any listed as a message
[I'll] just generate list of 98 and I was focusing testing on the case on suspicious of
[its] every message of Mentaiko drawing. Let's find out
[I'll] run these tests [that] everybody's hands and fate
And what you know some of them passed [38s] past obviously many things are correctly encoded
But this example is not found, or [is] this example now?
I want to look at the random example that failed
And [that's] that's [a] few more of them
one of the nice things about
Generating tests as you can see a lot of different examples of the same thing so is there anything in common between that first example
This other one can see that full of different values of numbers those are all different wait a minute. [look] at the last number there
[and] the last one here left one here. They all end in 0
[that's] very unlikely to happen by chance
so maybe the code works unless
the messages of multiple of 8 in length and the Last character 0
[let's] test that hypothesis. [I'll] just change this still [to] runtastic [Wednesday] put to say
provided the last character of the message
Is not 0 then the coach should work [ok]?
So am I right in my postures let's find out
Whoa [now] all of the test statistic?
And after 10 seconds, we'll see we run to [continue] two thousand tests without a 0 in the last position
[they] all passed
So that strong evidence that the hypothesis right [okay]?
So now without even looking at complicated code. We understand the problem very well
We know it happens when the messages of multiple date. We know that happens when the last character [0]
And if we think about it, we can actually understand. Why this is happening
When we pack a number of characters in two bites?
They're always going to be some bits left over
What happens if we pack 7 characters in 27 bites?
[what] happens is that there are seven bits left over?
70 bits
that looks very like
Eight characters where the last 10 [and] that's the problem
it's a problem in the standard [car] distinguish between
A message that contains eight characters with zero at the end as a message to [just] [contains] the first seven
And it's a known problem. So there's nothing found it except I suggest not putting their characters in your messages
We realized that we were all working with very much the same kind of programming language
But because we each had our own combiner. We couldn't share any [cooked]
We couldn't share our results
And that just like a waste of effort so that led to the proposal to just
Take the common core of what we're all doing