(chilled music)
- Today on BRStv Investigates, can you trust these
test kits, or even yourself to accurately
test your tank water?
(upbeat music)
Hi, I'm Randy with this Friday's BRStv Investigates,
where we put popular reefing gear,
theories, and methods to the test,
by experimenting on our own tanks, so you don't have to
experiment on yours.
And in today's experiment, we put six
test kit brands in the hands of five reefers
with different experience levels,
in hopes to answer the question of,
which is more important?
Saltwater test kit accuracy,
or testing consistency?
Accuracy meaning that the end result
and reading is right, or consistency,
meaning that you can expect consistent and
repeatable results with each test.
I'm willing to bet that at the end of this one,
one of these kits will be your clear favorite.
Okay, so who doesn't want super accurate
lab grade quality test results for
calcium, alkalinity and magnesium from their
hobby grade saltwater test kits,
specially if we can couple that with a reasonable cost,
that doesn't end up being more than the
entire reef tank setup on it's own.
Well, you can definitely add me to that list,
but for obvious reasons, that combination just
really isn't feasible in most cases.
And because of that, we need to refine
the pool of test kit options out there
down to just a small handful
that we can trust to be accurate,
as well as trust to be conducted consistently,
but let's face it. When it comes to making
any changes to water parameters in chemistry,
I'd like to have some amount of faith
that my test kit is not
steering me in the wrong direction.
So today we'll try to help clear away
some fog around saltwater test kit accuracy
and consistency, by handing five different reefers,
with different hobby experience,
our six most popular test kit brands,
for calcium, alkalinity, and magnesium,
and have them conduct each one using the same
saltwater sample from the BRS160, for a total of
almost 80 different test results.
Oh, and don't forget that we also have
the Neptune Trident installed on the BRS160 as well,
so of course, we'll be adding that testing data
from it into the mix.
In the end, we'll use the data to help determine
which test kit had the most accuracy
compared to an overall baseline average,
as well as which of those test kits were the most
consistent by looking at the differences between the
highest reading and the lowest reading.
Here's how we plan to get that done.
In order to determine a baseline for accuracy,
we'll combine everyone's readings from
Red Sea, Salifert, Hanna Checkers, Aquaforce, Nyos,
LaMotte, and the results from the Neptune Trident,
into an overall average, minus the highest and lowest
10% of readings to remove any extreme outliers,
for each calcium, alkalinity, magnesium water parameter.
We'll compare each test kit brand's accuracy
based on how close our combined results came
to hitting that baseline average, and then
break down the average of all five reefers'
individual results, to determine how consistent
each test was on it's own.
Our five reefers consists of Aaron and Josh,
who conduct all of the parameter testing
for every BRStv lab experiment tank,
as well as the BRS160, 750XXL, and the ULM tanks.
These two represent those types of reefers
who go the extra mile to ensure that each test
is done as controlled and consistently
as possible, by using tools such as gloves,
meter pipe heads, magnetic stirrers,
and a thorough sense of detail when it comes to
cleaning, rinsing, and filling
each sample vial carefully.
Next up is Ryan and I, who represent your average
reefer that will likely utilize the included
test kit components, perhaps perform the test
a little more loosely, but still conduct each test
with the same consistency and procedure,
time and time again.
Finally we have Dave, who more closely represents
a hobbyist who has some degree of understanding
how the test kits work, and how to perform them,
but may still be learning how to properly
read the endpoints of the test, and perform them
consistently to achieve the accurate result,
so we can expect some outlier results here.
So with that, let's get into it.
First up are the calcium test kit results,
where we find a baseline average of 512 ppm
from 24 test results, and right of the bat
we see that Aquaforest comes in the closest,
within two ppm with an average accuracy of 510.
The next closest in accuracy was
the Nyos calcium test at 520 ppm, for a difference
of eight ppm total, followed by the Neptune Trident's test,
which was taken right around the exact same time
that the large water sample was pulled from the 160,
with a calcium reading of 498, or 14 ppm away
from the baseline average.
The largest accuracy deviation from all the tests
ended up being the LaMotte calcium hardness test,
with a 45 ppm difference from the baseline average,
which was likely due to the difficulty
that all five of us had in reading
the color change endpoint from red to purple to blue.
And probably one of the leading reasons
we discontinued the kit, where even in the instructions,
it notes to disregard any color change
back to purple after five seconds,
giving you only a fraction of time
to determine when the test is complete.
Moving on to our consistency comparison,
and how far apart each of the five reefers'
test results where from each other,
we see that the Red Sea calcium test kit yielded
the closest results across the board,
with only a 10 ppm difference between
the highest test at 495, and the lowest at 485,
which is absolutely within a reasonable
amount of variance for a hobby-grade test kits,
and within almost 250 ppm more consistent
than the LaMotte calcium hardness test,
which had the lowest 464, and a high end
of 722 point four, and again, likely due to
the endpoint reading difficulty.
Aaron's results are not present, because after
three tries he still couldn't get an end result.
In between those two extremes of our consistency tests
was Aquaforest with a 50 ppm difference,
between the four results of 500
and one result of 550.
Salifert came in with a low test result of 520
and a high of 580, for a total of 60 ppm difference.
Nyos had a difference of 85 ppm
with 475 for the low and 560 for the high,
while the Hanna calcium Checker showed a
consistency difference of 101 ppm,
between the lowest reading at 422,
and the highest at 523.
This was likely due to some difficulty
in the testing procedure, where we all agreed
that the dilution step of using a 100:1 ratio of
reagent DI water, to sample water, added the most
difficulty to achieving a consistent result.
All right, let's take a look at
the alkalinity test kit results for accuracy next,
with the test kits that we had available at the time,
starting with our 20 test result baseline average
of seven point eight one dKH,
again with the top and bottom
10% of readings removed to control outliers.
Interestingly enough, Aquaforest again
comes the closest in regards to accuracy,
with a deviation of only four hundreds of a dKH point
at seven point eight five,
followed closely by the Neptune Trident
at seven point nine, for a difference
of zero point zero nine dKH.
The LaMotte total alkalinity test has the largest
difference of the group at just about a half
a dKH under the baseline average
at seven point three nine,
while on the flip side the Hanna alkalinity Checker
looks to be over the average
by zero point three five dKH,
coming in at eight point one six.
The results for alkalinity testing consistency
tell a different tale, where we see that
the Hanna Checker performs astonishingly well
with only 3/10 of a dKH difference,
between the lowest test at eight point oh,
and the highest test at eight point three,
and it's truly a testament to what can be achieved,
when you take the human eye out of the equation
to help reduce the margin of error,
when reading the testing end point.
Up next was the Red Sea Alkalinity Pro test kit
at a seven and eight dKH low and high reading
respectively, for a difference
of just one dKH total,
and although the LaMotte test had the largest
accuracy difference, it actually performed better
in consistency at just a one point one two
dKH difference, between six point seven two
and seven point eight four
for it's low and high result.
Salifert's testing results yielded a low result
of seven point one five dKH and a high of
eight point nine, for a difference
of one point seven five, and surprisingly
the top performer for accuracy with Aquaforest,
ended up being the furthest apart for consistency
of almost two dKH, coming in
with a one point eight five total difference
between seven point one five, and nine point oh.
Wrapping up the accuracy and consistency
experiment today are the test kits for magnesium,
which didn't include results from Hanna or LaMotte,
due to them not having one available
for saltwater at the time.
For the baseline average of the magnesium kits
we did have available, we calculated an average
of 1389 ppm from 18 test results,
with the top and bottom 10% results being removed.
For a third time in the accuracy category,
we find that Aquaforce performed the closest
to that baseline average, with an average reading
of 1384 and a total difference of just five ppm,
while Salifert was close to the same
at 1395, in only a difference of six ppm.
Red Sea's Magnesium Pro wasn't too far away
with a 15 ppm difference at 1404, while the Nyos
at 1425 was only 36 ppm away
from the baseline average, and still what I would
consider an acceptable accuracy range,
from a standpoint of the amount of effort
reefers would actually put into controlling magnesium,
down to less than plus or minus 50 ppm.
The largest difference in accuracy category
was the Trident at 1308 ppm, or 81 ppm difference
between the baseline, however I would be confident
that if we were able to manually test
just for magnesium multiples of times,
we'd likely see this closer to how
the other test kits performed.
Finally for magnesium test kit consistency,
we see that Aquaforest had the tightest
group of results, with the lowest reading
at 1350 and the highest at 1440,
for a difference of 90 ppm.
Salifert performed about the same with a 105 ppm
difference between 1335 and 1440,
while Nyos had a range of 180 ppm difference
from 1320 to 1500.
Red Sea's consistency was the furthest apart,
with a low reading of 1340,
and an outlier higher reading at 1600,
for a total range difference of 260 ppm.
I'd venture to guess here
that because of the experience difference
in reading endpoint color changes,
we see such a dramatic difference in consistency,
where if the 1600 higher result where removed,
the consistency drops significantly down
to just 60 ppm between the high and the low results.
However this test was one of the more difficult tests
to perform for most of us,
due to the various waiting times and shaking intervals
between reagent drops.
So which is more important,
testing accuracy or testing consistency?
I'm going to rate this one on two different fronts.
First, for those more advanced reefers
who choose to run their tanks right on the
outer edge of what most would consider
to be middle of the road parameters,
for calcium, alkalinity and magnesium,
for example 380 or 500 calcium,
seven or twelve dKH, or 1250 or 1400 magnesium,
then I'd rate testing accuracy to be more important
than overall consistency.
Although maintaining a tank at these parameters
is likely far less common, there is an increase
in potential risk to your tank if you start
making water chemistry adjustments,
based on the test results of a kit that consistently
over, under reports, specially if you don't have
the means to verify the results
against a known standard. Conversely to that,
and probably relating to a much larger
majority of reefers and their tanks,
where they aim for more common water parameters
like 440 calcium, nine dKH, or 1300 magnesium,
which also allow for a wider margin of error,
I'd place testing consistency
above testing accuracy, as you can make
finer, more precise adjustments to tank parameters,
which ultimately leads to an increased level
of overall tank stability.
Meaning that consistency isn't just
how you perform the test each time,
but really how each reefer can interpret
the endpoint of the test, where the best
test kits available will leave
little room for error, with the end results
that are very clear and definitive.
Okay, so with today's data, can we really
choose a clear favorite from the group?
I can tell you that for me personally,
I definitely fall into that large category
of reefer who will rely on stability
to create an amazing looking tank that can
withstand the test of time, which in that case
I'm banking on test kits that provide
the most consistent results, test after test.
For me, that's going to be the Red Sea Calcium Pro,
Hanna alkalinity Checker, and after today's results,
I'm switching to the Aquaforest's magnesium test kit.
One of the only questions that we're left with
after today's testing is, what are the long-term
costs of using these various test kits?
And is there a way to get them cheaper?
The only way to find out is to check out
this BRStv Investigates video Ryan did
which compares the cost per test,
versus cost per test reagent refill,
you'll be pretty surprised.