My name is Mingze Xi I'm a research scientist with CSIRO, the Australian National Research Agency.
Today I'm going to talk about how we used the smart glasses to support situated decision
making in an agriculture setup.
In the rest of the talk, we will briefly describe the operational challenges in prawn farming,
and how we used a practical guide to choose smart eyewear for a new proposed
Pond Visualisation System.
Then we describe the on-farm trials and the results.
And more importantly, we'll focus on the lessons we learned and how that can help others in the future.
The work I'm talking about today was part of a much larger project called
Ddigiscape precision aquaculture.
Overall, that project was looking at using smart glasses to allow farmers to make better
decisions, which will ultimately allow them to produce more prawns.
Then, we want to understand how the water quality management works.
Because, that, according to the farmers, were the most important thing.
Then we talked to farmers, shadowed them, lots of interviews. We found that these are the main steps: They firstly will collect
the data using handheld sensing units, for example, the one shown in the picture.
Then, they will analyse the data using Excel and look at the plots and the visualizations and then decide two critical tasks.
One is adjusting aeration control task.
And the second one is to adjust the water exchange by turning on and off the water pumps.
Then, you would think that farmers have been doing this for years they have tons of historical data.
And when they go on in the pond, they will have the current reading, t
hen they should be able to make a decision.
However, that isn't the case.
Because there is no way for them to access the plots in the field.
And there is a delayed decision because they need to travel back to the office to
do the analysis and make some notes about the changes. Then, they travel back to the field.
And, then we thought how about we let you wear smart glasses that is able to pull
historical data and also read your current reading and gives you the plot.
Then you will be able to make a decisions.
This was strongly validated by the farmers during brainstorming at early design stage.
The first challenge we had was which device we should use.
If you were doing this in an indoor environment, probably any smart glasses will do.
However, we are using this in an outdoor environment in a farm that's built in the coastal area.
It has extremely high level of humidity and salinity.
And we talked to the farmers to understand what we need to consider they say "Oh, these
are the things you need to consider, which are in the paper, please read that.
And, then we came up with a list of criteria: ruggedization, weights are the two most important things.
Because if the device is not ruggedized, to some degree, it will break quite easily.
Probably every three months, then the (light) weight allows you to wear that for hours instead of minutes.
We end up using Google Glass Enterprise Edition 2 at the time this study was conducted.
But new devices come out every day.
The criteria will guide you to select the best one for your project..
Having decided which device to use, we next want to understand how farmers visualize their
data currently.
And this, what you see is actually being used in a commercial farm.
It's an Excel visualization with multiple parameters.
So you see the plots which is nice.
However, anyone has used my glasses, you will know that it's impossible to do this with
smart glasses, because you simply can't see/read anything.
In order to design the new visualization that will fit smart glasses, we use the user-centered design approach.
We talked to the farmers several around the time we came up with multiple designs.
And this is what we end up with. In this particular example, we did not use any machine learning
model there. If you want, you can insert a machine learning model and then it gives you the forecast.
You will see the optimal range of this parameter.
And if any value is outside of that optimal range, then that allows them to make better
decisions.
After that, we conducted their own farm trial was five farmers.
We asked them to exam the visualizations for 15 ponds, and the for each pond, they need to
decide whether they want to adjust the aeration, or adjust the water pump.
We got the ground truth from their farm manager who used the conventional tool, sit in front of the
computer, spend a good amount of time, and made sure that decisions were confident
and sound.
We also collected the task completion time.
And to see how fast or slow this thing compared to the conventional way.
The results are quite promising. For the aeration task, we had 89% accuracy, and extremely
low standard deviation, only about 2%.
The same thing for the water exchange task, 86% accuracy and 3.5% standard deviation.
That means the farmers were able to make consistent decisions and good decisions during the trial.
We also found that task completion time was pretty good.
And the main TCT (task completion time) was about seven and a half minute.
And we did see a bit of a standard deviation here that and we found that younger generations
were able to blast through the process.
However, some people did struggle with the small display and some fitting issue and spend
a bit of time, about 10 minutes that includes the time they spent adjusting the device etc.
But even though taking that into consideration, is still at least 40% faster than the conventional
way, which is 20 minutes at least, sometimes 30 minutes.
And let along
It also brings other benefits like electricity saving labor saving , reduce the carbon footprint.
The top three lessons we learned from the on farm trials.
Firstly, simple static visualizations are enough for these kind of tasks.
And also, it is quite impossible to do interactive visualization on these types of smart glasses.
Because the users cannot select things accurately like using a mouse.
On-farm studies also need to be highly flexible and interruption-resistant.
This is unlike a lab- based study where you can have the participants, like, say 30 minutes uninterrupted.
whereas in a real world scenario, arm is a very dynamic environment, emergencies
unexpected things happen all the time.
And when that happens, the farmers need to go and your session will be interrupted.
So that has to be designed in a loosely coupled and nonlinear manner.
We also found that you need to provide really, really clear and concise information and instructions
for the farmers to learn how to use the device, especially when you're running this remotely.
The on-farm trial has strongly demonstrated that smart glasses has great potential in
supporting farmers to make efficient and informed decisions.
However, there are things that we need to consider.
before deploying this to a full a production environment.
For example, the first thing to consider is automatic pond localization.
In our study, we prescribed the order of the pawns the farmer go through, they will start
from pond number one, number two, to number 50.
However, this isn't the case, in the real world scenario, we need a mechanism to let
the device automatically figure out where it is.
And this will save farmers from manually entering the pond location.
The second thing we found was farmers told us they prefer voice command because they
have their both hands occupied.
But we found that, at least all the speech recognition engines that we tried, on device,
Google Cloud, Microsoft Azure and others.
All of them suffered from strong wind noise and very, very noisy machine like those water paddles
running in the background.
There's another thing to tackle.
And we're interested in seeing maybe today there are waterproof, rust resistant optical
see through augmented reality headsets, they might be able to provide more interesting
capabilities.
And we'll be interested in seeing how that looks and how that work.
On the right side is an updated version of the device that we kind of turn a version of it
here comes the end of my presentation.
Thanks for listening.