Dependence on electricity has become a way of life. Laptops, mobile phones, air conditioners,
stoves, televisions, electric vehicles ... you get the idea ... our daily lives are using
an ever increasing amount of electricity, but we tend not to think about how much energy
each of these things use. At least until we get our electric bill. Out of sight, out of
mind. But what if you could tap into the power of machine learning to help you monitor your
energy use? To find out where it's coming from and where it’s going? To potentially
let you know if one of those devices is failing and in need of being replaced? Would you be
willing to add that artificial intelligence to your home?
I’m Matt Ferrell ... welcome to Undecided.
When you get a car that requires charging, whether it’s a plug-in hybrid or full blown
EV, it always seems to send you down a path of wanting to know exactly where your energy
use is coming from. When I got my first plug-in hybrid car about six years ago, I started
down this path myself. And it only got more intense when I had solar installed and got
my Tesla Model 3. About two years ago I installed a Sense whole home energy monitor, which not
only helps me track how much energy my solar panels are generating, but also how much energy
I’m using ... and provides a picture of what devices are using that energy.
There are a bunch of home energy monitors on the market that you can use, but Sense
caught my attention because of the machine learning aspect of the product. I had a chance
to sit down and talk with Mike Phillips, the CEO of Sense, about how this type of system
works and how it came about.
What is it that got you to start the company? What was your motivation behind it?
It started from this broad notion of people want to be able to save money by saving energy
in the house, but they just don't have visibility into where it's going. So that's what we set
out to attack.
There's other systems like this out there. But the thing that drew me to Sense was the
simplicity of the system, because most of these systems I was looking at were like,
you have to put all these connections on every single line on your electrical box, but here
was Sense, just put two clamps around your mains and the machine learning will take care
of the rest. Machine learning seemed to be the secret sauce of Sense. Was that your basic
approach that pulled this all together?
Yeah. I also started by putting on these systems in my house that had the clamps on every single
circuit and it just was too cumbersome. And even then, knowing power by circuit wasn't
really what I wanted. I wanted to know what the different devices were doing. So we started
wondering, well, if we measure the power in a detailed enough way, can we figure it out
just from the power signals? We're making use of the fact that the different things
in your home use power slightly differently. So if we measure well enough, can we tell
the difference? And sure enough, it's not a new idea, it's called load disaggregation
and there've been tons of research done on this, but what we found was it's just much
harder than most people thought it was.
Things worked in the lab, but not in the real world, which is how speech recognition was
30 years ago. I don't know if you know or remember what that was like. Systems that
would work in a lab but not the real world. And this is just as hard. And we realized,
okay, first of all, we got to get really ambitious about the signals that we collect. So we have
very high resolution data and then just the machine learning problem on top is a tough,
tough problem, but we've been deep into it.
I'm not an expert. So my understanding is base level, but I know there's basically two
approaches you can take. There's the, I think you even described it on your website, is
there's the supervised and the unsupervised paths towards teaching something, supervised
being, showing a machine here's a picture of a dog, a picture of a cat, and then you
show them a whole bunch of pictures and you tell them when they got it right or wrong
versus unsupervised where they're figuring it out for themselves. Is that approach for
unsupervised a little bit more challenging approach, but with more upside, I'm curious,
why one approach over the other?
Yeah. A great point. Look, it's the case that supervised machine learning is much easier.
So if you can get a nice label data that tells you this is a refrigerator and this is a toaster
or whatnot, that's great. The problem was we can't do that here. In speech recognition
or image processing, speech recognition, you can play audio to people, they can listen
to it and say, "Oh, the person said this," or you can show people pictures of cats and
dogs, that was a cat, that was a dog. Here for the most part, you can't do that because
you show people electrical signals that they don't even know. And a lot of our users want
to help and it's great to get their help, but they don't know when the condensate pump
of their furnace is running. They're just not the so-called ground truth that we can
do. So we had to go down the unsupervised learning approach.
As an end user, when it detects a device in my home and it says, "We think it's a washing
machine." And then it says, "But it could be one of these other things," I'm supposed
to pick which one it is. Is that the training of the system or is there more to it than
that? Is there something that you guys are doing at Sense to really shepherd it along?
Yeah. There's a lot that goes on behind the scenes and that one particular thing, where
if we find something that we believe to be unique and we don't know what it is, we can
say, "We found something, it looks like maybe a motor," and then 30% of people call this
a blender, 20% people call it a coffee grinder. You go, "Oh, it must be my coffee grinder."
You type in coffee grinder, it's good. It says that in your app. And then that feeds
back to us.
So that is super helpful. But some of the other things behind the scenes that comes
before that is we can't even show you something in the app until we believe it to be unique,
because if we just did this with your... We found your hallway light, a 60 watt incandescent
in your hallway, and said, "Great, we found it," but if you also happen to have a 60 watt
light in your bathroom, we would get them confused. So we have a severely hard part
of what we do, which is a uniqueness test, to say, "Have we found something and we modeled
it correctly and do we believe it's unique?" We have to do all that first before we say,
"Oh, we found something, help us know what it is."
Why do I sometimes see duplicates that show up? It will detect my refrigerator, and then
six months later it detects my refrigerator. I've had that happen on occasion and I think
I have understanding as to why that happens on occasion, but I was curious if you had
any insight as to why that might happen.
By the way, one thing I didn't mention on the machine learning side that I think is
important to point out, in addition to having to be unsupervised, in addition to having
this uniqueness tests that we have to do, and wanting to do things in real time, you
got to realize the signals are all on top of each other. So it's like doing speech recognition
with 30 people all talking at the same time. It's not just one more layer of why this is
hard. Now back to your question about why do you sometimes get duplicates. I mentioned
that we have this uniqueness test to decide, is this a unique thing? Well, sometimes we
get that wrong. And sometimes we think that what we see now is a different device than
we saw before. This is actually quite active internally right now, we're working on that.
We know it's a problem. So you should see better performance around that here coming
up pretty soon.
That actually leads right into one of the things I was really excited to talk to you
about, which was a few months ago, I read in one of your blog posts on your website
about the challenges of detecting EVs as an EV owner, I am super interested in knowing
how much energy is going into my car, how much it's costing me. I was curious if you
could expand on what you just mentioned as well as that as to the challenges around actually
identifying devices, especially things complicated, like an EV.
So first of all, we are super excited about EVs also, both in terms of be able to detect
them and also be able to start to do things like load shifting. We can talk about that
later. But it starts with be able to detect them and look, everyone thinks, and we do
too, they should be super obvious. There are these great, big things in your home. We actually
made life a bit harder for ourselves, but for a good reason. And that is, we wanted
to be able to show you things real time in the app. There's a big benefit in the app
to be able to turn on your microwave, seeing turn it on right now and go, "Oh, that's my
microwave." Same with the coffee grinder and so on. So we put a lot of work into making
the app real time.
And a lot of what we do is focus on how do we make that real time? The reason that makes
things difficult now for EVs is when they turn on, they actually don't turn on in a
big, clear way, right away. They're very complicated devices inside the cars where the charger
is, and they start by, they do a first little test to make sure everything's connected.
And then they ramp up in a little slow way and then they ramp up to some intermediate
thing. So there's this complicated dynamic of how these things ramp up and down when
they start and even harder for us is, as the battery starts to get full, they start to
gradually ramp down their charged rates. Sorry for all the excuses here, but what it means
is trying to show what your EV is doing in real time is a tough, tough problem for us.
We're not giving up. We'll increasingly be able to do these things, but still not perfectly,
but it's a major focus of our team to better and better do EVs over time.
Where is it happening? Where does the model actually live? Is it on the device, in my
home or is it actually being processed out on a server?
Yeah, a great question. At least that one has an easy answer. A lot of your questions
have tough answers, that one's an easy one to answer, which is what we call the runtime
modeling. So the thing that all day long is saying, "Did my toaster just turn on?" That's
happening in the device, because we don't want to have to wait and we don't want to
send up data up to the server for doing that.
But the thing that says, "Do you have a toaster?" And, "What does your toaster look like?" because
it turns out to do this well enough. We have to not just have a generic model of toasters,
which we do, but we also have to know specifically what your toaster looks like. In order to
get that real time we see right now your toaster is turning on, we need a specific model for
your toaster in your house. Sorry. That model learning happens server side. So what that
means is we send enough data up to both drive the application, but also to then learn what
your toaster looks like. And then we push those models back to the device to say, "Okay,
keep looking for that toaster."
One of the things I mentioned in the beginning is my motivation for it was to understand
where my energy was going. I'm curious if what you see the average Sense user is saving
on energy based on what they're doing. If that makes sense.
Look, we think even in energy savings, there's a lot that we can do over time automatically
to really point out where your potential savings are. And that was our first notion that we
would just identify for you where there's savings opportunity and automatically tell
you. But the first thing we found was that most of the savings are not well-defined categories,
like utilities think it's around light bulbs and refrigerators. Well, that's done. They're
all pretty good now. And meanwhile, there's this big chunk that we just call energy hogs,
which is random stuff that if you knew about it you'd go, "Oh, I don't need to do that.
I can fix that thing." It's a really broad range. It ranges from you leaving the roof
coils on that melt your ice dams and you leave them all on all summer.
Or you didn't realize a dehumidifier uses a third of your power or there's something
wrong with the setting of your heat pump system or ... the list just goes on and on. And because
it's so broad, we realized, well, we can't automate all that. So let's start by giving
consumers just visibility. And then hopefully they'll be like you and actively use the app
and the functionality to go track these things down. So that's been the focus just based
on the relationships being so broad. And what we're finding is if you're really active doing
that, you can get 15% or so savings in most homes, but only about half of the people do
that. So if you average across everyone, it's half that.
I liked your examples because that actually happened to me. I had a dehumidifier that
was on a smart system in my garage and something had happened to the smart outlet and it wasn't
doing what it was supposed to be doing, the dehumidifier was running 24 hours a day and
I noticed in Sense that there was this huge uptick in energy use and it helped me track
down exactly what had happened.
And it really is that, that if you have no visibility... Look, if water was running in
your basement, you'd know. It'd be clucking. If electricity is leaking and it doesn't leak
in the same way, there's puddle on the floor in the basement, but it leaks in these other
ways. You have no way to know. You just have no visibility. So just giving people this
direct real time visibility where, "Oh, what's this?" They get savings.
I don't think you'll necessarily be able to answer this, but it's because it's probably
about future plans, but have you guys considered deeper integrations into other devices? We're
talking about load balancing. So let's say my solar system is over producing electricity.
It can automatically turn on my wall charger, the wall connector to start my car charging.
Have you guys considered something like that?
Yeah, absolutely. And we already have a few things that are live today that don't go as
far as what you just said, but we've already done integrations with if this and that, we've
done integrations with TP-link plugs, with MIMO plugs. And so we are already using those
where you can... We use that to get both a direct measurement of what's going on in the
application and in the app, you can control it. So if you see in the Sense app that you
left your garage lights on, and if it's on the Waymo plug, you can click there to turn
off. But the next step of starting to automate these things very much is where we're heading.
So in fact, your case is exactly... So in my house I have solar panels. I have two EVs
and I have a very early rigged up version of Sense where it's actually controlling my
EVs to only charge when solar is over producing.
That's awesome.
If not, it goes and finds the marginal carbon intensity. Now we haven't productized that
yet just due to priorities. That kind of thing is very much what we're excited about. We're
experimenting and absolutely is the future.
It's one of the reasons I was excited about the Sense when I got it, was it felt like
something that will definitely grow and evolve longterm. So I felt like I was investing a
little bit into the future of the product when I bought it.
A lot of these things that we want to be able to do, you were just talking about, we couldn't
really do until we start to get data and we couldn't start to get data until we got these
things out there. So there is this a virtuous loop now where we've got ton of data flowing
in, ton of engaged consumers like yourselves and just more to be done. And now we're limited
by how fast we can get this stuff out.
So where do you see, not just Sense, but in the broader picture of this type of system,
where do you see this going in five or 10 years? Where's the end game for you on this?
Well, look, we really do view that your home itself needs to become smarter in ways that
it hasn't before. Most people, when they talk about smart home, they're talking about automated
lights or their entertainment system automating, which are all fine. I'm not trying to knock
those, but that's to me just fundamentally different than your home itself or the core
systems in your home becoming smart for efficiency, for reliability and safety. Things like that
I think is what's coming and has real, real benefit to consumers from a saving money,
from a making their homes more reliable and more healthy.
So I think that's where the opportunity is. It's like what happened in cars. Cars, 50
years ago were independent mechanical systems and carburetors and points that you had to
go and tune up every couple of weekends. And since then they became incredibly instrumented
and automated resulting in huge efficiency gains and huge reliability gains. And the
same should be true of homes. Homes are harder because they are built out of independent
systems versus built by one as a system itself. But it doesn't mean that the same principles
don't apply that by making those things smart, we can make these big gains around efficiency,
reliability, and convenience.
Whenever I'd make smart home videos, one of the things I'm always telling people is, it's
more than making your lights flash green and red by talking to your voice assistant. Smart
homes have so much more value to our everyday lives than just that.
And I think one that's been under explored even by us also, is even things like health
of the occupants. Things like air quality, do you have a Radon fan and is it running
at the right rate? Do you have a ventilation system, and is it running appropriately? These
things will start to be... There's no reason your home can't know things like the indoor
air quality and adjust as needed to make sure you're healthy. And those are big, big benefits
to people, obviously. So one of the other things that we're excited about and working
on that's going to result in even more doing what we can do today, is just getting more
details about what's actually in your home, because right now what we're doing on the
machine learning side, on the unsupervised side is we get these data signals and a little
bit of hints from the consumer, but then we have to figure out everything about what's
going on in the home.
If we start to work with home builders, for example, where they already know what dishwasher
is there, what furnace is there, what hot water heater is there and knowing half time,
what systems are there, how they're supposed to be working, and now our job is to make
sure that they are working as expected. There's a lot more value that we can provide versus
our current handle every single house out there case.
So it'd be more curated because you'd know exactly what's going on ahead of time.
That's right. So that kind of curated, we think also becomes really important for things
like performance monitoring over time. If you have an air conditioner, maybe it was
super efficient when you got it installed. Is it still efficient three years later? Well,
if we already know we've been watching it the whole time, we can say, "Yeah, it's working
fine," or, "No, it looks like it's been degrading, maybe the refrigerants leaked out. Maybe you
should have it looked at."
Well, thank you so much for taking the time to talk to me today.
You're welcome. It was great to meet you.
I’d like to thank Mike Phillips for taking the time to talk to me. It’s companies like
Sense that are helping to raise awareness of not only where our energy use is going,
but building the foundation of AI systems that can eventually let us know when devices
are breaking down and help keep our homes safe and healthy. With knowledge comes power
... yeah, bad dad joke.
Now jump into the comments and let me know if you’re using Sense or a system like it.
And as always a special thank you to all of my Patrons. If you liked this video be sure
to check out one of the ones I have linked right here. Be sure to subscribe if you think
I’ve earned it. And as always, thanks so much for watching, I’ll see you in the next
one.