Ben Eubanks: So we just released the new findings of some
research we did. We're trying to understand employee
surveys, are humans or AI better at identifying emotion and
sentiment in the data? It was really interesting. So we
structured this almost experimentally, not just a survey of
people, but also to really try to analyze this, just like an
academic study. So we surveyed 1000 North American workers
to understand their sentiment and their issues at work, Are
you stressed? Tell me about that. How's your lifestyle, your
manager? Those kinds of questions. And we used that data to
then pass that to a handful of consulting firms to get the
human analysis piece of that, let them return, what hot
buttons they saw what friction points in the employee
experience, those kinds of things. And then we ran that same
data through ultimate software's NLP natural language
processing engine to look at the data and see what sort of
sentiment and emotions you can find in there. It was really
interesting. So we were able to prove with this research
that AI is better than humans at analyzing survey data for
those kinds of things. The initial data was actually
collected, we collected through a third party, we gather
that data from the open market. So these are just 1000 North
American workers that are currently employed. That was the
focus group that we targeted there. And then again, we
brought that data in, cleaned it and then use that to pass
to the parties and pass to alter the software. The goal, we
couldn't do this with actual company. We tried to recreate
this just like it would be 1000 person employer, though,
right? So we asked those same kind of questions they would
get, and then use that that data, pass it to each of the
parties to have them analyze it and see who could do a best.
There are a couple problems with the employee survey process
as we currently know it. So it's funny because back when I
worked in HR, these are the problems and here we are 10
years later, still the same problems. So one of those is a
timeliness. I talked to an employer a few months ago. They
have 20,000 employees from the point where they decide
they're going to start an employee survey to the point to
get the data back from their service provider, six months.
So they said, you know, hey, the problems those employees
had four months ago, they all quit now, right? There's,
they're not even here anymore. And we laugh about those
things. But that's that's a reality for a lot of companies.
So the timeliness is a big issue, being able to segment the
data and understand it. And the research, we also did a kind
of companion piece where we actually post what the 300 HR
leaders to ask them, how they're structuring their
engagement surveys. Most HR leaders are still doing an
annual look at that. And when we asked them what kind of
questions they're asking, they're doing a mix of closed
multiple choice questions, you know, on a scale of one to
five, how do you feel about this? But they're all doing a
little bit of open ended, unstructured questions. So what
are we doing? Well, tell us about that. What are we not
doing? Well, tell us about that. The problem is, I just an
hour ago, spoke to an employer here at the show that I know
that said, "Hey, the problem is we're doing the surveys. No
one actually looks at any of that data because we don't know
what to do with it. We are 10,000 employees, we don't know
what no one can take all that time to look through the data,
the unstructured questions. So we just end up averaging all
of the other multiple choice questions and saying, 'Hey, 70%
of our people are happy.' Let's just go with that." So AI is
better than humans. at analyzing it, I'm sure be careful
with this AIs better analyzing it, but obviously, they can't
go go forth and solve the problems. It's not going to go do
those things need to be done. So the human comes in there.
We've shorten the time from that one example six months down
to in our study, 87% reduction in time, we actually gave the
it took them approximate two days to turn this back around
to us. And it can do it faster. That was just us trying to
trying to work around all the all the schedules. But two
days to get this data back. It's better because it's
instant, near instant, being able to get this data and
decide what to do next. There's a lot of insight there. We
actually can segment the data in 1000 different ways. So
let's look at it by gender. Let's look at it by age. Let's
look at it by work location or job function. Anything that
you can think of, we can segment that. So in the data, it's
actually funny. For the for the study we presented, we broke
it down by people that had kids who didn't have kids, and
found the kinds of things they're stressed about, even at
work with their life with that work, even their family
stresses bleed over into affecting them at work, they can't
focus they're having, having a lot of stress and issues,
because those family things creep in. So it's an example of
how, even though that happens outside the bubble of work,
still can bleed over into that.
So there's a lot of ways that you can apply NLP across the
landscape of HR tech, for example, one of the providers that
I've been talking to they actually have it where gender
performance rating as the manager is giving her a rating,
right? I'll give you a two out of five. Sorry, I'm just
gonna give you two out of five on your on your performance,
but what most managers do they then go in and write a really
nice positive comment to make the person feel better. I just
gave you two but I'm going to say nice things about you
while the NLP engine actually analyzes the sentiment, the
words you're using and says, "Wait a minute, there's a
mismatch here. You gave them a negative rating, but you're
saying positive things, you need to change one of these, so
they they actually match up." So that's an example of how
NLP can be applied. And just those little micro interactions
to help coach managers to make better decisions, makes HR's
life better for sure. You can also use NLP to analyze things
like Glassdoor data to see, are the candidates that are
applying for jobs generally happy or not, and what can we do
about it? One of the issues is, strangely enough, they're
going to have to do something with the information. I think
one of the challenges that employers have always faced this
isn't new, is there a little concerned about asking
employees questions, asking them for feedback, asking them
about their problems, their concerns, because then they have
that information and there's an expectation they're gonna
have to do something with it. So once I've told you that
there's a problem, I expect some sort of response to that.
If I don't get it that creates a very negative experience
and it doesn't end well. So employers that are willing to go
down this route. Yes, the problems are there, asking about
them doesn't create them, the problems are already there. It
just allows you to know what they are and try to address
them. A couple of case studies we talked about yesterday, we
released the research, some of the case studies for people
that are using these tools, gigher employee engagement
broadly, reduced turnover, cost savings from having having
less training, less, less turnover, less recruiting costs,
things like that. So if you understand your employee better,
fundamentally, you should be able to serve them better. And
they'll want to stick around longer, they'll probably
perform better all those good metrics that HR wants to be
bringing to the leadership team and saying, hey, look at
this impact we're having on the business that enables those
kind of things, because you're getting past the just past
the basics and getting to be a little more strategic and the
work you're doing. So they're always privacy issues, right?
It seems like today even more than ever. One of the
interesting things though, I talked to a company recently,
they're they're based in Germany, and they actually have
works councils that kind of govern how what you can do as a
nation. They're advocating for the employees. And one of the
struggles that company had, they said, "We actually gathered
this survey feedback from our employees with all that
trouble, all the time and effort and expense of gathering
survey data on their employees and their satisfaction." And
then they tried to hire a consultant to come in and help
them analyze that and do something with it. And the workers
council will not allow that third party to see the employee
feedback, all that data. So privacy is already a problem. A
tool like this would help you get around that issue. Because
there's no need to send send that out someone else you're
using this in-house, the only people that will see it, are
those in-house. So there's, there's always privacy issues.
Those employees are sharing these things voluntarily, with
the hope the sincere hope that the company is going to
actually take action and help them to create a better
workplace. With the new laws that are coming out, you know,
California is about has just passed the law, Illinois has an
interesting things about AI and recruiting not necessarily
applicable to this, but other other laws as well. Employers
need to be really careful about how they're making
decisions. What tools are using, and I don't know if it's
necessarily policy, but they need to be. They need to be
educating themselves on how these laws affect the tools they
can use, what data is being collected, where they're storing
that data, who has access to it, how long they're going to
keep it, all those basic things, or maybe that is policy,
right around data retention, all those kinds of things. But
understanding from that perspective, because so many times I
talked to are excited about getting a solution, their global
firms, and they don't realize at that initial moment of
excitement and I kind of rain on their parade a little bit.
Just tell them that you need to double check the privacy
laws before you go all in. You don't want to find out like
the the company in Germany that your data is locked. You
can't actually access it anymore.