Hello everyone!
Welcome to the presentation of our work, titled A B2C Bias for the B2B World:
A Case Study of Employees’ and Customers’ UX of a B2B e-commerce site.
My name is Vassilis-Javed Khan and I am part of the Digital Transformation Team at Sappi Europe.
This work has been a collaboration between Sappi and the University of Patras.
I and Damianos, who interned at Sappi Europe, will be presenting this work.
One of the immediate and perhaps long-lasting implications of the Coronavirus pandemic
has been the increase of e-commerce solutions for businesses.
Although the broader public is probably more familiar with B2C e-commerce sites,
the B2B e-commerce is many times larger in value.
and evaluating a site’s usability and user experience is of utmost importance to its continuous improvement.
The case of a B2B e-commerce site is a particular one as employees of the host organization can also be regular users.
In our case study we present some of the results of the user studies we performed on the e-commerce site of Sappi.
Sappi is a global manufacturing company focused on providing dissolving pulp and other paper-based
and biorefinery solutions to its customers all over the world.
The first version of Sappi’s e-commerce site was developed in 2000 and it offers extensive functionality.
Having as a goal to develop a holistic understanding of Sappi’s e-commerce site,
we followed a mixed methods approach.
Given the scope of this work, in the remainder of the presentation,
we will only focus reporting on the User Testing.
We conducted two rounds of user testing with different user groups:
One consisted of employees of Sappi who frequently use the website and one with customers.
At first, we administered to the participants a short pre-task questionnaire capturing their
previous experience with Sappi’s own B2B e-commerce website and other B2B or B2C e-commerce websites.
We also included the Net Promoter Score, a single question that serves
as an indicator of user loyalty and growth potential.
Following the questionnaire, we observed the participants perform the user test tasks
that were designed to cover the site’s functionality.
With the completion of the user testing, we administered the System Usability Scale Questionnaire,
which yields a single score as a measure of the perceived usability,
and the short version of the User Experience Questionnaire which offers a comprehensive view of the User Experience.
As a final step, we asked again the NPS question.
The analysis of the recorded sessions and the annotation of usability issues yielded
a total of 114 findings for the employees’ group and 150 findings for the customers’ group
distributed among different parts of the website.
Employees evaluated the site’s usability as average
whereas customers evaluated the site’s usability as excellent.
We observe an even bigger difference in the UEQ-S results
with the employees evaluating the User Experience of the website as bad
while customers rated the website as excellent with a much higher score.
A similar difference is observed for the NPS question.
It is evident that customers score markedly better in all three measures.
The question that naturally arises, is why do we observe such differences?
In our effort to answer this question we reached again to the employees that participated in
the user test and we asked them their opinion.
The follow-up investigation with-in the employees group suggested that a potential reason of
this difference is that employees tend to compare a B2B website with B2C e-commerce sites
that they use in their personal life whereas customers will tend to compare with
other B2B websites that they regularly use as part of their professional activities.
We can corroborate this reason with data we collected before the user test.
We indeed observe that employees seem to be more experienced with B2C sites,
while customers seem to be equally experienced with both B2C and B2B sites.
When comparing the difference within the two groups we find that in the case of employees
the difference is statistically significant but in the case of customers it is not.
When comparing between employees' and customers' experience with B2C sites
we do not find a statistically significant difference,
but when comparing between the two groups experience with B2B sites we indeed find a statistically significant difference.
A possible implication of employees being more critical and transferring their expectations from B2C sites,
is that they might not wholeheartedly promote the B2B e-commerce site.
Thus, a direct implication of our results, would be to measure the NPS score with employees
who are responsible for promoting the site.
And if such a bias is discovered, UX professionals should make employees aware of this bias and
communicate that a direct comparison with B2C sites might not be entirely fair,
as in the case of a B2B e-commerce site, the company’s main business is not the site itself,
the site is usually one of the ways of selling and servicing customers.
A final recommendation and a positive interpretation of our finding would be that the employees
would have this rich experience from both worlds and therefore potentially have suggestions
from their B2C experience on how to further improve the B2B site.
Precisely those employees would therefore be a valuable source for designers to tap into.
The main takeaway message of this paper is that there seems to be a bias exclusive to employees,
a sort of implicit transfer of expectations from B2C to B2B.
Companies in B2B e-commerce should take into consideration this critical stance of employees
and leverage it when designing user studies of their e-commerce solutions.
We hope that with our work we will encourage other companies, especially in the manufacturing domain,
to report findings on such comparisons of their own B2B websites.
For more details, I would refer you to our Case Study's paper.