Hi, everyone.
This is Shaoyang Fan.
Today, I would like to talk about our study
on the improvement of worker experience in micro finance
crowdsourcing, by the point of cooperative reward method.
Using micro task crowdsourcing to obtain high-quality data
has become increasingly popular.
It also plays a significant role for worker on the platform.
According to a survey, almost 40% of workers
consider crowdsourcing as their primary source of income,
but the people on the crowdfund face
risks such as unfair rejection and unexpected losses.
This risk not only prevents worker from receiving rewards,
but also causes their reputation to decrease.
This directly affects the options
for this worker to access more tasks in the future.
Therefore, improving crowd workers' experience
are becoming essential research topic.
Some tools are already available to help workers
to meet mitigating risk by avoiding
bad task and the requester.
For example, Crowd-Workers is a browser
plug-in to show the average hourly rate
for each task, which allows worker
to find the high-paying tasks.
Another example is Turkopticon, which allows worker
to review requesters.
However, utilizing such tools may lead to social division
among workers, as those workers who are not using this tool
may end up carrying most of the risks.
This study proposed a novel crowdsourcing reward method
that add worker into a cooperative group
to share risk and reward that worker in the group receive
a [INAUDIBLE],, consider the sum of the reward allocated
to the HIT.
Individual workers are paid based on the amount of time
that they spent.
This allows workers to share the risk.
Reward sharing strategy do not create social division,
as the proposed method encourages risk-sharing,
rather than risk avoidance.
We built a new crowdsourcing platform, CrowdCOOP.
By leveraging the API provided by MTurk,
that allowed us to perform controlled experiments,
comparing different reward methods and task information.
This figures show a sketch of our method.
Workers are recruited from MTurk,
and then they are redirected to our platform.
Workers can select and finish tasks on our platform,
and the reward of finished tasks are
collected into a money pool.
Finally, the rewards are distributed to worker
based on working time.
To observe the impact of different reward sharing model,
we collect and analyze worker behavior and performance
across five different conditions.
The conditions we studied varied based on whether or not
the group of workers are exposed to the proposed reward model,
and whether or not the monetary incentive
is visible to workers in the group,
and whether or not a worker can observe the performance
of other workers in the group.
Now I will focus on some findings.
First, workers completed more tasks
when they were presented with more information.
That means visible reward and platform transparency
play a significant role in encouraging workers
to finish more tasks.
The different reward and risk sharing schemes
did not have a significant impact on the number of tasks.
For complicated tasks, workers from reward sharing group
have higher accuracy than workers from baseline.
When the visible incentive is available,
workers in the reward sharing group
put more effort into task.
This can be explained as income is related
to the time spent on the task, rather than the number of tasks
completed.
This result support our assumptions
that our alternative rewards allow the worker to concentrate
their effort on committing tasks effectively, rather
than efficiently.
We found that when platform provides the same transparency
of information, the reward sharing group
attract few cheaters.
We also found an absence of free-riding
and the influence of slow workers on others is minimal.
Overall, our work of reward sharing method
creates a win-win situation for both workers and the requester.
For workers, their risk are shared.
For requester, our newer method led
to a more productive workforce, and did not
lead to a low quality data.
Thank you for your time today.