hi
this is Zhefan Ye and Jorge Vilchis from
the University of Michigan
we are presenting human in the loop pose
estimation via shared autonomy
object manipulation in unstructured
environments often requires either
careful human registration of the object
pose or direct
teleoperation by humans in previous
approaches
this can be burdensome and demanding for
both expert and non-expert users
in this work we propose a human in the
loop approach to registering object
pose
our approach leverages both human
initialization and machine registration
to reduce user effort while preserving
registration quality
so that the robot can successfully carry
out the downstream task
in this presentation we first introduce
how humans initialize object pose
followed by snap to grid machine
registration
and then the use of Affordance Template
for motion planning and task execution
to reduce search space for the machine
we first leverage
humans to initialize object pose our
robot
perceives the environment using the depth
sensor and
visualizes the scene using the point
clouds given a 3D model of the object
a user can initialize the pose by
clicking and dragging the model as shown
in the video
once the pose is initialized we proceed
to use the machine
to perform snap to grid registration our
registration algorithm is inspired
by Monte Carlo localization which is
widely used for estimating positions and
orientations
of a robot given sensor input in our
case
the machine proposes multiple hypotheses
given the pose initialized by a human
and compares them with the depth image
of the object
captured by the robot for the next
iteration
the machine picks the best sample from
the previous iteration and performs
the same operation over time
the machine can converge to a pose that
is closest to the ground truth
the blurriness in the video indicates
the certainty
of the estimated pose hence we call this
machine registration process snap to
grid
after the object pose is registered we
use an off-the-shelf framework
Affordance Template
to define waypoints for the manipulation
task
and then perform motion planning for the
end effector of the robot
here given the pose of the water pot
the robot is tasked to pick up the pot
and pour its content into the red bowl
to understand how the shared autonomy
system can benefit from both
experts and non-experts we conducted a
user study with 8 novice users
and 8 expert users each participant
was instructed to complete the
registration task in both
quick and accurate mode the quick mode
only requires participants to initialize
the
object pose while the accurate mode
requires participants to carefully
register the object pose
we performed a Fitts' law-inspired analysis
to
examine the entire registration process
the x-axis
represents the registration error margin
from the ground truth pose
the left y-axis represents the amount of
human time needed to reach
an error margin the shaded area together
with the right y-axis
indicates the remaining number of
participants at a given error margin
we ran snap to grid registration using
human
provided poses at every 0.5 seconds
to simulate the machine taking over the
registration task
as the blue curves show STG reduces the
translation error
by a large margin given the same amount
of human time
conversely STG also reduces the amount
of human time given the same error margin
however estimating orientation is more
challenging
because the objects are often only
partially observable
from the depth sensor and are sometimes
symmetrically ambiguous
to remedy this problem we suggest that
the machine can focus
on translation estimation while humans
can focus on rotation
registration
to conclude we propose a human in the
loop approach to registering object pose
using human
initialization and Monte Carlo object
localization and our user study suggests
that the shared autonomy system can
reduce human effort
while preserving registration quality
thank you