we're in the mixed reality lab of the
university of nottingham's computer
science school
and this is actually the lab space where
quite a few things happen so it's in a
state of perpetual mess and chaos
but this is the little uh piece of
santee we've carved out we grab the
space to set up a virtual reality bay
running off a vive which is usually we
use the vive system for any sort of
virtual reality development but of
course the tech is pretty good the
hardware is pretty good but also it's
the most open system for development as
say opposed to the oculus and stuff like
that and it does allow you i will grab
us a prop to use stuff like the vive
tracker which allow you to bring in
other
objects accessories and more interesting
virtual reality interactions than you
would in a closed ecosystem which is why
we tend to use it more
than oculus or other technologies
last time we saw you on computer file we
were looking at scanning of miniatures
3d scanning miniatures for various uses
and augmented reality storytelling so we
moved on and we're trying to close the
loop a little bit you know take
something physical scan it into a
digital asset that you can use in
virtual worlds and stuff like that so
america in that video we scanned a bunch
of miniatures that people brought in we
put those in a virtual museum so people
could walk around them like godzilla
size though okay we've gone from the
physical to the digital why not go back
to the physical again so what we did is
we started 3d printing those miniatures
you might recall this guy i think he was
the most popular one on the website back
into
physical form at slightly different
scales and then we thought okay how do
we push that envelope a little bit and
what can we do with these things is in
fact we're still just a hunk of plastic
depending on your printing technology
but usually just that
so we thought we'd go use a bit of tech
to now start overlaying those
photorealistic 3d scans you miracle from
the previous video back onto these
physical objects it's a little bit
tricky because you have to know where
this physical object is in space and
there's various ways to do that but once
you've got that and somehow you can skin
this hunk of plastic with that
photorealistic scan you can have some
very interesting interactions the key
point there is locating it how do you
know where it is well the
various technologies all their new to do
this basically it comes back to
uh locating objects in space which is
not quite a solved problem i mean we
wish we had indoor gps but we're not
quite there
uh so you can do things like um
tracking with ultra wide band that's
used widely in robotics you can do stuff
with timer flight i mean anybody who's
used uh recent vr the controllers the
headsets attract somehow for example
this is a vive tracker which is
effectively the controller without the
controller bit and these allow you to
track something in space just by
attaching them to something and then
just locating them so this now
effectively can be brought into vr
you can use uh
newer optical tracking technologies but
those when they work the fantastic by
getting them to work can be a bit tricky
when you don't know what your final
object will be or when that object is
sort of malleable so
for now we have been using these in our
vr experiences because they tie in with
the rest of the ecosystem so one reason
to do this
would be to
haptically experience objects in
different scales for example here we
have that miniature the original thing
was uh what 28 millimeter scale about an
inch tall now we have something bigger
and we can take that in vr and
interact with it at a larger scale now
okay that's not a big step but if we
take something different
like for example the veil vessel virgin
from chatsworth house the actual statue
is about two meters tall i believe and
here she is in comfortable palm size
more and more museums for example which
is one of the contexts we work with a
lot
uh what they do is they have now they're
not really scanning the collections so
they have vast archives of 3d models of
things so you can interact with those
online on screen in vr the usual thing
walk around it but again you can't touch
it you can't actually really interact
with it things that would be at a
different scale like the veil vestals
much too large the real thing do
something like this or objects that are
really one-of-a-kind you would be able
to touch like say the crown jewels for
example you'd be able to interact with
them
in ways you'd never be allowed to for
example like say putting on the crown
jewels or see entire environments that
and buildings that don't exist there are
limitations to technology vr is has a
few unsolved problems so seeing as we
were using these uh trackers these do
have some issues
so of course they need line of sight
just like the controllers they are a
considerable physical
object that you have to work around you
can't you could just attach it to any
object but then
not all
objects
have this handy sort of way you can
attach them and even if you do what
happens the first time you do that
somebody sees that hand it to them what
they do they grab the track because they
think it's a handy handhold but of
course what that means is you break line
of sight and the tracking fails
so you need somehow to work around that
our sort of solution and we'll see here
some of the objects we ended up using
from our museum partners because this
was the um probably the best place to
launch this technology is
taking putting the trackers inside boxes
the tracking works right through them
but here are some very practical
benefits nobody well no bidding with
normal sized hands at least can possibly
grab these trackers and fully occlude
them from the vive lighthouses one
elephant of the room that i haven't
really mentioned is the 3d printing of
these objects
now some might say that is a solved
problem 3d printing being a world
reverse technology at the moment but
that's not always the case for starters
you are trying to 3d print 3d scanned
models 3d scan models
whatever technique you might use might
be photogrammetry like we use for the
for the miniatures or something like a
laser scanner which was used to scan
this engine for example
uh they never produced
optimized models optimized models as the
kind you would make from the ground up
for say use in
games or animations and stuff like that
those models always need some cleanup
they have
millions of vertices more than they
would need to be optimized and often the
models are not
ready to use they still have gaps in
them they still have errors stuff that
really doesn't
helps with
any 3d printing technique so you have a
few ways to go about that either you fix
the model or you try and use techniques
that ignore them in this case for
example
which was a sort of easy object to fix
this was printable on a standard fdm
printer an added process layer by layer
and everything works fine that's just by
depositing material and this is the kind
of commercial 3d printing technology
that you can have in the home right now
so most organizations do have access to
it but something so complicated as the
igalero engine where you have a lot of
very small parts that are not
necessarily very well supported the
problem with an fdm printer is that you
do need to support the overhangs and the
objects that
are not connected during the printing
and then remove those supports quite the
job this one actually was printed by our
partners in the advanced manufacturing
building with one of the
let me see if i get this right
sintering printer i believe and those
are very nice in the sense that
they use a laser to sinter powder
plastic powder inside basically a bin of
powder the entire object while it's
being made is suspended in that powder
and you come out with this fantastically
detailed and way stronger than it looks
3d model all these fine bits are
actually exactly as they came out the
model they may look like errors but
they're not errors in the 3d printing
process rather they are errors in the
laser 3d scanning process that were used
sadly this is not quite ready for
domestic use but maybe we'll get there
even with these technologies they're not
there are objects that are not suitable
for 3d printing so there is a
halfway house that you can use and that
was an empty acrylic box when you have
something that you can't 3d print but
you still want people to be able to
interact it with it in vr in some way
well what we did was basically place a
tracker inside this box so we have a
trackable box and in vr we can show a
glass box or something similar the exact
same dimensions but inside this
in vr we can place any 3d model we want
whatever it's something we can't print
we can change it dynamically whatever we
want you can get the halfway house where
people can pick this box up in the real
world picking it up in vr bringing it up
to the face rotating it to get any angle
they want even the bottom because the
bottom can be see through in vr and get
that much closer to an object they would
not normally be able to experience it
might be as we're saying something very
large
now again in the palm of the hand almost
so it is a bit of a compromise but an
interesting one of that what environment
are you putting how does it all work for
our purposes and uh to sort of begin off
with a museum context since we've got
objects from uh
the collection of the derby museum art
gallery
uh we've placed them in sort of the
setting of um
the
warehouse of a museum because after all
this is giving you access to stuff that
you normally would not even perhaps be
able to see at a museum any museum can
that most have
usually less than half of its collection
on display at any time so we use the
unity game engine which is together with
unreal probably the most popular game
engine or rather at this point
experience engine to make virtual
reality experiences a pretty
straightforward environment get help
gave us everything we needed to
pull together
the
various disparate parts of this it's
effectively really a standard virtual
reality setup the biggest problem you
might have for hardware specifications
is having enough usb ports
because each one of these trackers
requires its own usb dongle which means
you really quickly run out of them and
you don't even realize it
where it is in comparison to the other
one so we'll ideally get something like
that as you can see are three different
heights
around the model
then that will create a sparse point
cloud so basically
the between the images it will figure
out interesting points on the model so
features and put those as points in
space