hey everyone it's norm from tested and
welcome back to projections but this
week we're not talking about
a virtual reality or mixed reality
headset
we're actually talking about different
type of projection a
holographic display so this is the
looking glass
portrait and earlier the very beginning
of this year i had the chance to preview
this device uh early prototype of it
and now it started shipping to their
kickstarter backers this one actually
is on loan from our very own jeremy
williams who is one of their earliest
backers and i've had a chance to spend a
couple weeks with it
because it is a device that endlessly
fascinates me
and you might have seen some video of it
go viral
recently playing volumetric doom on
this device in case you're unfamiliar
this is made by a company called looking
glass factory and the
engineers and really the futurists over
there over the past
10 years have tried to find ways to
create
a volumetric display they started off
with
kind of the 3d matrix of led lights
they've done everything from things
encased in acrylic
to this now more refined uh device
that isn't just for developers but what
i really think is
makes sense for early adopters and
tech enthusiasts and i've been having a
ton of fun
with it so as you can see it's a display
you know it looks like a digital picture
frame and in some senses
it's kind of that also works as almost
like a external monitor you can plug
into
a computer but unlike your typical lcd
display or small screen or tablet or
whatever
uh this is a light field volumetric
display
and that sounds very fancy but you can
almost think of it as like
one of those old lenticular book covers
where
you see that classic you know lenticular
screen
and if you hold that book or that sheet
in front of you and move your head
you either see an image that's in three
dimensions or you might as you move your
head you see some
movement and the idea behind that is
that screen
which has that iconic kind of vertical
lines that striping
is a filter that then takes the backing
the print of whatever that was and
splits it into
different views so where your relation
to it
your left eye and your right eye would
see different parts of
the image and when at an appropriate
distance at a certain sweet spot
your left and right eye would see maybe
a sterile image stereo would be in 3d
or as you move your head around it would
shift from the different interlaced
pictures
uh to create the illusion of movement
and that's essentially what's kind of
happening here
there is a lenticular filter a screen
uh on top of instead of a printed
sheet it is an active display
and so it's lit up and this thing itself
contains right now i just have it
plugged into usbc for power and it's
running off of a built-in
raspberry pi and as i'm cycling through
uh these preloaded images i'll try to
give you a sense of that depth
and you can see as i rotate this left
and right here's an example of the the
field of view
right there's a sweet spot for sure but
as you move your head
around in this case the camera lens here
uh you can see that there's
not only depth there's parallax that it
looks like
there's a 3d object inside
this screen not only does it go deep
look like the illusion it goes beyond
the screen
but it looks like it pops out a little
bit as well
and the idea is that they're displaying
up to
a 100 different view viewpoints all at
once
in fact here it looks like based on the
source images there's a sweet spot
of about 48 different views six by
eight and the source images here are
basically displaying
48 different interpretations of this
scene
all at the same time along a horizontal
axis so you can think of
that old like when they made the matrix
they had
the bullet time cameras all those dslrs
along the road and took those pictures
all at once this is displaying
48 pictures all at once your eyes may
only
get you know half dozen or so of those
but as you move your head around
you're seeing different parts of the
scene
and because it's taking a standard lcd
screen and splitting up into those views
and going through a filter
you know resolution wise it's not as
maybe as crisp as you'd find
on like a retina display or something
but it's still sharp enough that
it's very visible there's no text here
but for
faces for 3d models for even little
rendered scenes it looks
vibrant and and crisp and
it doesn't have any of that ghosting
that you might see as a hallmark of
those old lenticular screens they're
doing some
interesting blurring as they render out
these images now these are all
pre-rendered so as i cycle through them
you know they might look like
3d rendered scenes right but what they
really
are are pre-rendered outputs
and basically think of it like a digital
picture frame
you're taking some type of asset whether
it's pre-rendered scene or photo
and then side loading it onto this
device from your computer
and this can store hundreds and hundreds
of those images
which i can cycle through by pressing
these buttons on the side i can pause it
one and i can even adjust this
ring light that goes around the outside
uh that light helps give more of an
illusion
of depth because uh the sides of the
display are mirrored and so
it has like um it has a framing effect
where it looks like there's like a
a a lit uh almost cube um
or some type of grid that the 3d image
the volumetric image
is living within and
it's super neat uh the images that come
out of the box
incredibly compelling and it's a mix of
rendered things and
also very powerfully notably also
photos taken with a standard smartphone
because phones these days your iphone
your android phone not only can they
take a
nice high resolution picture they can
also take a
depth map that's how apple would do its
portrait mode and
can blur out the background it's using
depth data whether it's from like a
lidar sensor or multiple cameras
to add an extra layer of information on
top of the photo
so that it can say that it can recognize
that part of the image whether it's
someone's face maybe
in the foreground and some part is in
the background
and that's maybe the coolest part of the
looking glass portrait the thing i was
most curious about
what is that workflow to get a photo off
of
your phone you know all those portrait
photos that you may have taken not only
with this phone but phones over the past
couple years starting with like the
iphone 10
what can you do with those photos how
long does it take to process that and
get that
onto the looking glass portrait that was
what i was able to explore that's what i
want to share
with you today so let's head over down
to my desktop pc
plug this into usb for data and hdmi
onto my computer and i'll run you
through the software
including doom all right so what i have
lit up
here is the holoplay studio application
they have a version of this for windows
and
mac and you install a service that
recognizes the
looking glass portrait and this is how
you would add
what they call a hologram to the looking
glass so i've already done some
pre-loaded ones but basically you go to
click holo add hologram
and you have the option of adding an
iphone portrait photo or you could take
a bunch of
sequential images almost like that
matrix uh bullet time camera
array but we'll do just an iphone
portrait photo we're gonna
import and i've just downloaded these
jpegs off of my icloud so
let's do that one and now it's loaded
so what you actually see on the looking
glass right now which i can see as well
is the photo we've loaded and
essentially there's a depth map
that they've converted into a 3d model
so
my son here is in the foreground and
it's basically looks like
almost like a texture wrap he's taken
they've taken the photo and
wrapped it around their modeling a 3d
model
of of the same image using that depth
map
and i can adjust some settings here for
what they call the depthiness so
flatten out that image here as you can
see or stretch it back
and it's essentially a 3d model at this
point with some dynamic
um attributes i'm adjusting so like
how deep do i want the scene to be and i
want it really deep
then where do i want the focus point to
be there's like a sweet spot a plane
where there's a sweep a spot of focus so
too far up front
and the convergence doesn't really work
um too far
back and it gets a little blurry but
right a sweet point right here
super easy to do these and then you also
have
uh the ability to you know toggle uh
inverting the depth which is really
weird or the depth position
but at that point this image is ready to
be
uploaded to the display and when this
gets exported
to a format that then the looking glass
itself can read
it's really interesting i'll pull an
example of one up right now
it's actually what they call a quilt
format and here's an example
of that this is an animated video but
you see this grid
of 8 by 6 48 different
views uh when i was talking about that
lenticular display
that's what i'm talking about here these
48 different views
are all displayed at the same time
in the the display here and um
and then if you move your head left to
right
you actually can see you know behind a
little bit you get some parallax
the the text is in the front
and there's more you can do with these
iphone portrait
photos as well there's a piece of
software that looking glass has called
diorama
and you can import an image so
i will load another depth photo
and you can then actually add
elements to it and what's cool is i can
actually move my mouse
all the way across because this actually
acts as a
third screen and i can manipulate this
in real time so it's taking this photo
of our dog which is another depth map
and
adding elements like these funky lights
but i can
adjust and i can do things like
change the background if i want to do a
more
crazy background let's see what that
looks like
and these backgrounds all these assets
are just pulled in
from jiffy so i can actually you know do
something
like search san francisco and pull in
uh here let's add this to the scene or
maybe
this one this one's not bad and again i
can also go back into
looking glass itself and not only adjust
the image here but move my head
physically around
this display to see what that parallax
looks like uh let's go add some
uh stickers let's see
there you go how about another little
boss interior
right here i can even
adjust the depth of this
bring it to the foreground shrink it to
the background
and then also adjust the lighting
you can actually see that shadowing
behind what's neat is i can actually
click these lights
and dynamically move it around basically
it's a 3d scene here
like again taking that portrait image
and turning it into a model
that's texture map and then when i'm
ready then i click record video
and then it'll start rendering out all
of those views
and with every frame here there's going
to be
48 different angles so that this can
display them all at once
next up beyond the scope of just the
images there's also
objects that i can load in objs and they
have some sample ones here so this is a
3d model it's textured and you can see
on the
flat screen here i can do a little bit
of manipulation of it
and as i do that it's actually rendering
out again
all those views on the looking glass so
i can actually look around this robot
and see it as if it's floating right
here
in front of the display here's another
example
of a sample robot
and as i manipulate this it's kind of
animating right there
on the display i can adjust some
settings and
change the shader effects just the
detail
the brightness even the lighting i can
actually
move that lighting around which is
really neat
you can see if i move this backdrop
behind
then i can dynamically adjust that
lighting
and if you 3d print files you're used to
stls you can use a program like
meshmixer
and export those as objs without
textures
and then load them in and it becomes a
really really high mesh
model that i can see in full 3d
on the looking glass once again i can
take a screenshot here or i can record
a video and basically flatten out
this particular animation or this view
and then get it something i can just
look at baked into
a a quilt image that they they have
and then finally here is doom
playing on the looking glass which is
just a unity version
of doom running the looking glass api
so it then creates a lenticular screen
and you can see that
looks like doom
and what's really cool here
is you can actually look around
the scene
so as i'm moving around
you can actually look
around the corner
so after going through with that
software and even running a game
like doom a very recognizable game but
being able to
do things like look around the pillars
and then
actually play it in a new and novel way
my big takeaway so far
with this is not only have they created
a
nicely packaged piece of hardware that
has this sweet spot in terms of it
running
right out of the box already compelling
that way
but also a software package that's
feels refined as well the workflow to
get photos and
dress them up in those dioramas and get
them onto the vice
is really simple now it does require a
little bit of processing power
and my desktop was much more capable at
doing those type of renders than
my laptop especially the real-time obj
viewer but it's doable and maybe
uh there'll be efficiencies in the
software because my hope is maybe run
this in a self-contained
compute box like one of those intel nux
uh
and be able to do uh view
objs and in a self-contained way without
having to plug into a big desktop
pc and in terms of games while doom is
certainly eye-catching and a real
wow moment for anyone who's played doom
before
i'm really more excited about the other
potential game opportunities for the
looking glass portrait with
the unity plug-in and then real plug-in
and developers making
isometric games and and vertical
shooters
you know classic games like galaga could
look really cool in a volumetric
way and it feels like we're just getting
started with this so
we'll be doing more coverage of the
looking glass portrait when i get
my backer unit in because i really want
to try out
like connect video conferencing with a
volumetric video
and and also as they drop more of
their holograms and more software is
released for if you have questions about
this feel free to comment post in the
comments
below but it is still available for that
backer price of
250 bucks and i think if you're an early
adopter
in a tech uh this is a safe bet because
it's something that
works and you can take advantage of
right now
with all the phones photos you've taken
on your
phone i want to thank jeremy for letting
me borrow his
early unit to check out and go hands-on
with
and i will see you next time