Hi I'm Alex and welcome to Super Make Something!
Today we are making amazing resin 3D printing timelapses using computer vision, machine
learning, and the Elegoo Mars 3.
Let's get started.
Magical-looking timelapses are all over 3D printing social media these days.
From manually shooting a series of photos using an intervalometer to the Octolapse OctoPrint
plugin, several methods exist to create awesome videos of your FDM prints.
The mechanics of how the build platform moves for resin printers has made it difficult to
make similar time lapses for MSLA machines -- until recently.
ResinLapse is a DSLR camera add-on developed by YouTubers UncleJessy and Andrew Sink, which
plugs into Canon cameras and acts as a remote trigger.
By using a photo resistor to sense UV light that spills out of a printer's enclosure when
a layer is being exposed, ResinLapse is able to trigger Canon DSLRs using an analog signal
that is sent to a camera through its 2.5mm TSR input jack.
Unfortunately I don't own a Canon camera, making it impossible for me to use this add-on
to make my own timelapses.
While I'm sure that ResinLapse cables for different camera models will be released eventually,
I realized that it should be possible to create a very similar result that works for any camera
and that doesn't require any special add-ons, using a bit of computer vision, Python programming,
and machine learning.
The requirement for any good looking 3D Printing timelapse is that the lighting conditions
need to be kept consistent for all photos in the sequence.
Because of this, it should be impossible to tell in which order photos were taken.
That is exactly what we are going to exploit to create this timelapse only using software.
In ResinLapse timelapses, the print bed consistenly moves upward to make it seems like the model
is being pulled out of a printer's resin vat in one continuous motion.
Therefore, if it's impossible to tell when a photo was taken, recreating this effect
should be as simple as re-arranging photos taken during printing into the correct order
to make it look like the print bed is always moving upward.
To test this method, Elegoo sent me their new Mars 3, a masked stereolithography or
MSLA printer with a 4K monochrome LCD Screen, an upgraded lighting and heatsink system,
and a new stylish curved exterior.
Retailing for $300, this printer is able to create models with an X-Y resolution of 0.035
mm or 35 microns, making it perfect for creating tiny, yet highly detailed prints.
The overall setup procedure has remained essentially identical for all of the printers in Elegoo's
Mars line, so if you're interested in learning more about this or the MSLA printing process
in general, please click the link in the top right corner of the screen to check out my
previous videos on this topic.
To re-arrange the photos in the correct order, it's necessary to be able to measure the position
of the print bed in each of the photos.
To do this, I first headed into CAD and modeled an enclosure for an LED throwie, a simple
circuit that literally consists only of an LED and a CR2032 watch battery.
In this design, the enclosure sandwiches the LED leads to the watch battery, causing the
LED to light up.
The LED's light is then diffused by a small sphere printed out of clear PLA, which shows
up in each of the photos captured for the timelapse as a uniformly lit circle.
I printed each of the components, assembled everything with a few M3 screws, and then
taped the throwie enclosure to the top of the Mars 3's build plate screw with some double
sided tape.
After this, I headed into my garage and set up my lighting and cameras.
Since the lighting needs to be consistent throughout the entire printing process and
timelapse photos need to be recorded with the printer's lid off, I found that my garage
was the perfect place to carefully control lighting while avoiding hazardous resin fumes.
For timelapses, I typically use my Nikon D3100 DSLR in conjunction with an intervalometer
that I can use to automatically take pictures after a specified amount of time.
Newer cameras, like Sony's a6400, often have internal intervalometers that can be accessed
through their menus, so you may not even need any extra accessories depending on your make
and model of camera.
After setting the picture interval to 15 seconds, I next started the print and let the Mars
3 do its thing.
Once the print had finished several hours later, I open Python code that I had written
previously to process each of the images.
After installing the required libraries, I transferred all of my photos into the 'Frames'
folder located in the root directory of the code, and got ready to create my timelapse.
It was now time to process each of the images and re-arrange them into the correct order,
which is where the LED throwie comes into play.
By extracting the location of the lit circle in each of the images and then sorting the
order of the photos so that the circle's y location always moves up, it's possible to
create an image sequence that looks like the print is being pulled out of the vat in one
continuous motion.
Circle locations are identified using the "Circle Hough Transform," an image processing
method that uses edge detection to find and extract circles with a user specified radius
in still images.
For more information on how this method works, be sure to check out the video "How Circle
Hough Transform" works by YouTuber Thales Sehn Koerting, which is linked in the video
description below!
In our application, we know that the lit LED will always travel along the z-axis of the
printer.
This allows us to further constrain the problem by only looking in a specific region of interest
in the image, which helps to eliminate false positives and also makes image processing
a bit faster, since each photo in my sequence is a 14 megapixel image with over 14 million
100 thousand pixels!
After selecting the region of interest with a mouse, the code then iterates through each
of the folders in the 'Frames' directory and saves the (x,y) location for each circle it
found, along with its corresponding filename, into an array.
After this, rows of the array are sorted so that the y location of the circle always increases,
leading to a sequence of images that makes it look like the print bed never moves down.
Unfortunately, there is a bit of an issue...
Because a picture is taken every 15 seconds, it's likely that the location of identified
circles in several images will be the same, which would make it look like the print bed
is pausing as it travels up the z-axis.
To account for this, the code only keeps the first image at each circle y location and
deletes the rest, since it will again not be possible to tell exactly when a photo was
taken if the lighting conditions within the timelapse sequence are kept the same.
Overall, this sequence already looks pretty good!
The camera seemed to capture at least one photo for many of the circle's pixel locations
along the printer's z-axis, so at this point we could simply take each of the photos, drop
it into our favorite video editing software, and create our timelapse as usual.
But...
seeing as we already have some working Python code, let's get a little "extra."
Frame interpolation is a video processing technique that allows artificial video frames
to be created that were not actually recorded by the camera.
Historically, this technique has been used to create fake slow motion video from footage
that was not taken with a high speed camera or to create a smooth animation from a relatively
low number of keyframes.
In our timelapse application we will be using it for a slightly different purpose.
Since the camera's intervalometer blindly takes a photo every 15 seconds, it's possible
that it may not capture a photo for every y-location of the LED marker.
This then results in a timelapse where the build plate's motion is not smooth, which
can be a bit jarring and make the video seem jerky.
Frame interpolation essentially allows us to generate photos that were not captured
and fill in any missing frames, resulting in a smooth timelapse even though we did not
capture a photo for every frame in the timelapse sequence.
The most straight forward way of interpolating frames is to simply average neighboring frames
together in order to create a new one.
Unfortunately, this create a "ghosting" effect and results in noticeably blurry images.
Instead, more sophisticated frame interpolation methods use a technique called "optical flow,"
which estimates how pixels and objects move between frames, and then propagates these
pixel values accordingly to create new frames without any blurring.
RIFE or "Real-Time Intermdiate Flow Estimation" is a technique developed by researchers at
Megvii and Peking University that uses neural networks to accurately estimate optical flows
at arbitrarily small timescales between frames.
In practice, this allows videos shot at standard movie frame rates to be slowed down to 1/16th
of their speed or less without any noticeable artifacting or graphical glitches.
While the details of how exactly RIFE works are a bit outside of the scope of this video,
be sure to check out my "Machine Learning 101: Intro to Neural Networks" episode which
explains the basics of what Neural Networks are and how you can implement one yourself
from scratch!
In our code, we first identify missing frames in our image sequence by looking for places
where the pixel position of identified circles bewteen neighboring frames is greater than
a user specified threshold.
The size of the gap is proportional to the number of frames that we would like to generate,
which is then passed to the RIFE interpolation method as the desired slow down factor of
our video.
At this point, our photos are still at their original 14 megapixel resolution, since this
will allow us to crop and pan in our final 1080p timelapse sequence without losing any
detail, so calculating the optical flows and generating new frames can take a while.
Once new frames are generated, the final sequence with all images is written to the 'Output'
folder in the root directory of the code, at which point it's time to drop the sequence
into your video editor of choice, perform any color correction, panning, cropping, and
zooming, and generate your timelapse as usual.
Let's see what the final timelapse looks like.
Overall, the timelapse looks pretty good!
With some basic computer vision and machine learning, this method is able to produce high
quality timelapses that are sure to be a hit on social media!
While we are using RIFE in this project to fill in missing video frames, it can be used
for a variety of other purposes, too!
For example, I used it in a few sequences in this video to create smoother B-roll footage.
See if you can spot where I used it and drop a timecode in the comments below!
A big thank you to Elegoo for sending their new Mars 3 to me in order to make this video.
It's a fantastic machine that can print highly accurate models for a great price!
The Washing and Curing machines in their new Mercury X Bundle are also great pieces of
hardware that allowed me to clean and cure my prints with significantly less mess, material
waste, and time.
In case you're interested in picking up any of these machines for yourself, be sure to
check out the links in the video description below!
The code for this project is available for free on GitHub, so that you can use it to
make their own resin 3D printing timelapses.
If you liked this project and found it useful, please consider giving it a like, subscribing
to the channel, and hitting that bell icon to be notified when I upload my next video.
If you end up using the code from this episode to make your own resin 3D printing timelapses,
please also consider becoming a patron on Patreon to help me make more episodes like
this in the future!
Your support ensures that Super Make Something episodes will remain free and open source
to anyone looking to learn more about 3D printing, programming, AI, and more!
To stay up to date with my latest projects, be sure to follow me on social media, where
I post behind the scenes updates while I'm working on a project!
Finally, please be sure tag me in posts where you use this software for yourself.
I'd love to see what you come up with!
Well, that's all there is to this episode!
Thanks for watching, now go Super Make Something!