- Status booster? - Go.
- GNC? - Go.
Usually nations tend to go first to the moon,
fly by the moon, and then orbit the moon, and then land on the moon.
And then fly by Mars, and then orbit Mars.
The United Arab Emirates decided to skip all that and go straight to Mars.
Nothing about this mission is easy.
50% of the missions that were sent to Mars failed.
- Every successful business starts with a great idea.
Today, moving from concept to reality is easier than ever before.
Join us as we travel around the world and meet developers, architects, and innovators,
to learn how they're building on the AWS Cloud.
Welcome to 'This Is My Architecture'.
Our journey starts over 7,000 miles away from Dubai here in Boulder, Colorado,
where the Emirates Mars missions...
Solutions Architect Manager, AWS
...Hope Probe was built.
Getting to Mars is incredibly hard
and requires a specially designed launch vehicle and spacecraft.
The vehicle has to withstand traveling a vast distance
in the harsh environment of space
and then make a precision burn to land in the orbit around the red planet.
After 11 long minutes of radio silence, cheers erupt in the Command Center
on the confirmation of a successful orbiting insertion.
The UAE has made it to Mars.
So Pete, you are the Emirates Mars Mission Project Manager.
What does that involve?
- So as the project manager, I'm responsible...
...to make sure that the program meets all its objectives on schedule and in budget.
- And what were some of those high level objectives?
- So the overall objective was to further understand
the atmospheric processes on Mars
and how those evolve globally over seasons, and over time.
In order to do that, we had to create three instruments to look at the planet
in three different colors of light or wavelengths,
and then send that data back to earth for analysis.
- A team from the Emirates work directly with the University of Colorado to build the spacecraft
drawing on the expertise from the university's laboratory for atmospheric and space physics.
- LASP is part of a knowledge partnership with the Emirates,
and they came to us saying...
..."Look, we want to understand what it means
to go into an interplanetary mission and to learn about space
from an engineering level and a scientific level."
So they said, "As a partner, can you teach us this
and can we work side by side with you?
We don't want to buy this expertise, we want to learn it and take it into our future."
- So I understand one of the key instruments, the EXI, was built here.
Tell me what does it do?
- So that's the Emirates eXploration Imager.
And so it's taking pictures of the planet, and it's doing so in six wavelength bands.
There's a red, green, blue,
and then there are three ultraviolet wavelength bands.
This actually gets accomplished because there's a filter wheel
that has a filter for each of those bands.
So the filter is moving in place while we're taking images of the planet.
Something that's different about the orbit of this mission
is that we're at a high enough altitude that we can take an image of Mars
all in one field of view.
And so over the course of the week,
we can look at what is happening in the atmosphere,
and of course over the course of a year, seasonally, we can see how that changes.
So this is an engineering model of EXI,
and it's an exact duplicate of what's in flight around Mars right now.
You can notice first off that there are two different shapes on the front.
Those are the two different lens systems.
We have a visible camera and a UV camera.
So the UV camera just cares about the Mars surface.
And so that's round, 'cause Mars is round.
And then with the visible camera, we're hoping to catch a moon, a star,
some other things in some of our pictures with Mars.
- A decision we made very early on was to ensure
that we didn't need to develop new technologies.
We actually used existing technologies in clever ways.
The innovation of the program lies in how we managed it and how we developed it,
how we formed the partnership and the collaboration.
The hardware and software itself, it's mostly heritage.
- The entire project was just six years in the making and cost under $200 million,
a fraction of previous projects.
But I still wanna understand why the UAE is making such an investment in space.
I spoke with Omran Sharaf,
the Emirates Mars Missions Project Director, to learn more.
- The UAE government wanted to see...
...a disruptive change in our ecosystem when it came to science and technology.
For us, it's about the future of our economy.
The leadership in the UAE believes that the future of economy
will depend on science and technology, will depend on innovation-creativity.
And to do that, we need to have a strong and advanced scientific policy.
So the UAE looked at Mars as a mean or a tool for that bigger objective
and also to send a very strong message to the Arab youth
is that if a young nation like the UAE is able to reach Mars
in less than 50 years, then you can do much more.
- Omran, an inspiration to so many nations and to our youth.
Thanks much. - Thank you, Adrian, thank you.
- The Hope Probe will spend one Martian year.
That's about two earth years gathering data from the red planet's atmosphere,
all of which will be ingested by the cloud.
But how they gonna do that from so far away?
Well, I've come to Dubai to find out.
So Muhammad, I see the Hope Probe here,
tell me about some of the data that it collects.
- Okay, basically we have three onboard instruments on the spacecraft.
The first one is EXI,
which is the Emirates eXploration Imager,
which takes the images of the planet Mars.
We have EMUS, which is the Emirates Mars Ultraviolet Spectrometer
which gives us ultraviolet data from the atmosphere.
And we have EMIRS, which is the Emirates Mars InfraRed Spectrometer.
Those instruments onboard the spacecraft are used to study
the upper and lower atmosphere of Mars.
- So it's a mixture of textual files as well as rich image files.
- Yes. - So how does that data get down to earth?
- The data gets transmitted through the Deep Space Network,
which is managed all around the world by NASA.
And then we have the Mission Operation Center here at the MBRSC
which uses the data.
It gets initially processed, and then it gets pushed to an S3 bucket we have,
which is called the landing bucket
- What size is some of that data?
- It could vary between a few megabytes of data,
and it could be very small files.
- And how frequently is the data coming?
- We have two contacts per week that is done by the MOC,
and each contact has a window of 16 hours.
-It's a very small window. - Yeah.
- A lot of data gets generated out there in space and needs to travel back down to earth.
How does that happen?
- It's very challenging, and it's mostly challenging because of the vast distances
that we're talking about.
So if you look at the fact that we have data that's stored in the spacecraft,
and it needs to be sent to earth, it is millions of kilometers away.
So what we actually do is we send radio frequency through a high gain antenna,
but that antenna concentrates the energy into a beam
that has to be precisely located on this planet.
When that energy hits the earth, well, now something has to pick it up.
And so we use NASA's Deep Space Network.
That information is collected through those antenna and then sent to a central site,
which in EMMs case is the Mission Operations Center at MBRSC.
- So the data sitting there in S3, what happens next?
So the first Lambda, it gets triggered by this, an S3 event,-
once we receive the data.
So this Lambda what it basically does it check is what type of data we are receiving
and whether we are receiving correct or incorrect data.
If the data is correct, it'll move the data into another bucket
which we store all of our science and ancillary data
that we receive from the Mission Operations Center.
As well as it will index
that data into an RDS database
we have, which is the SDC database.
Another thing that it does after this Lambda runs
it sends an SNS notification that tells us whether this process
was done successfully, or we have any errors that occurred during this process
in which this SNS gets sent to the development team we have.
- To know that it's actually arrived. - Yeah.
- To know that the data arrived. - Right.
Why are you using different S3 buckets?
- So the first S3 bucket we're using is the bucket that we first received the data
from the MOC into this bucket.
This bucket, the science data bucket,
we're using it to actually store the data that we have.
So the science data, which are the data from the instrument,
as well as the ancillary files.
- So different processes needs different permissions.
- Yes. - Right.
Okay, so what happens after you've got two different layers of storage here?
What happens next?
We have an S3 event that is also configured to trigger another Lambda.
This Lambda also sends in a SNS notification,
which tells us whether a new file has been received,
for instance, in this Science Data bucket. - So quite clever.
You're actually using
the S3 event trigger capability - Yeah.
- to actually chain all these
different Lambdas together. - Yeah.
- So we've got indexes sitting over here in RDS.
What about this S3 bucket?
- This S3 bucket, we have a Lambda that sends the data
that we have received from each instrument, and it sends like a file
that will be used as an input to the next component,
which is the processing component. - Great.
And we're gonna get into this a little bit later. - Yeah.
- Fantastic.
Muhammad, thanks for sharing your architecture with me.
- Thank you very much.
- So currently we are in the science phase of the mission.
We started the science phase in May of this year,
and we've been collecting data and calibrating data
and preparing it for the release.
That's gonna take place basically on the 1st of October 2021.
And we've been looking forward to sharing this data with the rest of the world
and hopefully have the science staff of the world use this data to benefit from it
and come up with new findings.
- It's apparent that all that data being transmitted from the Hope Probe is priceless.
But how does the team at the Space Center process all that data?
I spoke with Omran Ahmed Al Hammadi to learn more.
So, Omran, we'd learn how you collect data from the Probe and store it.
But I know you have some instrument developers that are keen to get their hands on it.
How do they do that?
- We have for each instrument on board
the spacecraft, a team that is responsible for developing
processing software that will generate high level scientific products.
So they develop the software, they created Docker image for the software
and then they access - securely access the EC2 instance through the SSH protocol
in order to push their Docker image to the ECR Service.
- Right, so each team has their own EC2 instance to do that.
And then using ECR, I imagine they can keep different versions of that software.
So we learned before about the science products that get stored.
Tell us a little bit about what those science products are.
- Once we receive level zero scientific products,
which are the binary files plus the SPICE files that are required by the processing software
to generate high developed scientific products.
And these SPICE files contain information about the geometry of the spacecraft,
the location, exact location of this spacecraft, with the respect to Mars
or with respect to the sun,
plus other information that is required by the processing software.
- All right, so you've got these science products sitting in S3 that we learned about before.
How does it all kick off?
- So once we have the level zero products, the SPICE files available
that will trigger the Lambda function.
And the idea behind this Lambda function is to kick off the step function,
which orchestrate the workflow of the data processing software.
So once the step function kick off the batch job
will grab the appropriate Docker image from ECR service
and then kick off the level one processing software,
which will generate un-calibrated scientific products.
And then once we have this un-calibrated data,
the level two process software will be kicked off to generate calibrated information.
Once we generate this information,
the output level two and level one products will be stored back to S3.
- Right, so I see you're continuing on the theme of event driven architecture,
but tell me why you chose batch?
The batch is a great service for asynchronous job running.
So we have a batch job
that is running for each instrument independently.
We don't have to wait for specific instrument data to be completed
in order to kick off the process for the data for other instruments.
- And of course, batch automatically provisions all the compute
and your memory that you need for the actual job,
so that's more heavy lifting taken away.
So I see you've got some more storage services.
Tell us about how you're using DynamoDB, RDS, and I see the EFS there as well.
- Yes. We mount an EFS service and storage to batch jobs
so that the instrument team and their processing software
can save logs into that storage service.
They can store intermediate data into that storage service,
or they can have access to the SPICE files
that we send from the S3 to EFS.
So that's why you use EFS.
For the RDS, we provide the instrument team facility and their processing software
access to a separate database, a separate RDS instance for them.
Once we complete the processing jobs,
the Lambda function will collect the status of processing jobs,
what time they run, how long they take, the status of them,
and store that into DynamoDB.
So this is the entire pipeline that we architect for the instrument team facility
to run their process software, to generate high level of scientific products
and store them into S3 that will be accessible later on from the science community.
- So the significance of the EMM data that is being returned
is that is completely unique.
Never before have we had data of this type.
The previous Mars missions have only
been able to take a snapshot of what's happening
in the Mars atmosphere because of their orbit.
Our orbit gives us an opportunity to view Mars atmosphere daily,
weekly, monthly, and over seasons to really be able to see
how the atmosphere is evolving and changing over time.
But one of the most important questions that this data will hope to aid the community
to better understand is essentially what happened to the Martian atmosphere.
One of the driving questions of Mars science is what happened to the water on Mars.
And one of the hypotheses is that it has been lost to space
through atmospheric interactions and atmospheric loss.
Eventually we will have a better understanding of how Mars lost its water,
at what point it may have been habitable,
which has implications for life on that planet outside of our own planet.
I think that's a very exciting topic in planetary science right now.
- One big aspect of this mission and why it was called Hope, it's again to inspire
and teach Arab youths,
but also it's to promote science diplomacy.
So it's one of the things that the UAE wanted at the end of the day -
us understanding Mars will help us as humans to better understand our own planet
and the changes happening around us.
So sharing it with the scientists around the world with no restrictions,
definitely will benefit humanity.
- So Omran, I've spoken to the science community
about how they use the data,
but let's dive into how you make it available to them.
- To do that, we developed a website that will be accessible by the science community
to retrieve data from the platform that we have.
We configured a S3 bucket to act as a web host.
Since we have the scientists across the globe,
we utilize CloudFront to cache
the content of the website closer to the scientist
and we have created a registration form
for the scientist to go
and access the data that will be managed by Cognito
to manage the user accounts of the registered scientist.
- Right, so that takes care of your static files.
What about all the dynamic data that you generate for them?
- We developed a secure API Gateway that is secured by Cognito,
so that only a registered user can access that API.
This API can be requested and interfaced with the static content.
So once the request hits the API gateway, it will trigger a Lambda function
that will handle the request, see what is the nature of the request,
is it a query request, is it a file download request?
And if it's a query request, it will query this SDC database that we have,
or indexes of the files stored there, and then send back the result to the scientist.
If the request is a download request, the Lambda function will retrieve the files
from the S3 bucket, and zip the files, send the files back to the scientist.
- When it comes to space exploration, usually makes this look at it as a space race.
The UAE never looked at it as a space race.
It looked at it as an opportunity for collaboration
and opportunity to work together with other nations.
- When the United Arab Emirates reached Mars,
they became just the fifth country in the world to reach that amazing feat.
And they did it with an unprecedented low budget and short timeframe.
It's a true testament to what a country can do with a strong vision
and investment in STEM for its people and embracing the spirit of global collaboration.
The UAE has proven that the democratization of space is possible,
and they're just getting started.
Dubai's ruler, Sheikh Mohammed bin Rashid Al Maktoum, has got big aspiration.
It's an exciting frontier for space applications
and to boldly go where no one has gone before with the Cloud.