Hello, and welcome to this presentation, on Active-Passive Disaster Recovery (or DR for
short), using Oracle Linux Virtualization Manager.
In this video, we'll use the Oracle Solution Center's workshop environment to demonstrate
the active-passive DR process. A description of who they are, and the workshops they offer,
can be found at the end of the video.
So, let's talk about what we will cover in the video. First, we'll define the active-passive
disaster recovery.
Next, we'll cover the site requirements for the active primary site, and the passive secondary
site.
Then, we'll follow the process of initiating a failover to the secondary site, followed
by a failback to the primary site.
Oracle Linux Virtualization Manager offers two DR solutions: Active-Active, and Active-Passive.
A discussion of the active-active process is beyond the scope of the video.
You, can find more information, on the active-active configuration, in the Oracle Linux Virtualization
Manager Administration Guide.
The Active-passive solution uses two separate Oracle Linux Virtualization Manager environments
configured with an active primary site, and a passive secondary (or backup) site.
When the environment experiences an outage, you manually trigger failover using Ansible
playbooks that you create.
And there are three required ones and one that is optional:
• A playbook to create the file that maps entities on the primary and secondary sites.
• A failover playbook, • And a failback playbook.
You can also create an optional playbook to clean the primary site before failing back.
We will use these in our demonstration.
Before we begin our demos, let's review site requirements.
We'll start with the primary site:
The primary site should be a fully functioning Oracle Linux Virtualization Manager environment.
Meaning, the site:
• Has an Engine host managing Data Centers and Clusters;
• Installed KVM hosts. inside the clusters, with attached Logical Networks, VNICs, and
virtual machines. • Storage domains that replicate its contents.
We'll explore more on this soon.
For this type of disaster recovery to work, we need a preconfigured secondary environment.
This environment needs:
• An active Engine host running the identical versions of the Engine and PostgreSQL software
as the primary,
• With data centers and clusters at the same compatibility levels as the primary.
• Of course, it needs an active KVM host for running virtual machines after the failover,
• With enough resources to run the VMs.
• And, networks with the same general connectivity as the primary site.
• So, about the storage we need for the VMs?
The virtual disks and templates stored on the primary's storage domain is replicated
to a location that is not attached to the secondary site.
During a failover, this replicated data is then imported to a data center on the secondary.
We'll see this transition in the upcoming demonstration.
It is during the demonstration that we'll also discuss the different playbooks.
Now, let's demonstrate active-passive failover and failback.
We begin by showing that our Very Critical Production System is up and running. This
is evident from the live system clock on the production VM's web page.
Next, we'll simulate a disaster by switching to the Virtual Machines page of the Administration
Portal and selecting our Production VM from the list, and then clicking Shutdown > Power
Off to bring the system to a sudden and complete halt.
We'll click OK to confirm our choice. The triangle icon to the left of the VM transitioned
from green to red.
Before we run the failover playbook, we need to reverse the replicated storage from the
primary site to the secondary site.
Though, not part of the failover playbook, it needs to be done because of the way our
storage appliance manages replication.
The Ansible server is where we'll run scripts for storage tasks and playbooks.
So, we'll flip over to our Ansible server and run the switch-s2-s1 script to perform
this reversal.
After the reversal completes, we can deploy the failover playbook.
A quick word about playbooks.
Creating playbooks is beyond the scope of this video.
Please refer to the Oracle Linux Virtualizaton Manager Administration Guide for examples
and detailed guidance.
What is missing in the video capture, is running the playbook that created a map file containing
primary's site entities.
A mapping file is crucial when restoring continuity on the secondary site, and returning it back.
The mapping playbook populates the primary's site entities into a yaml file,
Then you manually add to this yaml file, the site-specific entities for the secondary.
These are entities such as its IP addresses, cluster, affinity groups, and vNIC profiles.
You add the mapping file to the failover and failback playbooks.
To recap and continue the scenario.
Our production VM is down.
We have reversed replication, so now we can initiate the failover.
So, at the prompt on the Ansible server, we will paste
the command:
ansible-playbook failover-s2-s1.yml --tags "fail_over" --ask-vault-pass
Shortly, we'll see some iSCSI errors in red.
These are expected, as the playbook is attempting to log in to the iSCSI targets.
So, let's go over to the secondary site and observe what is happening on the Storage Domains.
Form the virtual machines page, we click on Storage and then Domains to view its page.
Notice the lock icon next to PROD-S1. The playbook is in the process of importing the
replicated storage.
Shortly a message appears telling us the that the storage domain has finished attaching
it to the data center, and the status icon turns green.
Let's go back to the virtual machines page and check the status of our production VM.
A quick refresh shows a gray status icon next to the VM.
Not too long after, we get a message letting us know that the import has completed and
the icon changes to indicate it is booting. Soon after that, the VM is up and running.
Let's assume repairs to our production site are complete, and we can restore operations
back to our primary site.
So, we begin preparing our environments for failback.
First thing we need to do is clean the primary site. This involves running the clean_primary
playbook.
We'll go back to the ansible server and from there, execute the clean_primary playbook
This will prepare the primary for our failback.
So, at the command line we enter:
ansible-playbook clean_primary.yml --tags "clean_engine" --ask-vault-pass
In our scenario, the playbook does not take too long to complete.
After it does, we take a quick look on the secondary.
The status of the of our production VM shows that it is still up and running.
Back on the primary site, we first see that our VM is in the same state it was in before
the failover.
A quick refresh of the browser now shows that the VM was removed during the cleanup.
Let's look on the storage domains page.
And, no remaining artifacts of our domain are shown.
We're ready to move on to the next step.
In this step, we restore the replication back to the production site.
So, at the command line on the Ansible server, we enter the replicate-s2-s1.sh command to
execute the restoration.
In short order, we're ready to move on to the failback step.
Now it's time to run the failback playbook and start restoring operations back to our
production site.
The command we'll run is, ansible-playbook failback-s1-s2.yml --tags "fail_back" --ask-vault-pass.
There's a pause coded in this playbook where we need to run the switch-s1-s2.sh to reverse
replication and restore access back to production.
We have reached to point where we need to run the reverse replication script.
So, we'll go to another window logged into the Ansible server and execute the script.
After that script completes, our storage domain is ready for use on the production system,
so we go back to the session running the playbook and hit Enter to continue.
In our demo environment, this takes aaout 10 seconds.
When we get back to the prompt, we'll switch to the playbook window and continue.
While the playbook runs, we'll monitor progress from the primary site.
We begin on the Storage Domains page.
Our PROD-S2 domain does not appear in the list yet, so I'll click the bell icon and
monitor system events.
Our storage domain appears showing the locked status.
Viewing the events is a helpful way of monitoring progress and troubleshooting events.
To save time, I shorten the elapsed time by about 30 seconds.
Eventually, the green notification window appears with a note saying the storage domain
is now attached to the data center, and the status icon turns green.
Next, we'll check the status of our production VM on the virtual machines page.
Our VM appears in the list with a status of down. It transitions from down to booting,
to eventually up and running.
As we can see when I switch to a web page monitoring the status of this VM.
Now, I would like to acknowledge the Oracle Support for their contributions to this video.
Here is more information about them. Please contact them with your needs.
This concludes the presentation.
Thanks for watching.