In this video, we'll find out the problems with the typical software
development and deployment process.
What is continuous integration and how continuous integration
can help resolve those problems?
Now, if you've watched my last video in full, where I showed a typical software
development and deployment process.
You might remember that near the end of the video, I mentioned that there
are many problems with the way EZSOFT develops and deploys its software.
Let's quickly recap, EZSoft's development and deployment process to
understand the problems a little better.
EZSoft,
as we discussed in the last video has three software development teams.
Each working on writing different code as a specific software module.
These modules, which are nothing but pieces of independent software written
in the human readable language, are then checked into separate folders or
repositories on the source control system.
This checking in the code to the source control system is also
called the committing the code.
The three teams may check in or commit the code to their folders
in the source control system, multiple times during the sprint.
Then at the end of the sprint, the integration team collects the code
from these different repositories, merges the code, and then stores the
merged code in a separate folder, also called the main line repository
on the same source control system.
The integration team then builds or compile the code on an available
computer that the integration team owns and then sends the compiled code along
with the deployment or installation instructions to the operations team.
The operations team takes this combined goal and follows the included instructions
to install it to various environments like test environment, staging environment,
and finally to the production environment that is accessible to the end user.
Let's zoom into the integration piece of the entire software development
and deployment process, which is this.
Here are a few things that stand out immediately.
Integrating the code from different teams with developers who might
have different coding styles can be a daunting task, especially when
doing it all at once in huge chunks,
at the end of the sprint.
This may lead to merge conflicts and failed builds requiring the
developers from different teams to rework even when they have already
moved on to working on the next piece of software in the following sprint.
I'm assuming that there are no gaps between the Sprints.
There shouldn't be any gaps.
If you look carefully, this process will not only cause a lot of confusion
among developers who will be required to go back to those pieces of the code
that they wrote in last sprint, but will also take multiple days for the
merging to complete involving the effort from the developers of the three teams
and members of the integration team,
all trying to figure out how to fit different pieces of the code together.
Let's imagine that all the merge conflicts are resolved and the
development team fixed all the bugs.
It's now time for the integration team to rebuild or recompile the code.
Since this recompilation happens on an available personal
computer owned by the integration team members, the final compiled
code often gets contaminated by the automatic operating system
updates and personal preferences.
This often results in releases that are unstable and error prone, when
transfered to the production environment.
For a release to be stable,
the integration environment must be identical to the
final production environment.
On top of this, the integration team can only deliver the final
compiled code to the operations team only once every sprint.
This, as you might imagine, causes lots of delays and often results in the code
not being deployed to production until it is fixed in a subsequent sprint.
Let's summarize.
With the current integration process, we have, number one, a
high chance of rework and bugs.
Number two, by the time bugs are fixed, it's often too late.
Number three, fixing the bugs and resolving the merge conflicts
requires a lot of effort.
Number four, it is not easy to reproduce the compiled code
with the same configuration as the production environment.
And number five, since release happens only once per sprint, it causes delays.
All right, now that we have a clear understanding of what's wrong with
the current integration process, let's see how the concept of
continuous integration can help.
But first, what is Continuous Integration?
Continuous integration is a development practice that requires developers, not a
separate team that does the integration.
It requires the developers to integrate units of code into a single
shared repository on the source control system multiple times a day.
The absence of a separate dedicated integration team requires the developers
to integrate the code themselves, meaning that each team and each developer has
to do a bit extra work to make sure the code works with the rest of the code.
Which makes sense since the developers are the ones who created the code,
they know the code better than the members of the integration team.
And hence must know how it should fit with the rest of the code, and what exactly
could go wrong reducing the time to fix the code in case something does go wrong.
As soon as the code is successfully merged the continuous integration
server automatically triggers the build on a dedicated build server.
Now, since this is a dedicated build server and not a personal computer of
a member of the integration team, it's easy to maintain its configuration in
a way that resembles the configuration of the production environment.
All developers in all the three teams now have access to the results of the build.
This makes it easy for the developers to see if their code is compiled, or if there
is a problem that they need to resolve.
A successful build, then triggers the automated tests on a different
automated test environment to ensure that we get the best release possible.
The whole build process, compilation and automated tests should be quick
so that the results of before check-in can be communicated to the developers
as soon as possible, who can then fix whatever needs fixing on a timely basis,
ensuring that the code base stays healthy and everybody can continue working.
This is what a typical continuous integration process looks like.
Pretty straightforward.
Although the continuous integration process may look different for
different teams, all teams must adhere to a few principals to implement
continuous integration successfully in their overall development process.
Principle, number one, maintaining one shared central code repository.
Continuous integration requires all code from different teams
to work together correctly.
For this reason, branching also call the temporarly storing code from
different developers in different teams into separate repositories
on the source central system, should be kept to a minimum.
And all the code, should be merged into the mainline
repository as soon as possible.
This leads us to principle number two.
Developers should commit their code to the mainline repository frequently.
The longer a developer holds onto the code without building and testing it
the more likely it is to be inconsistent with other changes that have been made
in the central mainline repository.
Principle number three.
Maintaining separate build and test servers.
Fast builds catch errors before it's too late.
To speed up the builds a different machine should be designated for build
purposes so that it does not impact the development pace of other developers.
Developers must also be notified as soon as possible if there is an issue,
with their specific integration.
Principle, number four.
Builds and tests must be automated.
Every commit of the code or in simple words, every check-in of the code
into the source control system should be built and tested automatically.
Continuous integration tools like Jenkins, Bamboo, GitLab, et cetera,
can help in constantly checking the main non-depository for changes.
When a change is detected, these tools perform a build
and later test the build code.
Principle, number five.
Use production like testing environments.
The testing environment should be as similar as possible to the eventual
production environment with configuration, as similar as will be encountered
in the production environment.
This ensures the usefulness of the test and keeps the expectations
consistent throughout the deployment.
Principle number six.
QA should have access to the builds.
Along with the developers, QA must have constant access to the builds so that any
failure to meet production requirements can be found as soon as possible.
Early detection will lessen the danger of having to rework everything later.
Okay.
Now that we know what should be done to get the continuous integration,
let's look at the benefits of having a continuous integration setup.
Here's how the EZSoft's continuous integration setup looks like.
The three EZSoft teams, check-in or commit their code multiple times every
day to a central source code repository also called the mailline repository.
This code then gets compiled and passed through specific tests.
Tests like, unit tests, automated UI tests, security tests, and
anything else that can be automated.
This simple process results in numerous benefits.
For example, the integration process takes less time and effort.
Every developer now integrates their own code into a central main line
repository instead of a dedicated integration team that merges the
code from different repositories.
Number two, automating builds as soon as new code is checked in results in a
code that is always compiled and tested.
Before continuous integration, this happened only once every Sprint.
Now it can happen many times a day.
This surfaces issues early on making them easier to be fixed.
Also since the build and test processes are automated, there are fewer chances
for humans to make mistakes resulting in a process that is less prone to errors.
Next.
Increased visibility allows everyone to see what has been checked in and what
are the results of the latest builds, making it possible for the developers
to get timely feedback confirming if everything fits together or not.
Next continuous integration also results in better overall team communication.
By having improved communication, the chances that everything fits
together well becomes higher resulting in better software.
Finally, because of the short feedback loops, the teams can make
changes more often allowing them to react to customer's needs better.
Also more importantly, the code can now be delivered to operations more often,
resulting in more releases.
This means a shorter time to market and a higher ability
to react to customer's needs.
So there you have it.
Now that we have a good understanding of what continuous integration is
in the following video, we will take a look at the second piece of the
CICD puzzle, continuous delivery.
If you've got any value out of this video, then do not forget to
smash that like button, subscribe and leave your valuable comments.
I'll see you in the next video.