Continuous integration makes sure that the software builds and
runs successfully and then you do the testing.
There are still steps between this point and the software release.
Automating the release pipeline down to the software release is
called continuous delivery.
But since some of QA is a manual process,
you cannot run the next step completely automatically.
You need a manual trigger in the pipeline to deploy to the next stage,
which is usually a staging server.
For example, testing against a real database.
You can run more automated and manual tests on staging.
Once successful you can manually trigger the deployment.
It's important to note that a manual trigger does not equal a manual build
or deployment.
Build and deployment is automatic,
it's just the trigger that starts the process is manual.
There are integrations with communication software that allows these
manual triggers to be launched in a more publicly visible and
convenient way than logging into a particular software program.
For example, there are chatbots for
popular team communication platforms like Slack and HipChat
that can be integrated with Jenkins, CircleCI, and other CI solutions.
These chat-bots can send build test information to a public chat room.
And react to commands written into the chat.
That makes it easy and fast to coordinate within the team.
If the CI build and test passes, using chat, you can poll all involved
developers with something like, my change is good to go.
And then issue that command into the chat-bot.
This makes for a fast and transparent release process.
We will not be setting up these chat services, but see instructor notes for
more information.
>> If you have a service that handles thousands of user transactions
every second, over dozens or hundreds of servers, you're not going to sit
there and watch the log files to see if it's working right.
You, as a human being, literally cannot read thousands of messages per second
in hundreds of log files.
You can't react the difference between an acceptable and
an unacceptable level of errors.
Without monitoring systems to be your eyes and ears you literally do not have
the ability to notice or care about something happening that fast.
There are a few different ways that we can gather data about our services.
Not all of them apply to all services, though.
One is external probing, or sending test queries.
Some outside system will send queries in to our system and
see if it performs as expected and record the results.
Another is application-level metrics that are exported by monitoring
interfaces of our serving processes.
This might include things like the number of queries they're handling per
second or what the latency is.
Those can be pushed into a monitoring system or
pulled into it from the server.
Another thing to monitor are stats that are exported by our runtime environment,
for instance, the memory profile in a Java JVM.
And then there's also information about the computers that we're actually
running on such as the load average or the number of disc errors.
You can actually very typically discover that a disc is about to fail
by monitoring the number of errors it's having.
Other sources of monitoring data can include logs analysis, query profiling
such as Google App engines app stats, and external analytic services.
There are also a number of data products that a monitoring system can provide.
The one everyone knows about is alerting.
Getting a page saying the system is too slow.
But your monitoring system can also provide charts and graphs for
performance analysis.
Say things like this release is 10 percent faster than the last one.
And also for capacity prediction.
For instance,
if it takes ten instances to support 1,000 transactions per second.
Then if we scale to 2,000 transactions per second,
we should expect to need ten more instances.
Also for growth measurement.
You can plot a graph out of your monitoring system and
see that your service is doubling traffic every three months.
You can combine that with a capacity prediction to get capacity needs
forecasting.
Tell you how many computers you're going to need next quarter.
And last but not least,
your monitoring system can also provide great metrics for debugging problems.
For instance, suppose you're trying to diagnose why your servers are crashing.
If you notice from your monitoring that each instance is using more and
more memory proportional to the number of user queries,
that might indicate to look for a memory leak.