The CI/CD process is now one of the essential paradigms in modern software development, oriented on delivery and quality. Continuous integration, continuous delivery, and continuous deployment benefit both software development and business. But what exactly stands behind the acronym CI/CD, and what is its added value?
CI/CD stands for Continuous Integration (CI) and Continuous Deployment or Continuous Delivery (CD). Although the basics of each are relatively similar, there are fundamental differences that define the key beneficiaries, and the results that can be achieved. The main goal is to automate the repetitive work engineers do and reduce the need to handle dull and detailed processes. From a business point of view, properly implemented CI/CD can help reduce costs and shorten time-to-market. As with all engineering, however, the devil is in the details.
What does Continuous Integration stand for?
Continuous Integration (CI) uses automation tools that enable development teams to build and test code after each merge, as seamlessly as possible.
Developers working in the CI paradigm merge their changes with the main code as often as possible. As you likely know, modern software is composed of various components built using multiple technologies including programming languages, platforms and clouds - to name only the obvious ones. This creates a complex environment to manage, in which developers have to operate, validate and integrate changes. A helpful solution here is a reliable framework that makes their job easier.
A key component of the CI/CD ecosystem is the test automation stage. When tests are implemented correctly in a test environment they ensure the most serious bugs are spotted at an early stage of software delivery.
Frequently merging small pieces of code is a good way to minimize the risk of future conflicts. All team members have access to the latest code base and can ensure the compatibility of commited code, on the go. Running regular integration testing is crucial to maintaining software consistency.
How can CI positively impact code quality?
A developer best practice says that if you want to keep your code in good condition and preserve high quality, you must commit it as often as possible. By doing so, it is definitely easier to detect bugs or other potential code vulnerabilities and fix them.
Teams who implement CI in their processes are testing code quality at the release phase and on the branch.
The team can use different strategies for new features deployment:
- version-control branching – a structured way of introducing and combining changes to the development, test, and production stages of work on the project,
- feature flag – an under-development functionality which might be unavailable to the end user, waiting for its moment to be finished, but is still part of the application’s code.
The choice of method depends on the team’s preferences and the project requirements and, to be honest, is a secondary matter. The result is the key, and it is the same with both solutions – new features and code are tested frequently, which translates into better quality.
What is Continuous Delivery?
Focused on keeping the code ready to deploy at any given time, Continuous Delivery is a natural follow-up to CI. Continuous Delivery is a stage between CI and Continuous Deployment. In this step, if the test automation stage is complete, and the current version of code works correctly, artifacts linked to the release are generated and stored for the purpose of further testing and/or deploying the system in the production environment. The release contains a full list of binaries and system documentation.
CD challenges the traditional means of releasing code. Instead of one big release, it delivers several smaller ones, possibly in a single day. Better still, the releases are available on demand. This ultimately makes the code more bug-resistant. With smaller batches released, it is easier to fix the bugs. The possibility of delivering an update that needs to be nuked due to major, unexpected bugs is significantly lower.
What is Continuous Deployment?
The second version of the acronym CD stands for Continuous Deployment. It is the constant and automated production deployment of every change made to the code. The changes should be automated, without human intervention.
-> Want to learn more about CI/CD? Check out our other articles:
- Sharing configuration between your CI, build and development environments
- How to build a test automation framework in the cloud
The differences between the two CDs
Both versions of CD focus on working in fast iterations and delivering artifacts based on binary codes. But as the following illustrates, there are also some differences.
Continuous Delivery focuses on releasing the code faster, but it does not define what will be released for production or when. Someone has to be responsible for accepting the new software releases and launching them to the public. But, as I mentioned before, this process could happen automatically. Continuous delivery is a safer approach to working on apps that need to be polished and will be released to a large group of users. A good example here is a B2C or B2B market app targeting small and medium businesses.
Continuous Deployment focuses on putting all the changes straight into production. In the ideal situation, the code is constantly evolving, automated tests keep the high quality, and the users get new features regularly.
What is the CI/CD process
Knowledge of how continuous integration and continuous delivery processes work during a project is important to understand why they are worth implementing.
The CI/CD process is continuous integration and CD (continuous delivery or continuous deployment) working together. The first component (CI) stands for building and testing code after every implemented change. Code changes from different contributors are automatically integrated into the project. With CI/CD these stages are run automatically and continuously - and that means up to months of time saved.
Open-ended automation and monitoring stages are active throughout the process - from integration and testing to the inclusive, delivery and deployment phases. A well-developed and properly used CI/CD process helps provide secure and high-quality software.
All that together creates the CI/CD pipeline, which we will be doubling back to a bit later.
How to (successfully) handle the CI/CD process
To ensure that the CI/CD process fulfils its purpose, consider the following:
- Put security in first place - consider isolating CI/CD by placing it in secure internal networks.
- Start with simpler automation at the beginning, and then develop your CI/CD pipeline by adding more steps.
- Focus on communication. Align with different development teams to determine common processes for production, testing and implementation. Make sure that everyone is on board regarding which elements will be automated, and which artifacts and reports will be needed. (Use tools such as Jira which can provide easier insight into the level of progress. A solution like Git could help you in tracking code changes.)
These are only a few, briefly described aspects. There are a lot more which are important in the efficient and successful management of the CI/CD pipeline.
Wondering how to handle the CI/CD process? Our previous article walks you through it.
CI/CD pipeline - definition
A CI/CD pipeline focuses on automated testing, including unit testing and integration tests. But these are only examples used in our simplified model - there are more possible choices. That means the code is checked regularly and at shorter intervals. If there is a bug in a particular CI/CD stage then there will be no trigger to start the next one. This allows engineers to get fast feedback about the code they have just delivered (a fail-fast approach).
Fig. 1 The scheme of a CI/CD pipeline
The diagram above shows a simplified CI/CD pipeline. A CI/CD pipeline works when the output of one stage triggers the next to start. So, once BUILD (stage 1) works properly, TEST (stage 2) is triggered. When all tests are passed, then the project is pushed to DEPLOYMENT (stage 3). But, as I mentioned earlier, our model is simplified. The CI/CD pipeline will not always be as linear and simple as this diagram. In some cases, a pipeline could fork and work in parallel, delivering various artifacts and reports as rapidly as possible.
How to set up a CI/CD pipeline
There is no such thing as a one-size-fits-all CI/CD pipeline. The tools must be adjusted to your project and team’s needs.
During this process, think about covering problems like:
- You need a control system - this advice may seem obvious, but it isn’t always followed. Remember, pushing code to the repository triggers the whole pipeline.Good practice is also to define the CI/CD pipeline as a code to get better tracking and keep the pipeline and system being developed in sync.
- Define staging environments based on key production environment elements. In most cases, we want to test software as close as it could be to the production. You can cut down a little on resources but remember to keep the same architecture.
- Consider the “fail-fast” approach, whereby you shorten the feedback loop by means of the quickest possible test. A fail-fast test could look like this: code quality → unit tests → build → staging deployment → e2e test. This is only an example. Depending on the project or the technology used, the “build” stage could come before “unit tests”, or the process could be extended by adding component testing and/or integration testing.
- Consider using container technology for testing. In this approach, tests are run in a clean environment, independent from other tests.
Remember, that your goal is to implement regular and fast deployments to avoid too many operational burdens at your project’s outset. Setting up a CI/CD pipeline is more complex than this quick introduction. The abovementioned aspects are only examples of those which should be taken into consideration - we can help you handle this process from the ground or to optimize the existing CI/CD pipeline - here you can find how we do it.
For a deeper dive, check out our article to learn how to conduct CI/CD optimization.
The CI/CD tools for your pipeline
The variety of continuous integration and continuous delivery engines can be overwhelming. To make picking one easier, here are some tools for your consideration.
- Jenkins X is an open source automation server. It automates tasks related to building, testing, and deploying software. Jenkins X has features like the automatic setup of a CI/CD pipeline, and the ability to create different virtual environments for development, staging, and production. The list of other benefits of using this tool is long. Among others, an easy and fast installation and faster recovery are Jenkins X’s significant strengths.
- TeamCity Cloud is a CI tool that helps to build and deploy different types of projects. One of the main benefits of using TeamCity is that it can be used in the SaaS model without incurring unnecessary upfront costs. What's also important, this tool runs in a Java environment and integrates with Visual Studio and IDEs.
- Microsoft Azure Pipelines is a cloud-based tool that can build and test any language, and deploy to any platform or any cloud. You can set as many stages as you need during the development and testing phases. That helps to better understand the root and find the bugs faster. That feature sets this tool apart from the others.
You can find everything about the tools which you will need to build a CI/CD pipeline here: Best CI/CD pipeline tools you should know.
Or maybe you want to support your DevOps team? Here, you can read more about DevOps CI/CD tools that can be helpful for your experts.
CI/CD pipeline in cloud environments
The development of the cloud brought huge changes and introduced a fresh way of hosting and delivering software. CI/CD pipelines especially benefit from elements of cloud technology like containers and infrastructure-as-code.
Many teams operate CI/CD pipelines with solutions like Docker containers and orchestration systems such as Kubernetes. They make it easier to scale up or reduce environments, run different applications on one server, and still keep them separate. Moreover, the build is a container image so that it can be deployed seamlessly to the chosen test environment before the release.
If you are interested in this topic, you can find more information about combining Kubernetes with CI/CD in our previous article.
Another option is using CI/CD with an infrastructure as code approach or serverless architecture to deploy and scale applications according to demand. The coded infrastructure can be put through a CI/CD pipeline, allowing implementation of more automation into the process – creating a new environment whenever it is needed and upgrading it according to the introduced changes.
This is easier to implement than it sounds. You can learn more about cost-optimization in cloud environments from our previous publications: Microsoft Azure cost management and AWS cost optimization.
Why is test automation worth your attention?
The answer to this question is simple. The CI/CD approach is more iterative in comparison to the linear waterfall approach. Processes are intertwined, and feedback loops are constant, so bugs are detected faster.
In the waterfall approach, the final product is passed to the tester for checking. If a bug is found, it can be extremely hard to fix after whole months, or even years have been sacrificed writing the application.
In this case study, we shown our client benefits from implementing CI/CD - Building CI/CD and testing for a full-stack monitoring and alerting service.
A test automation framework presents numerous important questions and daunting challenges. Before setting one up, pin down the most crucial requirements your testing framework will need to meet. When all the requirements are clear, decide if a preconfigured testing environment or a dynamically created environment will better meet your needs.
While CI/CD is a complex topic, we hope this post has helped you understand how to build a test automation framework in the cloud
For now, let’s focus on the process itself: how does testing work in the CI/CD pipeline?
Developers working simultaneously have to be sure that their contributions don’t create any bugs in the application. This is where Continuous Integration comes in: developers push written code into the code repository, then a build system tool (e.g. Jenkins) takes the code and builds it. The next step is an automation test. When that is passed, the Continuous Integration phase is complete. Then, there are two possible options. With a Continuous Delivery approach, in-house users test the application, and when the project meets our expectations, we can deploy it. Or we could choose Continuous Deployment, making the application available to the end-users in the target environment. But this approach demands a certainty that tests cover the application code properly, and also the fast fixing of potential bugs.
Monitoring and observability of the CI/CD pipeline
Setting up and deploying a continuous integration and continuous delivery pipeline is not enough. To reap real benefits, implement continuous monitoring and observability to collect metrics and important insights.
Despite their apparent similarities, monitoring and observability complement each other rather than merely duplicating results. Monitoring entails collecting, processing, aggregating, and displaying real-time quantitative data about the system to keep track of its condition and alert users about eventual problems. Observability, in a nutshell, provides a glimpse into the system’s internal workings by knowing its external outputs.
Everything you need to know about CI/CD monitoring and observability is in our previous article.
The benefits of CI/CD implementation for your company
I hope this article has convinced you of the hidden potential of the CI/CD approach.
CI/CD could also have a positive impact on your business. Here are a few examples of how:
- Code quality is higher - automated tests allow you to fix bugs immediately.The continuity and repeatability of the process make it faster.
- Costs and labor both fall - automation reduces both costs and the element of human error. In a perfect world, there would be no human intervention in deployment to the production environment.
- Today’s customers often expect quick feedback, bugs to be fixed immediately, and features to be regularly released and refreshed. They reward the companies that do these things with their trust and loyalty. CI/CD will help you meet these challenging demands and answer rapidly-changing customer needs.
But this is only a sneak peek of the real benefits for your business. You’ll find a far more exhaustive list here: The business benefits of CI/CD
You can find more information about CI/CD in our latest white paper „How to implement CI/CD in just six weeks”. If you still have doubts or questions, please get in touch!
Original post date 04/17/2019, update date 08/01/2022.
