The differences between Agile, DevOps, CI/CD and Pipelines

10-12 minute read

In today's complex technology landscape clients, customers, and business users expect a continual flow of updates to applications and functionality.

From an IT perspective, it is also expected that they are deployed seamlessly in near realtime.

Modern systems and applications have many moving pieces that comprise the solution. Speed of delivery and time to market is how IT is being judged by the business.

Understanding the differences between Agile, DevOps, and the CI/CD pipeline are critical aspects of how this can be accomplished.
Agile, DevOps, and CI/CD differ but they support each other and are closely related

Below, we give a high-level concept for each. For CD, it could possibly mean Continuous Delivery or Continuous Deployment, we'll cover both. We'll even include the Pipeline concept and how it traverses your CI/CD.

High-Level Concepts:

  • The Agile Method refers to a lightweight process framework and methodology based on incremental/iterative development(sprints) and mainly focuses on processes, change, and accelerating delivery
  • CI, or Continuous Integration, is the practice of developers merging their code changes to a central repository as often as possible or whenever needed. It drives build and test automation.
  • CD, or Continuous Delivery, is an extension of Continuous Integration. It's the practice of automating the entire release process to production at the push of a button or triggered event
  • CD, or Continuous Deployment, is a step beyond Continuous Delivery. Every change in the code is automatically deployed to production. No human intervention. Only a test failure can stop the new change from being deployed to production
  • A Pipeline is a deployable path on your CI/CD destined for production. This path is called a pipeline and typically includes an executable series of steps. Includes stages for Source, Build, Test, & Deploy.
  • The DevOps culture focuses on creating an agile relationship between development and IT Operations. The goal is to improve communication, collaboration, and responsiveness between the two

Making it even clearer:

Here are some quick points that help summarize Agile, DevOps, and CI/CD and their differences:
  • Agile focuses on processes emphasizing change while accelerating delivery
  • CI/CD focuses on the SDLC emphasizing tools that enable automation
  • DevOps focuses on culture emphasizing roles that improve responsiveness
  • Pipelines are executable paths on your CI/CD

Below, we dig deeper into what they are and how they are typically used by an organization.

Agile Methodology

refers to software development methodologies focused on the concept of iterative development. Requirements and solutions continually evolve through collaboration between cross-functional teams.

A key value is that Agile enables teams to deliver value faster, increased quality and predictability, and the ability to respond to change quickly.

Scrum, XP, and Kanban are three of the most widely used Agile methodologies.

Agile is a time-constrained, iterative approach to software development. Agile incrementally builds software from the start as opposed to delivering everything at once at the end of the project.

Agile breaks projects down into smaller pieces of functionality called user stories. The stories are prioritized. The Agile project then delivers them continuously in short two week cycles called iterations or sprints.



Agile focuses on the people that do the work and how they work together. Collaboration occurs between self-organizing cross-functional teams.

Where did Agile come from?

In 2001, 17 individuals wrote the agile manifesto. It sought an improved method for managing the progress of software projects.

The 12 principles of agile software development: Click to expand
  • Satisfying the customer with early and continuous delivery of valuable software is the highest priority 
  • Changing requirements are welcome, even late in the project 
  • Deliver software that works more frequently 
  • The business and developers must work closely together daily, throughout the project 
  • Projects should be built around motivated individuals, an environment of appreciation, trust, and empowerment 
  • Face-to-face conversations are the best way to convey information 
  • The primary measure of progress is working software 
  • An Agile process promotes sustainable development 
  • Agility is enhanced with continuous attention to technical excellence and good design 
  • Simplicity is essential 
  • The best designs, requirements, and architectures result from self-organized teams 
  • Teams should routinely reflect and adjust on how to become more effective


    • Continuous Integration

    I struggled with a definition and didn't want to rewrite or redefine the term Continuous Integration.  I ended up liking what I found below:

    Continuous Integration (CI) a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.
    - Thoughtworks.
    That being said, let's explore CI further.

    Continuous Integration is a development approach. It is not necessarily linked to a vendor or a specific tool. In reality, one could use PowerShell, bash scripts, and Perl to practice CI. Probably not practical but it could be done. CI is usually developed first as seen below in the image.



    In the past, integration has been a time consuming and costly activity.

    CI focuses on approaches and tools that drive build and test automation, continuously.
    Every code change committed to the repository is validated by creating a build and executing automated tests against the build.

    A Successful CI is when:

    • All features are directly merged into the repository/main branch (mainline) at least daily
    • All features are developed from the mainline
    • Developers do not work in isolation
    • The mainline is healthy after a new feature is integrated
    • Automatic testing occurs at the mainline and feature level

    As a result, build and integration efforts should drop considerably.





    Continuous Delivery

    I struggled with a definition and didn't want to rewrite or redefine the term Continuous Delivery. I ended up liking what I found below:

    Continuous Delivery (CD) is the natural extension of Continuous Integration: an approach in which teams ensure that every change to the system is releasable, and that we can release any version at the push of a button. Continuous Delivery aims to make releases boring, so we can deliver frequently and get fast feedback on what users care about.
    - Thoughtworks
    That being said, let's explore CD further.

    Continuous Delivery is an extension of Continuous Integration.  It's the practice of automating the entire software release cycle including build, configuration, packaging, release and orchestrating a production deployment in a software-defined manner.

    The goal is to automate the preparation and track a release all the way to production.

    The CD process typically includes at least one manual step of approval and deployment to production. Some complex CD systems with multiple dependencies may include additional manual or automatic steps.

    With CD in place, you can decide to release daily, weekly, or Monthly.  Whatever meets the business requirements. Production deployments can happen anytime by clicking a button.

    By eliminating nearly all manual tasks, developers and IT Operations become more productive.



    Continuous Delivery can be more challenging than the adoption of Continuous Integration. The full lifecycle has to be automated because every release candidate can potentially reach production.



    Continuous Deployment

    Continuous Deployment builds upon Continuous Delivery. The difference is that every change in the code is automatically deployed to production, no approval from a human required.

    Any release candidate that passes all testing and is found to be ready is immediately deployed to production. Only a failed test will stop a new change from being deployed to production.

    Each development approach builds on the previous ones.



    Automatic release and deployment to production shouldn't be taken lightly and I have only seen a few clients go to this level.  It is truly amazing when you reach this level of maturity though.

    Each foundation should be solid before adding Continuous Deployment.

    It's a bad idea to try and adopt Continuous Deployment without fully establishing Continuous Delivery first.  I have never seen it work out well.



    Making Continuous Integration, Continuous Delivery, and Continuous Deployment clearer:

    • Continuous Integration is a part of both Continuous Delivery and Continuous Deployment 
    • Continuous Deployment is similar to Continuous Delivery, except that releases happen automatically


    CI/CD Pipeline

    Once CI and CD in place, the deployable path is called a pipeline.

    A CI/CD Pipeline automates the steps in a software delivery process.  This includes initiating code builds, running automated tests, and deploying to staging or a production environment.

    Automated Pipelines eliminate manual errors, provide standardized development feedback loops and enable rapid releases.


    Here is a basic Pipeline example:

    • Pipelines are triggered when code is committed to a repository
    • Notification to a build system typically come next
    • The build system compiles the code and runs unit tests
    • Integration tests should be the next step for well-built pipelines
    • After testing, images can be created and pushed to a registry service, such as Docker Hub
    • Deployment typically follows


    Pipeline's often do additional things:

    • Performing security scans
    • Initiate quality checks on the code
    • Send notifications to Slack or MS Teams 
    • Approval steps within Change Control that stop a pipeline and ask for human approval before continuing operations
    • Use Continuous Deployment to automatically deploy to a test environment for UAT after packages are built


    CI/CD Pipeline stages:


    Source stage

    In most cases, execution of a pipeline is triggered by a source code repository. Code changes trigger the CI/CD tool, thus running the appropriate pipeline.

    Scheduled, user-initiated workflows and results of other pipelines are other common triggers


    Build stage

    Builds a runnable instance by combining the source code and its dependencies. Some need to be compiled, such as programs written in Java, C/C++, or Go. Languages like Ruby, Python, and JavaScript do not require this step.

    Cloud-native software is typically deployed with Docker, building the Docker containers.


    Test stage

    During this stage, we typically run automated tests to validate the quality of our code and the behavior of our program.

    The testing stage can last from seconds to hours. It depends on the size and complexity of the project.

    A common approach for larger projects is to run tests in multiple stages. They can start with smoke tests that do quick sanity checks or they can do end-to-end integration tests that test the entire system from the end user's perspective.  Running in parallel can reduce the time to test for large test suites.

    This stage quickly produces critical feedback to developers when failures occur.


    Deploy stages

    We're ready to deploy once the code has passed all defined tests. Typically there are multiple deployment environments, such as development, staging, and model.  These are used internally.  Finally, there is a production environment for actual use by end-users.



    DevOps

    Again, I struggled with a definition and didn't want to rewrite or redefine the term DevOps.  I ended up liking what I found below:

    DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology— especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.
    - Gartner

    That being said, let's explore DevOps further.

    DevOps isn’t a standard, process, or technology. Many sources refer to DevOps as a “culture” as seen above.  I personally agree.

    The DevOps concept is built on establishing a culture of collaboration between teams that historically functioned in relative "siloes".

    It is a set of practices that automates the processes between software development (Dev) and IT  Operations (Ops) teams.  The goal is that they can build, test, and release software faster and more reliably.



    Some DevOps models combine the two teams where developers and engineers work the entire application lifecycle.  This can include development, test, deployment, and operations.  This results in the development of skills not limited to a single function.

    QA and security teams may also become more tightly integrated with DevOps.

    The DevSecOps model is when security is the focus of everyone on a DevOps team.

    All DevOps teams and models use practices to automate processes that historically have been error-prone, slow, or manual.

    Benefits can include increased trust, faster releases, and quick resolution of critical issues.

    Increased agility allows organizations to compete more effectively in the market and better serve customers.

    DevOps teams typically use a tech stack and tooling which enable them to operate and evolve applications quickly and predictably.

    These tools allow engineers to accomplish tasks that typically would have required assistance from other teams, thus increasing the team's speed and efficiency.


    Key DevOps Practices

    Below are key practices that enable organizations to innovate faster through automating and streamlining the software development and infrastructure management processes.

    The majority of these practices can be achieved with the right tooling.

    • Continuous Integration
    • Continuous Delivery
    • Microservices
    • Infrastructure as Code
    • Monitoring and Logging
    • Communication and Collaboration

    full-width

    Tags

    Matthew Fiallos

    1 Comments

    Post a Comment

    Previous Post Next Post