Let’s take a stab at guessing how your CI/CD pipeline works today. Code commit triggers CI (build continuous integration), and CD (deployment of application) happens if the CI was able to create new application bits. Much of this is automated and depends on an environment consisting of the infrastructure, services, and configuration your application needs to be deployed to (or installed on).
DevOps teams will use a scatter of environment assets to orchestrate the “environment deployment” stage using popular tools: Terraform, Helm charts, configuration scripts, etc.
Now, let’s consider the realities of this process for DevOps teams:
- DevOps teams are typically outnumbered by developer teams by 1-10 or even 1-20. This naturally leads to wait times for environments, and important things like innovation and team culture suffer as a result.
- IaC scripts proliferate and lack traditional governance (e.g. developer changes compute size in Terraform script) or cost visibility (e.g. what is the infra cost for application dev?).
- The deployment sequence and destroy events are characterized as declarative code in some build servers (Jenkins Groovy), but not all (Team City). This indicates a lack of versioning and an inability to re-create environments reliably to address defects.
- Waiting for access to environments often takes up valuable cycles for app dev teams under pressure to meet deadlines. Permissions and keys are handled outside the IaC—typically by a DevOps team—controlling the developers’ access parameters and permissions.
These gaps can lead to “lower environments” (feature branch, CI, staging) becoming static environments (not ephemeral), which prevents them from being refreshed during every build/test cycle. As a result, once there is a build, the application is deployed into a stale environment with associated costs. This creates several challenges:
- It is uncommon to duplicate the build process and static environments to serve multiple, concurrent developer commits. As such, developers find themselves blocked from making frequent commits, delaying their work.
- Due to cost/need balancing, these environments rarely mimic production-like environments, making it harder for devs and testers to spot defects before the app is live in production.
- Different types of integration tests running sequentially on the single environment require additional time.
Your application must be built using CI/CD best practices and with the infrastructure, data, and configuration that make up your application’s underlying environment.
And applications and environments both have lifecycles. Therefore, efficient application development depends on the ability to update environments.
One way to accomplish this, while also freeing up DevOps resources for critical tasks and reducing delivery times, is to allow your teams to create environments in a self-service manner.
While IaC describes infrastructure, application, and configuration as part of the environment, it lacks the ability to control who executes the script and track application development costs.
You want your app environment created immediately after building your bits within the CI, and just before deploying your application to production (CD).
Historically, environment creation/maintenance/operation/monitoring/destroying was handled by IT, then DevOps, and now it is coming into its own lifecycle. Neglecting the environment has led to development delays, bugs escaping to production, and slower deployments of new app versions. Preventing this requires acknowledging that environments have their own lifecycle and integrating them (infrastructure + configuration) into the CI/CD pipeline.
This is where Quali’s Torque platform comes into the equation, acting as a control plane for infrastructure with an eye towards what the application needs at that specific stage of its lifecycle. With Torque, users can deploy new environments on demand, a cost-efficient approach that mimics production-like environments.
Torque uses automation to deploy environments reliably and efficiently, offering the DevOps team centralized and efficient control over the environments through blueprints (govern, audit, and change), with self-serve functionality that allows devs and testers to operate at their own cadence.
Torque blueprints are application-centric descriptions of application environments. You can view your blueprint in Torque by application:
Or you can see the blueprint in the native YAML:
Torque’s growing plugin environment includes (CI) build system integrations with Azure DevOps, GitHub, CircleCI, Jenkins, and TeamCity, with more on the way. These plugins are CI system-tailored to make the integration seamless and preferable to using Torque via command-line or by REST APIs, which are always there for any custom requirement you might need in your CI orchestration.
These integrations make it simple to create the environment after successful CI (build) of application. After the testing phase of your pipeline is complete, Torque will shut down the sandbox environment to save on costs automagically.
To see it in action, check out this brief video where I use Torque to launch and tear down a Minecraft server in just a few clicks.
You can try launching a sample sandbox environment on your own, with a trial account of Torque. Our samples don’t require connecting your cloud infrastructure to see how you can automate access to sandbox environments with pre-determined runtimes, auto-applied tags, and other customizations—all integrated with your CI/CD pipeline.
I get that thinking differently about your toolset and the way you operate can seem overwhelming. But building Torque into your pipeline will yield significant dividends—like bringing flowers to a spouse or partner, or going to therapy (with a spouse or partner!).
Hit us up on Twitter or LinkedIn and let’s discuss infrastructure automation, environments, and implementation within CI/CD pipelines. I’ll rope in the Product team, and we can both make them uncomfortable (especially Amir).