Electric cars may be stealing the limelight these days, but in this blog, we'll discuss a different kind of newsworthy plugin: Quali just released the TeamCity plugin, to help DevOps teams integrate CloudShell automation platform and JetBrains TeamCity pipeline tool.
This integration package is available for download on the Quali community. It adds to a comprehensive collection of ARA pipeline tool integrations that reflects the value of CloudShell in the DevOps toolchain - To name a few: Jenkins Pipeline, XebiaLabs XL Release, CA Automic, AWS Code Pipeline, Microsoft TFS/VSTS.
JetBrains is well known for its large selection of powerful IDEs. Comes to mind their popular PyCharm for Python developers. They've also built a solid DevOps offering over the years, including TeamCity, a popular CI/CD tool to automate the release of applications.
So what does this new integration bring to the TeamCity user? Let's step back and consider the challenges most software organizations are trying to solve with application release.
Application developers and testers have a mandate to release as fast and as possible. However, they are struggling to get in a timely manner, an environment that represents accurately the desired state of the application once deployed in production. On the other hand, IT departments do have budget constraints on any resource deployed during or before production, so the onus is on the DevOps team to meet these business needs.
The CloudShell solution provides environments modeling that can closely match the end state of production using standard blueprints. Each blueprint can be deployed with standard out of the box orchestration that can provision complex application infrastructure in a multi-cloud environment. As illustrated in the diagram above, the ARA tool (TeamCity) triggers the deployment of a Cloud Sandbox at each stage of the pipeline.
The built-in orchestration also takes care of the termination of the virtual infrastructure once the test is complete. The governance and control CloudShell provides around these Sandboxes guarantee the IT department will not have to worry about budget overruns.
As we've discussed earlier, when it comes to selecting a DevOps tool for Application Release Automation, there is no lack of options. The market is still quite fragmented and we've observed from the results of our DevOps/Cloud survey as well as our own customer base, that there is no clear winner at this time.
No matter what choice our customers and prospects make, we make sure integrating with Quali's CloudShell Sandbox solution is simple: a few configuration steps that should be completed in a few minutes.
Since we have developed a large number of similar plugins over the last 2 years, there are a few principles we learned along the way and strive to follow:
The Financial Services Industry (FSI) is in the midst of an application transformation cycle. This transformation involves modernizing FSI applications into fully digital cloud services to provide bank customers a significantly better user experience. In turn, an improved customer experience opens the door for the FSI to offer tailored products and services. To enable this application modernization strategy, Financial Institutions are adopting a range of new technologies hosted on Cloud infrastructure.
The technologies that are introduced during a Financial Service application modernization effort may include:
Together, these technology components provide the capability for FSI’s to meet market demands by offering mobile-friendly, scalable applications to meet the demand and requirements within a specific geographic region. Each region may have stringent compliance laws which protect the customer privacy and transactional data. The challenge is to figure out how to release these next-generation FSI applications while ensuring that validation activities have been performed to meet regulatory requirements. The net result is that any certification process for a financial application and the associated modernization effort can take weeks, if not months.
The approach to overcoming the challenges mentioned in the previous section is to streamline the application validation and certification process. Quali Cloudshell solution is a self-service orchestration platform that enables FSI’s to design, test and deploy modernized application architectures. It provides the capabilities to manage application complexity with standard self-service blueprints and validate compliance with dynamic environments and automation.
This results in the following business benefits:
Using the CloudShell platform, the FSI application release manager can now quickly automate and streamline these workflows in order to achieve their required application updates.
Ready to "Dive into DevOps"? Quali will be in San Francisco next week November 13-15 at the DevOps Enterprise Summit . We will showcase our latest DevOps integrations with Atlassian's Jira, Jenkins, CA Blazemeter, Microsoft VSTS, AWS codepipeline and many others.
Since I've covered details on our Jira, Jenkins and Blazemeter integrations in previous blogs, I wanted to introduce the two new kids on the block:
If you're going to be around, make sure you visit us at booth #15, to learn how you can accelerate your application release and scale your DevOps automation with CloudShell Dynamic Environments. You'll also have a chance to win one of these handy $100 Amazon gift cards (right before Christmas as it turns out) and bring home tons of colorful giveaways.
I am also looking forward to meet many of our technology partners who will also be attending the event such as JFrog, CA, and Atlassian.
As new networking concepts such as SD-WAN and NFV are picking up steam, there is a growing awareness among Service Providers that building a standard approach to the consumption of development and test environment is key to sustainable certification and innovation. That means not only getting fully on board with Automation and DevOps from a process standpoint, but adopting Orchestration solutions that meet the very specific needs of the industry vertical.
Applying a DevOps approach in the Telco industry is relatively new. Up until recently the development cycle had traditionally followed a multi-month (or year) release pattern quite remote from webscale companies leading the way such as Netflix and Airbnb. Certainly a service provider has to deal with regulatory constraints in terms of uptime and the delivery of services. However the dynamics are changing. Several driving factors are at play:
Making DevOps a reality for the network engineer and tester responsible for validation of new SD-WAN and NFV technologies is easier said than done.
To begin with, there is a mismatch expectation on skills: network engineers are not programmers (for the most part) so they need to be exposed to the proper level of resource abstraction for an effective implementation.
In addition, quite often, efforts to apply general compute based automation approach to industry specific challenges, such as NFV on boarding, have been stopped on their tracks. The complexity of software based network environments makes the translation non trivial. Instead of reaping the benefits and agility of software based approach, it merely compounds problems seen with physical based environments with additional challenges tied to virtual workloads . In the best cases, this leads to a successful (and well publicized) small scale pilot on a greenfield project, that never reaches full production at scale and expected ROI.
Taking a practical approach to network orchestration, Quali's CloudShell combines the power DevOps automation with the flexibility to provide on demand, dynamic environments to meet the inherent needs of service providers.
Now you might be intrigued and want to learn more about our solution. There are several ways to accomplish that: scheduling a short demo, watching a recent webinar we hosted on this topic, or read about the NFV/SDN solution space on our web site.
Who said trade shows have to be boring? That was certainly not the case at the 2017 Jenkins World conference held in San Francisco last week and organized by CloudBees. Quali's booth was in a groovy mood and so was the crowd around us (not mentioning the excellent wine served for happy hours and the 70's band playing on the stage right next to us).
The colorful layout of the booth certainly didn't deter from very interesting conversations with show attendees around how to make DevOps real and solving real business challenges their companies are facing.
This was the third Jenkins conference we were sponsoring this summer (after Paris and Tel Aviv) and we could see many familiar faces from other DevOps leaders such as Atlassian, Jfrog and CA Blazemeter that have partnered with us to build end to end integrations to provide comprehensive CI/CD solutions for application release automation.
This really felt like a true community that collaborate together effectively to benefit a wide range of software developers, release manager and devOps engineers and empower them with choices to meet their business needs.
To illustrate these integrations, we showed a number of short demos around some the main use cases that we support (Feel free to browse these videos at your own pace):
As you would expect at a tech conference, there was the typical schwag, such as our popular TShirts (although we can't just claim the fame of the legendary Splunk outfit). In case you did not get your preferred size at the show, we apologize for that and invite you to sign up for a 14 day free trial of newly released CloudShell VE.
Wrapping up the "Summer of Love" 50th anniversary, Quali will be at the 3 day JenkinsWorld conference this week from 8/29-8/31 in San Francisco. If you are planning to be in the area, make sure to stop by and visit us at booth #606!
We will also have lots of giveaways (cool Tshirts and schwag) as well as a drawing for a chance to win a $100 Amazon Gift Card!
Can't make it but still want to give it a try? Sign up for CloudShell VE Trial.
The process of automating application and IT infrastructure deployment, also known as "Orchestration", has sometimes been compared to old fashioned manual stitching. In other words, as far as I am concerned, a tedious survival skill best left for the day you get stranded on a deserted island.
In this context, the term "Orchestration" really describes the process of gluing disparate pieces that never seem to fit quite perfectly together. Most often it ends up as a one-off task best left to the expert, system integrator and other professional service organization, who will eventually make it come together after throwing enough time, $$ and resources at the problem. Then the next "must have" cool technology comes around and you have to repeat the process all over again.
But it shouldn't have to be that way. What does it take for Orchestration to be sustainable and stand the test of time?
IT automation over the years has taken various names and acronyms. Back in the days (early 2000s - seems like pre-history) when I got first involved in this domain, it was referred to as Run Book Automation (RBA). RBA was mostly focused around troubleshooting automatically failure conditions and possibly take corrective action.
Cloud Orchestration became a hot topic when virtualization came of age with private and public cloud offerings pioneered by VMWare and Amazon. Its main goal was primarily to offer infrastructure as a service (IaaS) on top of the existing hypervisor technology (or public cloud) and provide VM deployment in a technology/cloud agnostic fashion. The primary intent of these platforms (such as CloudBolt) was initially to supply a set of ready to use catalog of predefined OS and Application images to organizations adopting virtualization technologies, and by extension create a "Platform as a Service" offering (PaaS).
Then in the early 2010s, came DevOps, popularized by Gene Kim's Phoenix Project. For Orchestration platforms, it meant putting application release front and center, and bridging developer automation and IT operation automation under a common continuous process. The wide spread adoption by developers of several open source automation frameworks, such as Puppet, Chef and Ansible, provided a source of community driven content that could finally be leveraged by others.
Integrating and creating an ecosystem of external components has long been one of the main value add of orchestration platforms. Once all the building blocks are available it is both easier and faster to develop even complex automation workflows. Front and center to these integrations has been the adoption of RESTful APIs as the de facto standard. For the most part, exposing these hooks has made the task of interacting with each component quite a bit faster. Important caveats: not all APIs are created equal, and there is a wide range of maturity level across platforms.
With the coming of age and adoption of container technologies, which provide a fast way to distribute and scale lightweight application processes, a new set of automation challenges naturally occurs: connecting these highly dynamic and ephemeral infrastructure components to networking, configuring security and linking these to stateful data stores.
Replacing each orchestration platform by a new one when the next technology (such as serverless computing) comes around is neither cost effective or practical. Let's take a look at what makes such framework(s) a sustainable solution that can be used for the long run.
What is clear from my experience interacting with this domain over the last 10 years is that there is no "one size fits all" solution, but rather a combination of orchestrations frameworks that depend on each others with a specific role and focus area. A report on the topic was recently published by SDx Central covering the plethora of tools available. Deciding what is the right platform for the job can be overwhelming at first, unless you know what to look for. Some vendors offer a one stop shop for all functions, often taking the burden of integrating different products from their portfolio, while some others provide part of the solution and the choices to integrate northbound and southbound to other tools and technologies.
To better understand how this would shape up, let's go through a typical example of continuous application testing and certification, using the Quali's CloudShell Platform to define and deploy the environment (also available in a video).
The first layer of automation will come with a workflow tool such as the Jenkins pipeline. This tool will be used to orchestrate the deployment of the infrastructure for the different stages of the pipeline as well as trigger the test/validation steps. It delegates the next layer down the task to deploy the application. then orchestration will set up the environment and deploy the infrastructure and configure the application on top. Finally the last layer, closest to the infrastructure will be responsible for deploying the Virtual Machines and Containers onto the hypervisor or physical hosts, such as Openstack and Kubernetes.
When it comes to scaling such a solution to multiple applications across different teams, there are 2 fundamental aspects to consider in any orchestration platform: standardization and extensibility.
Standardization should come from templates and modeling based on a common language. Aligning on an industry open standard such as TOSCA to model resources will provide a way to quickly on board new technologies into the platform as they mature without "reinventing the wheel".
Standardization of the content also means providing management and control to allow access by multiple teams concurrently. Once the application stack is modeled into a blueprint it is published to a self service catalog based on categories and permissions. Once ready for deployment, the user or API provides any required input and the orchestration creates a sandbox. This sandbox can then be used for completing some testing against its components. Once testing and validation is complete, another "teardown" orchestration kicks in and all the resources in the sandbox get reset to their initial state and are cleaned up (deleted).
Extensibility is what brings the community around a platform together. From a set of standard templates and clear rules, it should be possible to extend the orchestration and customize it to meet the needs of my equipment, application and technologies. That means the option, if you have the skill set, to not depend on professional services help from the vendor. The other aspect of it is what I would call vertical extensibility, or the ability to easily incorporate other tools as part of an end to end automation workflow. Typically that means having a stable and comprehensive northbound REST API and providing a rich set of plugins. For instance, using the Jenkins pipeline to trigger a Sandbox.
Another important consideration is the openness of the content. At Quali we've open sourced all the content on top of the CloudShell platform to make it easier for developers. On top of that a lot of out of the box content is already available so a new user will never have to start from scratch.
Want to learn more on how we implemented some of these best practices with Quali's Cloud Sandboxes? Watch a Demo!
We tell our children, “Sharing is caring.” In the work place, though, it’s sometimes a different story. A big catch phrase in DevOps is to “eliminate silos” so that the work can be shared and continuously flow through the pipeline. In reality, what I see is that the silos are staying up and being fortified from within, and there’s not enough sharing.
Why is this the case? The problem is a focus on automation.
Hang on just a second. I know you’re thinking: “isn’t automating everything what DevOps is all about?”
Yes, it is. But maybe the DevOps community is too focused on the question of what to automate, rather than why they are really automating.
When the DevOps clarion call of automate everything echoed through the valley, teams and individuals started picking off low hanging fruit for quick automation wins. They introduced more open source tools, maybe a little GitHub, Jenkins, Ansible, and of course Python, as the glue that holds the whole thing together. Next thing you know, the developers automated themselves into a very cool, but also isolated, silo.
It turns out that a lack of automation was a symptom of lack of speed and agility, not the actual problem, and the root cause was something different. So, if a lack of automation is a symptom of a lack of agility, what is the root cause?
Maybe it’s a lack of self-service.
If you are delivering a service, either to an internal or external customer, it’s best you work on providing that service dynamically, and through self-service on-demand.
Until the mantra in DevOps becomes, “on demand, globally accessible, self-service” as a service delivery standard, I’m afraid the silos will remain, and new ones will be erected.
It’s in Our Nature
I notice that while parents are very adamant about teaching their children to share, they totally roll over every time a child doesn’t wait for his or her turn. Kid tries to grab toy. Parent tells kid to wait his or her turn. Kid looks at parent with a look on his or her face saying, “Wait? You’ve gotta be kidding, right?” Kid goes over to toy shelf, grabs similar toy and gets instant gratification.
Whether a toddler that just wants a toy, or a developer that just wants an environment, it’s very natural to take the path of least resistance. And, if there’s some other way for instant access to something, that other way instantly becomes the default way to go.
Shadow IT is the organization’s problem. The way to solve that is to become the path of least resistance. Not all services should be taken in house. You’ll still want to rely on third party service and tool providers and leverage Shadow IT visibility tools. But you can’t outsource DevOps.
If you’re in DevOps mode and are finding trouble breaking teams out of their silos, look at where they rely on manual IT for environments, data, feedback, etc., and see if you can’t work on providing one or more of these services on-demand via self-service.
I wanted to share my insights from DevOpsCon 2017, which took place in mid-June in Berlin. It was an excellent conference, with a lot of energy in the air. There were many people attending this year - the place was packed. I understood from one of the organizers that attendance doubled compared to last year, and the trend seems set to continue.
Most attendees came from German companies, but there was a fair participation from Nordic countries and eastern Europe. From the interactions I had with people, I noticed that while some were members of dedicated DevOps teams in their organization, the majority of participants still handle DevOps tasks as part of their other Development content.
We had several enterprises come up to share their DevOps journeys. What struck out strongly was the fact that almost all of them were expressing the complexity of setting up environments for DevOps, causing them delays. This was particularly acute in Enterprises with on-premise deployments and where they owned their data centers and had built up application stacks over the years. The dependency between infrastructure automation and application agility was on display there. Quali’s ability to quickly standardize environments via blueprints, model and deploy was fully resonating with this audience.
Our CMO Shashi Kiran also delivered a session on the top barriers Enterprises face on their path to deploying DevOps and he set the tone for best practices within the organization as well as a prescriptive approach to smoothening out those barriers. A houseful audience appreciated these tips based on a global survey conducted by Quali
After two days, I left the conference highly energized and with a big smile. I had a lot of fun, I learned a lot, and can’t wait to be here next year again.
This week our team is having a vibrant presence at Cisco Live in Las Vegas. Do visit them in booth #416 – they have some cool demos around cloud sandboxes and its applicability to deliver on-demand, self-service environments for a variety of different use-cases. Don’t miss out some of the cool swags that are only available at the Quali booth as well as our 2017 Cloud and DevOps survey.
3 in 1! we recently integrated CloudShell with 3 products from CA into one end to end demo, showcasing our ability to deploy applications in cloud sandboxes triggered by an Automic workflow and dynamically configure CA Service Virtualization and Blazemeter end points to test the performance of the application.
Financial and HR SaaS services, such as Salesforce and Workday have become de-facto standards in the last few years. Many ERP enterprise applications while still hosted on premises (Oracle, SAP…) are now highly dependent on connectivity to these external back end services. For any software update, they need to consider the impact on such back end application service that they have no control on. Developer accounts may be available but they are limited in functionalities and may not reflect accurately the actual transaction. One alternative is to use a simulated end point known as service virtualization that records all the API calls and responds as if it was the real SaaS service call.
We've discussed earlier on this year in a webinar about the benefits of using Cloud Sandboxes to automate your application infrastructure deployment using devOps processes. The complete application stack is modeled into a blueprint and publish to a self service catalog. Once ready for deployment, the end user or API provides any required input to the blueprint and deploys it to create a sandbox. This sandbox can then be used for completing some testing against its components. A Service virtualization component is yet a new type of resource that you can model into a blueprint, connected to the Application server template, to make this process even faster. The Blazemeter virtual traffic generator, also a SaaS application, is represented as well in the blueprint and connected to the target resource (the web server load balancer).
As an example let's consider a web ERP application using Salesforce as one of its end point. We'll use CA Service Virtualization product to mimic the registration of new Leads into Salesforce. The scenario is to stress test the scalability of this application with a virtualized Salesforce in the back end to simulate a large number of users creating leads through that application. For the stress test we used Blazemeter SaaS application to run simultaneous user transactions originating from various end points at the desired scale.
We used the Automic ARA (Application Release Automation) tool to create a continuous integration workflow to automatically validate and release end to end a new application built on dynamic infrastructure from QA stage all the way to production. CloudShell components are pulled into the workflow project as action packs and include the create sandbox, delete sandbox and execute test capabilities.
The way everything gets connected together is by using CloudShell setup orchestration to configure the linkage between application components and service end points within the sandbox based on the blueprint diagram. On the front end, the Blazemeter test is updated with the load balancer web IP address of our application. On the back end, the application server is updated with the service virtualization IP address.
Once the test is complete, the orchestration tear down process cleans up all the elements: resources get reset to the initial state and the application is deleted.
Want to learn more? Watch the 8 min demo!