post-img

DevSecOps Environments Deployed Secure and Fast

Posted by Dos Dosanjh August 25, 2018
DevSecOps Environments Deployed Secure and Fast

You've just implemented security tools that lower your organization's risk profile for your applications deployed on the Microsoft Azure Public Cloud.  End-user experience is compromised and you're trying to figure out why...sound familiar?  Responsibility rests squarely upon the DevOps, SecOps or DevSecOps teams who modified the application workflow behavior.

So where to start?  You contact the DevOps team to provide you a test environment so that you can start your troubleshooting efforts.  The DevOps team is busy updating the latest software updates and don't have cycles to spare due to production deployment deadlines.  At the same time, you are notified that the Azure Load Balancer is being replaced with Nginx and you have no idea what the ramifications will be on your security posture or end-user experience.

Data Applications Cloud NetworksThe initial troubleshooting activity occurs in the SecOps environment but you still require the involvement of the DevOps team.  DevOps will provide the latest updated software releases.  DevOps teams, responsible for cloud architectural components, re-platform the Azure environment to reflect network modifications.  These tasks are daunting without an ability to access self-service Azure test environments.  In order to address these challenges, test environments are required to isolate troubleshooting activities.  The following example outlines a microservice application deployed in a hybrid Azure cloud utilizing Quali CloudShell for orchestration.

Functionality:  The first step in any application and infrastructure deployment is to ensure that the baseline functions of the application are responsive per the requirements.  CloudShell provides the capability to introduce objects that represent the physical and virtual elements required within the solution architecture.  These objects are modeled and deployed in Azure Public Cloud and within the Azure Stack at the organization's datacenter or remote edge network.

Hybrid Cloud Microservices Application Functionality

Cybersecurity:  Once the functionality of the solution has been validated, the security software components are assessed to determine if they are the cause of the traffic bottlenecks.  In this example, a security scan utilizing Cavirin's Automated Risk Analysis Platform (ARAP) determines the risk posture.  If the risk score violates a regulation or compliance standard, a Polymorphic Binary Scrambling solution from Polyverse is installed to enable a Moving Target Defense.  The DevSecOps team utilizes the blueprint design to update the application software, determine risk posture and remediate as required.

Cavirin cybersecurity

Performance:  So we're feeling good, functionality is in place, security protection is enabled, but end-user experience is terrible!   Visibility is required at the data, application and network layers within the Azure Cloud environment to determine where the bottlenecks exist.  In this example, Accedian PVX  captures, analyzes and reports on the traffic workflows with test traffic from Blazemeter.  Environments are quickly stood up by the DevSecOps team and a root cause is identified for the traffic bottlenecks.

Accedian PVX

Automation:  To bring it all together requires a platform that allows you to mix and match different objects to build your solution architecture.  In addition, Self-service is a key workflow component that allows each team to conduct each operation.  This allows savings in time, resources and costs whereby solution validation can be achieved in minutes and hours rather than days, weeks and months.

Quali CloudShell

In summary, CloudShell automates environment orchestration, modeling, and deployments.  Any combination of public/private/hybrid cloud architectures are supported.  The DevOps team can ensure functionality and collaborate with SecOps to validate the security risk and posture.  This collaboration enables a DevSecOps workflow that ensures performance bottlenecks are addressed with visibility into cloud workloads.  Together this allows the DevSecOps team to deploy environments secure and fast.

To learn more on DevSecOps Environments please visit the Quali resource center to access documents and videos.

 

 

 

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

Continuous Integration — Only one Shade of Red

Posted by Tomer Admon February 18, 2018
Continuous Integration — Only one Shade of Red

How we shorten our nightly build time and enhance stability using TeamCity RESTful API

“What’s the first thing that comes to mind when you think about CI?”

If you are a developer, you probably think it should be FAST;

If you are an automation engineer, you probably think there should be lots of tests;

And if you are a DevOps engineer you ARE worried.

The problem

Back in 2016, we faced a problem. Our CI process became madness. Out of a culture where every small feature and bug needed to be automated, the type of culture where automation tests are a MUST and new infrastructure is added on a monthly basis, we found ourselves buried under a heap of tests.

Combining our top 5 automation infrastructures, we had more than 10K tests, that would have taken over 20 hours to run (had they run on a single CI agent). The need to cover so much ground forced us to write a powerful UI/Web test framework to allow us to easily test our full stack. However, despite the many enhancements we made, the execution of each web test still took anywhere between 1 and 5 min.

Our testing pyramid back in February 2016

Typically, when you think of a testing pyramid, you want to have a solid base of unit-tests, reasonable number of integration tests and a few high-quality web tests. In our case, with the power of super-infrastructure, on came the problem — a huge amount of web tests almost preventing our nightly suite from running on our JetBrains TeamCity (TC) instance. Back then, we had about 10 agents, and probably a dozen build configurations in TC, each running a specific type of tests, for a long time. For example, running our integration test assembly took 3.5 hours, while our web test suite took 5 hours.

A minimal number of build configurations sounds like a best practice when having a huge amount of tests, but we faced several problems:

  • Only one shade of red: The build was always red, full of unstable tests, execution failures, infrastructure issues, etc. The build never reflected the actual status of the product or the progress of the current iteration.
  • Extremely long nightly build, or should we say daily? We even considered only running it 3 times a week.
  • Execution time: Each build execution took anywhere between 1 to 5 hours.
  • Stability: It was impossible to identify unstable tests when the minimum cycle time is 5 hours.
  • Limited number of build executions: Infrastructure errors prevented entire build executions from running during the nightly. Losing one nightly build result can be crucial.
  • Unusually long response time for private builds: Developers had to wait a day or two to get execution results from TeamCity when changing code in a specific area of the product, even before committing the code to the main repository.
  • Build agent inaccessibility: Build agents were busy most of the time. As a result, other builds had to wait for long times in the queue.

The Beginning of Solution

We needed a solution or at least a way to provide the CI consumers a fresh breeze. A solution that would make their life easier, while achieving maximum stability and quick response time.

So we decided to SPLIT. Splitting everything. Class by class. web, integration, unit-tests. Instead of having less than 15 build configurations in TC, we would have more than 200.

Although TC easily handles 200 builds, we couldn’t manually deal with this setup, considering we had 4 branches in development, 3 DevOps engineers, and just 10 agents trying to keep up with the pace. Managing the new build configurations became our day today. As a result, a few days after the initial split we found ourselves in chaos. And as usual, when you don’t have enough, you need to automate what you have.

September 2016 — Splinter was born.

Splinter - our personal splitter. We realized we needed someone to keep up with the developers, review the changes in the code, observe new classes and new tests, and introduce them to TC while removing previously deleted ones. Time to write some automation.

Here’s the basic concept:

The Splinter utility should support any test assembly from our CI process and do the following:

1) Using reflection, analyze the assembly and find the relevant test classes. Test classes that are ignored, empty or contain only ignored tests wouldn’t be considered.

2) Clone an existing TC build configuration template and set the correct parameters in the new build configuration. If a matching build configuration already exists, just move to the next one in line.

3) Trigger the build configuration.

4) For cleanup, remove any build configurations that don’t have a matching class in the code.

Splinter utility should support any test assembly from our CI process
Iterating over each test class in the assembly, creating and triggering build configuration in TeamCity

Master Splinter (or Splinter Manager) is triggered every night and initiates all other Splinter, each for a specific test assembly.

The number of Splinters may vary between products and branches
Each class in the web test assembly have a corresponding build configuration in TeamCity

FluentTC

In Quali, we write most of our infrastructure code in C#. In the Splinter project, we couldn’t find a library that wrapped TeamCity’s rich RESTfull API for C# for advanced queries, so we decided to develop one — FluentTC.

FluentTC — Easy-to-use, readable and comprehensive library for consuming TeamCity REST API. Written using real scenarios in mind, enables various range of queries and operation on TeamCity

TeamCity’s rich RESTfull API allowed us to implement Splinter easily and add custom logic and enhancements in the following versions of Splinter.

FluentTC in JetBrains community: https://plugins.jetbrains.com/plugin/8980-fluenttc

FluentTC sources: https://github.com/QualiSystems/FluentTc

FluentTC Nuget: https://www.nuget.org/packages/fluenttc

Running build configuration with custom comment using FluentTC.

Conclusions

Introducing Splinter has created a new CI culture:

  • Agent utilization is amazing. Almost at any time, a developer can find a free slot to execute a build.
  • Nightly can be executed every 10 hours and still allow quick TeamCity response for developers during their workday commits.
  • Adding more agents to the agent pool immediately decimates the nightly execution time. Instead of having huge builds that are executed on a single agent, one at a time, Splinter allows the distribution and execution of the builds between available agents, in parallel.
  • Unstable tests are quickly detected, allowing for prompt investigation while unstable test classes can be executed in short intervals to collect actionable statistics.
  • Two-phase check-in made easy: Using TeamCity ReSharper integration for Visual Studio, developers can now select a specific set of test classes to execute with their private build, so areas affected by the code changes are fully tested before committing the code to the main repository. Response time for two-phase check-in is not likely to exceed 15–20 min.
  • Managers can enjoy a bird’s eye view of the product status and stability and quickly pinpoint defective areas in the product.
CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

TeamCity Plugin Development Journey

Posted by Tomer Admon February 18, 2018
TeamCity Plugin Development Journey

Jet Brains TeamCity & Quali CloudShell integration case study

This article is intended for developers who wish to perform 3rd party integration with TeamCity.

Following customer requests, Quali has decided to support custom integration between CloudShell (our main product) to JetBrains TeamCity CI.

In this article, we will dive into the design and development process of theCloudShell Sandbox TeamCity plugin, which is now available in JetBrains community.

Getting Started

CloudShell & TeamCity

CloudShell, a cloud sandbox platform by Quali, allows users to create and publish a CloudShell Sandbox, which is a replica of a specific computing infrastructure and application configurations, and use it for development, testing, demos, training, and support.

This plugin enables the consumption of CloudShell Sandboxes directly from TeamCity in order to be able to test complex testing environments.

When new code is pushed to the SCM system, TeamCity triggers a build configuration that starts a Sandbox using the CloudShell plugin to enable the execution of tests on that particular Sandbox’s components. Communication between TeamCity and CloudShell is provided by CloudShell’s Sandbox API, a RESTful API designed specifically for DevOps use cases that allows easy interaction with Sandboxes.

Here is an example of a Sandbox containing an application and some AWS services that mimic the staging configuration. This particular Sandbox is designed to be launched by TeamCity and will allow end-users to execute tests on the different components in the Sandbox.


Sandbox containing an application and some AWS services

Plugin Requirements & Design

The plugin comprises three main components: Admin configuration page, Build feature and a runner type.

  • Admin Configuration page - Administration page that allows the TeamCity administrator to configure, manage and verify the plugin’s integration and connectivity with CloudShell. The connection info provided in this page will be used in the plugin’s other two components.

  • Build Feature (single Sandbox) - Use this option if you want to use a build configuration with a specific Sandbox. The build feature will start a Sandbox prior to the execution of the build steps and will make sure to tear down the Sandbox just before the build completes.
  • “Start Sandbox” and “Stop sandbox” runner type (multiple Sandboxes)- When more than one sandbox is needed in a build execution, a build step for starting and stopping each sandbox helps to manage the consumption of the different Sandboxes.

Admin configuration page

Basics
To begin, we added an AdminPage class. This requires implementing the basic configuration of the page (the below basic configuration is from our plugin). Adding an admin page adds a new section in TeamCity’s Administration tab. Note that the new page is under the Integrations section as we override the getGroup() method and use the INTEGRATIONS_GROUP constant.

SandboxAdminPage.java in GitHub

Important: Any new resource you add to the project must also be described as a Spring bean so that TeamCity will load it. To include the resource, just add it to the plugin’s XML file. For example:

XML file - missing

Handling plugin data on TeamCity server

The Admin Page we created will be used only as the presentation layer of our plugin. In order to store and use some logic in the TeamCity server, we will create a new controller that will handle the configuration.

The Settings Controller extends BaseFormXmlController to allow easy HTTP POST and GET XML communication. In our case, only a POST request will be used to post data from the UI to the server.

The controller is added in the following way:

The controller gets a POST data form the main Admin Page and uses the Admin Settings class to save the configuration to a server-side config file.

In this plugin, we implemented the admin settings class to store the configuration data in a properties file. It is also possible to save the configuration data using built-in functionalities in TeamCity. When you save the configuration on the server, the user is automatically redirected to the Admin Page, which is reloaded with the data from the properties file.z

 

Admin Page user interface

The Admin Page UI is implemented using a JSP form that has two main functionalities, depending on whether the Save or Test Connection button is triggered.

The form includes a parameter that indicates the user-triggered functionality:

  • submitSettings — for saving the configuration.
  • testConnection — for test connection button.

In the Admin Page JSP, it looks like this:

Handling passwords securely

A key capability we wanted to achieve was to encrypt the password value during transmission between the client and the TeamCity server. To achieve this, we implemented an extra field in the form bean that stores a public key. When opening the Admin Page, TeamCity generates a public key on the server and sends it the page. Saving the configuration on the Admin Page encrypts the password (using the public key) in a way that allows the server to then decrypt it when it needs to use the same public key. In the properties configuration file, the password is stored encrypted as well.

When one of the fields in the form is a passwordField, the value of the field should be described as an encrypedPassword.

To decrypt the value sent from the client and encrypt on the way back, we implemented setter and getter methods called setEncryptedPassword and getEncryptedPassword that will be called when the bean object is accessed.

These methods use RSACipher to handle the values for us. The getHexEncodedPublicKey getter is implemented to ensure the public key is sent to the client in order to handle the encryption.

QsSandboxSettingsBean.java in GitHub

Test connection

After the administrator fills in all the required fields in the Admin Page form, we want to make sure the integration works and the two systems (TeamCity and CloudShell) can communicate with each other. In our case, the test connection sends a login request to CloudShell. This request verifies that CloudShell is accessible and that the user is authorized to use CloudShell.

TeamCity comes with few built-in options we can utilize to enhance the user experience during the test connection. For example, displaying a progress bar and a nice pop-up message once the test passes or fails.

In the Admin Page JSP file, we included a link to a javascript file that comes with TeamCity called testConnection.js.

//JSP file
<bs:linkScript>
     /js/bs/testConnection.js
    /plugins/sandbox/js/sandboxSettings.js
</bs:linkScript>
<script type="text/javascript">
    $j(function() {
    SandboxAdmin.SettingsForm.setupEventHandlers();});
</script>

In order to utilize the built-in testConnection javascript functionality that comes with TeamCity, we implemented sandboxSettings.js. This file extends the base form behavior in TeamCity and registers to TeamCity form events.

Here are the complete Sandbox settings we implemented:

sandboxSettings.js in GitHub

The built-in function BS.TestConnectionDialog.show() allows us to display pass/fail messages to the user.

Build Feature

As previously discussed, the CloudShell build feature was designed to start a Sandbox prior to running the user’s custom build steps and to stop the sandbox just before the build completes. To achieve this, we implemented the AgentLifeCycleAdapter class, which includes the buildStarted and beforeBuildFinish methods. The AgentLifeCycleAdapter is implemented on the agent side and will be packaged as another zip in the final plugin.

The BuildFeature registration on the server is added by implementing the BuildFeature class. As noted in the Admin Page section, make sure to add the new class to the plugin.xml file.

To implement field validations on the server, use the getParametersProcessormethod.

Passing the plugin configuration info to agent-side operations

To start our sandbox both in the CloudShell build feature and in the CloudShell Sandbox runner type that will be described below, we need to access the main configuration settings from the agent side. In other words, we need to send CloudShell connectivity details from the server to the agent prior to the build’s execution.

The BuildStartContextProcessor extension point allows us to manipulate the build context and to add additional information that will be accessible in the agent.

In the example below, we added the CloudShell connectivity settings:

To access the CloudShell connectivity details from the agent side, we need to retrieve the shared parameter we set on the server getSharedConfigParametersmethod on the AgentRunningBuild parameter. In this example, the buildStarted method in our build feature just retrieves the shared parameter directly from the AgentRunningBuild variable:

“Start Sandbox” and “Stop sandbox” runner type

Adding a build step to TeamCity requires implementing a new RunType and adding it to the plugin.xml file. In our plugin, we wanted to implement a single runner type with two different actions, each with different parameters. For example: the “Start sandbox” action should get a blueprint name while the “Stop sandbox” action should only get the sandbox id to stop.

Multiple actions in the same runner type

In our RunType JSP file, we added a drop-down that lists the different actions available in the runner type.

Note that the currValue of each of the options is ${modeSelected}. This way, the form is consistent while the user selection will not change after saving the build step.

To control specific fields that should be shown for each of the actions, we added a javascript code that will show and hide the needed fields.

sandboxRunnerParams.jsp in GitHub

On the agent side, we implemented an AgentBuildRunner that will house our custom BuildProcess implementation. The buildProcess implementation will then use the build parameters to locate and execute the relevant action (start sandbox or stop sandbox).

Helpful tips

  • Like every other TeamCity plugin out there, we started with an empty project, as described in JetBrains confluence guide.
  • We realized that the best approach for debugging and building the plugin was to use the TeamCity-SDK-Maven-plugin that can start afresh TeamCity server-agent setup for you directly from command line. This particular plugin was really useful for reloading our project whenever we performed agent-side code changes or changes to the project resources folder (buildServerResources).

Debugging TeamCity agent

During the development process, we needed to debug the TeamCity agent. To do that, we added the following configuration to the agent.bat file located in the TeamCity agent folder (On Windows: C:\TeamCity\buildAgent\bin). This code allows you to debug the agent over port 8001.

:agent_mem_opts_set_done
SET TEAMCITY_AGENT_OPTS_ACTUAL=-Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=n %TEAMCITY_AGENT_OPTS% -ea %TEAMCITY_AGENT_MEM_OPTS_ACTUAL% -Dlog4j.configuration=file:../conf/teamcity-agent-log4j.xml -Dteamcity_logs=%TEAMCITY_AGENT_LOG_DIR%

 

 

 

 

 

 

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

Pedal to the Metal – Putting 2017 in Perspective for Quali

Posted by Shashi Kiran January 22, 2018
Pedal to the Metal – Putting 2017 in Perspective for Quali

Whether you are an individual, or a corporate, success and interesting lives come to those that step outside their comfort zones, push the envelope and step a bit further out on the edge.

2017 was an interesting year for Quali with a lot of change - mostly self-driven, considerable business success and a chance to move the needle a bit further.

Here are 10 things where Qualians put pedal to the metal in 2017 and consciously stepped out.

Revenues - A record year and a record Q4

2017 was a year where CloudShell revenues started to mature and scale.  We had a 88% CAGR in CloudShell subscription monthly recurring revenue. Q4 also had a 55% Q/Q growth with strong revenues between service provider and enterprise customers

 

We added five more Fortune 1000 clients in Q4

Quali has been adding several Fortune 500 clients through the year and Q4 was no exception, with the addition of a few more Fortune 500 and Fortune 1000 clients. Our solutions are resonating quite strong with financial services in a number of areas and innovators that are investing in digitization efforts embracing cloud and DevOps. While we take pride in some of the marquee names that are continuing to come on board as our customers, we’re seeing innovation among our customers across the spectrum. The sense of urgency is felt by all to move faster, to reduce risk and to make employees more productive with strong cost control mechanisms.

A significant portion of our revenue started to be subscription based

This was a shift for Quali from its earlier model of selling based on perpetual revenue and annual maintenance contracts. While we still provide that flexibility to customers with CAPEX spends, we’ve predominantly shifted now to subscription revenue. This helps customers bring more users into the fold and it increases adoption within organizations and delivers ROI faster. It also is a shared commitment between us and our customers – a partnership of sorts where we are driven to help each other succeed faster. Quite a few of our subscriptions are 3-years now and over 75% of our customers including our older contracts have now moved to subscription.

Internally this has meant changing our revenue recognition, yearly targets, sales and channel compensation structure etc., touching backend sales and marketing automation processes but on the flip side, it has allowed us to scale much faster as an organization and to address customer budgets more innovatively. Many software companies begin as subscription companies – changing business models without impacting revenue is akin to repairing the wings of a plane while it is flying – an interesting exercise in conscious pivots.

Our customer diversity increased

As we expanded our global footprint and sandboxes found innovative use-cases, we continued our focus evenly on service providers and the enterprise. We’ve been fortunate to have three of the top financial services customers in the US, 8 of the top ten telcos, three of the top five public cloud providers and a range of new and innovative enterprises found value in cloud sandboxes. Who would have thought a paper company would have hundreds of software developers, or a company known for its best-of-breed hardware solutions would be building one of the most innovative developer focused sandbox solutions on the planet?

Some of the use-cases of sandboxes were a surprise to us. Our customers were using the notion of environments-as-a-service in so many innovative ways.

 

Cyber Ranges started to pick up – big time

While it is always easy to spin up something virtual, Quali was always known for its ability to integrate well with physical infrastructure and to automate and orchestrate an environment on any cloud, for any application or infrastructure. For modeling and orchestrating full-stack environments on-demand, this is a boon.

With increased cyber security threats globally, building up cyber ranges for training, security posture testing and hardening is very important. However, rising complexity of distributed environments and sophisticated communication equipment in all forms of connected security including on the battlefield has made this a difficult task to achieve paving the way for defense agencies, financial service institutions and larger enterprises to adopt cyber ranges as a credible and proactive defense mechanism.

We expanded our product offerings and launched a free trial

Our product expanded beyond the flagship CloudShell (re-branded as CloudShell Pro) to now accommodate the CloudShell Virtual Edition (VE). We launched CloudShell VE at VMworld, and were promptly awarded the “Best of VMworld” finalist award in the automation category. While CloudShell Pro was highly differentiated for full-stack offerings including integration with heterogeneous physical infrastructure, as well as hybrid and multi-cloud deployments, CloudShell VE was tailored toward virtual-only environments and functioned on any single cloud of choice. It also offered a different entry-level price point and the ability to set up environments in less than an hour!

CloudShell VE also allowed us to now launch a free-trial of our offering allowing prospects and partners to experience the value of the product first hand or to dip their toe into the sandbox. We’ve since had a strong traction for our trial offering coming in globally, and expect the momentum to further increase in 2018.

If you’d like to check out the 14-day free trial version, go to www.quali.com/trial

Our Multi-cloud story became stronger

Today’s clouds are like yesterday’s cell phone providers. You should be able to switch to the best cloud provider and be able to carry all your data, processes and operational models, as easily as you could port your cell phone number and move to a different provider. Except you can’t. Cloud providers have made their offerings quite stick – it’s easy to get in, but somewhat harder to get out.

So, products and offerings that can offer abstraction for multi-cloud offerings are becoming popular.  Even though our customers may not truly be operating in a hybrid or multi-cloud operational manner, the fact that we can spin-up environments consistently on private, public or hybrid clouds allows them a sense of future proofing. They can design blue prints once and deploy via a single click on any cloud/s. Last year we added support for Azure, OpenStack in addition to AWS, VMware and Baremetal offerings. Very soon we’ll be announcing support for Oracle cloud and the architecture is obviously extensible to accommodate Google Cloud platform as well.

Our partner ecosystem expanded

No man is an island and that goes for corporates too. We have always had strong technology partnerships. As we continued to expand our multi-cloud offerings, the ecosystem expanded with better technology integrations and go-to-market partnerships with the likes of Amazon Web Services (AWS), CA Technologies, JFrog, CGITower and more recently with Oracle Cloud. We have solution briefs, joint demonstrations and more with these partners and several others that are showcased on our website.

Introducing our Gen 2.0 "Shells" allowed our technology partners and customers to easily extend the power of Cloudshell to other products including their own do-it-yourself products in a seamless way. The CloudShell software development kit (SDK) got downloaded hundreds of times.

Our channel partners also switched to high-gear with greater training investment and helped expand our pipeline

Our market presence improved considerably

While we’re still scratching the surface in terms of market awareness, we took strong steps to further increase our presence in 2017

  • We won 9 awards and were finalists in several categories. From the Red Herring Global 100 winners, to Best of VMworld finalists, our award categories spanned automation, cloud management, security, SDN and more.
  • Our marketing and sales teams conducted 14 educational webinars with industry analysts, customers and partners and participated in 26 tradeshows across N. America, Europe and APAC
  • We also had over 60+ independent news articles with dozens of analyst mentions and media publications
  • We delivered several keynotes, tech sessions and participated in a few panel discussions with other industry thought leaders and subject matter experts
  • We had over 1300+ responses to our annual Cloud and DevOps survey, the results of which are expected to be published imminently

We let our hair down and had some some fun!

2017 wasn’t all a year of serious work for the Qualians. Across the different offices including our R&D headquarters in Israel, our business headquarters in the Silicon Valley, we carved out time for some fun stuff. Barbecue parties, magic shows, fun runs in crazy costumes and then some!

This is something I’m personally hoping we’ll do more of this year.

A big THANK you to our customers, partners and all the employees and their families for making 2017 a year to remember and a toast to make 2018 even more.

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

Quali wins Layer123 Network Transformation Award

Posted by Pascal Joly October 17, 2017
Quali wins Layer123 Network Transformation Award

Quali's CloudShell just won the Layer123 Network Transformation award in the Best Orchestration & Control Product category.

Winning this award was a significant achievement for Quali.  Telco decision makers attending the award diner at the Gemeentemuseum den Haag last week were also paying attention.

As new networking concepts such as SD-WAN and NFV are picking up steam, there is a growing awareness among Service Providers that building a standard approach to the consumption of development and test environment is key to sustainable certification and innovation. That means not only getting fully on board with Automation and DevOps from a process standpoint, but adopting Orchestration solutions that meet the very specific needs of the industry vertical.

Shifting Factors in the Telco industry are opening doors for accelerated innovation

Applying a DevOps approach in the Telco industry is relatively new. Up until recently the development cycle had traditionally followed a multi-month (or year) release pattern quite remote from webscale companies leading the way such as Netflix and Airbnb. Certainly a service provider has to deal with  regulatory constraints in terms of uptime and the delivery of services.  However the dynamics are changing.  Several driving factors are at play:

  • Business mandate to sustain innovation: This is a matter of survival for telcos.  keeping a competitive advantage when the boundaries of delivering network services are no longer solid.
  • Software Defined as a change agent: no just Software Defined Networking but in general the ability to commoditize the underlying infrastructure and overlay software based intelligence on top.
  • Automation takes center stage: now that software based technologies are prevalent, continuous updates through automation and orchestration can become reality for the network tester.

Making DevOps Automation a reality for Service Providers: not as simple as it seems

Making DevOps a reality for the network engineer and tester responsible for validation of new SD-WAN and NFV technologies is easier said than done.

To begin with, there is a mismatch expectation on skills: network engineers are not programmers (for the most part) so they need to be exposed to the proper level of resource abstraction for an effective implementation.

In addition, quite often, efforts to apply general compute based automation approach to industry specific challenges, such as NFV on boarding, have been stopped on their tracks. The complexity of software based network environments makes the translation non trivial. Instead of reaping the benefits and agility of software based approach, it merely compounds problems seen with physical based environments with additional challenges tied to virtual workloads . In the best cases, this leads to a successful (and well publicized) small scale pilot on a greenfield project, that never reaches full production at scale and expected ROI.

Can it scale beyond prototype?

A Practical Approach to Network Orchestration

Taking a practical approach to network orchestration, Quali's CloudShell combines the power DevOps automation with the flexibility to provide on demand, dynamic environments to meet the inherent needs of service providers.

  • A network designer can create visual blueprints to model the various network infrastructure scenarios based on the TOSCA standard for certifying NFV/SD-WAN roll out.
  • These environments, or "cloud sandboxes" can then be deployed from a self service portal to virtual or physical resources and accessed by the tester through a web interface for a simple interactive experience, or through API using modern CI/CD tools such as Jenkins.
  • Out of the box orchestration takes care of workload deployment, configuration and connectivity setup. Similarly, automated resource reclamation provides control on cost and resource allocations to large teams of users.
  • Extensibility provides a way to customize the experience based on what is most relevant for the end user and sustain the automation in the long run: when it comes to technology, there only one constant: change.

Now you might be intrigued and want to learn more about our solution. There are several ways to accomplish that: scheduling a short demo, watching a recent webinar we hosted on this topic, or read about the NFV/SDN solution space on our web site.

 

 

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

5 Best Cloud and DevOps Podcasts

Posted by admin October 4, 2017
5 Best Cloud and DevOps Podcasts

These days, commutes are getting longer and longer. In Silicon Valley where I work, even a short 10 miles drive can turn into a 40-minute slog. Or maybe your plane is stuck on a runway, or you've got time to kill waiting in line at Costco. Whatever it is - Podcasting can be your new friend. Over the last few years, I've discovered podcasting as an essential way to fill those otherwise dead times. Not only is it a way to catch up on all the latest fan theories for Game of Thrones, it's also a great way to brush up on all the latest IT tech and industry trends. Here are my 5 recommendations for the best cloud and DevOps podcasts. These great podcasts are related to all things cloud, DevOps, network, and emerging IT technologies. I can heartily recommend these as podcasts that are both engaging and highly educational.

The CloudCast

These guys have been going at it for a whopping 6 years! Aaron (NetApp) and Brian (RedHat) are great hosts - engaging and inquisitive. Each episode covers in-depth topics ranging from cloud automation, cloud architecture, devops, networking, containers, and tools. One thing that I love about this podcast is that they provide a wealth of helpful follow-up links after each episode. They'll reference white papers, videos, articles, and other podcasts that are related to topics covered in each show.

Another thing I appreciate about these guys is that they're genuinely trying to understand the landscape, get at fundamentals, and not just mouth off industry jargon. For example, a recent episode called "Making Sense of New Technologies" tries to do just that: make sense of all the new platforms, frameworks, and tools that make up today's rapidly changing technology landscape. They genuinely want to understand the technologies beneath the hype. Here are a some of my favorite episodes.

"Next Generation DevOps Tools" covers Ansible and how automation and orchestration tools are evolving from mere configuration management to offering infrastructure as code, while at the same time reducing complexity. "Evolving from Plumbers to Coders" is a great listen - it covers the topic of how DevOps and cloud are shifting everyone - IT admins to network engineers - from being CLI and tooling "plumbers" to true coders. "Large Scale Distributed Infrastructure" talks about Mesos and how Twitter is managing large-scale infrastructure. Lastly - "The Evolution of VARs in Cloud Computing" is an interesting earlier episode which discusses how WWT (World Wide Technology) is bringing more agility to how they sell cloud solutions through the development of their Advanced Technology Center.

Electric Cloud Podcast

Sure, it's a vendor podcast - but what I love about this podcast is how down to earth and informal it is - which creates an environment for very honest and enjoyable conversations about topics related to Continuous Delivery and DevOps. One of my favorite episodes - "Consistent Deployments Across Dev, Test, and Prod" - really gets into an honest of how hard and how critical it is to ensure environment constancy from development all the way into production.

Packet Pushers

Network engineers, carriers, and network equipment vendors will argue that the network *is* the cloud. Whether you agree or not, this network-centric and deeply technical podcast will teach you a lot about cloud and DevOps from a networking perspective. Host Greg Ferro is well known for his networking blog EtherealMind. What's nice here is that the hosts clearly identify which shows are vendor-sponsored.

Here are some suggested shows. "A Deep Dive into NFV" gives a good overview of what NFV is, how it's different than SDN, and why it's critical for cloud adoption. "Automation and Orchestration in Networking" is a great show because it talks about all the various automation and orchestration tools from Ansible & Puppet to OpenStack & Kubernetes, and discusses how the newer technologies will connect with older, legacy hosts. "SRE vs Cloud Native vs DevOps" is a great show to become familiarized with the term SRE (site reliability engineer) which is a term that comes out of Google and aims to put operations teams on an equal footing with developers.

Speaking in Tech (SiT)

This podcast is a bit more on the fun side and covers a broader range of topics that the other podcasts listed here. Greg Knieriemen (HDS), Ed Saipetch (Blue Box), Melissa Gurney (DellEMC), Peter Smallbone (Cloud Architect), Sarah Vela (Dell) do a great job covering a breath of topics with interesting perspectives. Show #230 is a good take on DevOps, with guest Gene Kim, author of the Phoenix Project.

The Doppler Cloud Podcast

Lastly, the Doppler Cloud Podcast covers all the latest cloud topics while taking an angle of helping traditional enterprises embrace the cloud. Some good recent episodes include "Multi-Cloud vs Hybrid Cloud: What's the Difference?", "Kubernetes Wins at Orchestration Engines", and "Edge Computing and How it Transforms Enterprises".

So put on your headphones and start listening! These podcasts will keep you up to speed on latest industry trends and genuinely teach you something new with every listen.

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

VMworld 2017: AWS Was Very Loud

Posted by admin September 14, 2017
VMworld 2017: AWS Was Very Loud

And I mean that literally. The AWS booth was blasting their presentations so loudly, those of us attending the Best of VMworld Awards Ceremony in the Solutions Exchange Theater could barely hear who the winners were.

A rep from the theater area eventually got them to turn down, just in time to hear that Quali won the Finalist award for "Agility and Automation", second only to Puppet. We were super excited to win the Finalist Award a second year in a row; and to share the accolades with a sexy startup like Puppet was really an honor.

VMware + AWS - It's Real

As I left the Award Ceremony I couldn't get the sound of AWS out of my head. It wasn't just their excessive volume. It was that, for many of us, seeing the technical fruit of the AWS and VMware strategic partnership being made available to actual customers was quite shocking. As Forbes recently reflected, VMware's CEO Pat Gelsinger really swallowed his pride when he made the decision to move away from public cloud and align his company with one of its key competitors. That takes a lot of guts.

I've always admired Pat Gelsinger. In 2015, he was already suggesting that in order to succeed "Elephants must learn to dance." And great tech leaders like Gelsinger, Chambers, and Jobs know that the best dance move is the pivot. So here we are with VMware and AWS dancing together, and very loudly.

In many ways it makes sense. For both VMware and AWS, their loud dance helps block an elephant more than triple their combined size: Microsoft. Now VMware can pivot back to what it does best without the threat of the public cloud dampening core data center technologies like vSphere and NSX. I spoke with one of the Product Directors for VMware Cloud on AWS. According to him, the "it just magically works" approach will actually help on-prem customers stay on-prem. Often the flight to public cloud is driven by fear based "what-if" scenarios, and knowing that migration can happen securely when needed will help assuage existing vCenter customers' anxiety about migrating to public cloud.

Secure Cloud Migration

Another interesting announcement was the partnership between VMware and Dell EMC to bring Dell EMC data protection to the VMware Cloud on AWS. This gives customers end to end data protection across public and private cloud with Dell EMC’s Data Domain and Data Protection Suite for Applications. Another reason to not worry about moving everything to the public cloud for now.

VMware and Containers?

During the day two keynote, a big announcement was made about VMware and containers. Containers are often touted as the lightweight alternative to bulky hypervisors. However, VMware announced PKS: Pivotal Container Services. The "K" stands for Kubernetes - the rapidly growing container management platform that is largely responsible for driving enterprise adoption of container technology. It's also the technology Google's cloud services runs on. PKS makes it super easy to deploy Kubernetes on vSphere. And what's more cool is that PKS has VMware's NSX SDN technology built in. Networking has long been the Achilles heel of containers. With NSX's ability to provide seamless cross-cloud networking, PKS offers a solution that allows applications to seamlessly span on-prem data centers and public cloud.

Cloud Automation and Self-Service IT

I wish I could have spent the entire VMworld attending keynotes and seminars. I particularly wanted to get into the gory details of how VMware Cloud on AWS actually works. Unfortunately booth duty called. However, this year I had many enriching and fun conversations. The transformation from legacy IT to customer based on-demand IT is in full swing. Every one I talked to is trying to figure out how to transition to an "as-a-service" approach. That's great to see. People were very eager to catch a live demo - especially when I talked about how critical it is to get cloud automation and self-service IT solutions in place that deliver fast time to value. I was also happy to be able to offer a Free Trial of our award winning cloud automation software for the first time. While the past couple of years I would say most folks were contemplating self-service, this year many folks were truly evaluating and re-evaluating soup to nuts solutions.

It's an exiting time for IT. The partnerships and technology developments across today's IT landscape continue to benefit the consumer. However, it's still a changing world, with technologies like hybrid cloud still evolving. Abstracting that complexity away while making services available to end users via on-demand, self-service portals will help businesses ride the wave of innovation.

Hot though it was - another great time was had in Vegas! If you didn't make it out to see us at VMworld, you can find us at the upcoming Delivery of Things in San Diego in October as well as the DevOps Enterprise Summit in San Francisco in November. See you all again soon!

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

Self-Service IT Environments at VMworld 2017!

Posted by admin August 26, 2017
Self-Service IT Environments at VMworld 2017!

Vegas here we come! It's the end of summer and still sunny in Las Vegas, which means it's time for another VMworld. The Quali team will be out in force, ready to talk all things cloud, automation, and what the "as-a-service model" means for modernizing IT.

When you visit our booth you'll:

  • Learn how providing self-service, on-demand access to IT environments on public and private cloud makes the lives of developers, testers, DevOps engineers, security teams, and other key IT users way, way easier!
  • Learn why effective self-service IT needs to deploy apps and VMs across public and private clouds as well as the power to provision both virtual and bare-metal infrastructure.
  • See how Quali's self-service IT software (CloudShell) allows you to easily model and deploy complex, full-stack application and IT environments for a multitude of use cases.
  • See how Quali's BI plugin (InSight) gives IT Ops the ability to plan effectively, manage better, and reduce costs.
  • See Live Demos of Quali's new CloudShell 8.1 and CloudShell VE releases

So come join us at booth #1821 to meet some great people, grab t-shirts, stickers, and other fun swag, catch a live demo, as well as enter our Cloud & DevOps drawing for a chance to win a $100 Amazon Gift Card!

In the meantime - watch the demo video introducing CloudShell VE.

See you soon!

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

Speaking of Sandboxes

Posted by Roni Dover August 9, 2017
Speaking of Sandboxes

I came to the Jenkins Users Conference in Tel Aviv last month (JUC) expecting to do a lot of explaining around the concept of sandboxes, what they are and why they are needed. Surprisingly, I found that sandboxes were already a big part of the conversation at the conference, so I spent most of my time listening instead. To be fair, only a handful of people used the term 'sandboxes'. Some referred to them as environments, other as 'production replicas’. Similarly, I heard about many diverse, seemingly unconnected use cases, from applications testing to physical network topologies. These different use cases and terms all revolve around a similar pain that surfaced again and again in the conversation: How to create production like environments and make them easy to use for testing and dev without adding a ton of complexity. How to make use of these environments across the devops pipeline, from dev to test - from CI to CD.

So, what are sandboxes, and why do we need them?

I think of sandboxes as the missing abstraction for creating on-demand, environments based on your production which are scoped for a specific usage context. Basically, making both production and its subsets something easily accessible to everyone by making it reproducible via automation. The three basic layers of sandbox environments are the apps (in testing these are the apps under test), the required infrastructure and the orchestration code – the magic gluing everything together.

One of the reasons sandboxes are becoming so important is that tests are evolving. In the presentation I gave at this conference, I discussed how the term ‘continuous integration’ should be considered as broader than simply continuously running unit tests. Continuous integration, to me, means continually integrating the code with different aspects of production – load, infrastructure, different browsers and viewports, security restrictions as well as failures and server crashes. Unit testing as code validation are only a small part of the continuous integration spectrum.

As organizations ‘move right’ from continuous integration towards continuous deployment, the need for better and more varied integration tests now becomes critical. By accurately testing different aspects of production, much of the uncertainly and risk of the final deployment act is removed and the organization can transition to an automated release process. Unlike unit tests, which run in isolation on any MV, this varied spectrum of tests requires continually and accurately recreating aspects of production. These tests require sandboxes.

There seem to be many issues with making sandboxes work. Some of the people I've spoken with at JUC have already implemented custom solutions for setting up environment using various tools and products, but are struggling to release it for the rest of the team in a way that would be standardized, controlled and easy to use. There is no shortage of tools which can help hand string production environments together, making the life of an admin or devops engineer much easier. However, packaging this orchestration, and defining the environment specifications and parameters in a way that an IT or devops engineers would be comfortable letting anyone in the organization use, is something else entirely.

This conflict is at the heart of devops these days. On the one hand, maintaining control and ensuring standardization, on the other hand realizing that the role of the devops team is to enable others - not to do the work for them. Devops teams cannot afford to become a bottleneck by being personally responsible for each environment. Devops after all does not mean every developer is now responsible for ops. Having each team member create their own environment using whatever means they see fit would result in hard to support non-standard environments, not to mention in a mess of orphaned networks and forgotten VMs.

A maintainable devops process requires, therefore, to turn the pyramid on its head. To provide sandboxes as ‘self-service’ to users -  but maintain restrictions and policies on their consumption.

The complexity of production environment and the problems in recreating it for testing was a popular topic at JUC. Even organizations who have succeeded in creating a stable continuous deployment process find it hard apply that process to dev/test environments. Most of the time, the way production is modeled is a big impediment. Different dev/test activities require different levels of similarity to production, in terms of scale, configuration, data and external services production environment may depend on. It’s difficult to create an environment modeling that is both single sourced and based on the push to production assets and at the same time flexible and easily parameterized.

The user should be able to selectively scale up/down certain elements of the environment or even mock or replace some of them altogether. Some integrated cloud solutions make this even more frustrating by closely coupling infrastructure with the application. This is makes it very difficult, for example, to shift between running an app on multiples instances and running many apps on once instance without massive changes. Simple yet flexible modeling is a make or break feature for any sandboxing platform.

My takeaway from JUC was that sandboxes are ever present and increasingly an important part of the conversation in the devops world. They are also a big pain in many organizations with the strain falling directly on the devops team. This may seem a bit disparaging, yet I think it’s a good sign and an indication of the growing maturity of devops and testing processes.

To be effective, sandboxes need to be available ‘as-a-service’ to all users, modeling is key as is implementing way for the devops team to enforce policies for consumption and standardization. At Quali, we are excited to be working on improving our own sandboxing platform based on these principles. Judging by the conversations I’ve had at JUC TLV, cloud sandboxes are going to get even more interesting in the coming year.

Want to learn more? Contact us or Schedule a demo.

CTA Banner

Learn more about Quali

Watch our solutions overview video
post-img

More Human – CloudShell Business Transformation

Posted by Pascal Joly July 25, 2017
More Human – CloudShell Business Transformation

Netflix killed Blockbuster. Why? They had the same movies available. Amazon is killing brick and mortar retail before our very eyes. Why? They offer the same products. Uber and Lyft are crushing traditional taxi services? Why? It’s the same deal—a point A to point B ride. I think you know the answer to the question “why?”. These companies turned the consumer experience on its head, leveraged innovation and technology to create a level of operational efficiency their competition literally couldn’t imagine in their wildest dreams, and are accelerating business transformation at a massive pace.

That’s called disruption.

When we look at technology and disruption, ease of use and quality of experience are paramount, and the human element is too often overlooked. For example, people love AIR BNB because it's very easy to rent your living space and make a few bucks, or find a place to rent and save a lot of money over hotels all through a basic smart phone application. AIR BNB did not just materialize out of the ether and appear in the App Store with renters ready and waiting. We all know that not to be true, but no one marvels or appreciates what was a tremendous amount of human energy and visionary decision making that truly created the market disruption.

The next time you engage one of your favorite apps (maybe you are right now), consider that the easier you are able to consume that application and its services, the more ability that application has to add a force multiplier to the human energy you invest in it.

“…while these [automation] tools offer new strengths and capabilities, they are meant to complement and enhance human skills.” - Marshall Wells & Aiha Kralj, Accenture

The organizations we are working with realize that they need to embrace digital transformation, and the reasons they have for taking action are not trivial. Too much critical work is coming in and resources are sparse. The business needs to show Wall Street better management of expense. They are losing critical talent to more forward leading technology organizations. The business issues are real, and the time lines to solve them are tight.

They know that baby-steps won’t cut it, and that a paradigm shift in operations and service delivery are required to gain significant ground on the competition, or to remain the market leader. These issues are not problems, but opportunities. And when our most visionary customers are able to see our software through the lens of opportunity, business transformation can occur.
Though our cloud orchestration software is a critical part of our customers’ digital transformation, its most important role is its ability to support the humans who are collaborating, taking visionary risks, and making big time business decisions.

“[Automation] is better, predictable, faster, typically cheaper, and a much lower error rate as long as our algorithms work.” - Catherine Bessant, COO, CTO Bank of America

Business transformation is a monumental endeavor, but it doesn’t necessarily have to be a daunting task. In fact, the more difficult or awkward, or the less certain the desired outcome is possible, the better likely hood that particular endeavor was a bad idea and that it’s time to go back to the drawing board. And that’s not a bad thing, failing is simply part of success (see: failing fast).

Our customers are visionaries who don’t just see a piece of software to solve an immediate pain. What they see is an opportunity to deliver services better, faster, cheaper. Period. Note that in the last paragraph I qualified that business transformation “is” a monumental task. Our customers find that we’ve made much of that task consumable in an application. So without having to ask the business for more people, we make it possible for customers to increase output and the number of users supported by…pick a multiplier.

It’s a good day when we hear from our customers that they are taking on and successfully completing more work without straining existing resources. That’s what makes the investment of seemingly immeasurable development hours’ worth while.

While automation has transformed and will continue to transform many industries, it largely redefines rather than eliminates jobs.  - Gene ZainoMBO Partners

The executives and management will plan out and execute the high level business activities and make the important decisions involved in steering the ship through the choppy seas of transformation, but it’s the team of doers that are critical to successful execution. And the attitude of management will often reflect in the team. For example, I have a customer who reminds everyone on his team in his email signature, “Enjoy your day and be sure to disrupt.” Great leader!

It is the team that embraces disruption that will enthusiastically take on their part of extending our orchestration software the extra 10% of the way. Like I said, the monumental task of business transformation doesn’t have to be daunting. An extra 10% across a small team doesn’t seem like a lot of work, but the output of their work has a major impact.

The human element is crucial to Quali’s cloud sandbox software. We need conscientious and motivated users, and in return for their earnest engagement, we’ll continue to make the product easy to consume. And with their effort, they can continually improve their customers experience. As I discussed in Stop Automating! the results are not as powerful when addressing automation from a mechanical angle, rather than empowering the human element that is so critical.

To say that cloud automation software like Quali is disrupting the market in the same way as Netflix, Amazon and Uber is not an overstatement. What would be an over statement is to say that we have it all figured out, right here right now, and have exactly what you need to transform your business. We’re a big part of that transformation, but the human element, the creativity, intelligence, vision and passion are essential.

“Enjoy your day and be sure to disrupt.”

CTA Banner

Learn more about Quali

Watch our solutions overview video