Automation of tests has reached satisfying level, which enables agility from the testing side. However, one of the main reasons why testing results are often being delayed is because of waiting time spent for deployment of testing environments. Therefore our main focus should be on how to strengthen this weakest link so it does not brake in this highly competitive Agile race. We will share our DevOps and Continuous Delivery experience at a large financial institution and identify the required tools to support those processes.
Audience: DevOps engineers, testers, scrum masters
Prerequisites: Basic understanding Agile methods, SOA,
You will learn:
1) The main constraints that hold back the deployment of applications & test environments. Based on a survey of many real-life test situations in particular in a DevOps and Continuous Delivery process.
2) A tool based solution (processes and methodology) consisting of service virtualization to create more flexible staged environments.
3) And a new architecture that supports automated deployment and provisions of test-environments.
Modern software development methods (Agile, Continuous Delivery) require flexible test environments, whilst applications (like SOA) become more complex and have more external dependencies. How can these constraints be resolved and taken away?
In 2012, a survey asking software developers, testers, and performance test engineers a series of questions about their access to test environments. Key findings showed that testing today’s composite, distributed applications requires interacting with a number of dependent (connected) applications, which are difficult to access. Respondents reported needing access to an average of 8 dependent applications, but having consistent access to only 3 of them. An overwhelming majority of respondents (76%) reported having restricted access to the test environments required for completing their development and testing tasks. The time available to access test environments is extremely limited and 30% of that limited time was consumed by configuration/setup tasks. Finally, testers had time to execute only 50% of the available test plan. These results indicate that development and testing teams lack the resources required to complete the expected level of testing.
To tackle this complexity of testing today’s composite applications, we will link the world of Agile development with the operational aspects of providing flexible test environment (DevOps) as building blocks for Continuous Delivery. Today, development and testing teams have limited control over the (sub)systems that are crucial for reaching the expected quality level of the services. There are a number of constraints that limit the output of dev/test teams, in particular in distributed architectures based on SOA and using an ESB. Our focus is mostly based on external constraints such as back-end systems that our system under test depends on. Those constraints include limited availability, unfinished development, limited capacity (to support load tests for example) or inappropriate (test) data.
The core technology to remove these constraints is Service Virtualization. Service Virtualization provides testers unconstrained access to dependent applications via test environments that are easily configured for distinct testing requirements. Rather than to attempt to replace the complete functionality of the dependent applications, Service Virtualization focuses on simulating only the portion of the dependent applications behavior that is necessary for completion of development and testing tasks. So, for each (web) services used by the system under test, we have a virtual (simulated) service available that can be used for testing.
We will demonstrate how scheduled manipulation with back-end data, performance and functional parameters can be implemented. Manual work needs to be done only a single time, to model the behavior of a service and to define the performance profiles relevant for the tests. Then, once the instances of Virtualized Services are prepared (functionality) and are defined how they should behave (performance), deployment and provisioning can be scheduled, and therefore included as part of automated process.
These methods have a significant value for the continuous integration of service-oriented systems. To show that extended functionality can be obtained, an open-source tool is used which implements scheduled automatic deployment of virtualized services by building dynamic infrastructure through SOAP interface. Additionally, through the web service API, we demonstrate how dynamic changes of back-end data, performance and functionality of each service can be easily done.
It is certain that high degree of automation is needed for obtaining consistent and reliable staged environments required by today’s agile testing. During the presentation, we will discuss if this automation approach can move organizations from reactive or proactive to managed or even optimized level of maturity.