On-Demand Test Environments: Solving the Problems of Traditional Testing Environments.
Senior Content Marketer & Technical Expert
Senior DevOps Engineer
Tech Leaders Corner
Quality Assurance (QA)
Testing is sometimes viewed in an ambivalent way within the business community, quite wrongly so
The developers often test the software on their own PCs, or testing environments are shared among several different projects.
The primary problem with test environments is that it takes a lot of time and money to set them up and keep them running properly.
The Solution to the Problem are Ephemeral (short-lived) on-demand test environments
Vagrant is a tool that allows you to create on-demand environments.
Testing is viewed in an ambivalent way within the business community. Because of the popular notion that the practice does not bring anything of value to the organization, it is typically overlooked. There is no financial evidence to support this belief. As a direct consequence of this, testing is frequently avoided. The developers test the software on their own PCs, or the testing environment is shared among several different projects.
Sometimes we will ask our clients to create more environments for us, so that we can test more often, and iterate quicker, which is sometimes frowned upon. We often hear that it's both difficult and costly to do so. Luckily, the new on-demand test environments are there for us.
Why are test environments problematic? Why test?
Testing is an important part of making software. It helps us find problems quickly, before they become considerably larger issues that render the product less stable or difficult to use. The main problem with test environments is that they take a lot of time and money to set up and keep running well. Also, if developers use the same environments for more than one project, it can be hard to keep track of who has access when. This could be frustrating for developers whose work is slowed down by coworkers who first send their code to testing. If there is a bottleneck, we could quickly run into a major issue of either slowing everybody down, lowering the quality of software, or both.
Let's look at what it is to better understand why just making more permanent environments isn't the answer, and instead why on-demand is the way to go.
What is an environment, and what is a test environment?
An environment is, simply put, a server. Sometimes, a set of co-processors will be available under the same configuration to run multiple tasks at the same time.
A test environment is like a dress rehearsal for the real thing. It’s a place where software, hardware, or technology can be tested in an isolated and controlled setting before releasing it to the public. Just like actors practice their lines on stage with props during dress rehearsals before performing in front of a live audience, software engineers use test environments to make sure everything works properly before launching it into production.
When a software development team works on a product, there are at the very least four different sets of servers, or, in other words, four different environments.
The best-case scenario is having a few of testing environments at once, just so that when one person uploads tests, this action does not block another person from doing the same. As you can predict, costs rise rapidly—we are essentially multiplying the costs X number of times.
The Solution to the Problem
Ephemeral (short-lived) on-demand test environments are the solution to the problem. They solve the issue of costly server uptime and allow for more frequent testing. We won’t block the work of others anymore. They are configured to run in the Continuous Integration pipeline – every time we create a new PR, we know whether the changes we introduce broke the app!
The tests are carried out in an environment that has already been configured, and this environment is deleted as soon as the tests are finished. We can go back and look at the results whenever we choose, for as long as we require them to be stored.
They are available to us whenever we require them but vanish as soon as our requirements have been satisfied. Tests can be done quickly and effectively using on-demand testing, which eliminates the need to keep up with an ongoing testing environment.
Superior in Quality
Because we can test more regularly, we will discover any problems much earlier. We won't have to waste time waiting around for a distinct setting to become accessible for our use.
Perhaps the most important part – we don’t block anybody’s work for them. Multiple developers may test at the same time, and nothing is going to happen. There will be no more frustrations or messages exchanged.
Vagrant – The Tool for the Job
Vagrant is a tool that allows you to create on-demand environments. You can use it to configure your test environment just as you want it, and then upload the code into the virtual machine running in this environment.
What is Vagrant? How Does Vagrant Work?
Vagrant is like a virtual machine for developers.
It provides an easy way to create and configure lightweight, reproducible, and portable development environments. Just like a regular machine, you can install applications onto it, but instead of being physical hardware stored somewhere in your home or office, this one exists virtually on your computer. It's perfect for testing out changes before they are applied to a live server.
You create a Vagrantfile and you put the configuration for your test environment in there, including the software that is going to be installed.
Then you run the command to run everything, which will create a virtual machine instance with all the preconfigured settings and install all the required software.
It is important to differentiate Vagrant from another HashiCorp product, Packer. Running Vagrant means everything that is defined in config file must be installed during deployment. Packer on the other hand is an app that creates an image of a Virtual Machine that contains the exact state of a computer. This means Vagrant files are very lightweight, especially compared to images, but the installation might be time-consuming and might require constant internet access.
Some have speculated that the solution's higher price tag stems from the fact that it requires the hiring of a full-time DevOps Engineer.
That line of thinking obviously doesn't account for the future. Even if the task at hand is somewhat minor, you will almost certainly need one. A trained DevOps expert will have the on-demand solution ready in no time for programmers and testers to use.
Then, when an engineer is satisfied with the results of their adjustments and is ready to implement them, an automated tester will check to see if everything went smoothly, and if not, it will determine what went wrong and how to fix it.
The configuration outlined below allows for rapid detection and resolution of any issues that may develop during the delivery of projects.
The first step to setting up an on-demand test environment is to create a Vagrantfile, which defines the virtual machine configuration as well as any provisioning scripts that need to be run when the VM is created. This can include things like installing required packages, downloading, and running tests, etc.
2. Configure Continuous Integration (CI) Pipeline
Once you have your Vagrantfile set up, it's time to configure your CI pipeline for managing the on-demand test environments. Although specific steps depend on what platform you're using for CI (e.g., Azure Pipelines, Jenkins or TravisCI), the main principle remains the same. Vagrantfile is a list of requirements and instructions for a Virtual Machine (VM), that is provider-agnostic. This means the pipeline must define the destination and run the Vagrant, thus deploying the VM.
3. Provision and Run Tests within Test Environments
Once the CI job has triggered successfully and built out an instance of your VM with all necessary software installed via provisioners defined in step 1, it's then time to execute any tests against those environments before proceeding with other stages of deployment
In short, testing is an essential part of the process of developing software, but it is often overlooked due to the popular notion that it does not bring value to the organization.
On-demand test environments are problematic due to the time and money it takes to set them up and keep them running properly, and the difficulty of organizing who has access when.
An environment is a server that can run multiple tasks at the same time, and when a software development team works on a product, there are at least four different sets of servers. Ephemeral (short-lived) on-demand test environments are the solution to the problem of costly server uptime and more frequent testing. They are short-lived and run in a pre-configured environment that disappears right after the tests are done.
360° IT Check is a weekly publication where we bring you the latest and greatest in the world of tech. We cover topics like emerging technologies & frameworks, news about innovative startups, and other topics which affect the world of tech directly or indirectly.