Changes to load test functionality in Visual Studio and cloud load testing in Azure DevOps

Azure Test Plans

Web performance and load test functionality in Visual Studio is deprecated. Visual Studio 2019 will be the last version of Visual Studio containing web performance and load test features (more details).

Cloud load testing (CLT) functionality in Azure DevOps is deprecated. The changes to cloud load testing functionality in Azure DevOps affect the following products for the load testing service:

1. Web-based experience for load testing in Azure DevOps (URL-based, HAR file, Apache JMeter and Visual Studio web test).

2. Running a load test in the cloud using Visual Studio (both auto-provisioned agents and self-provisioned agents).

3. Running load tests in a CI/CD pipeline using the load testing tasks (Cloud-based Load Test task, Apache JMeter Test task, and Cloud-based Web Performance Test task).

Timeframe

On-premises load testing with Visual Studio and Test Controller/Test Agent

Visual Studio 2019 will be the last version of Visual Studio with the web performance and load test capability. Visual Studio 2019 is also the last release for Test Controller and Test Agent (installed through the 'Agents for Visual Studio SKU') for setting up a load test rig on-premises.

While no new features will be added, load test in VS 2019 will continue to be supported for any issues that may arise during the support lifecycle of the product. See Product Lifecycle and Servicing.

Cloud-based load testing service (CLT) availability timeframe for:

1. Existing customers of CLT:

   • For existing Azure DevOps customers using cloud-load testing functionality in the Azure DevOps portal, this feature currently continues to be available and will be officially deprecated on April 29, 2021.

   • For Visual Studio users leveraging cloud-based load testing, this feature is no longer supported effective March 31, 2020. Visual Studio Enterprise customers can, however, continue to run load tests on-premises using Test Controller/Test Agent and will be supported for any issues that may arise during the support lifecycle of the Visual Studio version.

2. New customers of CLT:

   Azure DevOps organizations and Azure subscriptions with no prior usage of cloud-based load testing functionality will not have access to this feature effective October 12, 2020.

Alternatives

For customers who prefer to continue using a self-service model for load test and cloud-load testing, there are a number of free and commercial tools available.

For example, Apache JMeter is a free, popular open-source tool with a strong community backing. It supports many different protocols and has rich extensibility that can be leveraged to customize the tool to your needs. In addition, if you have been using Apache JMeter to run load tests using the cloud-based load testing service, you will be able to use these load tests with alternatives that support them.

The following table outlines how the concepts in Visual Studio load test map to those in Apache JMeter.

Many commercial services such as Blazemeter support running Apache JMeter tests.

If you use code-based tests for load testing and .NET is your platform of choice then tools such as Neoload, Micro Focus Silk Performer, and Micro Focus Load Runner, and RadView WebLOAD are options to consider.

Other open-source alternatives that support code-based tests are Artillery, Gatling, k6, and Locust.

In addition, extensions from several load test vendors such as SOASTA (now Akamai CloudTest), Apica Loadtest, and k6 are available in the Azure DevOps and Azure Marketplace.

Download load tests and results

If you are using URL, HAR, or Visual Studio webtest-based load tests, you can download the tests by using the Export test to Visual Studio option and use the test through Visual Studio during the timeframe outlined above.

If you are using the cloud-based load testing service and want to access to results of any existing runs after the service goes offline, you can download the reports:

• If you use Visual Studio load tests, URL-based load tests, or HAR-based load tests, download the reports in Visual Studio Enterprise for offline viewing.

• If you use Apache JMeter, download the results file (.csv) for offline viewing using the reporters in JMeter.

Visual Studio web performance test (.webtest file)

Visual Studio web performance test (.webtest file) is tied to the load test functionality and is deprecated. Some customers have used .webtest for other purposes such as running API tests, even though it was not designed for that purpose. Many API testing alternatives are available in the market. SOAP UI is a free, open-source alternative to consider, and is also available as a commercial option with additional capabilities.

Deprecation FAQ

Q: What will happen to my pipelines dependent on Load Test Tasks after deprecation?

A: After April 29, 2021, you will no longer be able to run any new load tests through the Load Test Portal. Pipelines dependent on Cloud-based Load Test tasks, Cloud-based Web Performance Test tasks, and Cloud-based Apache JMeter Load Test tasks will experience errors.

Q: If I need support in migrating my tests to another alternative, where can I go?

Deprecating features is always hard and therefore not something we ever do lightly. We recognize that this announcement will be disruptive for the people that are using the service today and we will be working with affected customers to ensure they are aware of the changes. Premier support for Enterprise can be engaged for help with migrating tests to alternatives – they can be reached via email on [email protected].