Share via

How to: Run Automated Tests on Multiple Computers at the Same Time From a Test Plan

  • Article

If you have many automated tests to run from your test plan using Microsoft Test Manager, you might want the automated tests in your test plan to run at the same time on multiple computers. This way the test run can take less time to complete. When you run automated tests from a test plan, the test controller for the environment that you use manages a set of test agents to run tests, as shown in the following illustration. To do this, you must set up your environment to include multiple machines for the role to run your tests, for example, a Client role. Additionally, you must use this environment in your test settings and then select the Client role to run the tests. This is shown in the following illustration where the Client role runs the tests. For more information about environments and test settings, see Setting Up Test Machines to Run Tests or Collect Data.

Test Controller and Test Agents

The test controller communicates with test agents to start tests, stop tests, track test agent status, and collect test results. When the test controller receives the tests for a test run, if there are more than 100 tests the test controller divides the tests into groups ready to be sent to a test agent machine. Each of these groups is called a bucket. The default bucket size is set to 100. To what agent the buckets are sent is based on the number of available machines in the role to run tests that was selected in the test settings. For example, if you have a test run of 250 tests, the test controller divides these into three buckets of tests with sizes of 100, 100 and 50. If there are three available test agents, the first bucket of 100 tests will be sent to agent1, the next bucket of 100 tests will be sent to agent2, and the remaining bucket of 50 tests will be sent to agent3.

Note

You cannot change the default bucket size when you are running tests from a test plan.

When the test agent notifies the test controller that it has completed the bucket of tests, the test controller sends the test agent the next bucket of tests based on the buckets that are remaining. For example, if you have a test run of 600 tests then the test controller divides this into 6 buckets of 100 tests each. If you have three available agents, one bucket of 100 tests will initially be sent to each agent. When an agent has completed these tests, the test controller will send the agent the next bucket of 100 tests from the remaining 3 buckets.

If you have less than 100 tests (the default bucket size) in a test run, this is considered a partial bucket. In this case the test controller divides the tests in the run based on the value of the minimum number of tests per agent. This value is 10. For example, if you have 27 tests in your test run and three available test agents, the first 10 tests will be sent to agent1, the next 10 tests will be sent to agent2, and the remaining 7 tests will be sent to agent3. The test controller also tries to divide the tests in a partial bucket evenly. If you have 33 tests in your test run and three available test agents, each test agent will get 11 tests.

Warning

If you have 10 tests or less for the test run from your test plan, these tests will only be sent to one agent to run the tests even if you have multiple agents available in your environment.

Important

You cannot control which tests run on a specific machine in your environment for a test run. If you need to run some of your automated tests on different test configurations, you cannot do this with one test run. You can create different environment to use for each of the test runs. You can start test runs for multiple environments at the same time and your automated tests will run on the machines for each different environment at the same time.

Run Automated Tests on Multiple Computers at the Same Time

  1. Create an environment including multiple machines in the same role that will be used to run tests. For more information about how to create an environment, see Creating a Physical Environment to Use for Testing or How to: Create an Environment from Virtual Machines or Templates.

  2. Create a test settings that has the same roles as this environment and uses this role with multiple machines as the role to run tests. For more information about how to create test settings, see Create Test Settings for Automated Tests as Part of a Test Plan.

  3. Start the test run using the test settings and the environment that you created.

    You will see that the test agents are now all busy running tests.

See Also

Concepts

Setting Up Test Machines to Run Tests or Collect Data

Creating a Physical Environment to Use for Testing

How to: Create an Environment from Virtual Machines or Templates

Running Automated Tests