MedianStoppingPolicy Class
Defines an early termination policy based on running averages of the primary metric of all runs.
Initialize a MedianStoppingPolicy.
- Inheritance
-
azureml.train.hyperdrive.policy.EarlyTerminationPolicyMedianStoppingPolicy
Constructor
MedianStoppingPolicy(evaluation_interval=1, delay_evaluation=0)Parameters
| Name | Description |
|---|---|
|
evaluation_interval
|
The frequency for applying the policy. Default value: 1
|
|
delay_evaluation
|
The number of intervals for which to delay the first policy evaluation.
If specified, the policy applies every multiple of Default value: 0
|
|
evaluation_interval
Required
|
The frequency for applying the policy. |
|
delay_evaluation
Required
|
The number of intervals for which to delay the first policy evaluation.
If specified, the policy applies every multiple of |
Remarks
The Median Stopping policy computes running averages across all runs and cancels runs whose best performance is worse than the median of the running averages. Specifically, a run will be canceled at interval N if its best primary metric reported up to interval N is worse than the median of the running averages for intervals 1:N across all runs.
The Median Stopping policy takes the following optional configuration parameters:
evaluation_interval: The frequency for applying the policy. Each time the training script logs the primary metric counts as one interval.delay_evaluation: The number of intervals to delay policy evaluation. Use this parameter to avoid premature termination of training runs. If specified, the policy applies every multiple ofevaluation_intervalthat is greater than or equal todelay_evaluation.
This policy is inspired from the research publication Google Vizier: A Service for Black-Box Optimization.
If you are looking for a conservative policy that provides savings without terminating promising jobs,
you can use a Median Stopping Policy with evaluation_interval 1 and delay_evaluation 5. These are
conservative settings, that can provide approximately 25%-35% savings with no loss on primary metric (based
on our evaluation data).
Attributes
delay_evaluation
Return the value for the number of sequences the first evaluation is delayed.
Returns
| Type | Description |
|---|---|
|
The delay evaluation. |
evaluation_interval
POLICY_NAME
POLICY_NAME = 'MedianStopping'