Edit

Share via

CLI (v2) schedule YAML schema for model monitoring (preview)

  • Article

APPLIES TO: Azure CLI ml extension v2 (current)

The YAML syntax detailed in this document is based on the JSON schema for the latest version of the ML CLI v2 extension. This syntax is guaranteed only to work with the latest version of the ML CLI v2 extension. The comprehensive JSON schema can be viewed at https://azuremlschemas.azureedge.net/latest/monitorSchedule.schema.json. You can find the schemas for older extension versions at https://azuremlschemasprod.azureedge.net/.

YAML syntax

Key Type Description Allowed values
$schema string The YAML schema.
name string Required. Name of the schedule.
description string Description of the schedule.
tags object Dictionary of tags for the schedule.
trigger object Required. The trigger configuration to define rule when to trigger job. One of RecurrenceTrigger or CronTrigger is required.
create_monitor object Required. The definition of the monitor that will be triggered by a schedule. MonitorDefinition is required.

Trigger configuration

Recurrence trigger

Key Type Description Allowed values
type string Required. Specifies the schedule type. recurrence
frequency string Required. Specifies the unit of time that describes how often the schedule fires. minute, hour, day, week, month
interval integer Required. Specifies the interval at which the schedule fires.
start_time string Describes the start date and time with timezone. If start_time is omitted, the first job will run instantly and the future jobs will be triggered based on the schedule, saying start_time will be equal to the job created time. If the start time is in the past, the first job will run at the next calculated run time.
end_time string Describes the end date and time with timezone. If end_time is omitted, the schedule will continue to run until it's explicitly disabled.
timezone string Specifies the time zone of the recurrence. If omitted, by default is UTC. See appendix for timezone values
pattern object Specifies the pattern of the recurrence. If pattern is omitted, the job(s) will be triggered according to the logic of start_time, frequency and interval.

Recurrence schedule

Recurrence schedule defines the recurrence pattern, containing hours, minutes, and weekdays.

  • When frequency is day, pattern can specify hours and minutes.
  • When frequency is week and month, pattern can specify hours, minutes and weekdays.
Key Type Allowed values
hours integer or array of integer 0-23
minutes integer or array of integer 0-59
week_days string or array of string monday, tuesday, wednesday, thursday, friday, saturday, sunday

CronTrigger

Key Type Description Allowed values
type string Required. Specifies the schedule type. cron
expression string Required. Specifies the cron expression to define how to trigger jobs. expression uses standard crontab expression to express a recurring schedule. A single expression is composed of five space-delimited fields:MINUTES HOURS DAYS MONTHS DAYS-OF-WEEK
start_time string Describes the start date and time with timezone. If start_time is omitted, the first job will run instantly and the future jobs will be triggered based on the schedule, saying start_time will be equal to the job created time. If the start time is in the past, the first job will run at the next calculated run time.
end_time string Describes the end date and time with timezone. If end_time is omitted, the schedule will continue to run until it's explicitly disabled.
timezone string Specifies the time zone of the recurrence. If omitted, by default is UTC. See appendix for timezone values

Monitor definition

Key Type Description Allowed values Default value
compute Object Required. Description of compute resources for Spark pool to run monitoring job.
compute.instance_type String Required. The compute instance type to be used for Spark pool. 'standard_e4s_v3', 'standard_e8s_v3', 'standard_e16s_v3', 'standard_e32s_v3', 'standard_e64s_v3' n/a
compute.runtime_version String Optional. Defines Spark runtime version. 3.3 3.3
monitoring_target Object Azure Machine Learning asset(s) associated with model monitoring.
monitoring_target.ml_task String Machine learning task for the model. Allowed values are: classification, regression, question_answering
monitoring_target.endpoint_deployment_id String Optional. The associated Azure Machine Learning endpoint/deployment ID in format of azureml:myEndpointName:myDeploymentName. This field is required if your endpoint/deployment has enabled model data collection to be used for model monitoring.
monitoring_target.model_id String Optional. The associated model ID for model monitoring.
monitoring_signals Object Dictionary of monitoring signals to be included. The key is a name for monitoring signal within the context of monitor and the value is an object containing a monitoring signal specification. Optional for basic model monitoring that uses recent past production data as comparison baseline and has 3 monitoring signals: data drift, prediction drift, and data quality.
alert_notification String or Object Description of alert notification recipients. One of two alert destinations is allowed: String azmonitoring or Object emails containing an array of email recipients
alert_notification.emails Object List of email addresses to receive alert notification.

Monitoring signals

Data drift

As the data used to train the model evolves in production, the distribution of the data can shift, resulting in a mismatch between the training data and the real-world data that the model is being used to predict. Data drift is a phenomenon that occurs in machine learning when the statistical properties of the input data used to train the model change over time.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here. data_drift data_drift
production_data Object Optional. Description of production data to be analyzed for monitoring signal.
production_data.input_data Object Optional. Description of input data source, see job input data specification.
production_data.data_context String The context of data, it refers model production data and could be model inputs or model outputs model_inputs
production_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify production_data.data_window.lookback_window_offset and production_data.data_window.lookback_window_size properties. For using fixed data windows, please specify production_data.data_window.window_start and production_data.data_window.window_end properties. All property values must be in ISO8601 format.
production_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if production_data.data.input_data.type is uri_folder, see preprocessing component specification.
reference_data Object Optional. Recent past production data is used as comparison baseline data if this isn't specified. Recommendation is to use training data as comparison baseline.
reference_data.input_data Object Description of input data source, see job input data specification.
reference_data.data_context String The context of data, it refers to the context that dataset was used before model_inputs, training, test, validation
reference_data.data_column_names.target_column Object Optional. If the reference_data is training data, this property is required for monitoring top N features for data drift.
reference_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify reference_data.data_window.lookback_window_offset and reference_data.data_window.lookback_window_size properties. For using fixed data windows, please specify reference_data.data_window.window_start and reference_data.data_window.window_end properties. All property values must be in ISO8601 format.
reference_data_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if reference_data.input_data.type is uri_folder, see preprocessing component specification.
features Object Optional. Target features to be monitored for data drift. Some models might have hundreds or thousands of features, it's always recommended to specify interested features for monitoring. One of following values: list of feature names, features.top_n_feature_importance, or all_features Default features.top_n_feature_importance = 10 if production_data.data_context is training, otherwise, default is all_features
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds Object List of metrics and thresholds properties for the monitoring signal. When threshold is exceeded and alert_enabled is true, user will receive alert notification.
metric_thresholds.numerical Object Optional. List of metrics and thresholds in key:value format, key is the metric name, value is the threshold. Allowed numerical metric names: jensen_shannon_distance, normalized_wasserstein_distance, population_stability_index, two_sample_kolmogorov_smirnov_test
metric_thresholds.categorical Object Optional. List of metrics and thresholds in 'key:value' format, 'key' is the metric name, 'value' is the threshold. Allowed categorical metric names: jensen_shannon_distance, chi_squared_test, population_stability_index

Prediction drift

Prediction drift tracks changes in the distribution of a model's prediction outputs by comparing it to validation or test labeled data or recent past production data.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here. prediction_drift prediction_drift
production_data Object Optional. Description of production data to be analyzed for monitoring signal.
production_data.input_data Object Optional. Description of input data source, see job input data specification.
production_data.data_context String The context of data, it refers model production data and could be model inputs or model outputs model_outputs
production_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify production_data.data_window.lookback_window_offset and production_data.data_window.lookback_window_size properties. For using fixed data windows, please specify production_data.data_window.window_start and production_data.data_window.window_end properties. All property values must be in ISO8601 format.
production_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if production_data.data.input_data.type is uri_folder. For more information on preprocessing component specification, see preprocessing component specification.
reference_data Object Optional. Recent past production data is used as comparison baseline data if this isn't specified. Recommendation is to use training data as comparison baseline.
reference_data.input_data Object Description of input data source, see job input data specification.
reference_data.data_context String The context of data, it refers to the context that dataset was used before model_inputs, training, test, validation
reference_data.data_column_names.target_column Object Optional. If the 'reference_data' is training data, this property is required for monitoring top N features for data drift.
reference_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify reference_data.data_window.lookback_window_offset and reference_data.data_window.lookback_window_size properties. For using fixed data windows, please specify reference_data.data_window.window_start and reference_data.data_window.window_end properties. All property values must be in ISO8601 format.
reference_data_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if reference_data.input_data.type is uri_folder, see preprocessing component specification.
features Object Optional. Target features to be monitored for data drift. Some models might have hundreds or thousands of features, it's always recommended to specify interested features for monitoring. One of following values: list of feature names, features.top_n_feature_importance, or all_features Default features.top_n_feature_importance = 10 if production_data.data_context is training, otherwise, default is all_features
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds Object List of metrics and thresholds properties for the monitoring signal. When threshold is exceeded and alert_enabled is true, user will receive alert notification.
metric_thresholds.numerical Object Optional. List of metrics and thresholds in 'key:value' format, 'key' is the metric name, 'value' is the threshold. Allowed numerical metric names: jensen_shannon_distance, normalized_wasserstein_distance, population_stability_index, two_sample_kolmogorov_smirnov_test
metric_thresholds.categorical Object Optional. List of metrics and thresholds in 'key:value' format, 'key' is the metric name, 'value' is the threshold. Allowed categorical metric names: jensen_shannon_distance, chi_squared_test, population_stability_index

Data quality

Data quality signal tracks data quality issues in production by comparing to training data or recent past production data.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here data_quality data_quality
production_data Object Optional. Description of production data to be analyzed for monitoring signal.
production_data.input_data Object Optional. Description of input data source, see job input data specification.
production_data.data_context String The context of data, it refers model production data and could be model inputs or model outputs model_inputs, model_outputs
production_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify production_data.data_window.lookback_window_offset and production_data.data_window.lookback_window_size properties. For using fixed data windows, please specify production_data.data_window.window_start and production_data.data_window.window_end properties. All property values must be in ISO8601 format.
production_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if production_data.input_data.type is uri_folder, see preprocessing component specification.
reference_data Object Optional. Recent past production data is used as comparison baseline data if this isn't specified. Recommendation is to use training data as comparison baseline.
reference_data.input_data Object Description of input data source, see job input data specification.
reference_data.data_context String The context of data, it refers to the context that dataset was used before model_inputs, model_outputs, training, test, validation
reference_data.data_column_names.target_column Object Optional. If the 'reference_data' is training data, this property is required for monitoring top N features for data drift.
reference_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify reference_data.data_window.lookback_window_offset and reference_data.data_window.lookback_window_size properties. For using fixed data windows, please specify reference_data.data_window.window_start and reference_data.data_window.window_end properties. All property values must be in ISO8601 format.
reference_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if reference_data.input_data.type is uri_folder, see preprocessing component specification.
features Object Optional. Target features to be monitored for data quality. Some models might have hundreds or thousands of features. It's always recommended to specify interested features for monitoring. One of following values: list of feature names, features.top_n_feature_importance, or all_features Default to features.top_n_feature_importance = 10 if reference_data.data_context is training, otherwise default is all_features
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds Object List of metrics and thresholds properties for the monitoring signal. When threshold is exceeded and alert_enabled is true, user will receive alert notification.
metric_thresholds.numerical Object Optional List of metrics and thresholds in key:value format, key is the metric name, value is the threshold. Allowed numerical metric names: data_type_error_rate, null_value_rate, out_of_bounds_rate
metric_thresholds.categorical Object Optional List of metrics and thresholds in key:value format, key is the metric name, value is the threshold. Allowed categorical metric names: data_type_error_rate, null_value_rate, out_of_bounds_rate

Feature attribution drift (preview)

The feature attribution of a model may change over time due to changes in the distribution of data, changes in the relationships between features, or changes in the underlying problem being solved. Feature attribution drift is a phenomenon that occurs in machine learning models when the importance or contribution of features to the prediction output changes over time.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here feature_attribution_drift feature_attribution_drift
production_data Array Optional, default to collected data associated with Azure Machine Learning endpoint if this is not provided. The production_data is a list of dataset and its associated meta data, it must include both model inputs and model outputs data. It could be a single dataset with both model inputs and outputs, or it could be two separate datasets containing one model inputs and one model outputs.
production_data.input_data Object Optional. Description of input data source, see job input data specification.
production_data.input_data.data_column_names Object Correlation column name and prediction column names in key:value format, needed for data joining. Allowed keys are: correlation_id, target_column
production_data.data_context String The context of data. It refers to production model inputs data. model_inputs, model_outputs, model_inputs_outputs
production_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify production_data.data_window.lookback_window_offset and production_data.data_window.lookback_window_size properties. For using fixed data windows, please specify production_data.data_window.window_start and production_data.data_window.window_end properties. All property values must be in ISO8601 format.
production_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if production_data.input_data.type is uri_folder, see preprocessing component specification.
production_data.data_window_size String Optional. Data window size in days with ISO8601 format, for example P7D. This is the production data window to be computed for data quality issues. By default the data window size is the last monitoring period.
reference_data Object Optional. Recent past production data is used as comparison baseline data if this isn't specified. Recommendation is to use training data as comparison baseline.
reference_data.input_data Object Description of input data source, see job input data specification.
reference_data.data_context String The context of data, it refers to the context that dataset was used before. Fro feature attribution drift, only training data allowed. training
reference_data.data_column_names.target_column String Required.
reference_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify reference_data.data_window.lookback_window_offset and reference_data.data_window.lookback_window_size properties. For using fixed data windows, please specify reference_data.data_window.window_start and reference_data.data_window.window_end properties. All property values must be in ISO8601 format.
reference_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if reference_data.input_data.type is uri_folder, see preprocessing component specification.
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds Object Metric name and threshold for feature attribution drift in key:value format, where key is the metric name, and value is the threshold. When threshold is exceeded and alert_enabled is on, user will receive alert notification. Allowed metric name: normalized_discounted_cumulative_gain

Custom monitoring signal

Custom monitoring signal through a custom Azure Machine Learning component.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here. custom custom
component_id String Required. The Azure Machine Learning component ID corresponding to your custom signal. For example azureml:mycustomcomponent:1
input_data Object Optional. Description of the input data to be analyzed by the monitoring signal, see job input data specification.
input_data.<data_name>.data_context String The context of data, it refers model production data and could be model inputs or model outputs model_inputs
input_data.<data_name>.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify input_data.<data_name>.data_window.lookback_window_offset and input_data.<data_name>.data_window.lookback_window_size properties. For using fixed data windows, please specify input_data.<data_name>.data_window.window_start and input_data.<data_name>.data_window.window_end properties. All property values must be in ISO8601 format.
input_data.<data_name>.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if input_data.<data_name>.input_data.type is uri_folder, see preprocessing component specification.
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds.metric_name Object Name of the custom metric.
threshold Object Acceptable threshold for the custom metric.

Model performance (preview)

Model performance tracks the objective performance of a model's output in production by comparing it to collected ground truth data.

Key Type Description Allowed values Default value
type String Required. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the type specified here model_performance model_performance
production_data Array Optional, default to collected data associated with Azure Machine Learning endpoint if this is not provided. The production_data is a list of dataset and its associated meta data, it must include both model inputs and model outputs data. It could be a single dataset with both model inputs and outputs, or it could be two separate datasets containing one model inputs and one model outputs.
production_data.input_data Object Optional. Description of input data source, see job input data specification.
production_data.input_data.data_column_names Object Correlation column name and prediction column names in key:value format, needed for data joining. Allowed keys are: correlation_id, target_column
production_data.data_context String The context of data. It refers to production model inputs data. model_inputs, model_outputs, model_inputs_outputs
production_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify production_data.data_window.lookback_window_offset and production_data.data_window.lookback_window_size properties. For using fixed data windows, please specify production_data.data_window.window_start and production_data.data_window.window_end properties. All property values must be in ISO8601 format.
production_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if production_data.input_data.type is uri_folder, see preprocessing component specification.
production_data.data_window_size String Optional. Data window size in days with ISO8601 format, for example P7D. This is the production data window to be computed for data quality issues. By default the data window size is the last monitoring period.
reference_data Object Optional. Recent past production data is used as comparison baseline data if this isn't specified. Recommendation is to use training data as comparison baseline.
reference_data.input_data Object Description of input data source, see job input data specification.
reference_data.data_context String The context of data, it refers to the context that dataset was used before. Fro feature attribution drift, only training data allowed. training
reference_data.data_column_names.target_column String Required.
reference_data.data_window Object Optional. Data window of the reference data to be used as comparison baseline data. Allow either rolling data window or fixed data window only. For using rolling data window, please specify reference_data.data_window.lookback_window_offset and reference_data.data_window.lookback_window_size properties. For using fixed data windows, please specify reference_data.data_window.window_start and reference_data.data_window.window_end properties. All property values must be in ISO8601 format.
reference_data.pre_processing_component String Component ID in the format of azureml:myPreprocessing@latest for a registered component. This is required if reference_data.input_data.type is uri_folder, see preprocessing component specification.
alert_enabled Boolean Turn on/off alert notification for the monitoring signal. True or False
metric_thresholds.classification Object Optional List of metrics and thresholds in key:value format, key is the metric name, value is the threshold. Allowed classification metric names: accuracy, precision, recall
metric_thresholds.regression Object Optional List of metrics and thresholds in key:value format, key is the metric name, value is the threshold. Allowed regression metric names: mae, mse, rmse

Remarks

The az ml schedule command can be used for managing Azure Machine Learning models.

Examples

Monitoring CLI examples are available in the examples GitHub repository. A couple are as follows:

YAML: Out-of-box monitor

APPLIES TO: Azure CLI ml extension v2 (current)

# out-of-box-monitoring.yaml
$schema:  http://azureml/sdk-2-0/Schedule.json
name: credit_default_model_monitoring
display_name: Credit default model monitoring
description: Credit default model monitoring setup with minimal configurations

trigger:
  # perform model monitoring activity daily at 3:15am
  type: recurrence
  frequency: day #can be minute, hour, day, week, month
  interval: 1 # #every day
  schedule: 
    hours: 3 # at 3am
    minutes: 15 # at 15 mins after 3am

create_monitor:

  compute: # specify a spark compute for monitoring job
    instance_type: standard_e4s_v3
    runtime_version: "3.3"

  monitoring_target: 
    ml_task: classification # model task type: [classification, regression, question_answering]
    endpoint_deployment_id: azureml:credit-default:main # azureml endpoint deployment id

  alert_notification: # emails to get alerts
    emails:
      - [email protected]
      - [email protected]

YAML: Advanced monitor

APPLIES TO: Azure CLI ml extension v2 (current)

# advanced-model-monitoring.yaml
$schema:  http://azureml/sdk-2-0/Schedule.json
name: fraud_detection_model_monitoring
display_name: Fraud detection model monitoring
description: Fraud detection model monitoring with advanced configurations

trigger:
  # perform model monitoring activity daily at 3:15am
  type: recurrence
  frequency: day #can be minute, hour, day, week, month
  interval: 1 # #every day
  schedule: 
    hours: 3 # at 3am
    minutes: 15 # at 15 mins after 3am

create_monitor:

  compute: 
    instance_type: standard_e4s_v3
    runtime_version: "3.3"

  monitoring_target:
    ml_task: classification
    endpoint_deployment_id: azureml:credit-default:main
  
  monitoring_signals:
    advanced_data_drift: # monitoring signal name, any user defined name works
      type: data_drift
      # reference_dataset is optional. By default referece_dataset is the production inference data associated with Azure Machine Learning online endpoint
      reference_data:
        input_data:
          path: azureml:credit-reference:1 # use training data as comparison reference dataset
          type: mltable
        data_context: training
        data_column_names:
          target_column: DEFAULT_NEXT_MONTH
      features: 
        top_n_feature_importance: 10 # monitor drift for top 10 features
      metric_thresholds:
        numerical:
          jensen_shannon_distance: 0.01
        categorical:
          pearsons_chi_squared_test: 0.02
    advanced_data_quality:
      type: data_quality
      # reference_dataset is optional. By default reference_dataset is the production inference data associated with Azure Machine Learning online endpoint
      reference_data:
        input_data:
          path: azureml:credit-reference:1
          type: mltable
        data_context: training
      features: # monitor data quality for 3 individual features only
        - SEX
        - EDUCATION
      metric_thresholds:
        numerical:
          null_value_rate: 0.05
        categorical:
          out_of_bounds_rate: 0.03

    feature_attribution_drift_signal:
      type: feature_attribution_drift
      # production_data: is not required input here
      # Please ensure Azure Machine Learning online endpoint is enabled to collected both model_inputs and model_outputs data
      # Azure Machine Learning model monitoring will automatically join both model_inputs and model_outputs data and used it for computation
      reference_data:
        input_data:
          path: azureml:credit-reference:1
          type: mltable
        data_context: training
        data_column_names:
          target_column: DEFAULT_NEXT_MONTH
      metric_thresholds:
        normalized_discounted_cumulative_gain: 0.9
  
  alert_notification:
    emails:
      - [email protected]
      - [email protected]

Appendix

Timezone

Current schedule supports the following timezones. The key can be used directly in the Python SDK, while the value can be used in the YAML job. The table is organized by UTC(Coordinated Universal Time).

UTC Key Value
UTC -12:00 DATELINE_STANDARD_TIME "Dateline Standard Time"
UTC -11:00 UTC_11 "UTC-11"
UTC - 10:00 ALEUTIAN_STANDARD_TIME Aleutian Standard Time
UTC - 10:00 HAWAIIAN_STANDARD_TIME "Hawaiian Standard Time"
UTC -09:30 MARQUESAS_STANDARD_TIME "Marquesas Standard Time"
UTC -09:00 ALASKAN_STANDARD_TIME "Alaskan Standard Time"
UTC -09:00 UTC_09 "UTC-09"
UTC -08:00 PACIFIC_STANDARD_TIME_MEXICO "Pacific Standard Time (Mexico)"
UTC -08:00 UTC_08 "UTC-08"
UTC -08:00 PACIFIC_STANDARD_TIME "Pacific Standard Time"
UTC -07:00 US_MOUNTAIN_STANDARD_TIME "US Mountain Standard Time"
UTC -07:00 MOUNTAIN_STANDARD_TIME_MEXICO "Mountain Standard Time (Mexico)"
UTC -07:00 MOUNTAIN_STANDARD_TIME "Mountain Standard Time"
UTC -06:00 CENTRAL_AMERICA_STANDARD_TIME "Central America Standard Time"
UTC -06:00 CENTRAL_STANDARD_TIME "Central Standard Time"
UTC -06:00 EASTER_ISLAND_STANDARD_TIME "Easter Island Standard Time"
UTC -06:00 CENTRAL_STANDARD_TIME_MEXICO "Central Standard Time (Mexico)"
UTC -06:00 CANADA_CENTRAL_STANDARD_TIME "Canada Central Standard Time"
UTC -05:00 SA_PACIFIC_STANDARD_TIME "SA Pacific Standard Time"
UTC -05:00 EASTERN_STANDARD_TIME_MEXICO "Eastern Standard Time (Mexico)"
UTC -05:00 EASTERN_STANDARD_TIME "Eastern Standard Time"
UTC -05:00 HAITI_STANDARD_TIME "Haiti Standard Time"
UTC -05:00 CUBA_STANDARD_TIME "Cuba Standard Time"
UTC -05:00 US_EASTERN_STANDARD_TIME "US Eastern Standard Time"
UTC -05:00 TURKS_AND_CAICOS_STANDARD_TIME "Turks And Caicos Standard Time"
UTC -04:00 PARAGUAY_STANDARD_TIME "Paraguay Standard Time"
UTC -04:00 ATLANTIC_STANDARD_TIME "Atlantic Standard Time"
UTC -04:00 VENEZUELA_STANDARD_TIME "Venezuela Standard Time"
UTC -04:00 CENTRAL_BRAZILIAN_STANDARD_TIME "Central Brazilian Standard Time"
UTC -04:00 SA_WESTERN_STANDARD_TIME "SA Western Standard Time"
UTC -04:00 PACIFIC_SA_STANDARD_TIME "Pacific SA Standard Time"
UTC -03:30 NEWFOUNDLAND_STANDARD_TIME "Newfoundland Standard Time"
UTC -03:00 TOCANTINS_STANDARD_TIME "Tocantins Standard Time"
UTC -03:00 E_SOUTH_AMERICAN_STANDARD_TIME "E. South America Standard Time"
UTC -03:00 SA_EASTERN_STANDARD_TIME "SA Eastern Standard Time"
UTC -03:00 ARGENTINA_STANDARD_TIME "Argentina Standard Time"
UTC -03:00 GREENLAND_STANDARD_TIME "Greenland Standard Time"
UTC -03:00 MONTEVIDEO_STANDARD_TIME "Montevideo Standard Time"
UTC -03:00 SAINT_PIERRE_STANDARD_TIME "Saint Pierre Standard Time"
UTC -03:00 BAHIA_STANDARD_TIM "Bahia Standard Time"
UTC -02:00 UTC_02 "UTC-02"
UTC -02:00 MID_ATLANTIC_STANDARD_TIME "Mid-Atlantic Standard Time"
UTC -01:00 AZORES_STANDARD_TIME "Azores Standard Time"
UTC -01:00 CAPE_VERDE_STANDARD_TIME "Cape Verde Standard Time"
UTC UTC UTC
UTC +00:00 GMT_STANDARD_TIME "GMT Standard Time"
UTC +00:00 GREENWICH_STANDARD_TIME "Greenwich Standard Time"
UTC +01:00 MOROCCO_STANDARD_TIME "Morocco Standard Time"
UTC +01:00 W_EUROPE_STANDARD_TIME "W. Europe Standard Time"
UTC +01:00 CENTRAL_EUROPE_STANDARD_TIME "Central Europe Standard Time"
UTC +01:00 ROMANCE_STANDARD_TIME "Romance Standard Time"
UTC +01:00 CENTRAL_EUROPEAN_STANDARD_TIME "Central European Standard Time"
UTC +01:00 W_CENTRAL_AFRICA_STANDARD_TIME "W. Central Africa Standard Time"
UTC +02:00 NAMIBIA_STANDARD_TIME "Namibia Standard Time"
UTC +02:00 JORDAN_STANDARD_TIME "Jordan Standard Time"
UTC +02:00 GTB_STANDARD_TIME "GTB Standard Time"
UTC +02:00 MIDDLE_EAST_STANDARD_TIME "Middle East Standard Time"
UTC +02:00 EGYPT_STANDARD_TIME "Egypt Standard Time"
UTC +02:00 E_EUROPE_STANDARD_TIME "E. Europe Standard Time"
UTC +02:00 SYRIA_STANDARD_TIME "Syria Standard Time"
UTC +02:00 WEST_BANK_STANDARD_TIME "West Bank Standard Time"
UTC +02:00 SOUTH_AFRICA_STANDARD_TIME "South Africa Standard Time"
UTC +02:00 FLE_STANDARD_TIME "FLE Standard Time"
UTC +02:00 ISRAEL_STANDARD_TIME "Israel Standard Time"
UTC +02:00 KALININGRAD_STANDARD_TIME "Kaliningrad Standard Time"
UTC +02:00 LIBYA_STANDARD_TIME "Libya Standard Time"
UTC +03:00 TÜRKIYE_STANDARD_TIME "Türkiye Standard Time"
UTC +03:00 ARABIC_STANDARD_TIME "Arabic Standard Time"
UTC +03:00 ARAB_STANDARD_TIME "Arab Standard Time"
UTC +03:00 BELARUS_STANDARD_TIME "Belarus Standard Time"
UTC +03:00 RUSSIAN_STANDARD_TIME "Russian Standard Time"
UTC +03:00 E_AFRICA_STANDARD_TIME "E. Africa Standard Time"
UTC +03:30 IRAN_STANDARD_TIME "Iran Standard Time"
UTC +04:00 ARABIAN_STANDARD_TIME "Arabian Standard Time"
UTC +04:00 ASTRAKHAN_STANDARD_TIME "Astrakhan Standard Time"
UTC +04:00 AZERBAIJAN_STANDARD_TIME "Azerbaijan Standard Time"
UTC +04:00 RUSSIA_TIME_ZONE_3 "Russia Time Zone 3"
UTC +04:00 MAURITIUS_STANDARD_TIME "Mauritius Standard Time"
UTC +04:00 GEORGIAN_STANDARD_TIME "Georgian Standard Time"
UTC +04:00 CAUCASUS_STANDARD_TIME "Caucasus Standard Time"
UTC +04:30 AFGHANISTAN_STANDARD_TIME "Afghanistan Standard Time"
UTC +05:00 WEST_ASIA_STANDARD_TIME "West Asia Standard Time"
UTC +05:00 EKATERINBURG_STANDARD_TIME "Ekaterinburg Standard Time"
UTC +05:00 PAKISTAN_STANDARD_TIME "Pakistan Standard Time"
UTC +05:30 INDIA_STANDARD_TIME "India Standard Time"
UTC +05:30 SRI_LANKA_STANDARD_TIME "Sri Lanka Standard Time"
UTC +05:45 NEPAL_STANDARD_TIME "Nepal Standard Time"
UTC +06:00 CENTRAL_ASIA_STANDARD_TIME "Central Asia Standard Time"
UTC +06:00 BANGLADESH_STANDARD_TIME "Bangladesh Standard Time"
UTC +06:30 MYANMAR_STANDARD_TIME "Myanmar Standard Time"
UTC +07:00 N_CENTRAL_ASIA_STANDARD_TIME "N. Central Asia Standard Time"
UTC +07:00 SE_ASIA_STANDARD_TIME "SE Asia Standard Time"
UTC +07:00 ALTAI_STANDARD_TIME "Altai Standard Time"
UTC +07:00 W_MONGOLIA_STANDARD_TIME "W. Mongolia Standard Time"
UTC +07:00 NORTH_ASIA_STANDARD_TIME "North Asia Standard Time"
UTC +07:00 TOMSK_STANDARD_TIME "Tomsk Standard Time"
UTC +08:00 CHINA_STANDARD_TIME "China Standard Time"
UTC +08:00 NORTH_ASIA_EAST_STANDARD_TIME "North Asia East Standard Time"
UTC +08:00 SINGAPORE_STANDARD_TIME "Singapore Standard Time"
UTC +08:00 W_AUSTRALIA_STANDARD_TIME "W. Australia Standard Time"
UTC +08:00 TAIPEI_STANDARD_TIME "Taipei Standard Time"
UTC +08:00 ULAANBAATAR_STANDARD_TIME "Ulaanbaatar Standard Time"
UTC +08:45 AUS_CENTRAL_W_STANDARD_TIME "Aus Central W. Standard Time"
UTC +09:00 NORTH_KOREA_STANDARD_TIME "North Korea Standard Time"
UTC +09:00 TRANSBAIKAL_STANDARD_TIME "Transbaikal Standard Time"
UTC +09:00 TOKYO_STANDARD_TIME "Tokyo Standard Time"
UTC +09:00 KOREA_STANDARD_TIME "Korea Standard Time"
UTC +09:00 YAKUTSK_STANDARD_TIME "Yakutsk Standard Time"
UTC +09:30 CEN_AUSTRALIA_STANDARD_TIME "Cen. Australia Standard Time"
UTC +09:30 AUS_CENTRAL_STANDARD_TIME "AUS Central Standard Time"
UTC +10:00 E_AUSTRALIAN_STANDARD_TIME "E. Australia Standard Time"
UTC +10:00 AUS_EASTERN_STANDARD_TIME "AUS Eastern Standard Time"
UTC +10:00 WEST_PACIFIC_STANDARD_TIME "West Pacific Standard Time"
UTC +10:00 TASMANIA_STANDARD_TIME "Tasmania Standard Time"
UTC +10:00 VLADIVOSTOK_STANDARD_TIME "Vladivostok Standard Time"
UTC +10:30 LORD_HOWE_STANDARD_TIME "Lord Howe Standard Time"
UTC +11:00 BOUGAINVILLE_STANDARD_TIME "Bougainville Standard Time"
UTC +11:00 RUSSIA_TIME_ZONE_10 "Russia Time Zone 10"
UTC +11:00 MAGADAN_STANDARD_TIME "Magadan Standard Time"
UTC +11:00 NORFOLK_STANDARD_TIME "Norfolk Standard Time"
UTC +11:00 SAKHALIN_STANDARD_TIME "Sakhalin Standard Time"
UTC +11:00 CENTRAL_PACIFIC_STANDARD_TIME "Central Pacific Standard Time"
UTC +12:00 RUSSIA_TIME_ZONE_11 "Russia Time Zone 11"
UTC +12:00 NEW_ZEALAND_STANDARD_TIME "New Zealand Standard Time"
UTC +12:00 UTC_12 "UTC+12"
UTC +12:00 FIJI_STANDARD_TIME "Fiji Standard Time"
UTC +12:00 KAMCHATKA_STANDARD_TIME "Kamchatka Standard Time"
UTC +12:45 CHATHAM_ISLANDS_STANDARD_TIME "Chatham Islands Standard Time"
UTC +13:00 TONGA__STANDARD_TIME "Tonga Standard Time"
UTC +13:00 SAMOA_STANDARD_TIME "Samoa Standard Time"
UTC +14:00 LINE_ISLANDS_STANDARD_TIME "Line Islands Standard Time"