- Add support for additional include in spark component.
- Fix error message while resolving mlflow url in get workspace details
- #3620407 - Fix Datastore credentials show up as NoneCredentials
- #38493 - Fix error NoneType object is not subscriptable
- Added support to select firewall sku to used for provisioning azure firewall when FQDN rules are added in
AllowOnlyApprovedOutbound mode. FirewallSku options are
StandardorBasic, defaults toStandard - Update TLS version from 1.0 to 1.2
- Added support for Distillation jobs. Can be created by importing
disillationfromazure.ai.ml.model_customization - Added Workspace property
ProvisionNetworkNowto trigger the provisioning of the managed VNet with the default Options when creating a Workspace with the managed VNet enabled, or else it does nothing
- Fix send email notification issue in model monitoring
- #37857 - Fix online deployment registry issue
- Cross subscription storage account support for workspace and feature store. Developer can provide a storage account from another subscription while creating a workspace or storage account.
- #35820 - using compute location attribute to fill compute location to align the experience with UI.
- When a workspace is created with
managed_networkenabled or haspublic_network_accessset to disabled, the resources created with the workspace (Key Vault, Storage Account) will be set to have restricted network access settings. This is only applicable when the user does not specify existing resources. - Added support of
fqdnsproperty for managed networkPrivateEndpointDestinationoutbound rule objects. Enabling the support of Application Gateway as a Private Endpoint target in the workspace managed network. - Added support of
address_prefixesproperty for managed networkServiceTagDestinationoutbound rule objects. - Removed experimental tag from
managed_networkwhich is a GA feature.
- Added enable_sso operation under compute operation that will allow user to enable sso setting of a compute instance without any write permission set on compute.
- Workspace update no longer broken for older workspaces due to deprecated tags.
- Support credential-less fileshare datastore
- Expose
public_ip_addressinAmlComputeNodeInfo, to get the public ip address with the ssh port when callingml_client.compute.list_nodes - Uploads to account key access datastores will be authorized via a SAS token retrieved from a call to
DatastoreOperations._list_secrets. Key-based authentication for uploads for such datastores is no longer used. Identity-based datastores will use user identity authentication retrieved from the MLClient. - Support
update_sso_settingsinComputeOperations, to enable or disable single sign-on settings of a compute instance.
- InputTypes exported in constants module
- WorkspaceConnection tags are now listed as deprecated, and the erroneously-deprecated metadata field has been un-deprecated and added as a initialization field. These two fields still point to the same underlying object property, and actual API usage of this value is unchanged.
- Workspace Create operation works without an application insights being provided, and creates a default appIn resource for normal workspaces in that case.
- Project create operations works in general.
- WorkspaceConnections are officially GA'd and no longer experimental. But its much newer subclasses remain experimental.
- Workspace Create operation works without an application insights being provided, and creates a default appIn resource for normal workspaces in that case.
- Project create operations works in general.
- Add experimental support for working with Promptflow evaluators:
ml_client.evaluators. - Many changes to the Connection entity class and its associated operations.
- Workspace Connection
list,get, andcreate_or_updateoperations now include an optionalpopulate_secretsinput, which causes the operations to try making a secondary call to fill in the returned connections' credential info if possible. Only works with api key-based credentials for now. - Many workspace connection subtypes added. The full list of subclasses is now:
AzureBlobStoreConnectionAzureBlobStoreConnectionMicrosoftOneLakeConnectionAzureOpenAIConnectionAzureAIServicesConnectionAzureAISearchConnectionAzureContentSafetyConnectionAzureSpeechServicesConnectionAPIKeyConnectionOpenAIConnectionSerpConnectionServerlessConnection
- Many workspace connections only accept api keys or entra ids for credentials. Since Entra IDs require not inputs, these have been refactored to not required a full credential object. Instead they only accept an api_key as a top-level input, and default to an entra credential otherwise. Their YAML schemas have been similarly altered.
- Client-side credential-type validation added for some workspace connection types.
- Added new credential type:
AadCredentialConfiguration - Renamed WorkspaceHub class as Hub.
- Added Project entity class and YAML support.
- Project and Hub operations supported by workspace operations.
- workspace list operation supports type filtering.
- Add support for Microsoft Entra token (
aad_token) auth ininvokeandget-credentialsoperations. - Add experimental support for working with indexes:
ml_client.indexes
- Removed WorkspaceHubConfig entity, and renamed WorkspaceHub to Hub.
- workspace_hub input of Workspace class hidden, renamed to hub_id, and re-surfaced in child class Project.
- Removed Workspace Hub Operations from ML Client.
- The following classes will still be able to be imported from
azure.ai.ml, but the import is deprecated and emits a warning. Instead, please import them fromazure.ai.ml.entities.AmlTokenConfigurationManagedIdentityConfigurationUserIdentityConfiguration
- The following classes will still be able to be imported from
azure.ai.ml.entities, but the import is deprecated and emits a warning. Instead, please import them fromazure.ai.ml.sweep.ChoiceUniformLogUniformQLogUniformQUniformQLogNormalQNormalLogNormalNormalRandint
- Remove
experimentaltag forml_client.jobs.validate. - Workspace Connection has new read-only subclass: AzureBlobStoreWorkspaceConnectionSchema.
- Workspace Connection supports 2 new types under main class: gen 2 and azure_one_lake.
- Workspace Connection LIST operation can return data connections via new optional flag: include_data_connections.
- Support
ml_client.schedules.trigger(name='my_schedule')function to trigger a schedule once.
- Fix pipeline job
outputsnot load correctly whencomponent: <local-file>exists in pipeline job yaml. - Workspace ListKey operation serialization issue fixed.
- Workspace Diagnose result now can be print in to Json format.
- Support for Python 3.12
- Workspace Connections had 3 child classes added for open AI, cog search, and cog service connections.
- Workspace Connections replaced metadata with tags, and surfaced api_version, api_type, and kind for certain connection types.
- Workspace Hubs now properly create various endpoints, and surface a variable to select the resource they connect to via the 'endpoint_resource_id' kwarg.
- pydash dependency version was upgraded to >=6.0.0 to patch security vulnerability in versions below 6.0.0
- Workspace hub deletion no longer fails if delete_dependent_resources is true.
- Now, when you specify
delete_dependent_resourcesas True when deleting a workspace, the log analytics resource associated with the workspace application insights resource will also be deleted. - Now, when creating or updating a workspace, you can provide a
serverless_computeconfiguration object. This allows configuring a custom subnet in which all Serverless computes will be created. You can also specify whether or not these Serverless computes will have public IP addresses or not.
- Python 3.7 reached end-of-life on June 27th 2023. Consequently, 3.7 will be deprecated in azure-ai-ml starting in October 2023 and azure-ai-ml will end support for 3.7 in February 2024.
- Feature sets can now be registers after being dumped and reloaded.
- SDK feature store create/update can now assign materialization identities to cross RG offline stores and online stores.
- Added support of features that are known into the future/at forecast time for dnn in AutoML Forecasting jobs.
- Added support for new workspace connection types: azure_open_ai, cognitive_search, and cognitive_service.
- Added support for new credential type: ApiKeyConfiguration.
- Added support of
downloadfor component operations.
- Local job runs will no longer fail if Docker registry has no username/password
- Fixed an issue that code asset doesn't work with relative symbol links.
- Fixed Issue 31319: can't accept
PathLikeforCommandComponent.code.
azure-ai-mlnow performs all file i/o onutf-8encoded files per Azure SDK guidance. (instead of the default behavior for python < 3.15, which uses locale specific encodings)- Removed references to deprecated "feature_store" workspace connection type.
- Added support to enable gpu access (local_enable_gpu) for local deployment.
- Improved the output when printing a workspace object to be more clean and readable.
- Log level of unknown field notifications for pipeline nodes raised from INFO to WARNING.
- Added support to enable set workspace connection secret expiry time.
- Added support for
stageon model version
- Fixed an issue affecting authentication to registry-related services in sovereign regions.
- Made job_tier and priority values case insensitive
- Public preview support for new schedule type
MonitorSchedule
- Fixed an issue where
OnlineDeployment.provisioning_statewas incorrectly deserialized and set asNone
- Added data import schedule. The class added is
ImportDataSchedule. - Added support to enable data isolation feature at workspace creation stage.
- Added auto_delete_setting support for asset version in data import job.
- Switched code snapshot upload from directory-based to container-based design in order to allow finer RBAC within workspaces. A container will be created for each new snapshot. This change does not affect storage costs or snapshot functionality.
- Added experimental scatter gather node to DSL package. This node has a unique mldesigner dependency.
- Added support to make JobService and ServiceInstance objects serializable when printed
- Support Singularity compute in pipeline job
- Added purge operation support for workspace resource
- Added Feature Store, its dedicated classes and updated the docstrings, now available in public interface. The classes added are
FeatureStoreOperations, FeatureSetOperations, FeatureStoreEntityOperationswith properties classes specific to the new features. - Support additional_includes in command component
- Added experimental
distribution: raysupport in command job.
- Fixed issue where show_progress=False was not being respected for uploads when set via MLClient
- Fixed issue of spark input/output mode validation doesn't take effect because of wrong type assertion
- Fixed the bug when setting
node.limits.timeoutto a pipeline input. - Removed Experimental Tag from Idle Shutdown, Custom Applications, Setup Scripts, and Image Metadata on Compute Instances.
- Removed Experimental Tag from JobService classes
- Renamed
JobServiceBase.job_service_typetotype
- Remove the default placeholder for CommandComponent.code
- Added support for
tagson Compute Resources. - Added support for promoting data asset from a workspace to a registry
- Added support for registering named asset from job output or node output by specifying name and version settings.
- Added support for data binding on outputs inside dynamic arguments for dsl pipeline
- Added support for serverless compute in pipeline, command, automl and sweep job
- Added support for
job_tierandpriorityin standalone job - Added support for passing
locationsvia command function and set it toJobResourceConfiguration.locations - Added support for modifying SSH key values after creation on Compute Resources.
- Added WorkspaceConnection types
s3,snowflake,azure_sql_db,azure_synapse_analytics,azure_my_sql_db,azure_postgres_db - Added WorkspaceConnection auth type
access_keyfors3 - Added DataImport class and DataOperations.import_data.
- Added DataOperations.list_materialization_status - list status of data import jobs that create asset versions via asset name.
- Fix experiment name wrongly set to 'Default' when schedule existing job.
- Error message improvement when a local path fails to match with data asset type.
- Error message improvement when an asset does not exist in a registry
- Fix an issue when submit spark pipeline job with referring a registered component
- Fix an issue that prevented Job.download from downloading the output of a BatchJob
- Added dependency on
azure-mgmt-resource - Added dependency on
azure-mgmt-resourcegraph - Added dependency on
opencensus-ext-azure<2.0.0 - Update job types to use MFE Dec preview rest objects.
- Added classifiers for Python version 3.11.
- Added warning for reserved keywords in IO names in pipeline job nodes.
- Added telemetry logging for SDK Jupyter Notebook scenarios with opt-out option (see README.md)
- Added dedicated classes for each type of job service and updated the docstrings. The classes added are
JupyterLabJobService, SshJobService, TensorBoardJobService, VsCodeJobServicewith a few properties specific to the type. - Added Custom Applications Support to Compute Instances.
- Update data asset list, show and create operations to support data assets in registry.
- Added Managed Network features to workspace to include
ManagedNetwork,FqdnDestination,PrivateEndpointDestination,ServiceTagDestinationas well as relevant schema.
- Fixed an issue where the ordering of
.amlignoreand.gitignorefiles are not respected. - Fixed an issue that attributes with a value of
FalseinPipelineJobSettingsare not respected. - Fixed an issue where ignore files weren't considered during upload directory size calculations.
- Fixed an issue where symlinks crashed upload directory size calculations.
- Fixes a bug where enable_node_public_ip returned an improper value when fetching a Compute.
- Update workspace creation to use Log Analytics-Based Application Insights when the user does not specify/bring their own App Insights.
- Upgraded minimum azure-core version to 1.23.0.
- Change print behavior of entity classes to show object yaml in notebooks, can be configured on in other contexts.
- Added property to enable/disable public ip addresses to Compute Instances and AML Computes.
DeploymentandScheduleOperationsadded to public interface.
- Fixed issue with date-time format for utc_time_created field when creating models.
- Added stricter behavior for ArmStr schemas when parsing 'azureml:' prefix.
- Fixed issue where AmlComputes could only be created in a workspace's default region.
- Improved intellisense with VS Code for fields supporting local paths and datastores.
- Added validation for token generation with aml scope when user_identity is used in job definition aka OBO flow.
- Fixed duplicate node name error in pipeline when two node names assigned to the same node and get renamed by node.name='xx'.
- Resolve the cross references for MLClient, Resource and OnlineDeployment.
- Explicit use of Optional (or a Union with None), as per PEP 484.
- Fixed print on Command objects when job id is empty
- Fixed issue where
SasTokenConfigurationcannot be used as credential forWorkspaceConnection
- Removed dependency on API version 2021-10-01 and 2022-06-01-preview to reduce side of azure-ai-ml package.
- Removed description from Registry.
- Disable sdk telemetry logging
- Enable updating the CMK encryption key (workspace.encryption.keyVaultProperties.keyIdentifier) for a workspace.
- Mark JobService class and services param to command() as experimental.
- Added a replication_count value to the schema of SystemCreatedStorageAccount in Registry.
- Added support for Fairfax and MoonCake cloud for the registry discovery baseurl.
- Added support for variable args as pipeline input in DSL Pipeline.
- Added OS Patching Parameters to Compute Instance.
- Update the upper bound dependencies version for tqdm, strictyaml, colorama and opencensus-ext-azure.
- Added missing "properties" to batch deployment.
- Retain the cases for the names of system job services (Tracking and Studio).
- Update registry begin_delete method return type.
- Fixed sweep job optional input cannot be empty.
- Fixed bool test for output in download operation.
- Fixed Compute Instance schedule not being created
- Removed erroneous experimental warning from Compute Schedules
- Restored idle_time_before_shutdown property for Compute Instances.
- Deprecated idle_time_before_shutdown property in favor of idle_time_before_shutdown_minutes.
- Fixed idle_time_before_shutdown appearing as None for Compute Instances returned by
showorlist. - Fixed idle_time_before_shutdown_minutes preventing creation of Compute Instances when set to None.
- Renamed idle_time_before_shutdown to idle_time_before_shutdown_minutes and changed input type to int.
- Fixed idle_time_before_shutdown_minutes not appearing in GET calls for Compute Instances.
- Registry list operation now accepts scope value to allow subscription-only based requests.
- Most configuration classes from the entity package now implement the standard mapping protocol.
- Add registry delete operation.
- The values of JobService.job_service_type are now using the snake case. e.g jupyter_lab, ssh, tensor_board, vs_code.
- Command function now accepts services param of type Dict[str, JobService] instead of dict.
- MLClient.from_config can now find the default config.json on Compute Instance when running sample notebooks.
- Fixed job inputs not accepting datastores or job inputs.
- Registries now assign managed tags to match registry's tags.
- Adjust registry experimental tags and imports to avoid warning printouts for unrelated operations.
- Make registry delete operation return an LROPoller, and change name to begin_delete.
- Prevent registering an already existing environment that references conda file.
- Fix ARM ID logic for registry environments (ex: Creating a registry component that references a registry environment).
- Fix ARM ID logic for passing models and environments with ID (ex: Creating endpoint deployment for a registry model should return said model's ID immediately)
- Switched compute operations to go through 2022-10-01-preview API version.
- GA release
- Dropped support for Python 3.6. The Python versions supported for this release are 3.7-3.10.
- OnlineDeploymentOperations.delete has been renamed to begin_delete.
- Datastore credentials are switched to use unified credential configuration classes.
- UserAssignedIdentity is replaced by ManagedIdentityConfiguration
- Endpoint and Job use unified identity classes.
- Workspace ManagedServiceIdentity has been replaced by IdentityConfiguration.
- Switched Compute operations to use Oct preview API version.
- Updated batch deployment/endpoint invoke and list-jobs function signatures with curated BatchJob class.
- Support passing JobService as argument to Command()
- Added support for custom setup scripts on compute instances.
- Added a
show_progressparameter to MLClient for enable/disable progress bars of long running operations. - Support
month_daysinRecurrencePatternwhen usingRecurrenceSchedule. - Support
ml_client.schedules.listwithlist_view_type, default toENABLED_ONLY. - Add support for model sweeping and hyperparameter tuning in AutoML NLP jobs.
- Added
ml_client.jobs.show_services()operation.
- ComputeOperations.attach has been renamed to begin_attach.
- Deprecated parameter path has been removed from load and dump methods.
- JobOperations.cancel() is renamed to JobOperations.begin_cancel() and it returns LROPoller
- Workspace.list_keys renamed to Workspace.get_keys.
- Fix identity passthrough job with single file code
- MLClient.from_config can now find the default config.json on Compute Instance when running sample notebooks.
- Removed declaration on Python 3.6 support
- Added support for custom setup scripts on compute instances.
- Updated dependencies upper bounds to be major versions.
- Spark job submission.
- Command and sweep job docker config (shmSize and dockerArgs) spec support.
- Entity load and dump now also accept a file pointer as input.
- Load and dump input names changed from path to 'source' and 'dest', respectively.
- Load and dump 'path' input still works, but is deprecated and emits a warning.
- Managed Identity Support for Compute Instance (experimental).
- Enable using @dsl.pipeline without brackets when no additional parameters.
- Expose Azure subscription Id and resource group name from MLClient objects.
- Added Idle Shutdown support for Compute Instances, allowing instances to shutdown after a set period of inactivity.
- Online Deployment Data Collection for eventhub and data storage will be supported.
- Syntax validation on scoring scripts of Batch Deployment and Online Deployment will prevent the user from submitting bad deployments.
- Change (begin_)create_or_update typehints to use generics.
- Remove invalid option from create_or_update typehints.
- Change error returned by (begin_)create_or_update invalid input to TypeError.
- Rename set_image_model APIs for all vision tasks to set_training_parameters
- JobOperations.download defaults to "." instead of Path.cwd()
- Show 'properties' on data assets
- Support for AutoML Component
- Added skip_validation for Job/Component create_or_update
- Dataset removed from public interface.
- Fixed mismatch errors when updating scale_settings for KubernetesOnlineDeployment.
- Removed az CLI command that was printed when deleting OnlineEndpoint
- Allow Input/Output objects to be used by CommandComponent.
- Added MoonCake cloud support.
- Unified inputs/outputs building and validation logic in BaseNode.
- Allow Git repo URLs to be used as code for jobs and components.
- Updated AutoML YAML schema to use InputSchema.
- Added end_time to job schedule.
- MIR and pipeline job now support registry assets.
- Have mldesigner use argparser to parse incoming args.
- Bumped pyjwt version to <3.0.0.
- Reverted "upload support for symlinks".
- Error message improvement when a YAML UnionField fails to match.
- Reintroduced support for symlinks when uploading.
- Hard coded registry base URL to eastus region to support preview.
- First preview.