Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
APPLIES TO:
Azure CLI ml extension v1
Python SDK azureml v1
Important
This article provides information on using the Azure Machine Learning SDK v1. SDK v1 is deprecated as of March 31, 2025. Support for it will end on June 30, 2026. You can install and use SDK v1 until that date.
We recommend that you transition to the SDK v2 before June 30, 2026. For more information on SDK v2, see What is Azure Machine Learning CLI and Python SDK v2? and the SDK v2 reference.
Important
Some of the Azure CLI commands in this article use the azure-cli-ml, or v1, extension for Azure Machine Learning. Support for the v1 extension will end on September 30, 2025. You're able to install and use the v1 extension until that date.
We recommend that you transition to the ml, or v2, extension before September 30, 2025. For more information on the v2 extension, see Azure Machine Learning CLI extension and Python SDK v2.
An Azure Machine Learning pipeline is a workflow that automates a complete machine learning task. It standardizes best practices, supports team collaboration, and improves efficiency.
Why are Azure Machine Learning pipelines needed?
- Standardizes machine learning operations (MLOps) and supports scalable team collaboration
- Improves training efficiency and reduces cost
A pipeline breaks a machine learning task into steps. Each step is a manageable component that can be developed and automated separately. Azure Machine Learning manages dependencies between steps. This modular approach:
- Standardizes MLOps and supports team collaboration
- Improves training efficiency and reduces cost
- Standardizes machine learning operations (MLOps) and supports scalable team collaboration
- Improves training efficiency and reduces cost
Standardize the MLOps practice and support scalable team collaboration
MLOps automates building and deploying models. Pipelines simplify this process by mapping each step to a specific task, so teams can work independently.
For example, a project may include data collection, preparation, training, evaluation, and deployment. Data engineers, scientists, and ML engineers each own their steps. Steps are best built as components, then integrated into a single workflow. Pipelines can be versioned, automated, and standardized by DevOps practices.
Training efficiency and cost reduction
Pipelines also improve efficiency and reduce costs. They reuse outputs from unchanged steps and let you run each step on the best compute resource for the task.
Getting started best practices
You can build a pipeline in several ways, depending on your starting point.
If you are new to pipelines, start by splitting existing code into steps, parameterizing inputs, and wrapping everything into a pipeline.
To scale, use pipeline templates for common problems. Teams fork a template, work on assigned steps, and update only their part as needed.
With reusable pipelines and components, teams can quickly create new workflows by cloning or combining existing pieces.
You can build pipelines using the CLI, Python SDK, or Designer UI.
Which Azure pipeline technology should I use?
Azure provides several types of pipelines for different purposes:
| Scenario | Primary persona | Azure offering | OSS offering | Canonical pipe | Strengths |
|---|---|---|---|---|---|
| Model orchestration (Machine learning) | Data scientist | Azure Machine Learning Pipelines | Kubeflow Pipelines | Data -> Model | Distribution, caching, code-first, reuse |
| Data orchestration (Data prep) | Data engineer | Azure Data Factory pipelines | Apache Airflow | Data -> Data | Strongly typed movement, data-centric activities |
| Code & app orchestration (CI/CD) | App Developer / Ops | Azure Pipelines | Jenkins | Code + Model -> App/Service | Most open and flexible activity support, approval queues, phases with gating |
Next steps
Azure Machine Learning pipelines add value from the start of development.
- Define pipelines with the Azure Machine Learning CLI v2
- Define pipelines with the Azure Machine Learning SDK v2
- Define pipelines with Designer
- Try out CLI v2 pipeline example
- Try out Python SDK v2 pipeline example
- Learn about SDK and CLI v2 expressions that can be used in a pipeline.