adf cost reduction

azure_learner 420 Reputation points
2024-11-12T11:43:53.45+00:00

Hello , I am trying to use the Powershell script to reduce the costs in ADF. I am not a Powershell expert,  below are the snippets of PowerShell

Invoke-AzDataFactoryV2Pipeline -DataFactory $df -PipelineName "Adfv2QuickStartPipeline" -ParameterFile .\PipelineParameters.json -ResourceGroupName "myResourceGroup"

But I need the code should iterate through different parameters and run the Powershell script. I would be grateful if any Powershell script experts would share a working elaborate script. Thank you

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,873 questions
0 comments No comments
{count} votes

Accepted answer
  1. Luis Arias 7,376 Reputation points
    2024-11-12T11:58:16.03+00:00

    Hello azure_learner ,

    Use this PowerShell script to iterate through parameter sets for cost reduction in ADF:

    • Create a ParameterSets.json file with parameter sets:
          [
              { "Parameter1": "Value1", "Parameter2": "Value2" },
              { "Parameter1": "AnotherValue1", "Parameter2": "AnotherValue2" }
          ]
      
    • PowerShell script:
          Import-Module Az.DataFactory
          
          $DataFactoryName = "YourDataFactoryName"
          $ResourceGroupName = "myResourceGroup"
          $PipelineName = "Adfv2QuickStartPipeline"
          $ParameterSets = Get-Content -Path ".\ParameterSets.json" | ConvertFrom-Json
      
          foreach ($ParameterSet in $ParameterSets) {
              $ParameterHash = @{}
              foreach ($key in $ParameterSet.PSObject.Properties.Name) {
                  $ParameterHash[$key] = $ParameterSet.$key
              }
              
              Invoke-AzDataFactoryV2Pipeline `
                  -DataFactoryName $DataFactoryName `
                  -PipelineName $PipelineName `
                  -ResourceGroupName $ResourceGroupName `
                  -Parameter $ParameterHash
      
              Write-Host "Pipeline executed successfully with parameter set:" $ParameterHash
          }
      

    References

    If the information helped address your question, please Accept the answer.

    Luis

    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.