There are not enough vcores resources available in the workspace while running the synapse dataflows

Hi @SaiSekhar, MahasivaRavi (Philadelphia)

The error you're encountering:

"Your job requested 8 vcores. However, the workspace only has 4 vcores available out of quota of 60 vcores for node size family [MemoryOptimized]."

Indicates that there are insufficient available vCores in your Synapse workspace at the time of Data Flow execution, especially when running within a ForEach activity, which may be triggering multiple parallel executions.

Even though your workspace has a 60 vCore quota, at the time your job runs, only 4 vCores are free. This could be due to:

• Other Data Flows or Spark jobs consuming resources.
• Too many parallel executions within the ForEach loop.
• Data Flows using a node size configuration requesting more cores than available.

Suggested solutions:

• Set the Batch Count property in the ForEach activity to a lower value (e.g., 2 or 3) to reduce simultaneous executions and resource contention.
• In your Data Flow activity, adjust the integration runtime settings to use a smaller compute size (e.g., General Purpose Small or Medium) or lower the core count under Core Count per node.
• Check the Monitor hub for any active or stuck jobs consuming vCores, and cancel them if not needed.
• Although you mentioned not seeing the Usage + quotas section, here's how you can request more vCores:

Steps:

• Go to Azure Portal.
• Open "Subscriptions" > Your Subscription.
• In the left-hand pane, go to "Usage + quotas"

If the adjustable column indicates 'Yes', you can request the adjustment. Otherwise, if it indicates 'No', you should raise a support ticket. Based on the region, you can select where your resources are relying.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Hi @Smaran Thoomu,
Thanks for the update, we will check for the above methods implementation, can you please guide for Usage + quotas In the below screen shot which metrics we have to check.

Hi @SaiSekhar, MahasivaRavi (Philadelphia)

You're correct that it can be tricky to find the right vCore metrics, especially if the provider or region filter is not set properly.

Please find below a screenshot for guidance. Here are the key things to note:

• Provider: Make sure it's set to the correct resource type - for Synapse, that would be Microsoft.Synapse.
• Region: Should match the region where your Synapse workspace is deployed (e.g., East US 2).
• In this screenshot, the example shown is for Azure PostgreSQL just as a reference — you should switch to Microsoft.Synapse under the provider filter.

If visible, you can click on the Request adjustment option (top-right) to request a quota increase for vCores.

If you still don’t see the relevant quotas even after applying filters, you may want to raise a support ticket to request the vCore increase for Synapse.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

@SaiSekhar, MahasivaRavi (Philadelphia) We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Hi @Smaran Thoomu,
Thanks for the update, it is already stated as yes, thank you for helping us,
But now we are facing one more issue the record count in delta table 126022273, we have around 5 tables with this count, it is running more than 5 hours how to optimize the performance for this Hugh data in dataflow, can you please share any other options.

Thanks in advance