How to add external Cache in Azure data explorer?

Shubham Jaunjal 0

Currently, I have two instances of the "Standard_E8ads_v5" SKU, with CPU utilization ranging between 40% and 60%. However, the cache utilization is at 100%, and this is causing slow load times for cold cache data.

I need to increase the CPU cache externally. How can I accomplish this?

PRADEEPCHEEKATLA-MSFT 89,296 Reputation points Microsoft Employee

2024-08-08T10:28:07.8866667+00:00

@Shubham Jaunjal - Just checking in to see if the below answer provided by @Sander van de Velde | MVP helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Sander van de Velde | MVP 32,556 Reputation points MVP

2024-08-04T19:14:05.36+00:00

Hello @Shubham Jaunjal,

welcome to this moderated Azure community forum.

You can scale either vertically (having powerful instances with more cache) or horizontally (having more instances).

It's recommended to try out the optimized autoscale option on the cluster:

By auditing this setting, you learn about how scaling affects the performance.

Check the documentation for more details on how this works.

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.
Please sign in to rate this answer.
Shubham Jaunjal 0 Reputation points

2024-08-13T07:14:22.0133333+00:00

"Hi Sander van de Velde | MVP,

Thanks for the reply.

Actually, I don't want to add more instances. My CPU is working fine, but the hot cache is getting overloaded when I try to load data for 10 tags over a year with a 1-minute interval. It's taking 30-40 minutes, and the customer wants the results in under 1-2 minute.

Is there any way to increase the cache without adding more instances?"

Sander van de Velde | MVP 32,556 Reputation points MVP

2024-08-13T20:23:43.94+00:00

Hello @Shubham Jaunjal,

either when this is a one-time load or a query that is repeated over time, you need above average caching compared to the current situation.

This can be achieved by either scale out, having multiple instances of the same cluster item or you can scale up, using more powerful cluster items.

Because you do not want to scale out, check out these scale up options, seen under scale up in the cluster menu:

Note: This is an example, I expect this list can differ per region.

I checked the West Europe region. Your Standard_E8ads_v5 has a 240GB cache. If you only need one availability zone, has the same amount of vCPUs and memory but offers 1750GB. The price is only slightly higher (please check this in your region, under your subscription).

Keep in mind you can scale up and down whenever you want but the hot cache will not be filled with 1 year of historical data by default.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How to add external Cache in Azure data explorer?

1 answer

Your answer