azure eventhub to prism integration

Hi @Sunil Sigroha ,

Thanks for reaching out to Microsoft Q&A.

In most cases you don’t need 800 separate Event Hubs, one Event Hub inside a single namespace with your private endpoint configured can ingest logs from all 800 VMs, provided you size your throughput units (TUs) and partitions correctly.

1. Single Event Hub vs. Multiple
   • Azure Event Hubs is built for high-volume ingestion (millions of events/sec).
   • You can host all VM log senders in one Event Hub and simply scale up by adding TUs or partitions if you hit throughput limits.
   • If you need tenant or workload isolation (audit separation), you could spin up additional hubs or consumer groups, but it’s not a performance requirement.
2. Throughput Units and Partitions
   • Each TU provides 1 MB/sec ingress (or up to 1,000 events/sec, whichever comes first) and 2 MB/sec egress.
   • You can purchase up to 20 TUs per standard namespace by default.
   • An Event Hub supports up to 32 partitions, which you’d leverage if you have multiple concurrent readers or high-parallel ingestion.
3. Private Endpoint Scope
   • Private endpoints are scoped at the namespace level, so once you’ve configured your private link, any Event Hub in that namespace can be reached privately no extra P.E. needed per hub.
4. When to Add More Hubs
   • If your combined VM log traffic exceeds what the max TUs (or partitions) can handle in a single namespace, you can:
   • Increase TUs (up to your subscription/namespace limit)
   • Or deploy a second namespace/event hub pair (and P.E.) to split the load

Limitations:

• Max 32 partitions per Event Hub
• Default max of 20 TUs per standard namespace (can be raised via support request)
• Single event max size is 1 MB
• Retention defaults (1 day up to 90 days) and consumer group counts (7 standard per hub)

Hope this helps!

If the resolution was helpful, kindly take a moment to click on and click on Yes for was this answer helpful. And, if you have any further query do let us know.

one hub is enough — the only real failure case is a hot‑partition outrunning PRISM’s read‑lag, so start Std/Prem with TU +1 and 4–8 partitions, and if it still slows just add TU +1 again and move the noisiest sender to its own partition…that’s always held up in the field.

A single Event Hub is typically sufficient for ingesting security logs from 800 VMs, provided throughput and partition limits are respected. The main design dimension is not the number of VMs but throughput units (TUs/PUs/CUs) and partition count.

Key points for sizing and limitations:

1. Number of Event Hubs

• One Event Hub (in one namespace) can handle logs from hundreds or thousands of VMs as long as the aggregate throughput stays within tier limits.
• Multiple Event Hubs are generally only required for:
  • Isolation by workload/tenant or environment (prod vs non-prod)
  • Different retention or access policies
  • Very high throughput that needs to be split across hubs

2. Partitions and parallelism

• Partitions provide parallelism for consumers (such as PRISM) and scale-out capacity.
• Recommended starting point for monitoring/log scenarios is 4 partitions if unsure, as each partition can support up to 20 MB/s or ~20,000 messages/sec.
• Maximum partitions per Event Hub by tier:
  • Basic/Standard: up to 32 partitions per event hub
  • Premium: up to 100 partitions per event hub (with namespace-level limits)
  • Dedicated: up to 1,024 partitions per event hub
• Choose enough partitions to cover peak ingestion and PRISM’s consumption model. If PRISM can consume from multiple partitions, 4+ partitions is appropriate; if it expects a single stream, fewer partitions may be used.

3. Throughput and quota limits

• Throughput is governed by the namespace capacity:
  • Basic/Standard: per throughput unit (TU), ingress is 1 MB/s or 1,000 events/s; egress is 2 MB/s or 4,096 events/s.
  • Premium/Dedicated: no fixed per-unit throughput limit, but constrained by partitions and resource allocation.
• Subscription-level limits (per region) include:
  • Max TUs/PUs/CUs per namespace/tier
  • Max partitions per event hub and per namespace
  • Max event hubs per namespace
• Ensure the chosen tier and number of TUs/PUs/CUs can handle the combined log volume from 800 VMs.

4. Private endpoint and network limitations

• Private endpoints for Event Hubs are not supported in the Basic tier; use Standard, Premium, or Dedicated.
• Up to 120 private endpoints per Event Hubs namespace.
• When private endpoints are enabled:
  • Public access is effectively blocked at the application layer, but TCP-level checks (like nslookup or TCP connect) to the public endpoint may still succeed.
  • Other Azure services, Azure portal, and logging/metrics may be blocked unless explicitly allowed as trusted services.
• All traffic from PRISM to Event Hubs will traverse the private endpoint over the Microsoft backbone network.

5. General design guidance for this scenario

• Use a single Standard (or higher) Event Hubs namespace with one Event Hub dedicated to security logs from the 800 VMs.
• Configure an appropriate number of partitions (for example, 4–8) based on expected peak log rate and PRISM’s ability to consume from multiple partitions.
• Scale TUs (or PUs/CUs) as needed if ingestion or egress approaches limits.
• Use a private endpoint to connect PRISM to the Event Hub, and review the impact on other Azure services that may need to access the namespace.

There is no hard limit in the provided information that directly restricts the number of VMs sending to a single Event Hub; design is driven by throughput, partitions, and private endpoint constraints.

References:

• Stream Azure monitoring data to an event hub and external partner
• Azure Event Hubs quotas and limits
• Azure subscription and service limits, quotas, and constraints
• Allow access to Azure Event Hubs namespaces via private endpoints
• [Allow access to Azure Event Hubs namespaces via private endpoints

Limitations and design considerations](https://learn.microsoft.com/azure/event-hubs/private-link-service#limitations-and-design-considerations)

• What is Azure Event Hubs?
• Scaling with Event Hubs