Intermittent Telemetry Loss in IoT Device Hub – Messages Not Reaching Endpoint

Sai Manasa Ivaturi 0 Reputation points Microsoft Employee
2026-05-27T20:59:33.67+00:00

I’m encountering an issue with an IoT solution where device telemetry is intermittently not being delivered to the configured backend endpoint.

Scenario:

  • Devices are connected to an IoT Hub and sending telemetry at regular intervals.
  • Message routing is configured to forward telemetry data to downstream services (e.g., storage/event processing).
  • While most messages are processed successfully, there are sporadic gaps where telemetry from certain devices is missing.

Observed Behavior:

  • Devices appear connected and no connection errors are reported.
  • No consistent pattern in message loss (affects different devices at different times).
  • Metrics show a discrepancy between messages sent from devices and messages received at the endpoint.

Troubleshooting Done:

  • Verified device connectivity and authentication.
  • Checked routing configuration and endpoints.
  • Reviewed logs for throttling, quota limits, or dropped messages.
  • Monitored IoT Hub metrics (e.g., ingress, egress, throttling).I’m encountering an issue with an IoT solution where device telemetry is intermittently not being delivered to the configured backend endpoint. Scenario:
    • Devices are connected to an IoT Hub and sending telemetry at regular intervals.
    • Message routing is configured to forward telemetry data to downstream services (e.g., storage/event processing).
    • While most messages are processed successfully, there are sporadic gaps where telemetry from certain devices is missing.
    Observed Behavior:
    • Devices appear connected and no connection errors are reported.
    • No consistent pattern in message loss (affects different devices at different times).
    • Metrics show a discrepancy between messages sent from devices and messages received at the endpoint.
    Troubleshooting Done:
    • Verified device connectivity and authentication.
    • Checked routing configuration and endpoints.
    • Reviewed logs for throttling, quota limits, or dropped messages.
    • Monitored IoT Hub metrics (e.g., ingress, egress, throttling).
Azure IoT Hub
Azure IoT Hub

An Azure service that enables bidirectional communication between internet of things (IoT) devices and applications.

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. kagiyama yutaka 4,670 Reputation points
    2026-05-28T11:14:37.81+00:00

    I think IoT Hub only guarantees device→Hub ingestion and downstream delivery depends on endpoint health. A temporary true route and a time‑aligned ingress vs endpoint count comparison pinpoints the drop.

    Was this answer helpful?

    0 comments
  2. Karnam Venkata Rajeswari 4,925 Reputation points Microsoft External Staff Moderator
    2026-05-27T22:00:22.6066667+00:00

    Hello @Sai Manasa Ivaturi ,

    Welcome to Microsoft Q&A .Thank you for reaching out to us.

    The described behavior indicates that device telemetry is being successfully ingested into Azure IoT Hub, while intermittent gaps are observed in downstream visibility.

    Similar patterns have been observed in a few recent platform scenarios where telemetry ingestion remains successful, but intermittent delays are seen in downstream visibility due to routing or endpoint-level conditions.

    The backend product teams are actively investigating these scenarios, and continuous improvements are being made to stabilize routing behavior and ensure consistent downstream delivery.

    In this architecture, ingestion and downstream delivery operate as separate stages. IoT Hub guarantees at-least-once ingestion, while final visibility depends on routing evaluation, endpoint health, and downstream processing capacity. Therefore, apparent “missing messages” typically originate from routing rules, delivery retries/delays, endpoint constraints or lag rather than ingestion loss.

    Meanwhile please check if the following help-

    1. Routing evaluation and filter matching A primary cause of intermittent gaps is messages not consistently matching routing rules due to property or schema variations. Recommended checks are to :
      • Validate all route filter conditions (system and custom properties)
      • Ensure payload/schema changes are not affecting routing logic
      • Confirm fallback route is enabled to capture unmatched messages
      • Review routing diagnostic logs for:
      • Route evaluation failures
      • Unmatched messages - no route matched scenarios
    2. Routing delivery behavior and retry lifecycle When endpoints are temporarily slow or unavailable, IoT Hub retries delivery for a limited period. During this window, messages may be delayed and repeated delivery failures can align with the observed telemetry gaps. Recommended checks are to :
      • Review routing logs for delivery-related signals such as:
      • EndpointUnhealthy
      • Route evaluation success vs delivery failure patterns
      • Correlate routing latency metrics with missing telemetry windows
      • Identify retry delays that align with endpoint degradation
    3. Endpoint health and availability Downstream endpoints may experience transient degradation that impacts message delivery performance. Recommended checks are to :
      • Use endpoint health APIs and diagnostics to assess status
      • Validate for intermittent:
      • Throttling
      • Temporary failures
      • Reduced ingestion capacity
      • Ensure endpoint scaling aligns with peak ingestion patterns
    4. Downstream throttling and processing constraints Even when routing succeeds, downstream systems can experience temporary slowdowns due to capacity or quota limits. Recommended checks are to :
      • Compare incoming telemetry rate vs processing rate
      • Check throttling metrics and quota utilization
      • Validate scaling configuration for burst traffic handling
      • Ensure no sustained backpressure in downstream services
    5. lag and partition distribution -event-driven pipelines In Event Hubs–based architectures, telemetry may be present but not yet processed due to lag or uneven partition load. Recommended checks are to:
      • Monitor consumer lag across partitions
      • Identify hot partitions receiving disproportionate load
      • Ensure consumers scale with partition count
      • Validate balanced processing across consumer groups
    6. Device-side retry and transient connectivity behavior Even when devices appear connected, short-lived network interruptions can affect delivery timing. Recommended checks:
      • Validate SDK retry and offline buffering configuration
      • Check for transient MQTT/AMQP reconnect patterns
      • Ensure local buffering is enabled for unstable networks
      • Confirm messages are not expiring before successful delivery
    7. End-to-end correlation and validation To isolate the exact break point in the pipeline Recommended approach are to :
      • Compare IoT Hub ingress metrics with routing success metrics
      • Enable routing and diagnostics logs for delivery tracking
      • Correlate timestamps between ingestion and downstream processing
      • Identify whether delays originate in routing or consumption layers

    The following references might be helpful , please check them out

     

    Please let us know if the response was helpful

     

    Thank you

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.