Analyzing job queue lifecycle telemetry

APPLIES TO: Business Central 2020 release wave 2, version 17.2, and later

Job queue lifecycle telemetry gathers data about the following operations:

• A job queue entry was enqueued.
• A job queue entry was started.
• A job queue entry finished. Provides information as to whether it was successful or failed.
• A job queue entry was stopped.
• A job queue entry was cancelled.
• A job queue entry was scheduled on login.
• A job queue entry was run by a user.

For information about creating and managing job queue entries, see Use Job Queues to Schedule Tasks in the Business Central application help.

For technical information about how the Job Queue works, see How the Job Queue works in the in the Business Central developer content.

Job queue telemetry events

The following table describes the event IDs that are currently emitted by the job queue system.

Common custom dimensions to all job queue events

The following table explains custom dimensions that are common to all job queue entry telemetry events described below.

Job queue entry enqueued

Occurs when a job queue entry is sent to the job queue to eventually be run. A job queue entry is sent to the queue when its status is changed from On Hold to Ready or if it's a recurring job queue entry. Recurring job queue entries are automatically enqueued after each run.

General dimensions

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

Job queue entry failed to be enqueued

Information if the job queue entry fails to be sent to the queue.

General dimensions

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

Job queue entry started

Occurs when a job queue entry starts to run.

General dimensions

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

Job queue entry finished (request)

Occurs when a request for a job queue entry finishes running.

General dimensions

Custom dimensions

Job queue entry finished (run)

Occurs when a job queue entry finishes running.

General dimensions

Custom dimensions

Job queue entry failed with error

Occurs when a job queue entry fails to run.

General dimensions

Custom dimensions

Sample KQL code

This KQL code can help you get started troubleshooting job queue errors. Note that these failed entries might be restarted.

// Job queue entry run failed
traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'AL0000HE7'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
// jobQueueApp* dimensions contains the information about the app where the jobQueueObject* is located
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
// jobQueueObject* dimensions contains the information about the object containing the code being run
, jobQueueObjectId = customDimensions.alJobQueueObjectId 	
, jobQueueObjectName = customDimensions.alJobQueueObjectName // added in 22.0 
, jobQueueObjectType = customDimensions.alJobQueueObjectType
// jobQueueExecution dimensions contain information about the job queue entry definition
, jobQueueEntryId = customDimensions.alJobQueueId 	
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription // added in 22.2
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun // added in 21.5
// jobQueueExecution dimensions contain information about the attempted run of the job queue entry
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun // added in 21.5
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId // you can join to task scheduler telemetry on the taskId

Job queue entry {Job Queue Id} errored after {attempt number} attempts

Occurs when a job queue entry fails to run. This telemetry event was introduced in version 21.5.

General dimensions

Custom dimensions

Sample KQL code

This KQL code can help you get started troubleshooting job queue errors where the entry has been vstopped because it failed for the last time before being set in Error state. As an administrator, you want to know about job queue entries that are going into the final failed state and require manual intervention (restarting etc).

// Job queue entry {Job Queue Id} errored after {attempt number} attempts
traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'AL0000JRG'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
// jobQueueApp* dimensions contains the information about the app where the jobQueueObject* is located
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
// jobQueueObject* dimensions contains the information about the object containing the code being run
, jobQueueObjectId = customDimensions.alJobQueueObjectId 	
, jobQueueObjectName = customDimensions.alJobQueueObjectName // added in 22.0 
, jobQueueObjectType = customDimensions.alJobQueueObjectType
// jobQueueExecution dimensions contain information about the job queue entry definition
, jobQueueEntryId = customDimensions.alJobQueueId 	
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription // added in 22.2
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun // added in 21.5
// jobQueueExecution dimensions contain information about the attempted run of the job queue entry
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun // added in 21.5
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId // you can join to task scheduler telemetry on the taskId

Job queue entry run by user

Occurs when the "Run once (Foreground)" action is clicked in Job Queue Entries/Card pages by a user. This telemetry event was introduced in version 22.2.

General dimensions

Custom dimensions

Job queue entry rescheduled on login

Occurs when a job queue entry was rescheduled on login of a user. This telemetry event was introduced in version 22.0

General dimensions

Custom dimensions

Job queue entry cancelled

Occurs when a job queue entry was cancelled. This telemetry event was introduced in version 23.0.

General dimensions

Custom dimensions

Job queues and performance

Scheduled tasks or job queue entries that are set to run on a recurring schedule can impact the performance of Business Central under either of the following conditions:

• They run too frequently.

  Consider how often the task or job needs to run. Especially for polling scenarios, you might have better and more performant ways to react, such as using webhooks.

• They run heavy jobs while many users are also using Business Central.

  Consider running them outside working hours. This might decrease locking and deadlock issues both for users and for the task or job itself.

Troubleshooting Job queue issues with telemetry

The reason for a failed job queue entry could be in the code that is running. You can find the owner (also called publisher) of the code in the AL stack trace dimension in the job queue error event AL0000HE7. If the error comes from an app/extension, you need to contact the owner of the code to get it resolved. If the error comes from the Microsoft application, then you need to open a support request with Microsoft to get it resolved.

See event AL0000HE7 above for sample KQL code to analyze further.

If a job queue entry fails after the maximum number of retry attempts has been done, the telemetry event AL0000JRG is emitted and the job queue entry is set to permanently being stopped. As an administrator, you want to know about these job queue entries that are going into the final failed state and require manual intervention (restarting etc).

See event AL0000JRG above for sample KQL code to analyze further.

Alerting on Job queue issues with telemetry

If something happens in your environment or app that you need to act on, you can set up a system that sends you an alert. Azure Application Insights makes it easy to define such alerts.

You can use the following tools to define and set up alerts on telemetry events:

• Azure Application Insights Alerts
• Azure Logic Apps
• Power Automate

All three approaches need a Kusto (KQL) query to define the alerting condition.

There exists at least three job queue error scenarios where you might want to setup alerts:

1. Some job queue entries fail
2. Some job queue entries failed and have been stopped
3. No job queue entries are running

In the table below, you can read more about the scenarios and get sample KQL code for the alerting conditions in your alerts.

For more information about how to setup alerts on telemetry, see Alert on Telemetry.

See also

Use Job Queues to Schedule Tasks
How the Job Queue works
Telemetry overview
Enabling Telemetry
Alert on Telemetry