Queues and messages in queues in Exchange Server

A queue is a temporary holding location for messages that are waiting to enter the next stage of processing or delivery to a destination. Each queue represents a logical set of messages that the Exchange server processes in a specific order. In Exchange 2016 and Exchange 2019, queues hold messages before, during, and after delivery. Queues exist in the Transport service on Mailbox servers and on Edge Transport servers. Mailbox servers and Edge Transport servers are called transport servers throughout this topic.

Like all previous versions of Exchange, a single Extensible Storage Engine (ESE) database is used for queue storage.

You can manage queues and messages in queues by using the Exchange Management Shell and Queue Viewer in the Exchange Toolbox. You can use these interfaces to view the status and contents of queues and detailed message properties. You can also perform actions that modify queues or the messages in queues. For more information, see Procedures for queues and Procedures for messages in queues.

Types of queues

The following types of queues are used in Exchange 2016 and Exchange 2019, which are the same as Exchange 2013:

Queue database files

All the different queues are stored in a single ESE database. By default, this queue database is located on the transport server at %ExchangeInstallPath%TransportRoles\data\Queue.

Like any ESE database, the queue database uses log files to accept, track, and maintain data. To enhance performance, all message transactions are written first to log files and memory, and then to the database file. The checkpoint file tracks the transaction log entries that have been committed to the database. During an ordinary shutdown of the Microsoft Exchange Transport service, uncommitted database changes that are found in the transaction logs are committed to the database.

Circular logging is used for the queue database. This means that transaction logs that are older than the current checkpoint are immediately and automatically deleted. Therefore, the transaction logs can't be replayed for queue database recovery from backup.

The following table lists the files that constitute the queue database.

Exchange uses generation tables for storage and clean-up of messages in the queue database. Instead of processing and deleting individual message records from one large table, the queue database stores messages in time-based tables, and only deletes the entire table after all the messages in the table have been successfully processed. For example, consider the following example:

• All messages queued from 1:00 PM to 2:00 PM, regardless of the queue or destination, are stored in the 1p-2p_msgs table.

• At 2:00 PM, new messages are stored in the 2p-3p_msgs table.

• At 4:00 PM, a new table named 4p-5p_msgs is created. The entire 1p-2p_msgs table is deleted, but only if all messages in the table have been successfully processed.

This approach of deleting entire messages tables instead of individual messages helps improves the I/O performance of the drive that holds the queue database.

Options for configuring the queue database

You configure the queue database by adding or modifying keys in the %ExchangeInstallPath%Bin\EdgeTransport.exe.config XML application configuration file. This file is associated with the Microsoft Exchange Transport service. Changes you make to the EdgeTransport.exe.config file take effect after you restart the Microsoft Exchange Transport service.

The <appSettings> section of the EdgeTransport.exe.config file is where you can add new keys or modify existing keys. If a specific key doesn't exist, you can add it manually to change its value.

The keys for the queue database that are available in the EdgeTransport.exe.config file are described in the following table.

Queue properties

A queue has many properties that describe the purpose and status of the queue. Some queue properties are applied to the queue when the queue is created, and don't change. Other properties contain status, size, time, or other indicators that are updated frequently.

NextHopSolutionKey

The routing component of the categorizer in the Microsoft Exchange Transport service selects the destination for a message, and this destination is used to create the delivery queue. The destination is stamped on every recipient as the NextHopSolutionKey property. Every unique value of the NextHopSolutionKey property corresponds to a separate delivery queue.

The NextHopSolutionKey property contains the following fields:

• DeliveryType: Represents the results of the categorization of the message, and how the Transport service intends to transmit the message to the next hop, which could be the ultimate destination of the message, or an intermediate hop along the way. The Transport service uses a predefined list of values for DeliveryType.

  Based on the value of DeliveryType, the NextHopCategory property is added to the queue:

  • The value External indicates the next hop for the queue is outside the Exchange organization.

  • The value Internal indicates the next hop for the queue is inside the Exchange organization.

    Note that a message for an external recipient may require one or more internal hops before the message is delivered externally.

• NextHopDomain: Uses specific values based on the value of the DeliveryType field. For delivery queues, the value of this field is effectively the name of the queue.

  The value of NextHopDomain isn't always a domain name. For example, the value could be the name of the target Active Directory site or database availability group (DAG). Think of this field as the next hop name.

• NextHopConnector: Uses specific values based on the value of the DeliveryType field. The value is always expressed as a GUID. If this field isn't used, the value is a GUID with all zeroes.

  The value of NextHopConnector isn't always the GUID of a connector. For example, the value could be the GUID of the target Active Directory site or DAG. Think of this field as the next hop GUID.

The values of DeliveryType, NextHopCategory, NextHopDomain and NextHopConnector are described in the following table.

IncomingRate, OutgoingRate, and Velocity

Exchange measures the rate of messages entering and leaving a queue and stores these values in queue properties. You can use these rates as an indicator of queue and transport server health. The properties are described in the following table:

At a basic level, a positive value of Velocity indicates a healthy queue that's efficiently draining, and a negative value of Velocity indicates a queue that isn't efficiently draining. However, you also need to consider the values of IncomingRate, OutgoingRate, and MessageCount, as well as the magnitude of Velocity.

For example, consider a queue that has the following property values.

• Velocity: -50
• MessageCount: 1000
• OutgoingRate: 10
• IncomingRate: 60

Based on the property values for this queue, the negative value for Velocity clearly indicates that the queue isn't draining properly.

Now consider a queue that has the following property values.

• Velocity: -0.85
• MessageCount: 2
• OutgoingRate: 0.15
• IncomingRate: 1

Although the value for Velocity is negative, it's very close to zero, and the values of the other properties are also very small. Therefore, a negative Velocity value for this queue doesn't indicate a problem with the queue.

Queue status

The current status of a queue is stored in the Status property of the queue. A queue can have one of the status values that's described in the following table:

Other queue properties

There are other queue properties that are self-explanatory. You can use most of the queue properties as filter options. By specifying filter criteria, you can quickly locate queues and take action on them. For a complete description of the filterable queue properties, see Queue properties.

An important queue property that's also worth mentioning here is the MessageCount property that shows how many messages are in a queue. This property is an important indicator of queue health. For example, a delivery queue that contains a large number of messages that continues to grow and never decreases could indicate a routing or transport pipeline issue that requires your attention.

Message properties

A message in a queue has many properties. Many of the properties reflect the information that was used to create the message. Some of the messages status and information properties are heavily influenced by corresponding properties on the queue. However, an individual message may have a different value than the corresponding property of the queue. Other properties contain status, time, or other indicators that are updated frequently.

Message status

The current status of a message is stored in the Status property of the message. A message can have one of the status values that's described in the following table:

Other message properties

There are other message properties that are self-explanatory. You can use most of the message properties as filter options. By specifying filter criteria, you can quickly locate messages and take action on them. For a complete description of the filterable message properties, see Properties of messages in queues.

Manage queues and messages in queues

Queue Viewer and the historical queue and message management cmdlets in the Exchange Management Shell are restricted to a single Exchange server. You can view or operate on individual queues or messages, or multiple queues or messages, but only on a specific server.

The Get-QueueDigest cmdlet was introduced in Exchange 2013 to provide a high-level, aggregate view of the state of queues on all servers within a specific scope. The scope could be a DAG, an Active Directory site, a list of servers, or the entire Active Directory forest. Note that queues on a subscribed Edge Transport server in the perimeter network aren't included in the results. Also, Get-QueueDigest is available on Edge Transport servers, but the results are restricted to queues on the Edge Transport server.

The following table describes the management tasks you can perform on queues or messages in queues.