Orchestrate agent behavior with generative AI (preview)
You can configure an agent to use either of the following types of orchestration. With classic orchestration, an agent responds to users by triggering the topic whose trigger phrases match most closely the user's query. With generative orchestration enabled your agent can choose the best actions, knowledge, and topics to answer user queries, or respond to event triggers.
Important
This article contains Microsoft Copilot Studio preview documentation and is subject to change.
Preview features aren't meant for production use and may have restricted functionality. These features are available before an official release so that you can get early access and provide feedback.
If you're building a production-ready agent, see Microsoft Copilot Studio Overview.
Important
Enabling generative orchestration can impact how billing is calculated. Learn more about billing for generative mode.
The following table compares agent behavior between classic orchestration and generative orchestration.
| Behavior | Classic orchestration | Generative orchestration |
|---|---|---|
| Topics | Topics are selected based on matching a user query with trigger phrases | Topics are selected based on the description of their purpose |
| Actions | Actions can only be called explicitly from within a topic | The agent can choose to call actions based on their name and description |
| Knowledge | Knowledge can be used as a fallback when no topics match a user's query (or called explicitly from within a topic) | The agent can choose to proactively search knowledge to answer a user's query. |
| Use of multiple topics, actions, knowledge sources | Agent tries to select a single topic to respond to the user, falling back to knowledge if configured | The agent can use a combination of topics, actions, and knowledge |
| Asking users for input | You must use question nodes in topics to author messages prompting the user for any required information | The agent can automatically generate questions to prompt users for any missing information required to fill inputs for topics and actions |
| Responding to a user | You must use message nodes in topics to author messages responding to the user (or call an action from a topic) | The agent automatically generates a response, using the available information from topics, actions, and knowledge that it has called |
Tip
There are key differences between classic and generative orchestration, such as how knowledge is searched, and the supported data sources. Before turning on generative mode for an existing agent, read about the known limitations.
How does generative mode work?
Using generative AI to determine how your agent responds can make the conversation more natural and fluid for the user. Generative AI can also make it possible for an agent to perform actions autonomously.
Selecting the right topics, actions, and knowledge sources
When a user sends a message, your agent selects one or more actions, topics, or knowledge sources to prepare its response. Multiple factors determine the selection. The most important factor is the description of the topics, actions, and knowledge sources. Other factors include the name of a topic, action, or knowledge source, any input or output parameters, and their names and descriptions. Descriptions make it possible for your agent to be more accurate when it associates the user intent with actions and topics. You don't need to predict all of the ways a user might indicate what they need.
When an agent is configured to use generative orchestration, it can select one or more actions or topics, or choose from its knowledge sources, to handle user queries (including multi-intent queries), or to autonomously respond to events. If multiple actions or topics are selected, the agent calls them in sequence, after generating any questions to ask the user for missing information.
Learn more about how agents search across knowledge sources when generative orchestration is enabled.
Responding to user input or event triggers
The agent takes the information returned from all knowledge sources, actions, and topics that it selected in response to user input or to an event trigger, and summarizes an answer to any originating user query.
Tip
When an agent is configured with generative orchestration, because the agent can use information provided by knowledge, actions, and topics to generate a response for the user, you can make your topics more flexible by not sending their final response in a message node, but instead return it as an output variable to the agent. This lets your agent provide contextual responses to your users. Learn more about configuring topic inputs and outputs.
Testing
When you test an agent that uses generative mode in Copilot Studio, you can open the activity map to follow how your agent is responding.
Turn on generative orchestration for an agent
Open your agent.
Select Settings.
Select the Generative AI tab.
Select the Generative option.
(Optional) Select your preferred level of strictness for content moderation.
Select Save.
Important
Using generative AI for orchestrating actions with both user and event triggers is a preview feature. Preview features aren't meant for production use and may have restricted functionality. These features are available before an official release so that customers can get early access and provide feedback.
Tip
It's a good practice to inform your users that some of the conversation (for example, questions generated when running an action) might be generated by AI. For example, you could add an extra message in the Start conversation system topic, which controls the message shown to your users when a new conversation is started with the agent.
Authoring descriptions
If an agent is configured to use generative orchestration, it's important to provide a high-quality description for each of its topics, actions, and knowledge sources. Good descriptions ensure the agent selects the right topics, actions, and knowledge sources to respond to users.
For actions, authoring a description is part of the wizard used to add them to the agent. The description is often prepopulated for you, but you can make changes as appropriate. To learn more about adding and managing actions, see Use actions with custom agents (preview).
For topics, once generative orchestration is enabled, Triggered by agent appears on Trigger nodes (instead of Phrases), which allows you to add or edit the description for the topic.
Tip
When you turn on generative orchestration, Copilot Studio automatically generates a default description for each existing topic, based on the topic's trigger phrases. The generated description is often good enough to allow these topics to be selected in response to relevant user queries. However, it's a good practice to follow the advice provided in this article to revise the generated descriptions.
Best practices
Here are some best practices for naming topics, actions, and knowledge sources, and drafting clear, concise, and relevant descriptions for them.
Writing style
Use simple and direct language. Avoid jargon, slang, or technical terms.
Use the active voice and the present tense for descriptions. For example, write "This action provides weather information" instead of "Weather information is provided by this action."
Use bulleted and numbered lists to clearly separate a series of items, actions, or considerations.
Relevance
Use keywords that relate to the action or topic's functionality and the user's intent. For example, if an action provides weather information, use keywords like "weather," "forecast," "temperature," "rain," "snow," and so on.
For the description, write a short and informative summary of the action's or topics's functionality. The summary should be limited to one or two sentences, explain what the action or topic does, and how it benefits the user.
Use a descriptive and unique name that's a short phrase. Avoid using generic or ambiguous names that could be confusing. For example, instead of naming an action "Weather," name it "Weather Forecast" or "Weather Report."
Use specific language to prevent ambiguity between similar topics and actions.
For example, if your agent already has an action that provides information about current weather conditions, but you want to add another topic to provide the weather forecast for tomorrow, make sure the names and descriptions of these topics are specific enough to avoid ambiguity. Make sure such similar topics have a different name and description. It can also help to indicate what they can't do. Here are examples of names and descriptions you might use.
Name: Current Weather
Description: This topic provides weather forecast for any location in the world. You can ask for the current weather, including temperature and if it's raining or snowing. It doesn't get weather forecasts for future days.
Name: Weather Forecast for Tomorrow
Description: This topic provides weather information for any location in the world for the next day. It provides the temperature. It doesn't get the current weather for today.
Examples of what not to do
Here, we include a few examples that don't follow the guidelines. This example is too vague, since it doesn’t specify what types of questions the action can answer.
Name: Answer Question
Description: This action can answer questions.
The next example uses jargon. Instead, it should spell out earnings per share (EPS).
Name: Get EPS
Description: Gets EPS for any stock ticker.
Examples of interactions with an agent using generative orchestration to handle user requests
The following examples are based on an agent that has two custom topics—one for finding store hours and another for locating a nearby store—and an action based on the prebuilt MSN Weather connector.
Example 1
In this example, the user asks, "What is the weather like in Seattle?". The agent selects the current weather plugin action and also prepopulates the Location input with "Seattle," which it recognized from the user's question.
Example 2
In this example, the user asks, "I need to get store hours and find my nearest store." Here, the agent selects two items, the Store Hours topic and the Store Locator topic, and chains them together to respond to both parts of the user's query.
Example 3
In this example, the user found their nearest store, which was identified as Kirkland, earlier within the conversation. The user then asked, "what's the weather like there?". Here, the agent selects the current weather action, but prepopulates the location with "Kirkland," based on the recent conversation context.
Known limitations for generative orchestration
The following known issues and limitations exist when using the generative orchestration mode.
Knowledge
When your agent searches knowledge sources in generative mode, it no longer uses the Conversational boosting system topic. Therefore any modifications you might have made to this system topic to customize how your agent searches knowledge won't be used. This limitation also applies to classic data sources configured in generative answers nodes, including custom data sources. Learn more about how knowledge works in generative mode.
Custom entity support for topic and action input parameters
Actions and topics don't yet support custom entities (closed lists and regex entities) as input parameters. To collect information using a custom entity, you can use a Question node in a topic.
Disambiguation
An agent might fail to disambiguate between topics, when there's more than one topic that closely matches a user's intent. Normally, the agent asks the user to choose between one or more topics that match their intent via the Multiple Topics Matched system topic. However, agents configured to use generative orchestration currently don't call this topic. If you leave the Multiple Topics Matched system topic enabled, your agent will start disambiguating between topics automatically when this issue is resolved. If you wish to prevent your agent from automatically disambiguating between topics in the future, turn off the Multiple Topics Matched system topic. Turning off this topic allows you to test your agent. Optionally, you can opt back in to using disambiguation by turning the topic back on after testing your agent.