ToolCallAccuracyEvaluatorContext Class

Definition

Namespace:: Microsoft.Extensions.AI.Evaluation.Quality

Assembly:: Microsoft.Extensions.AI.Evaluation.Quality.dll

Package:: Microsoft.Extensions.AI.Evaluation.Quality v9.7.0

Source:: ToolCallAccuracyEvaluatorContext.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Contextual information that the ToolCallAccuracyEvaluator uses to evaluate an AI system's effectiveness at using the tools supplied to it.

public ref class ToolCallAccuracyEvaluatorContext sealed : Microsoft::Extensions::AI::Evaluation::EvaluationContext

[System.Diagnostics.CodeAnalysis.Experimental("AIEVAL001")]
public sealed class ToolCallAccuracyEvaluatorContext : Microsoft.Extensions.AI.Evaluation.EvaluationContext

public sealed class ToolCallAccuracyEvaluatorContext : Microsoft.Extensions.AI.Evaluation.EvaluationContext

[<System.Diagnostics.CodeAnalysis.Experimental("AIEVAL001")>]
type ToolCallAccuracyEvaluatorContext = class
    inherit EvaluationContext

type ToolCallAccuracyEvaluatorContext = class
    inherit EvaluationContext

Public NotInheritable Class ToolCallAccuracyEvaluatorContext
Inherits EvaluationContext

Inheritance: Object

EvaluationContext
ToolCallAccuracyEvaluatorContext

Attributes: ExperimentalAttribute

Remarks

ToolCallAccuracyEvaluator measures how accurately an AI system uses tools by examining tool calls (i.e., FunctionCallContents) present in the supplied response to assess the relevance of these tool calls to the conversation, the parameter correctness for these tool calls with regard to the tool definitions supplied via ToolDefinitions, and the accuracy of the parameter value extraction from the supplied conversation history.

Note that at the moment, ToolCallAccuracyEvaluator only supports evaluating calls to tools that are defined as AIFunctions. Any other AITool definitions that are supplied via ToolDefinitions will be ignored.

Constructors

ToolCallAccuracyEvaluatorContext(AITool[])	Initializes a new instance of the ToolCallAccuracyEvaluatorContext class.
ToolCallAccuracyEvaluatorContext(IEnumerable<AITool>)	Initializes a new instance of the ToolCallAccuracyEvaluatorContext class.

Properties

Contents	Gets or sets a list of AIContent objects that include all the information present in this EvaluationContext. (Inherited from EvaluationContext)
Name	Gets or sets the name for this EvaluationContext. (Inherited from EvaluationContext)
ToolDefinitions	Gets set of tool definitions (see Tools) that were used when generating the model response that is being evaluated.
ToolDefinitionsContextName	Gets the unique Name that is used for ToolCallAccuracyEvaluatorContext.

Applies to