F1Evaluator Class

Definition

Namespace:: Microsoft.Extensions.AI.Evaluation.NLP

Assembly:: Microsoft.Extensions.AI.Evaluation.NLP.dll

Package:: Microsoft.Extensions.AI.Evaluation.NLP v9.7.0-preview.1.25356.2

Source:: F1Evaluator.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

An IEvaluator that evaluates the quality of a response produced by an AI model by comparing it to a reference response using the F1 scoring algorithm. F1 score is the ratio of the number of shared words between the generated response and the reference response.

public ref class F1Evaluator sealed : Microsoft::Extensions::AI::Evaluation::IEvaluator

public sealed class F1Evaluator : Microsoft.Extensions.AI.Evaluation.IEvaluator

type F1Evaluator = class
    interface IEvaluator

Public NotInheritable Class F1Evaluator
Implements IEvaluator

Inheritance: Object
F1Evaluator

Implements: IEvaluator

Remarks

The F1Evaluator computes the F1 score of a response ("hypothesis") in relation to a ground-truth reference supplied by GroundTruth. The score is returned in a NumericMetric with a value between 0.0 and 1.0 where 0.0 represents no match at all and 1.0 indicates a perfect match. By default, the score is interpreted with a pass/fail cutoff of 0.5. So a score of 0.5 or higher is passing and a score below 0.5 is failing.

Constructors

F1Evaluator()

Properties

EvaluationMetricNames	Gets the Names of the EvaluationMetrics produced by this IEvaluator.
F1MetricName	Gets the Name of the NumericMetric returned by F1Evaluator.

Methods

EvaluateAsync(IEnumerable<ChatMessage>, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)

Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.

Extension Methods

EvaluateAsync(IEvaluator, ChatMessage, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.

Applies to