GLEUEvaluator Class

Definition

Namespace:: Microsoft.Extensions.AI.Evaluation.NLP

Assembly:: Microsoft.Extensions.AI.Evaluation.NLP.dll

Package:: Microsoft.Extensions.AI.Evaluation.NLP v9.7.0-preview.1.25356.2

Source:: GLEUEvaluator.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

An IEvaluator that evaluates the quality of a response produced by an AI model by comparing it to a reference response using the GLEU (Google-BLEU) algorithm. The GLEU evaluator measures the similarity between the generated response and one or more reference responses using n-gram overlap.

public ref class GLEUEvaluator sealed : Microsoft::Extensions::AI::Evaluation::IEvaluator

public sealed class GLEUEvaluator : Microsoft.Extensions.AI.Evaluation.IEvaluator

type GLEUEvaluator = class
    interface IEvaluator

Public NotInheritable Class GLEUEvaluator
Implements IEvaluator

Inheritance: Object
GLEUEvaluator

Implements: IEvaluator

Remarks

The GLEUEvaluator computes the GLEU score of a response ("hypothesis") compared to one or more reference responses supplied via References. The score is returned in a NumericMetric with a value between 0.0 and 1.0 where 0.0 represents no match at all and 1.0 indicates a perfect match. By default, the score is interpreted with a pass/fail cutoff of 0.5. So a score of 0.5 or higher is passing and a score below 0.5 is failing.

Constructors

GLEUEvaluator()

Properties

EvaluationMetricNames	Gets the Names of the EvaluationMetrics produced by this IEvaluator.
GLEUMetricName	Gets the Name of the NumericMetric returned by GLEUEvaluator.

Methods

EvaluateAsync(IEnumerable<ChatMessage>, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)

Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.

Extension Methods

EvaluateAsync(IEvaluator, ChatMessage, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.

Applies to