BLEUEvaluator Class

Definition

Namespace:: Microsoft.Extensions.AI.Evaluation.NLP

Assembly:: Microsoft.Extensions.AI.Evaluation.NLP.dll

Package:: Microsoft.Extensions.AI.Evaluation.NLP v9.7.0-preview.1.25356.2

Source:: BLEUEvaluator.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

An IEvaluator that evaluates the quality of a response produced by an AI model by comparing it to a reference response using the BLEU (Bilingual Evaluation Understudy) algorithm. It is often used to evaluate the quality of machine translation or text generation tasks.

public ref class BLEUEvaluator sealed : Microsoft::Extensions::AI::Evaluation::IEvaluator

public sealed class BLEUEvaluator : Microsoft.Extensions.AI.Evaluation.IEvaluator

type BLEUEvaluator = class
    interface IEvaluator

Public NotInheritable Class BLEUEvaluator
Implements IEvaluator

Inheritance: Object
BLEUEvaluator

Implements: IEvaluator

Remarks

The BLEUEvaluator computes the BLEU score of a response ("hypothesis") compared to one or more reference responses supplied via References. The score is returned in a NumericMetric with a value between 0.0 and 1.0 where 0.0 represents no match at all and 1.0 indicates a perfect match. By default, the score is interpreted with a pass/fail cutoff of 0.5. So a score of 0.5 or higher is passing and a score below 0.5 is failing.

Constructors

BLEUEvaluator()

Properties

BLEUMetricName	Gets the Name of the NumericMetric returned by BLEUEvaluator.
EvaluationMetricNames	Gets the Names of the EvaluationMetrics produced by this IEvaluator.

Methods

EvaluateAsync(IEnumerable<ChatMessage>, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)

Evaluates the supplied modelResponse and returns an EvaluationResult containing one or more EvaluationMetrics.

Extension Methods

EvaluateAsync(IEvaluator, ChatMessage, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatMessage, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, ChatResponse, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.
EvaluateAsync(IEvaluator, String, String, ChatConfiguration, IEnumerable<EvaluationContext>, CancellationToken)	Evaluates the supplied `modelResponse` and returns an EvaluationResult containing one or more EvaluationMetrics.

Applies to