Edit

Generate text embeddings with Foundry Local

The Foundry Local SDK provides an embedding API that converts text into numerical vectors on-device. Use these vectors for similarity search, classification, clustering, and retrieval-augmented generation (RAG).

The SDK supports both single-input and batch embedding generation through a dedicated embedding client.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

pip install foundry-local-sdk-winml openai

Generate text embeddings

Copy and paste the following code into a Python file named app.py:

from foundry_local_sdk import Configuration, FoundryLocalManager



def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Select and load an embedding model from the catalog
    model = manager.catalog.get_model("qwen3-embedding-0.6b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True,
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get an embedding client
    client = model.get_embedding_client()

    # Generate a single embedding
    print("\n--- Single Embedding ---")
    response = client.generate_embedding("The quick brown fox jumps over the lazy dog")
    embedding = response.data[0].embedding
    print(f"Dimensions: {len(embedding)}")
    print(f"First 5 values: {embedding[:5]}")

    # Generate embeddings for multiple inputs
    print("\n--- Batch Embeddings ---")
    batch_response = client.generate_embeddings(
        [
            "Machine learning is a subset of artificial intelligence",
            "The capital of France is Paris",
            "Rust is a systems programming language",
        ]
    )

    print(f"Number of embeddings: {len(batch_response.data)}")
    for i, data in enumerate(batch_response.data):
        print(f"  [{i}] Dimensions: {len(data.embedding)}")

    # Clean up
    model.unload()
    print("\nModel unloaded.")


if __name__ == "__main__":
    main()

Run the code by using the following command:

python app.py

Troubleshooting

  • ModuleNotFoundError: No module named 'foundry_local_sdk': Install the SDK by running pip install foundry-local-sdk.
  • Model not found: Run the optional model listing snippet to find an alias available on your device, then update the alias passed to get_model.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/cs/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

The C# samples in the GitHub repository are preconfigured projects. If you're building from scratch, you should read the Foundry Local SDK reference for more details on how to set up your C# project with Foundry Local.

Generate text embeddings

Copy and paste the following code into a C# file named Program.cs:

using Microsoft.AI.Foundry.Local;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

// Initialize the singleton instance.
await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
var mgr = FoundryLocalManager.Instance;

// Get the model catalog
var catalog = await mgr.GetCatalogAsync();

// Get an embedding model
var model = await catalog.GetModelAsync("qwen3-embedding-0.6b") ?? throw new Exception("Embedding model not found");

// Download the model (the method skips download if already cached)
await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f)
    {
        Console.WriteLine();
    }
});

// Load the model
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");

// Get an embedding client
var embeddingClient = await model.GetEmbeddingClientAsync();

// Generate a single embedding
Console.WriteLine("\n--- Single Embedding ---");
var response = await embeddingClient.GenerateEmbeddingAsync("The quick brown fox jumps over the lazy dog");
var embedding = response.Data[0].Embedding;
Console.WriteLine($"Dimensions: {embedding.Count}");
Console.WriteLine($"First 5 values: [{string.Join(", ", embedding.Take(5).Select(v => v.ToString("F6")))}]");

// Generate embeddings for multiple inputs
Console.WriteLine("\n--- Batch Embeddings ---");
var batchResponse = await embeddingClient.GenerateEmbeddingsAsync([
    "Machine learning is a subset of artificial intelligence",
    "The capital of France is Paris",
    "Rust is a systems programming language"
]);

Console.WriteLine($"Number of embeddings: {batchResponse.Data.Count}");
for (var i = 0; i < batchResponse.Data.Count; i++)
{
    Console.WriteLine($"  [{i}] Dimensions: {batchResponse.Data[i].Embedding.Count}");
}

// Tidy up - unload the model
await model.UnloadAsync();
Console.WriteLine("\nModel unloaded.");

Run the code by using the following command:

dotnet run

Troubleshooting

  • Build errors referencing net8.0: Install the .NET 8.0 SDK, then rebuild the app.
  • Model not found: Run the optional model listing snippet to find an alias available on your device, then update the alias passed to GetModelAsync.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

npm install foundry-local-sdk-winml openai

Generate text embeddings

Copy and paste the following code into a JavaScript file named app.js:

import { FoundryLocalManager } from 'foundry-local-sdk';

// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');

const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});
console.log('✓ SDK initialized successfully');

// Get an embedding model
const modelAlias = 'qwen3-embedding-0.6b';
const model = await manager.catalog.getModel(modelAlias);

// Download the model
console.log(`\nDownloading model ${modelAlias}...`);
await model.download((progress) => {
    process.stdout.write(`\rDownloading... ${progress.toFixed(2)}%`);
});
console.log('\n✓ Model downloaded');

// Load the model
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');

// Create embedding client
console.log('\nCreating embedding client...');
const embeddingClient = model.createEmbeddingClient();
console.log('✓ Embedding client created');

// Generate a single embedding
console.log('\n--- Single Embedding ---');
const response = await embeddingClient.generateEmbedding(
    'The quick brown fox jumps over the lazy dog'
);

const embedding = response.data[0].embedding;
console.log(`Dimensions: ${embedding.length}`);
console.log(`First 5 values: [${embedding.slice(0, 5).map(v => v.toFixed(6)).join(', ')}]`);

// Generate embeddings for multiple inputs
console.log('\n--- Batch Embeddings ---');
const batchResponse = await embeddingClient.generateEmbeddings([
    'Machine learning is a subset of artificial intelligence',
    'The capital of France is Paris',
    'Rust is a systems programming language'
]);

console.log(`Number of embeddings: ${batchResponse.data.length}`);
for (let i = 0; i < batchResponse.data.length; i++) {
    console.log(`  [${i}] Dimensions: ${batchResponse.data[i].embedding.length}`);
}

// Unload the model
console.log('\nUnloading model...');
await model.unload();
console.log('✓ Model unloaded');

Run the code by using the following command:

node app.js

Troubleshooting

  • Cannot find module 'foundry-local-sdk': Run npm install foundry-local-sdk to install the SDK.
  • Model not found: Verify the model alias is correct. Use manager.catalog.getModels() to list available models.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/rust/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

Generate text embeddings

Replace the contents of main.rs with the following code:

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use foundry_local_sdk::{FoundryLocalConfig, FoundryLocalManager};

const ALIAS: &str = "qwen3-embedding-0.6b";

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("Native Embeddings");
    println!("=================\n");

    // ── 1. Initialise the manager ────────────────────────────────────────
    let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;

    // ── 2. Pick a model and ensure it is downloaded ─────────────────────
    let model = manager.catalog().get_model(ALIAS).await?;
    println!("Model: {} (id: {})", model.alias(), model.id());

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                std::io::Write::flush(&mut std::io::stdout()).ok();
            }))
            .await?;
        println!();
    }

    println!("Loading model...");
    model.load().await?;
    println!("✓ Model loaded\n");

    // ── 3. Create an embedding client ───────────────────────────────────
    let client = model.create_embedding_client();

    // ── 4. Single embedding ─────────────────────────────────────────────
    println!("--- Single Embedding ---");
    let response = client
        .generate_embedding("The quick brown fox jumps over the lazy dog")
        .await?;

    let embedding = &response.data[0].embedding;
    println!("Dimensions: {}", embedding.len());
    println!(
        "First 5 values: {:?}",
        &embedding[..5]
    );

    // ── 5. Batch embeddings ─────────────────────────────────────────────
    println!("\n--- Batch Embeddings ---");
    let batch_response = client
        .generate_embeddings(&[
            "Machine learning is a subset of artificial intelligence",
            "The capital of France is Paris",
            "Rust is a systems programming language",
        ])
        .await?;

    println!("Number of embeddings: {}", batch_response.data.len());
    for (i, data) in batch_response.data.iter().enumerate() {
        println!("  [{i}] Dimensions: {}", data.embedding.len());
    }

    // ── 6. Unload the model ─────────────────────────────────────────────
    println!("\nUnloading model...");
    model.unload().await?;
    println!("Done.");

    Ok(())
}

Run the code by using the following command:

cargo run

Troubleshooting

  • Build errors: Ensure you have Rust 1.70.0 or later installed. Run rustup update to get the latest version.
  • Model not found: Verify the model alias is correct. Use manager.catalog().get_models().await? to list available models.
  • Slow first run: Model downloads can take time the first time you run the app.

Related content

  • Last updated on