How to pass dynamic parameters (eg, index name, endpoint, key) from Semantic Kernel API to MCP server without involving LLM decision-making in enterprise production environment?

Kaja Sherief 135 Reputation points
2025-08-28T09:40:31.1466667+00:00

Problem Description:

I'm building a production-grade enterprise application using Semantic Kernel with Model Context Protocol (MCP) servers for vector search capabilities. I need to dynamically pass an Azure AI Search /Pinecone index name from my FastAPI service to the MCP server based on user requests, but I want this routing to be deterministic and not involve the LLM in the decision-making process.

Current Architecture:

  • API Layer: FastAPI service with Semantic Kernel ChatCompletionAgent
  • MCP Layer: FastMCP server handling Pinecone vector searches
  • Infrastructure: Azure OpenAI, Azure AI Search and Pinecone vector databases with multiple dbs and indexes

Current Implementation:

# API Side (semantic-kernel-client/main.py)
  @app.post("/chat", response_model=ChatResponse)
  async def chat_endpoint(
      request: ChatRequest,
      conversation_id: str = Depends(get_conversation_id),
      client_id: str = Depends(get_client_id),
      bot_id: str = Depends(get_bot_id)
  ):
      # How to pass index name to MCP without LLM involvement?
      result_item = await agent.get_response(
          messages=request.message,
          thread=channel.thread,
      )
  # MCP Server Side (common-mcp-server/main.py)
  @mcp.tool(structured_output=True)
  async def search_pinecone_tool(query: str, top_k: int = 5) -> str:     
      # Currently uses hardcoded index
      # Need dynamic index selection
      pinecone_index = _pinecone_client.Index("azure-docs-chat")

Requirements:

  • Enterprise-Grade: Must be production-ready, scalable, and maintainable
  • Deterministic: Index selection should not depend on LLM interpretation
  • Performance: Minimal latency overhead
  • Security: Support authorization and audit trails
  • Observability: Compatible with enterprise monitoring tools

What I've Considered:

  1. HTTP Headers Approach: Pass index name via request headers
  2. Service Registry Pattern: Create separate MCP connections per index
  3. URL Query Parameters: Append index to MCP server URL
  4. Session-Based Storage: Store index preference in session data

Specific Questions:

  1. What is the recommended Microsoft/enterprise pattern for passing context from Semantic Kernel to MCP servers in production environments?
  2. How can I modify the MCPStreamableHttpPlugin to dynamically pass parameters without involving the LLM in routing decisions?
  3. Is there a way to intercept and modify MCP requests before they reach the server to inject context headers?
  4. What's the best practice for managing multiple MCP connections with different configurations in a single Semantic Kernel application?
  5. Are there any built-in Semantic Kernel features for context passing to MCP servers that I might be missing?

Environment Details:

  • Semantic Kernel Version: Latest Python version with MCP support
  • Framework: FastAPI + Semantic Kernel + Azure OpenAI
  • MCP Implementation: FastMCP with Pinecone integration
  • Deployment: Azure Container Apps (planned)
  • Scale: Enterprise production with multiple concurrent users Any guidance on Microsoft's recommended patterns for this use case would be greatly appreciated!
Azure
Azure

A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.

0 comments
Answer accepted by question author
Ofek Ben Eliezer 75 Reputation points MVP
2025-08-28T14:00:28.8533333+00:00

Do the routing in your API, not in the model. In Semantic Kernel you can call MCP tools directly with explicit arguments and keep the LLM out of the decision loop. For transport-level context, front the MCP server with an SSE-capable proxy like Azure API Management and inject headers there. Details below.

A recommended, deterministic pattern

  1. Programmatic tool invocation - no LLM routing

Disable auto function calling and invoke the MCP tool yourself with the exact args you want to pass, like index_name, endpoint, top_k. In SK this is done with Function Choice Behavior or by directly invoking the function. This keeps routing deterministic and auditable.

Python - API side (FastAPI + SK)

from semantic_kernel import Kernel
from semantic_kernel.functions import KernelArguments
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion())

# Add MCP plugin (SSE)
from semantic_kernel.connectors.mcp import MCPSsePlugin
pinecone_plugin = MCPSsePlugin(name="pinecone_mcp", url="https://mcp.example.com/mcp")  # SSE endpoint
await kernel.add_plugin(pinecone_plugin)  # tools become kernel functions
# Tip: you can register multiple MCP plugins with different names for different indices

# Deterministic call - no LLM involved
fn = kernel.plugins.get_function("pinecone_mcp", "search_pinecone_tool")
args = KernelArguments(query=request.query, top_k=5, index_name=request.index)  # pass index deterministically
result = await fn.invoke(kernel, arguments=args)  # returns tool result directly
  • If you still want an agent reply around the tool output, run a second LLM step that formats result into text, but keep the tool call itself outside the LLM.

Make the index a first-class parameter

On the MCP server, update the tool signature to accept index_name (and optionally service_id or tenant_id). Route to the correct Pinecone or Azure AI Search connection in server code.

  • Do not pass secrets from SK to MCP. Resolve keys in the server via Managed Identity + Key Vault.

Python - MCP server side (FastMCP)

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("vector-search")

@mcp.tool()
async def search_pinecone_tool(query: str, index_name: str, top_k: int = 5) -> dict:
    # look up credentials securely
    from azure.identity import DefaultAzureCredential
    from azure.keyvault.secrets import SecretClient
    kv = SecretClient(f"https://{KV_NAME}.vault.azure.net", DefaultAzureCredential())
    api_key = (await kv.get_secret(f"pinecone-{index_name}-key")).value
    endpoint = (await kv.get_secret(f"pinecone-{index_name}-endpoint")).value
    # instantiate client and query...
    return {"hits": [...]}

Key Vault + Managed Identity is the Microsoft-recommended pattern for secrets at runtime.

Connection registry - many indexes

If you have many indexes, either:

Register one MCP plugin per logical route (for example pinecone_sales, pinecone_support) and pick the plugin by tenant in the API, or

Keep a single tool that accepts index_name and look up connection details on the server.

Was this answer helpful?

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.