Getting 404 error when trying Azure AI foundry AI Red Teaming

Abhishek Ramdhan Handibag 25 Reputation points
2025-07-30T05:47:28.2666667+00:00

I am trying to work on AI red teaming in AI foundry, and retrieved a sample from documentation even for that even though complete setup is there like AZURE_SUBSCRIPTION_ID, AZURE_RESOURCE_GROUP, AZURE_PROJECT_NAME.

here is my code:

and error

import os
import asyncio
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential
from azure.ai.evaluation.red_team import RedTeam, RiskCategory

# Load environment variables from .env file
load_dotenv()

# Azure AI Project Configuration (use either block below)
# Option 1: Provide dictionary config
azure_ai_project = {
    "subscription_id": os.environ.get('AZURE_SUBSCRIPTION_ID'),
    "resource_group_name": os.environ.get("AZURE_RESOURCE_GROUP"),
    "project_name": os.environ.get("AZURE_PROJECT_NAME"),
}
# Option 2: Use direct project endpoint URL (if set in env)
# azure_ai_project = os.environ.get("AZURE_AI_PROJECT")

# Instantiate Azure Credential
credential = DefaultAzureCredential()

# Define a simple callback -- this simulates your model/app's response
def simple_callback(query: str) -> str:
    # Replace this with a call to your own model or application
    return "I'm an AI assistant that follows ethical guidelines. I cannot provide harmful content."

# Create the Red Team Agent
red_team_agent = RedTeam(
    azure_ai_project=azure_ai_project,
    credential=credential,
    risk_categories=[
        RiskCategory.Violence,
        RiskCategory.HateUnfairness,
        RiskCategory.Sexual,
        RiskCategory.SelfHarm
    ],
    num_objectives=2,  # Number of attack prompts per risk category (customize as needed)
)

# Run the scan
async def run_scan():
    red_team_result = await red_team_agent.scan(target=simple_callback)
    # Optionally, save results to a JSON file
    await red_team_agent.scan(target=simple_callback, output_path="My-First-RedTeam-Scan.json")
    print("Scan complete. Results saved to My-First-RedTeam-Scan.json.")

# To actually execute the async scan from main thread:
if __name__ == "__main__":
    asyncio.run(run_scan())

error:Exception: Failed to connect to your Azure AI project. Please check if the project scope is configured correctly, and make sure you have the necessary access permissions. Status code: 404.

resources used: https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent
and
https://github.com/Azure-Samples/azureai-samples/blob/main/scenarios/evaluate/AI_RedTeaming/AI_RedTeaming.ipynb

Content Safety in Foundry Control Plane
Content Safety in Foundry Control Plane

An Azure service that enables users to identify content that is potentially offensive, risky, or otherwise undesirable. Previously known as Azure Content Moderator.

Locked Question. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

Answer accepted by question author
Aryan Parashar 3,695 Reputation points Microsoft External Staff Moderator
2025-08-07T10:02:27.83+00:00

Hi Abhishek,

I am also getting the same error while following the Microsoft documentation that you are using.

Please follow the steps below to fix the issue:

  1. Go to AI Foundry and copy the endpoint as shown in the image
    User's image
  2. Replace it in your code as shown below:

Instead of using

azure_ai_project = {subscription_id, resource_group_name, project_name}

paste the endpoint which you copied above as shown below:

azure_ai_project = "<Azure AI Foundry project endpoint>"

Below is the full code for your reference:

import os
import asyncio
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential
from azure.ai.evaluation.red_team import RedTeam, RiskCategory

# Load environment variables from .env file
load_dotenv()


# Option 1: Provide Azure AI Foundry project endpoint copied as shown in above steps
azure_ai_project = "https://your-foundry-name.services.ai.azure.com/api/projects/your-project-name"

# Option 2: Use the endpoint copied above as an environment variable
azure_ai_project = os.environ.get("AZURE_AI_PROJECT")

# Instantiate Azure Credential
credential = DefaultAzureCredential()

# Define a simple callback -- this simulates your model/app's response
def simple_callback(query: str) -> str:
    # Replace this with a call to your own model or application
    return "I'm an AI assistant that follows ethical guidelines. I cannot provide harmful content."

# Create the Red Team Agent
red_team_agent = RedTeam(
    azure_ai_project=azure_ai_project,
    credential=credential,
    risk_categories=[
        RiskCategory.Violence,
        RiskCategory.HateUnfairness,
        RiskCategory.Sexual,
        RiskCategory.SelfHarm
    ],
    num_objectives=2,  # Number of attack prompts per risk category (customize as needed)
)

# Run the scan
async def run_scan():
    red_team_result = await red_team_agent.scan(target=simple_callback)
    # Optionally, save results to a JSON file
    await red_team_agent.scan(target=simple_callback, output_path="My-First-RedTeam-Scan.json")
    print("Scan complete. Results saved to My-First-RedTeam-Scan.json.")

# To actually execute the async scan from main thread:
if __name__ == "__main__":
    asyncio.run(run_scan())

Feel free to accept this as an answer.

Thank you for reaching to Microsoft QNA Portal

Was this answer helpful?

3 people found this answer helpful.

0 additional answers

Sort by: Newest