Validity of Scan Reports Despite Logged Errors

Hi Karthickumar Karuppiah,

Upon reviewing the provided report, it is my recommendation that it be considered invalid for the purposes of an actual red team assessment. Below are the key reasons supporting this conclusion:

• The logs indicate that HTTP 404 errors were encountered across all risk categories, meaning that the prompts could not be sent to the target system due to the unavailability of the Azure resource (project). As a result, no meaningful responses from the model were received or evaluated.
• The Attack Success Rate (ASR) for all tests is recorded as zero. However, this result should not be interpreted as a success of the model; rather, it reflects that the tests could not be executed due to backend errors, preventing any meaningful attack simulations.
• Although evaluation results are present, they are based on empty or failed conversations. These outputs do not reflect a legitimate red team assessment or testing of the target system.

Feel free to accept this as an answer.
Thankyou for reaching out to The Microsoft Q&A Portal