Microsoft 365 Certification sample evidence guide overview

This sample evidence guide helps ISVs produce the correct evidence required to complete the Microsoft 365 Certification. This document includes the intent of each control and all sub-parts of a control, potential ways in which evidence could be collected for controls with sample evidence documents, policies, and screenshots. Additionally, this guide provides assistance on how to structure submitted evidence.

Any examples shared in this document do not represent the only evidence that can be used to prove that controls are being met. These are guidelines for the type of evidence that can help certification analysts decide if the control has been met by the ISV.

Please note: The actual interfaces, screenshots, and documentation used to satisfy the requirements for certification will vary depending on product use, system setup, and internal processes. Please note that where policy or procedure documentation are required, the ISV must send full versions of the actual documents and not screenshots as maybe shown in some of the examples.

Any screenshots that are to be submitted must be full screen screenshots with any URL, logged in user (please ensure that the name of the logged in user is visible in the screenshot) with the time and date stamp included. For Linux based systems, please include this information with/on the control evidence by using the command prompt to produce it.

ALL evidence when submitted needs to be less than 3-months old to ensure that by the time you complete the certification the evidence is still relevant and not stale. If this is not followed you may be asked to collect new evidence.

Please also note: No beta API’s can be used for the purposes of this certification or any sample used within this process.

It is recommended that you follow these guidelines to avoid having your assessment delayed due to insufficient evidence.

Microsoft 365 Certification structure

The certification has been structured around three security domains:

• Application security (Including connectivity checks where necessary)

• Operational security

• Data security and privacy

A penetration test is required for certification and will be reviewed under the application security domain. Please see sections 3 and 4 of evidence submission structure for more information.

Security domains are broken into control groups to help ISVs understand the structure of tasks and to split the evidence collection into small, manageable pieces. These control groups have been aligned with common enterprise resourcing structures to help identify internal teams for support, while allowing the teams to work in parallel to expedite the evidence gathering process.

Control(s): Assessment Activity Description - These control(s) and associated number (No.) are taken directly from the Microsoft 365 Certification checklist.

Intent: The intent of why the security control is included within the program and the specific risk that it is aimed to mitigate. The hope is that this information will provide ISVs with the reasoning behind the control to better understand the types of evidence that needs to be collected and what ISV’s must pay attention to and have awareness and understanding of in producing their evidence.

Example Evidence Guidelines: Given to help guide the evidence collection tasks on the Microsoft 365 Certification. This allows the ISV’s to clearly see examples of the type of evidence that can be used by the Certification Analyst who will use it to make a confident determination that a control is in place and maintained – it is by no means exhaustive in nature.

Example Evidence: This section gives example screenshots and images of potential evidence captured against each of the controls within the Microsoft 365 Certification, specifically for the Operational Security and Data Security and Privacy Security Domains. Please note any information with red arrows and boxes within the examples is to further aid your understanding of the requirements necessary to meet any control.

Evidence submission structure

Evidence submission against all applicable controls will be carried out through Partner Center, except for assessments in support of the Microsoft compliance recording and contact center certifications. These will typically need to go through a manual process, please skip this section and see next section regarding completing the certification if you are compliance recording & contact center.

To ensure certification analysts can easily identify the evidence supplied and successfully review it, please follow these recommendations:

1. For each of the sections ensure that the evidence is clearly labelled before submission. If there is more than one piece of evidence per control, please put the evidence into a single word/pdf file, including a commentary of what the evidence is showing. If the evidence consists of multiple word/pdf documents, i.e., supporting documentation, then please upload these as individual files. Please do not put these into a zip file for uploading into Partner Center as we do not accept zip files due to the risk of malware.

2. All external framework information should be given in full without redactions (we will only permit partial name of individuals to be redacted from these reports as this would fall under Personally Identifiable Information (PII)) - All parts of the reports should be included, for example, ISO 27001 Statement of Applicability (SOA) and Certificate, full SOC 2 Type 2 report and/or full PCI-DSS Attestation of Compliance (AOC). All documents are covered by Microsoft Partner Center agreement under Clause 7.

Section 7(a) includes the following NDA:

3. A full unredacted penetration test report must be sent through Partner Center when requested - PLEASE NOTE if this is not provided, the certification analysts will not be able to complete the audit for your Microsoft 365 Certification.

4. Internal and external infrastructure and web application penetration test – A penetration test report is required for all hosting environments.

   • For Infrastructure as a Service (IaaS) or ISV hosted (on-premises, private data center) an internal and external infrastructure and web application test is required.

   • For Platform as a Service ‘PaaS/Serverless’ the pen test should be from your web application and the underlying supporting infrastructure.

Note: Complimentary Penetration Testing – The free Microsoft pen test is limited to twelve days only. If when we scope your pen test, your app requires more than 12-days of testing then you will be asked to pay for the additional days, additionally if you require pen testing out of hours this will also incur additional costs. The pen test service provided is also limited to one pen test (including one re-test) per year, for example if you had your penetration test performed on 1st of September 2022 then you will be unable to obtain another one until 31st of August 2023.

This is applicable for all submission attempts, which means that this applies to your current submission cycle as well as if you decide to close your current submission and restart the process later within the same year, you will not be entitled to an additional pen test as one had already been performed for you.

5. All ISVs must complete their initial document submission within 14 days (This includes any backwards and forwards with your analyst) after they receive the ticket start email from the Microsoft admin team. The 14-day timeframe is for full completion, review, and moving the submission to full evidence stage. ISVs should be checking regularly to see if their analyst has required any amendments or requested any additional information or documents. During the 14-day period ISVs may submit the required information as many times as is necessary, however, please bear in mind that if the submission is not actively being worked on, it will be deemed as stale and closed in Partner Center and certification will need to be restarted. Under certain circumstances, an ISV might be provided up to 30 days for completion. If an ISV is unable to complete initial document submission within the given timeframe, their submission will be closed.

Additionally, all ISVs must complete their certification within 60 days of their submission being moved from the initial document submission stage to the full evidence collection stage. This includes any revisions and feedback provided by the assessor/auditor throughout the process. The 60-day timeframe is for full completion, review, and final QA of your evidence, this means that ISVs should be submitting their evidence at least two weeks before the completion date to ensure that certification is completed on time. During the 60-day period ISVs may submit evidence to Partner Center as many times as necessary. However, bear in mind that final submission as stated is at least two weeks before the completion date to give certification analysts time to review and QA the submission. After your 60-days are over, ISVs will be required to start the process again.

If issues arise during the certification process the certification analyst can grant a potential extension for valid concerns. An analyst may at their discretion grant an extension of time up to a maximum of an additional 30 days if ISVs are seen to be actively working on your submission. Please note if given an extension from 0 – 30 days the ISV will need to ensure that evidence is available for review two weeks before the end date of the extension.

If granted an extension but evidence is more than 3-months old, it may potentially fail QA as that evidence maybe potentially deemed as stale due to the length of time between when it was provided and final QA which will mean that new evidence for that control(s) will be required. Once the extension time is up there will be no further extensions and the ISV will be expected to submit their evidence (at least two weeks before the end of the extension), if not submitted the submission will be classed as failed and will be closed. If no active work on the submission has commenced at any time during the initial 60-days or during the extension time (up to 30-days), the submission will be classified as abandoned and will be closed.

If the submission has been classified as abandoned the ISV will need to restart the process with a new attestation and fresh evidence as the current evidence will be deemed as stale. Please note that ISVs are allowed only one submission in a year. If for any reason an ISV abandons the process once they are in the full evidence collection stage and their submission has been marked abandoned, they will be unable to restart the process until the following year.

Manual evidence submission structure – compliance recording & contact center

To help ensure certification analysts can easily identify the evidence supplied and successfully review it, please follow these recommendations for the evidence submission structure if submitting manually by request from the certification team.

1. Create a single document, which can be easily reviewed (i.e., in Word or PDF), for each security control group (e.g., anti-virus, patch management, etc.).

2. Name the single document after the Security Control Group to make it clear what the document contains.

3. Add your evidence artifacts to it, reference your organization’s supporting documentation that supports the control group and any additional notes for the certification analyst explaining what the artifact is and how this evidence meets the control (it will help your Certification Analysts if you name the images with their control numbers i.e., Data Security and Privacy Control No.1).

Note: Remember, where sampling is used, artifacts need to be taken from each device in the sample set, ensure the artifact also displays the system name to validate that the artifact is from the device being assessed and that no data is obscured or redacted.

4. All external framework information should be given in full without redactions (we will only permit name of individuals to be redacted from these reports as these falls under PII) - All parts of reports should be included, for example, ISO 27001 Statement of Applicability (SOA) and Certificate, full SOC 2 Type 2 report and/or full PCI-DSS Attestation of Compliance (AOC) Full HIPAA Report or FedRAMP. All documents are covered by Microsoft Partner Center agreement under section 7.

Section 7(a) includes the following NDA:

5. A full unredacted penetration test report must be sent to the certification analyst when requested - PLEASE NOTE if this is not provided, the certification analyst will not be able to complete your Microsoft 365 Certification.

6. Internal and external infrastructure and web application – A penetration test report is required for all hosting environments.

   • For Infrastructure as a Service (IaaS) or ISV hosted (on-premises, private data center) an internal and external infrastructure and web application test is required.

   • For Platform as a Service ‘PaaS/Serverless’ the pen test should be from your web application and the underlying supporting infrastructure.

Complimentary penetration testing - Please note that the free Microsoft pen test is limited to twelve days only. If an app requires more than 12-days of testing then the ISV will be asked to pay for the additional days. Additionally, if the ISV requires pen testing out of normal business hours based on the auditor’s location, this will also incur additional costs. The pen test service provided is limited to one free pen test (including one re-test) per year per submission attempt.

7. All ISVs must complete their initial document submission within 14 days (This includes any backwards and forwards with your analyst) after they receive the ticket start email from the Microsoft admin team. The 14-day timeframe is for full completion, review, and moving the submission to full evidence stage. ISVs should be checking regularly to see if their analyst has required any amendments or requested any additional information or documents. During the 14-day period ISVs may submit the required information as many times as is necessary, however, please bear in mind that if the submission is not actively being worked on, it will be deemed as stale and closed in Partner Center and certification will need to be restarted. Under certain circumstances, an ISV might be provided up to 30 days for completion. If an ISV is unable to complete initial document submission within the given timeframe, their submission will be closed.

Additionally, all ISVs must complete their certification within 60 days of their submission being moved from the initial document submission stage to the full evidence collection stage. This includes any revisions and feedback provided by the assessor/auditor throughout the process. The 60-day timeframe is for full completion, review, and final QA of your evidence, this means that ISVs should be submitting their evidence at least two weeks before the completion date to ensure that certification is completed on time. During the 60-day period ISVs may submit evidence to Partner Center as many times as necessary. However, bear in mind that final submission as stated is at least two weeks before the completion date to give certification analysts time to review and QA the submission. After your 60-days are over, ISVs will be required to start the process again.

If issues arise during the certification process the certification analyst can grant a potential extension for valid concerns. An analyst may at their discretion grant an extension of time up to a maximum of an additional 30 days if ISVs are seen to be actively working on your submission. Please note if given an extension from 0 – 30 days the ISV will need to ensure that evidence is available for review two weeks before the end date of the extension.

If granted an extension but evidence is more than 3-months old, it may potentially fail QA as that evidence maybe potentially deemed as stale due to the length of time between when it was provided and final QA which will mean that new evidence for that control(s) will be required. Once the extension time is up there will be no further extensions and the ISV will be expected to submit their evidence (at least two weeks before the end of the extension), if not submitted the submission will be classed as failed and will be closed. If no active work on the submission has commenced at any time during the initial 60-days or during the extension time (up to 30-days), the submission will be classified as abandoned and will be closed.

If the submission has been classified as abandoned the ISV will need to restart the process with a new attestation and fresh evidence as the current evidence will be deemed as stale. Please note that ISVs are allowed only one submission in a year. If for any reason an ISV abandons the process once they are in the full evidence collection stage and their submission has been marked abandoned, they will be unable to restart the process until the following year.

Creating the folder structure for compliance recording & contact center

Follow these instructions to create the folder structure needed for the analyst to review the evidence supplied.

1. Complete the initial document submission so the controls can be scoped. Please provide as much detail as possible and that the architectural diagram and data flow diagram also have the same level of detail.

2. Create either an internal or online folder and add all your documents to it.

   • Label the actual shared folder Microsoft Certification

   • Add your Initial Document Submission information into the Microsoft Certification folder in a folder called IDS.

   • Inside the Certification folder create four folders with the following names:

     • Application Security (this is for your Pen Test information)

     • Compliance (for your external frameworks such as SOC 2)

     • Operational Security (OS)

     • Data Security & Privacy (DS&P)

   • Inside the OS and DS&P folders please create control groups for each set of controls i.e. Malware, Patching etc. where you can add documents for each of the controls (if you wish to be specific you can create a folder for each individual control making it easier for yourselves to track evidence by the control name – in this case please call these folders Control X where X stands for the control number). Please note you will not be able to create the secondary folder structure until after your controls have been scoped.

An example of a folder structure Root Folders:

Subfolders within the "Evidence" folder:

3. After you upload documents to the share, please give permission to the shared folder and email the certification analyst with the details. Please send any passwords in a separate email.

4. When collating evidence please remember to take full screen screenshots showing any logged in user, URL and date and time stamp. If using Linux this can be done from the command prompt. Provide any explanation where necessary for each control and remember that for policies a full copy of the policy and not snippets or screenshots are required.

5. Reference this document to help understand the control and the type of evidence we require.

6. Any pen test that may be required will not be scheduled and you will not receive documentation to get the process started until you have had 50% of your controls approved. The pen test will be free for 12 days only, however if your pen test runs over 12 days then you will be required to pay the additional days upfront before the test starts.

7. Once you receive a copy of your scoped controls to gather evidence against the controls, your ticker will start – you will receive an email to that effect, and you will have 60-days to complete the whole certification process including the pen test.

8. Please try to submit the first iteration of the controls within 30 days to allow any issues to be identified and revisions requested before the 60-days run out.

Learn more

• Application security (Including connectivity checks where necessary)
• Sample evidence: operational security
• Sample evidence: data security and privacy