School Data Sync Overview

Article
05/10/2024

School Data Sync

School Data Sync (SDS) is a free service for Education that helps to automate the process of synchronizing user and roster data from Student Information or Management Systems with Microsoft 365. SDS is a tool that helps you manage your educational organization, users, classes, and roles. SDS syncs your data with Microsoft Entra ID and Microsoft 365, so you can use Microsoft Teams, Intune for Education, Exchange Online, SharePoint Online, OneNote Class notebooks, and enable third-party apps with Single Sign On integration. SDS helps make learning more engaging and efficient.

SDS prerequisites

Microsoft 365 Education tenant
Need Global Administrator Permissions
Global Administrator accessing SDS needs SDS Plan 1 (A1) or SDS Plan 2 (A3 / A5) License

SDS core concepts

School Data Sync models the best practices of the Education Customer Success Team that helps institutions successfully set up and use School Data Sync to empower Microsoft 365 experiences globally. We believe that applying these best practices, to improve the IT administrator’s experience for onboarding and monitoring, along with better understanding of the health of their data, provides more time for administrators to help educators focus on innovations to improve learning outcomes.

The onboarding experience is broken down into connect data and managing data to enable scenarios to improve learning outcomes.

Connect data

First, you need to connect to your institution’s data. You're defining how we're bringing in or connecting to your Student Information System’s (SMS) / Student Management System’s (SMS) data. Here you define your source system, the format of the incoming data, your user identity matching rules, and your active academic year. By default, your sync activates twice (2x) daily, called a run.

When the run starts, it connects to the defined source and perform basic validation. Basic validation ensures that the connection is correct for OneRoster API as a source, or the filenames and headers are correct for SDS CSV as a source.

Next, the system transforms the data for import in preparation for advanced validation.

If the source system/data doesn't provide the users' organization role entry date and exit date, the system will not infer the entry date or exit date based on the presence or nonpresence of the record.
If the source system/data doesn't provide the users enrollment entry date and exit date, the system will not infer the entry date or exit data based on the presence or nonpresence of the record.
If the source system/data doesn't provide a valid organization/enrollment/ or contact role for a user, the system does not add the user record to the system.
A user's association with a session is based on its role that is tied to an organization.
A user's association with a class is based on its role tied to an enrollment, which also includes a link to a session.

Additionally, the system stores an updated copy of your Microsoft Entra ID into the cache. The copy of Microsoft Entra assists with user matching between your SIS/ SMS and Microsoft Entra user object. At this stage, the match link isn't written to Microsoft Entra user object.

In the instance that a user has multiple roles, the following rules are used to determine what staff or student match rules should be used between the user record and the Microsoft Entra user object.

Additionally, the system stores an updated copy of your Microsoft Entra ID into the cache. The copy of Microsoft Entra assists with user matching between your SIS / SMS and Microsoft Entra user object. At this stage, the match link isn't written to Microsoft Entra user object.

If isPrimary is set for all student roles, even if association to a staff role exists, the match is made based on the student role.
If isPrimary is set for any staff role, even if association to a student role exists, the match is made based on the staff role.
If isPrimary is set for both staff and student role, the match is made based on the staff role.
If isPrimary isn't set for any roles, especially with a mix for both staff and student roles, the match is based on the staff role.

Validate and data health

Next, your SDS performs advanced validation to determine the health of your data. Validation focuses on identifying errors and/or warnings. Validation follows the concept of bringing in good data and keeping out bad data.

Errors mean that a record didn't pass validation based on required data and was removed from further processing
Warnings mean that the value on an optional field on a record didn't pass so the value was removed from the record, but the record was included for further processing. Errors and warnings are used to help you better understand the health of your data.

For the good data that passed validation, when the data is stored in our internal representation in the cache, it identifies when first seen by SDS, and for data linked with users' org, role association and class association, it's identified as active in session. In future runs, for the same connected data source, SDS identifies if the record is still seen, and based on the presence or absence of record, and keep the record active or mark it as no longer active.

The data reflects when a new record is presented for the first time. FirstSeenDateTime is the time the record was first seen by SDS and the creation data of that row in the table. This does NOT mean it's the creation date of the record in the SIS / SMS as SDS might have run well after the data was added to the external system.
- SDS sets the first seen date (time) and last modified date (time) to now, and if appropriate, it marks record as "is active in session" to be true.
The data reflects when the same record is present in the subsequent run. LastSeenDateTime is the moment SDS last saw the data in a sync.
- SDS preserves the first seen date (time) value, set the last modified date (time) to current, and leave "is active in session" to be true.
The data reflects when the same record isn't present in a subsequent run. This isn't the deletion date of the record from the SIS / SMS, but the date (time) of the last time SDS saw the record in a sync.
- SDS preserves the first seen date (time) and the last modified date (time) values and if appropriate, mark the record as "is active in session" to false.
The following are General Data Handling rules for data in the cache when data is missing on subsequent run, same source, and same academic session.
- Orgs: If missing on subsequent run, system does nothing and record persists.
- Users: If missing on subsequent run, system sets IsActiveInSession to False on the person organization role and enrollment.
- Roles: If missing on subsequent run, system sets IsActiveInSession to False on the person organization role.
- Academic sessions: If missing on subsequent run, do nothing and record persists.
- Classes: If missing on subsequent run, system sets IsActiveInSession to False for section, section session, and enrollment.
- Enrollments: If missing on subsequent run, system sets IsActiveInSession to False for enrollment.
- Courses: If missing on subsequent run, system sets IsActiveInSession to "False" for course.
- Demographics: If missing on subsequent run, system does nothing and record persists.
- User flags: If missing on subsequent run, system does nothing and record persists.
- Relationships: If missing on subsequent run, system does nothing and record persists.
There are rolling updates for "inactivated" records. For example, if a user record isn't present, the system preserves the existing first seen date (time) and last modified date (time) values. The system sets "is active in session" to false for the users' org/role and enrollment records.
- Organization, user, and session records persist over time and aren't inactivated.
- Only the association records to organization, user and session records, as described prior, and the association records IsActiveInSession status are impacted based on General Data Handling rules.
When the source data stops providing a user’s section enrollment record, and then provides the same user's section enrollment within the same academic session, the system updates the existing record.

At the end of each run, the data health is determined. If there are no errors and/or warnings, the system determines that no data actions are needed by the administrator. If there are errors and/or warnings, the system notifies that it found some issues with the data and encourage you to investigate the data health. Error and warnings statistics are produced to help provide an initial impact to data health. The administrator, when investigating data health, is able to download a log file that contains information based on the errors and warnings found. The administrator can then begin the data investigation process to correct the data in the source system.

Run Status
- Running: Actively executing
- Completed: Completed without any errors or warnings
- Completed with Errors: Completed but errors were found
- Completed with Warnings: Completed but only warnings were found
- Error: Run canceled by system
Counts
- Errors: Total number of errors encountered. Error means that a record didn't pass validation and the entire record was removed before processing, after validation. If a record couldn't be processed post validation, the count will also include that record as well.
- Warnings: Total number of warnings encountered. A warning means that one or more portions of data on the record didn't pass validation. The field that had the data for the record was removed but the rest of the record and validated data remain before processing post validation.

Manage data

Managing data to improve learning outcomes with SDS provides dynamic provisioning and roster updates for virtual classrooms in Microsoft 365, to assist with simplified deployment and adoption of Microsoft 365 Group-enabled apps like Teams, SharePoint, Exchange, and OneNote Class Notebooks. SDS also performs provisioning of education institutions, users and groups for IT to assist with simplified Microsoft 365 identity management, app management, and device management. SDS also provides the option to store SIS/SMS values it syncs within Microsoft Entra ID. Provisioning to Microsoft Entra ID can also make roster information available to third party applications through the Education APIs (Application Programming Interfaces) and Microsoft Graph.

You can set up your Manage data configuration immediately after defining your connect data configuration, during the active first run, or later after the first run is finished.

Based on data in the cache, associated to the active academic session, and is active.
Uses the sync end date defined during connect data configuration to also know when to stop running changes for managed data.
- Following academic session transition and defining a new sync end date, the managed data configuration starts up again.

To help support different Microsoft 365 scenarios, while also simplifying the admin experience, admins can define what provision type configurations they would like to enable.

Users provision type allows you to automate management and mapping of your Microsoft 365 users from your SIS users. This writes forward the mapping link, based on the sourcedId value to the user’s object in Microsoft Entra ID.

Based on data in the cache and is identified as active in session.
Write the matched user link to the Microsoft Entra user object with default and optional attributes.
- These default values are written for every mapped user:
  - User External Id: Data found in the sourcedId field for the user.
  - Organization External Id: Data found in the orgSourcedId field for the user’s associated organization and role.
  - Organization Role: Data found in the role field for the user and the associated organization.
- These optional values are written if selected and there's a value provided in the data:
  - User Grade Level: Data found in the grade field for the user for the associated organization and role.
  - User Number: Data found in the user number field for the user for the associated organization and role.
When a user has multiple roles, rules are used to determine the value when writing role to Microsoft Entra user object.
- If isPrimary is set for all student roles, even if association to a staff role exists, the role attribute value is made based on the student role.
- If isPrimary is set for any staff role, even if association to a student role exists, the role attribute value is made based on the staff role.
- If isPrimary is set for both staff and student role, the role attribute value is made based on the staff role.
- If isPrimary isn't set for any roles, especially with a mix for both staff and student roles, the role attribute value is based on the staff role.
If the user is also associated with multiple organizations, another rule is used to determine the value when writing the role to the Microsoft Entra user object.
- Utilize the organization role sort order value for the highest organization role to set to the Microsoft Entra user object.
There are another options when managing users with the role of 'Student' that can be enabled.
- Mark all students as minors: Based on the rules to determine what user role value is written to the Microsoft Entra user object if the role is Student. This setting identifies the user as a minor so that Microsoft and third-party applications identify them as such. The outcome sets the AgeGroup and ConsentProvidedForMinor user properties. The net result of setting the two properties is the attribute of LegalAgeGroupClassification set to MinorWithParentalConsent.
- Student contacts associations: Based on the rules to determine what user role value is written to the Microsoft Entra User object, if the user is a Student and if connected data from the SIS is included to bring in Contact Relationships, also known as Guardians or Parents. If this option is on when processing data, this includes the contact information with the Microsoft Entra user object for educator communication.
SDS also provides the ability for creating unmatched users for tenants not using AD Sync, or other user management tools, to manage user creation. You can enable if you would like SDS to create new users.
- By default, the setting is off. Don't enable this option if users are being created and managed in Microsoft Entra ID through other automated means.
- The Microsoft Entra UserPrincipalName value, also referred to as username, is constructed from the User identity rule selections, as part of the Connected data source configuration linked to the user’s SIS record.
  - This is because, after the user is created, the user identity rules are used to continue linking to the same user object in subsequent runs. This is to prevent creating a new user with one set of rules and then trying to identify them with a different set of rules.
- For Staff and Student users that weren't matched to existing Microsoft Entra Users and are to be created, you define their default password.
  - Password construction follows Microsoft Entra ID Password Protection guidance and assists with scoring calculation for default value to be accepted to successfully create the user.
  - The password configuration can be modified through the edit flow experience for those tenants that wish to rotate default passwords when creating new users at later dates.
- License selection is optional and doesn't have to be set. If selected this will set the default license when the user is created. If you're planning on using Group Based License assignment and management don't select a value.

Provision type for Class groups is for managing class groups and provides a space for users to connect with each other, communicate, and collaborate across various Microsoft 365 Apps including Teams. This creates a Class group for every active class found (where at a minimum, an active user that is linked with a mapped owner role is associated with the class). The admin also has the option to enable the ability to automatically create the Class team based on the corresponding class group.

Creates a Microsoft 365 group for each class or section.
Write group link to the Microsoft Entra group object with default and optional attributes.
- These default values are written for every class group managed:
  - Organization external ID: Data found in the orgSourcedId field of the section or class.
  - Class external ID: Data found in the sourcedId field of the section or class.
  - Class title: Data found in the title field of the section or class.
- These optional values are written if selected and there's a value provided in the data:
  - Class code: Data found in the code field of the section or class.
  - Course title: Data found in the title field of the course associated to the section or class.
  - Course code: Data found in the code or course code field of the course associated to the section or class.
  - Course subject: Data found in the subject or subjectCodes field of the course associated to the section or class. As a reminder, values provided from the SIS must match the corresponding List of Values to be accepted with the connected data. If not, the value isn't ingested, therefore no data is written forward to the group object until corrected.
  - Course grade level: Data found in the grade or grades code field of the course associated to the section or class. As a reminder, values provided from the SIS must match the corresponding List of Values to be accepted with the connected data. If not, the value isn't ingested, therefore no data is written forward to the group object until corrected.
  - Course external ID: Data found in the courseSourcedId field of the course associated to the section or class.
  - Academic session external ID: Data found in the sessionSourcedId code field of the academic session associated to the section or class.
  - Academic session title: Data found in the title field of the academic session associated to the section/class.
Define owners and members based on class or section enrollment and their enrollment role.
- A user's association with the Class Group is based on the enrollment role associated with sections or classes.
- Selecting a role means that for any user with that role for the associated group, the user is included to that section or class.
- User privileges for a class group are based on the associated enrollment role and if that role is selected in the Owners or Members list.
  - For any roles that aren't selected, they won't be linked or have access to the corresponding group, section, or class--even if there are users that have enrollment role records for the corresponding group.
- Owners have editing privileges to manage their groups, change group display names, and add or remove group members.
  - The owner will also be seen and listed as a member of the group, which enables them to see the section or class in Microsoft applications.
- Members that aren't Owners, have read-only permissions for their groups. They only gain access to the Class team when an Owner activates the class.
There are more options when managing class groups:
- Automated Teams creation: Selecting this option creates a Microsoft Group and Class team for each class or section synced. If the option isn't selected, only a Microsoft 365 Group is created.
  - If the toggle is on:
    - For Educators, or group owners that are using the created Class Teams from SDS, they have early access before the students and other group members. When the educator is ready, they can select Activate to allow students and other group members access.
  - If the toggle is off:
    - For Educators, or group owners, where SDS isn't creating Class Teams, they can create Class Teams based on the SDS Class groups manually withing teams. When the educator is ready, they can select Activate to allow students and other group members access.
- Class group display name: Selecting this option writes the class group display name during initial creation. This option will allow administrators and the group owner to update the group and class team display name and preserve the change from updates in future runs.

Security groups provision type is for managing Security groups and provides a grouping construct for use within various identity management, app management, and device management scenarios in Microsoft 365.

Role Groups: Creates Security Groups by User role groups. Example: One for all Students, another for all Staff.
- Creates all Students Security Group and manages users with roles in the Students role group.
- Creates all Staff Security Group and manages users with roles in the Staff role group.
Organizations, Organizations + Role Groups: Creates Security Groups by Organization + Role Groups combinations. Example: Contoso School containing Students - Contoso School and Staff - Contoso School.
- Creates a Security Group for each Organization present with active users.
- Nests corresponding Role group + Organization Security Groups to the Organization Security Group.
  - Creates a Security Group for each combination of users with roles in the Students role group + Organization present.
  - Creates a Security Group for each combination of users with roles in the Staff role group + Organization present.
SDS Security Groups can be used in various administrative functions within Microsoft Entra ID and Microsoft 365. Here are some of the most common uses of the SDS Security Groups:
- Intune for Education Device Policy - What is Intune for Education?
- Mobile Device Management - Create and deploy device security policies
- Group Based Licensing - Assign licenses to users by group membership in Microsoft Entra ID
- Conditional Access - What is conditional access?
- Group and Team Creation Policy - Manage creation of Groups
- Self Service Password Reset - Let users reset passwords
  - To target Security groups for Password Reset, choose "Selected" instead of "All" on step 5.

Administrative units provision type is for managing Administrative units and provides a grouping construct for delegated IT administration and scoped role assignments. Scoped role assignments allow admins to manage subsets of the broader Microsoft 365 directory.

Organizations: Creates Administrative Units by Organization and link organization users, organization class groups and organization security groups. The selection provides Organizations with the type ‘school’ to be available in the EDU Graph Schools endpoint. Example: Contoso School containing Contoso School users + class groups + security groups.
- Creates an Administrative Unit for each Organization present with active users.
- Links all Students with an active association to the Organization with roles in the Students role group.
- Links all Staff with an active association to the Organization with roles in the Staff role group.
- If also provisioning Class groups, links all Classes with an active association to the Organization.
Organizations + Role Groups: Creates Administrative Units by Organization + Role Group combination. SDS allows the ability for permitted staff to perform delegated IT administration for students of school administrative unit. Example: Students - Contoso School, Staff - Contoso Schools.
- Creates Students - Organization Administrative Unit for each Organization present with active users for each combination for users with roles in the Students role group.
- Creates Staff - Organization Administrative Unit for each Organization present with active users for each combination for users with roles in the Staff role group.