GDPR for SharePoint Server

Applies to:

• SharePoint Server 2013
• SharePoint Server 2016
• SharePoint Server 2019
• SharePoint Server Subscription Edition

As part of safeguarding personal information, we recommend the following:

• Classify your data, using Azure Information Protection.

• Run SharePoint Server in a least-privileged configuration. See Plan for least-privileged administration in SharePoint Server and Security for SharePoint Server for more information.

• Enable BitLocker encryption on your servers.

User Generated content

The basic recommended approach for user generated content contained in SharePoint Server sites and libraries is:

• Use Azure Information Protection to label sensitive data.

• Use SharePoint Server search and eDiscovery to retrieve sensitive data.

The recommended approach for files shares and SharePoint sites and libraries includes these steps:

1. Install and configure Azure Information Protection scanner.

   • Decide which sensitive data types to use.

   • Specify which SharePoint sites to use.

2. Complete a discovery cycle.

   • Run the scanner in discovery mode and validate the findings.

   • If needed, optimize the conditions and sensitive information types.

   • Assess the expected impact of automatically applying labels.

3. Run the Azure Information Protection scanner to apply labels to qualifying documents.

4. For protection:

   a. Configure Exchange data loss prevention rules to protect documents with the desired label.

   b. Be sure permissions to limit who can access files.

   c. For SharePoint, use IRM-protection for libraries.

5. For monitoring, integrate Windows Server logs with a SIEM tool.

   a. To find personal data for data subject requests, use Search Center or eDiscovery.

When applying labels to sensitive data, be sure to use a label that is not configured with protection. Protection includes encryption which prevents services from detecting sensitive data in the files.

For more information on using Azure Information Protection scanner to find and label personal data, see the Microsoft GDPR Data Discovery Toolkit (https://aka.ms/gdprpartners).

For information on configuring the scanner for conditions and using the Microsoft Purview Data Loss Prevention (DLP) sensitive information types, see How to configure conditions for automatic and recommended classification for Azure Information Protection. Note that new Office 365 sensitive information types will not be immediately available to use with the scanner and custom sensitive information types cannot be used with the scanner.

Removing personal information from Office files

Removing personal information (such as metadata or comments in a Word document) from Office files that are stored in a SharePoint document library must be done manually. Follow these steps:

1. Download a copy of the document from SharePoint Server to your local disk.

2. Delete the document from the SharePoint document library.

3. Follow the steps in Remove hidden data and personal information by inspecting documents.

4. Upload the document back to the SharePoint document library.

Telemetry and log files

ULS Logs

Unified Logging Service (ULS) and Usage logging in SharePoint Server track a variety of system functions and can contain user information. ULS logs and usage logs are text files and can be searched using a variety of searching tools. The Merge-SPLogFile PowerShell cmdlet provides a way to return records from the ULS logs on multiple servers in a farm.

Consider setting log retention policies to the minimum value needed for your business purposes. For information about configuring logging in SharePoint Server, see Configure diagnostic logging in SharePoint Server.

Note that some system events are also logged to the Windows Event Log.

Usage Database

The SharePoint Server Usage database (default name WSS_Logging) contains a subset of the information found in the ULS logs. The maximum retention of data in this database is 30 days. We recommend that you configure it for the shortest duration allowable by your business needs. For more information, see Configure diagnostic logging in SharePoint Server.

Personal information and search

The search query history and usage records contain references to user names.

Query history and favorite queries

In SharePoint Server, query histories and 'favorite' queries automatically expire after 365 days. If a user leaves your organization, it is possible to remove references to a user's name from the query history using the steps below.

The following SQL queries apply to SharePoint Server and make it possible to:

• Export a user's query history or favorite queries

• Remove references to user names in the query history

Export a user's queries since a specific date

Use the following procedure to export queries from the Link Store query log tables, performed by @UserName since @StartTime.

[In dbo].[LinkStore_<ID>]:
CREATE PROCEDURE proc_MSS_GetQueryTermsForUser 
( 
    @UserName nvarchar(256), 
    @StartTime datetime 
) 
AS 
BEGIN 
    SET NOCOUNT ON; 
    SELECT searchTime, queryString 
    FROM 
        dbo.MSSQLogPageImpressionQuery 
    WITH 
        (NOLOCK) 
    WHERE 
        userName = @UserName AND 
        searchTime > @StartTime 
END 
GO 

Export a user's queries from the past 100 days

DECLARE @FROMDATE datetime 
SET @FROMDATE = DATEADD(day, -100, GETUTCDATE()) 
EXECUTE proc_MSS_GetQueryTermsForUser '0#.w|domain\username', @FROMDATE 

Export a user's favorite queries

Use the following procedure to export a user's favorite queries from the Search Admin DB personal result tables, performed by @UserName, since <DateTime>.

In [dbo].[Search_<ID>]:
CREATE PROCEDURE proc_MSS_GetPersonalFavoriteQueries 
( 
    @UserName nvarchar(256), 
    @SearchTime datetime 
) 
AS 
BEGIN 
    SET NOCOUNT ON; 
    SELECT max(queries.SearchTime) as SearchTime, 
           max(queries.querystring) as queryString, 
           max(url.url) as URL 
    FROM MSSQLogOwner owners WITH(NOLOCK) 
    JOIN MSSQLogPersonalResults results WITH(NOLOCK) on owners.OwnerId = results.OwnerId 
    JOIN MSSQLogUrl url WITH(NOLOCK) on results.ClickedUrlId = url.urlId 
    JOIN MSSQLogPersonalQueries queries WITH(NOLOCK) on results.OwnerId = queries.OwnerId 
    WHERE queries.SearchTime > @SearchTime 
        AND queries.UserName = @UserName 
        GROUP BY queries.QueryString,url.url 
END 
GO 

Export a user's favorite queries from the past 100 days

DECLARE @FROMDATE datetime 
SET @FROMDATE = DATEADD(day, -100, GETUTCDATE()) 
EXECUTE proc_MSS_GetPersonalFavoriteQueries '0#.w|domain\username', @FROMDATE 

Remove references to user names that are more than X days old

Use the following procedure to remove references to all user names that are more than @Days old, from the Links Store query log tables. The procedure only removes references backwards in time until it reaches the @LastCleanupTime.

In [dbo].[LinksStore_<ID>]:  
CREATE PROCEDURE proc_MSS_QLog_Cleanup_Users 
( 
    @LastCleanupTime datetime, 
    @Days int 
) 
AS 
BEGIN 
    DECLARE @TooOld datetime 
    SET @TooOld = DATEADD(day, -@Days, GETUTCDATE()) 
    DECLARE @FromLast datetime 
    SET @FromLast = DATEADD(day, -@Days, @LastCleanupTime) 
    BEGIN TRANSACTION 
         UPDATE MSSQLogPageImpressionQuery 
    SET userName = 'NA' 
    WHERE @FromLast <= searchTime AND searchTime < @TooOld 
    UPDATE MSSQLogO14PageClick 
    SET userName = 'NA' 
    WHERE @FromLast <= searchTime AND searchTime < @TooOld 
    COMMIT TRANSACTION 
END 
GO 

Remove references to a specific user name that's more than X days old

Use the following procedure to remove references to a specific user name from the Links Store query log tables, where the references are more than @Days old. The procedure only removes references backwards in time until it reaches the @LastCleanupTime.

In [dbo].[LinksStore_<ID>]:
CREATE PROCEDURE proc_MSS_QLog_Cleanup_Users 
( 
    @UserName nvarchar(256),
    @LastCleanupTime datetime, 
    @Days int 
) 
AS 
BEGIN 
    DECLARE @TooOld datetime 
    SET @TooOld = DATEADD(day, -@Days, GETUTCDATE()) 
    DECLARE @FromLast datetime 
    SET @FromLast = DATEADD(day, -@Days, @LastCleanupTime) 
    BEGIN TRANSACTION 
         UPDATE MSSQLogPageImpressionQuery 
    SET userName = 'NA' 
    WHERE @FromLast <= searchTime AND searchTime < @TooOld AND userName = @UserName
    UPDATE MSSQLogO14PageClick 
    SET userName = 'NA' 
    WHERE @FromLast <= searchTime AND searchTime < @TooOld AND userName = @UserName
    COMMIT TRANSACTION 
END 
GO 

Remove references to all user names in the query history from a date and up to the past 30 days

EXECUTE proc_MSS_QLog_Cleanup_Users '1-1-2017', 30 

Delete usage records

SharePoint Server automatically deletes usage records after 3 years. You can manually delete such records using the procedure below:

To delete all usage records associated with deleted documents: 

1. Ensure that you have the latest SharePoint update installed. 

2. Start a SharePoint Management shell.

3. Stop and Clear the Usage Analytics analysis: 

   $tj = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition 
   $tj.DisableTimerjobSchedule()
   $tj.StopAnalysis() 
   $tj.ClearAnalysis() 
   $tj.EnableTimerjobSchedule()
   

4. Wait for the analysis to start again (might take up to 24 hours). 

5. On the next run of the analysis, it will dump all records from the Analytics Reporting database. This full dump might take a while for a large database with many entries.

6. Wait for 10 days. The analysis runs daily, and records associated with deleted documents will be removed after the 10^th^ run. This run might take longer than normal if many records need to be deleted. 

Personal information and search in SharePoint Server 2010

FAST Search Server 2010 for SharePoint 

In addition to storing files in the index, the FAST Search Server 2010 Add-On also stores files in an intermediate format called FixML. FiXML files are compacted regularly, by default between 3 am and 5 am every night. Compaction removes deleted files from the FiXML files automatically. To ensure timely removal of information belonging to deleted users or documents, ensure that compaction is always enabled.

Hybrid Search 

The recommended actions for hybrid search solutions are the same as for search in SharePoint Server or SharePoint Online. There are two hybrid search solutions:

The cloud hybrid search solution - With the cloud hybrid search solution for SharePoint, you index all your crawled content, including on-premises content, in your search index in Office 365. When users query your search index in Office 365, they get search results from both on-premises and Office 365 content. When documents are deleted from the SharePoint Server environment, they are also deleted from the search index in Office 365. Read more about the cloud hybrid search solution and how search components and databases interact in cloud hybrid search to understand better how GDPR affects the hybrid environment.

The hybrid federated search solution - With the hybrid federated search solution, you use both your index in SharePoint Server and your index in Office 365. Both SharePoint Server and SharePoint Online Search services can query the search index in the other environment and return federated results. When users search from a Search Center, the search results come from both your search index in SharePoint Server and your search index in Office 365. Read more about the hybrid federated search solution to understand better how GDPR affects the hybrid environment.

On Prem to Cloud Migrations

While migrating data from SharePoint Server to SharePoint Online, duplicate data might exist in both locations for a time. If you have data that you need to delete that is in mid-migration, we recommend that you complete the migration first, and then delete the data from both locations. You can query data for export from either location.

User Profile data

The User Profile Service allows for import of profile data from a variety of external sources. Queries for and update of such user profile data should be handled in the systems in which the data is mastered. If you make updates to the external system, be sure to synchronize the user profiles in SharePoint Server again.

Follow these basic steps to remove a user's personal information from their SharePoint Server user profile:

1. Remove the user information from any external systems that feed into the SharePoint Server user profile. If you are using directory synchronization, the user must be removed from the on-premises Active Directory environment.

2. Run a profile synchronization on SharePoint Server.

3. Delete the profile from SharePoint Server. Once this is done, SharePoint Server will fully remove the profile from the User Profile Database in 30 days. The user's profile page and personal site will be deleted.

After deleting a user's profile, some limited information (such as user ID) might still be recorded in site collections that the user has visited. If you choose to delete this data from a given site collection, this can be done using CSOM. A sample script is provided below:

$username = "<[email protected]>"
$password = "password"
$url = "<https://site.sharepoint.com>"
$securePassword = ConvertTo-SecureString $Password -AsPlainText -Force

# the path here might need to change if you use the SharePoint Server Client Components SDK in a different location.
Add-Type -Path "$env:CommonProgramFiles\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "$env:CommonProgramFiles\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

# connect/authenticate to SharePoint Online and get ClientContext object.
$clientContext = New-Object Microsoft.SharePoint.Client.ClientContext($url)
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword)
$clientContext.Credentials = $credentials
if (!$clientContext.ServerObjectIsNull.Value)
{
    Write-Host "Connected to SharePoint Online site: '$Url'" -ForegroundColor Green
}

# Get user
$user = $clientContext.Web.SiteUsers.GetByLoginName("i:0#.f|membership|[email protected]")

# Redact user
$user.Email = "Redacted"
$user.Title = "Redacted"
$user.Update()
$clientContext.Load($user)
$clientContext.ExecuteQuery()

# Get users
$users = $clientContext.Web.SiteUsers

# Remove user from site
$users.RemoveById($user.Id)
$clientContext.Load($users)
$clientContext.ExecuteQuery()