Share via

DFSR Error : Error: 9098 (A tombstoned content set deletion has been scheduled)

Question

Monday, August 19, 2013 6:44 PM | 1 vote

Hello,

We are getting this error on one of our DFSR shares and am not sure how to get this fixed. We have over 30 different shares replicating and all of them are fine except for one. I have tried to remove the Replication and the namespace and have recreated them both and continue to get the same error. This is what is in the event logs on the server;

The DFS Replication service stopped replication on the replicated folder at local path E:\SHARE\Sharelocation.

Additional Information:

Error: 9098 (A tombstoned content set deletion has been scheduled)

Additional context of the error:  

Replicated Folder Name: Sharelocation

Replicated Folder ID: 9BDA3E5A-8103-481F-84B2-269AE8A3BBC8

Replication Group Name: SHARE

Replication Group ID: C25E73B7-5B65-4299-A1BE-76E723443B02

Member ID: 568CC517-3329-42A4-ABAD-633CB3D1EF2E

If anyone has any suggestion on how to get this working again it would be greatly appreciated.

All replies (11)

Wednesday, August 21, 2013 7:18 AM | 4 votes

Hi,

Have a try with steps below to reset database on failed server:

1. Remove the failed server from replication group. Perform an AD replication to replicate the change to all members before adding it back to replication group.

2. Go to the failed server, in Windows Explorer please open the specific drive stored replicated folders.

3. Right click on the "System Volume Information" directory and select Properties\Security

Note: You might need to select the option for "Show hidden files, folders or drives" and also uncheck "Hide protected operating system files" in the folders view options to be able to even see the "System Volume Information" directory.

4. Grant your user account that you're logged in with (if a member of Administrators group this will also suffice) "Full Control" to the "System Volume Information" directory.

Note: You may get an error on setting security on some files - this is expected.

5. Open an elevated/Administrative command prompt. Switch to the "<drive letter>:\System Volume Information" directory

6. Type the command "rmdir DFSR /s"

7. Add it back to replication group. Some files are still exist so it will not perform a full replication but just the changed ones. See if the issue could be fixed.

TechNet Subscriber Support in forum |If you have any feedback on our support, please contact [email protected].

Wednesday, August 21, 2013 4:21 PM

I will give this a try. The one question I have is what impact will this have on the existing shares that are replicating without any problems? I don't want to make the problem worse by doing this.

Thursday, August 22, 2013 10:09 AM

Hi,

The first step is to remove the affect server from the replication group. After clear database, it will actually like add a new server to an exists replication group. It will ask to choose a healthy server as primary server so it will not affect current folders.

However you could still perform a backup or create a snapshot incase anything goes wrong.

TechNet Subscriber Support in forum |If you have any feedback on our support, please contact [email protected].

Monday, September 2, 2013 6:31 AM

How are things going? Please let us know if there is any progress.

TechNet Subscriber Support in forum |If you have any feedback on our support, please contact [email protected].

Tuesday, September 3, 2013 11:09 PM

Hello,

Unfortunately that did not work. I will be recreating the share with a new name and will have to copy the data.

Monday, September 30, 2013 8:14 PM | 2 votes

So after a month of trying to figure out why I kept getting this error message, I finally figured out what was causing the problem. For some reason (may have been another admin that did this), the folders were at some point previously setup for replication and then was turned off. After looking in the DfsrPrivate\Staging folder there was a ton of stuff waiting to replicate. What I did was delete the entire DfsrPrivate folder and let DFSR recreate the folder and replication started to work and the error went away.

Monday, March 23, 2015 7:46 PM | 5 votes

Here is a short summary of what I did:

  1. Look in Event Viewer and identify all the replication groups/folders that are giving the tombstone error. Once you have them identified, go into DFS Management GUI and completely delete the replication group associated with that folder. You do not need to delete the DFS Namespace for that folder, just the replication functionality of that namespace folder. If you have other replication groups in your DFS-R that do not get the 9098 errors, then you do not have to do this for these folders.

  2. Stop DFSR services (you may need to kill the service using the taskkill command if it hangs when it tries to stop).

  3. Give yourself permissions to the hidden System Volume Information folder. If you're account is under the domain admins group, you can simply add the security group.

    This folder exists on all servers that is a member of the replication group. In my situation, 2 of the 3 servers didn't show this folder as existing even when I enabled to see hidden folders. If this happens to you, the server is lying to you that it's not there. It is there. Don't listen to it. My suggestion is to download and use the 7-zip file manager. It will see the folder and will help you set the permissions to it as well as delete files that are longer than 256 characters, which is an issue if you do the next step from the command line).
    Note, after you set the permissions, it might tell you that you still don't have access to that folder. Just close out of 7-zip and open it back up. It should let you in that folder as well as its subfolders.

  4. Once you have access to that folder, go ahead and delete the DFSR folder that resides underneath it. You will want to do this on all servers that has the DFSR role installed and is a member to any replication groups. You can use the command line command "rmdir", but it fails to delete files/folders that are longer than 256 characters. This is why the 7-zip file manager is a better option to delete the DFSR folder under System Volume Information. However, there are instances where 7-zip is unable to delete a file or folder. If you run in that scenario, use the rmdir command in an elevated command prompt. Essentially, a combination of these two will eventually clear out everything you need to clear out.

  5. Turn DFSR services back on. This will begin the process of recreating the DFSR hash and virtual tree that you had just deleted.

  6. Recreate the replication group that you want.

  7. On the replication groups that you did not delete, you may get the warning: "The DFS Replication service initialized the replicated folder at local path <path> and is waiting to perform initial replication. The replicated folder will remain in this state until it has received replicated data, directly or indirectly, from the designated primary member."

    If you do, what you need to do is run the command line to set one of the DFSR servers as the primary server for that replication group, and then once set - this is important - you will have to go in the DFS Management GUI, click on the replication group with the associated warning, select the connections tab, and then right click the the sending member that you just made as primary and choose "Replicate now..." This will initialize the replication and you will have to do this just that once for it to replicate here on out. You will need to do choose the "Replicate now..." option for each receiving member that the sending member/primary member server is attached to in that replication group.

  8. Wait about 5-10 minutes and run the dfsrdiag backlog command on each replicationgroup and see if a backlog for replication/sync gets created. Run this command each 5 to 10 minutes to see if the backlog file count value decreases. If it does, it's syncing/replicating.

Don't forget that when testing replication, you cannot create test files via the local path and see if it replicates. You must create it in the DFS Namespace (\domain\folder\share) to see if replication works. Otherwise you get false negatives and you think replication is still broken. Just an FYI.

*
*

Monday, December 14, 2015 3:43 AM

Excellent post persianmanIT. Fixed my issue with a tombstoned DC/DFS server. Thanks alot.

Monday, July 31, 2017 8:22 PM

Thank you!! This is exactly what I needed to get my DFSR working again!!

Tuesday, January 23, 2018 9:53 AM

Hello,

We have encountered with the same issue. But we are deleting the complete DfsrPrivate folder and again trying to create the DFSR group. And again we are getting the same issue.

On node 2 server DfsrPrivate folder is getting created but on node 1 DfsrPrivate folder is not getting created. And we are getting the Error on node1 "

A tombstoned content set deletion has been scheduled"

Tuesday, January 15, 2019 6:01 AM

Perfect Steps and full Details.

As per the mentioned steps problem did resolved and I am back to normal with the replication.