Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Question
Saturday, July 18, 2015 8:44 PM
Hi,
we do have an rds farm with roaming profiles (not profile disks) consisting of 6 terminal servers ("srvts001" to "srv006" 2012 R2), 2 domain controllers ("srvdc01", "srvdc02" 2012 R2), 1 SQL/Fileserver ("srvsql01" 2012 R2).
Roaming profiles are stored on srvsql01 in share RDS. Oplocks are disabled, directory caching is disabled.
Everything was running without problems for 1 or 2 years. Recently (about 2 weeks ago) users began complaining about being logged on with a temporary profile. Investigation showed a lot of open file handles from previous (no longer existing) sessions on the file server.
E.g. user1 had logged on to rds one day and logged off in the evening (not just disconnected, really logged off) and when he logs in the other day on the rds farm he is logged on with a temporary profile due to the profile service being unable to access the roaming profile as it is (event log) "in use by another process".
Looking at the fileserver's open files we can see that a lot of files of that user profile (and of other users, too!) are shown as open - all with read option and no apparent locks.
When user1 loggs off these locks stay! Manually closing the open files on the file server allows the user to log on to the rds servers normally. Subsequent logging off may or may not create those stale open file handles.
We have no clue as to when these enormous amount of open file handles (we talk about 100s of files per user) happen. It seems to be at random and not hitting every user.
Has anyone ever met a similar problem? Or at least an idea on how to prevent logged off users to still have open files on the file server?
Any answer pointing in the direction of a possible solution or source of this problem is greatly appreciated!
All replies (28)
Tuesday, July 21, 2015 9:13 AM
Hi,
The problem is that the session between RDS server and file server still exists after using logged off. It may not directly related to RDS.
What kind of files are locked on file server? For example, whether they are all files in mapped folders, roaming profile etc?
A quick workaround is to set a logoff script to run "net use * /delete" so that all accesses will be disconnected during log off.
Please remember to mark the replies as answers if they help and un-mark them if they provide no help. If you have feedback for TechNet Support, contact [email protected].
Thursday, July 23, 2015 7:21 AM
Hi Shaon,
the open files are all part of the users' roaming profile. Files on mapped network drives do get closed.
So we could - as a workaround - add a logoff script to every terminal server with "net use * /d /y" ? Wouldn't that cause problems with saving the roaming profile to the network share?
Of course we'd prefer a solution rather than a workaround. Unfortunately the problem seems to occur totally at random so it might be difficult to track down or even to reproduce it...
Friday, July 24, 2015 12:10 PM
We also experience similar problems, resulting in temporary profiles randomly.
A quick (temporary) fix is closing al session on the share for that specific user this will allow them to log back on with their regular profile. But doing that on (semi)daily basis not an option and leads to a lot of user frustration.
In our environment we don`t use profile disk, and the profile share on file server does have Oplocks disabled and directory caching is disabled.
There is one 2012 R2 fileserver en two RDS servers.
I did try the above option of creating a logoff script (like suggested above) but this destroys the roaming of profiles completely and on top of that even more file handles are left over !
Any ideas ?
Tuesday, August 4, 2015 11:06 AM
Hi,
Sorry for the delay in reply.
As you said, terminate the syncing will cause data lose. I it is a roaming profile, can you find the exact file which was still connecting?
Please remember to mark the replies as answers if they help and un-mark them if they provide no help. If you have feedback for TechNet Support, contact [email protected].
Tuesday, August 4, 2015 1:47 PM
Hi,
the open files we observed actually weren't files but directory handles. After logging off from a terminal server the profile is synced to the file server. In order to do that, the user profile service sets "write" locks on the directories of the user's profile:
Unfortunately those seem not to get removed from time to time so that a later read (say next morning) on logon fails and leads to a temporary profile on the terminal server.
At first we thought it might be related to one terminal server but disabling logon to that server only slightly reduced the problem.
For sake of completeness here are our registry settings of lanmanserver (server side):
And for lanmanworkstation (on terminal servers):
Maybe one could tune some timeout parameters - but I do not want to experiment on that.
Slightly off topic: we'd love to use profile disks as that might reduce or even solve the problem but unfortunately profile disks are a per rds farm setting where you cannot leave out certain users (read administrator). And being able to only log on to ONE terminal server at a time in order to do updates is not feasable at all.
Thursday, August 6, 2015 11:31 PM
I've seen similar issues. Most of the time it is as you describe on the profiles, but once in a while we also see it on other files that are in-use during logoff. (One particular application presents a "Are you sure you want to quit?" when the session is being logged off and we find that a file handle to the EXE persists even after the session has successfully logged off)
Best solution we've found is to kill the handle from the file server. We temporarily removed anti-virus from the server thinking that maybe it was causing the problem but the issue continued after this change.
Tuesday, August 18, 2015 8:49 AM
Killing the open file handle on server side works. But this is something that cannot be done on a daily basis as the customer's work time starts WAY before ours.
We are going to reboot the terminal servers each night at around 4am and see if that remediates the problem.
Nevertheless I'd love to have someone from Microsoft comment on this issue here.
Tuesday, August 18, 2015 3:46 PM
The reboot works over here, but we have had one or two cases where we end up with locks in the middle of the day.
We're trying to come up with a reproducable case so we can open an issue with Microsoft, but it is so sporadic we have yet to come up with a pattern.
Wednesday, September 9, 2015 11:19 AM
We are experiencing the same issue and have not been able to fix it, tried several workarounds like daily reboots and closing all open files.
Does anyone have a solution for this issue?
Wednesday, September 9, 2015 7:36 PM
Rebooting the server on a daily basis is working but as Damon Durand pointed out it happens from time to time that a user logs off in the middle of the day and gets a problem logging on in the afternoon.
So a comment / suggestion / hint / whatever from MS would be greatly appreciated.
We cannot experiment on that customer's site as this has been a rather touchy subject in the past weeks.
Otherwise I would have already tried switching from a round-robin session host DNS to a broker server approach in the rdp connection files as per http://it-gotsch.com/rds2012-upd/ (unfortunately only available in german language).
I tried that in a demo environment and it seemed not to have any unwanted side effects. However if a user had the "require password change at next logon" flag set in active directory I only got a mean error from rdp... though that might have had to do with the aged (as in really really old) rdp client I used...
As I never had the temp profile issue in the demo environment I cannot say for sure if that is a solution to our problem.
Tuesday, September 22, 2015 12:23 PM
I've had the same problem for about a year.
Still no solution.
It's become more prevalent in recent months.
Friday, October 16, 2015 6:15 AM
We have the same issue in nearly all of our environments where the server for storing the roaming profiles is running under 2012 R2 .. when the server for storing the roaming profiles is running 2008 R2 everything is fine.
Depending on the infos from our customers it only happens when the user is connected for 10 hours and more, so we changed two options via GPO and the error seems to be gone.
The two options we set to 18 hours/1080 minutes are:
Maximum lifetime for service ticket
Maximum lifetime for user ticket
Friday, October 23, 2015 10:18 AM
We have the same issue in serveral environments where we have Windows 2012 R2 servers as a fileserver for the profiles. We have no problems with 2008 Servers.
We checked the Maximum lifetime parameters as shown on some spotted Servers but some of the servers had already the 18 hours Setting and still the same Problem.
It seems that this is not the needed solution.
we still need some help.
Wednesday, October 28, 2015 10:12 AM
Hi,
We're having the same issues like everyone else. Even opened a ticket with Microsoft Professional Support but they suggested premier support for a root cause analysis. Decided to continue to have a look and are currently trying a workaround.
On the file server we have created a task that executes a powershell command around 3AM every night:
net files | where { $_.Contains("username") } | foreach { $_.Split( ' ' )[0] } | foreach { net file $_ /close }
This closes all the open files of a specific username. It should also work for specific folders:
net files | where { $_.Contains("c:\users") } | foreach { $_.Split( ' ' )[0] } | foreach { net file $_ /close }
This way, in the morning, the users shouldn't have any locked files and won't get temporary profiles.
Sunday, November 1, 2015 6:05 PM
Update: I did try the round robin variant at another customer site. This was a complete new installation so I decided to test it there. Unfortunately I did get the temporary profile issue within the first week.
So using a broker server approach instead of session host round robin does not fix the problem.
I decided to disable logon with temporary profile via gpo in order to prevent users from losing data when working with temporary profiles. Reason for this is that there is a document management software in use that writes documents that are checked out to the user's temporary files folder...
So in short:
- rebooting terminal servers each morning reduces the problem but it migh occur in the middle of the day if a user dares to log off and on.
- switching to broker server instead of session host round robin does not fix the problem.
Sunday, November 1, 2015 6:09 PM | 1 vote
So MS Professional Support decided to just kill the file handles via powershell instead of a real fix?!? Great. If I were you I wouldn't accept that as an outcome of a support call. You paid for the support call so MS has to fix this problem. Once and for all.
Monday, November 2, 2015 2:13 PM
Hey Frederik J, the professional support ticket got closed because after some basic troubleshooting they expected it would be an issue that goes beyond their knowledge and advised to open a premier call. As we only have SPLA licenses, we do not have the possibility to open premier calls. That's why we created the workaround. It has been working good so far, luckily a majority of the users won't log out / in during the day and only in the morning.
Wednesday, August 3, 2016 11:40 AM
Several of our customers also have this problem, in our case with Windows Server 2012 (without R2) in connection with roaming profiles.
We also tried different registry changes regarding to SMB-protocoll on the terminalserver and fileserver, but without any success on the affected servers.
As a workaround we deployed new "blank" Windows Servers 2012 fileservers for the roaming profile share, but we cannot tell our customers to deploy new fileservers only because of not working roaming profiles. That's not accurate.
As it seems to be a general problem, we need a solution.
Thank you in advance
Tuesday, October 4, 2016 9:25 AM
Hi,
we are experiencing the exact same Problem with Roaming Profiles. The User gets a temporary profile, because folders are locked on the Fileserver. We only have 1 RDS Server, no farm.
Our Setup is:
1x RDS Server (Windows Server 2012 R2)
1x File Server (Windows Server 2012 R2)
The User logs on to the RDS Server and receives a temporary Profile. In the Application Event Log of the RDS Server there are 9x Event ID 1509 stating (with every event stating a different Folder):
Windows cannot copy file \?\UNC\server\profileshare$\username.V2\Desktop\ to location \?\C:\Users\username\Desktop\ This error may be caused by network problems or insufficient security rights.
DETAIL - The process cannot access the file because it is being used by another process.
Than Event ID 1534:
There are too many profile copy errors. Refer to the previous events for details. Windows will not log any additional copy errors for this copy process.
and last Event ID 1504:
Windows cannot update your roaming profile completely. Check previous events for more details.
Checking on the File Server we can see a open session of the user experiencing the problem with over a 100 folder locks. Killing those open folders will fix the Problem temporary but we experiencing this more often and by more Users.
Friday, February 3, 2017 3:03 PM
Hi,
any news here? We got several customers with exactly the same issue.
Complete fresh installations, all updates installed.
Monday, April 24, 2017 8:31 PM
We have been working with Microsoft since October 2016. We have spent an exorbitant amount of time troubleshooting this issue. Unfortunately, there doesn't seem to be a resolution in sight. They have no idea why this is occurring. This has caused a major productivity issue. Very frustrating.
Friday, April 28, 2017 3:19 PM
I have encountered the same issue with a client this week and have had a support case with Microsoft for last days with no solution in site and am considering scrapping farm and roaming profiles all together.
Scenario. Replaced DC and File server SBS2008 with 2012 R2 server that hosts roaming profiles/redirected folders. Performed in-place upgrade of 2008 R2 Terminal Servers to 2012 R2 and configured in a farm. Temporary profiles occur randomly. Some relation to open files from roaming profiles but nothing definitive.
Thursday, August 24, 2017 3:51 PM
Has anyone found a solution for this on server 2012 R2?
Thursday, December 28, 2017 9:35 PM
I worked with Microsoft for over a year, and they were never able to resolve the issue. They recommended we upgrade to Server 2016. We have upgraded our fileserver to 2016, and so far, there has not been any reported profile issues.
Thursday, May 3, 2018 4:45 PM
We've the same issue with a pure Server 2016 environment.
Our setup is:
1x 2016 AD Server
1x 2016 File Server
2x 2016 Terminal Server
We've enabled roaming profiles per GPO and now encounter the exact same issue with over 100 directories in write lock on our file server.
The odd thing is: Regardless if I restart the terminal server(s), the files keeps locked on the file server. Really annoying thing and I have currently no clue on which screw I need to screw...
Thursday, July 26, 2018 9:43 AM
Problem is in SMB 3.xx, try workaround:
HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DormantDirectoryTimeout
Set 1-5 sec.
Default is 600 sec.
See https://msdn.microsoft.com/en-us/library/windows/hardware/dn567661(v=vs.85).aspx
Saturday, April 13, 2019 11:53 AM
Is this supposed to be done on the server ? Or on the LAN clients ?
I created this setting on a Win 2012 R2 server (using SMB 3.02), with value 3, but it has no effect.
User logon sessions still remain open on the server for minutes after logging the users off (net sessions), with hundreds of the user's roaming profile files staying open on the server.
I have timed this: the files stay open for 10 minutes (= 600 seconds)
Found these logoff-related events on the client (Win 8.1 x64 also using using SMB 3.02):
- immediately after clicking logoff: Security log: event 4647 "User initiated logoff"
- a few seconds after this: System log: 7002 "Logoff" (now on the client the Alt-Ctl-Del lock screen appears, from users viewpoint the session has logically ended )
- after 10 minutes: Security log: 4634 "An account was logged off"
So apparently there is still communication between server and client regarding logoff 600 seconds after the user initiated the logoff. I am assuming that the server waits 10 minutes before sending some confirmation to the workstation that the session was terminated (+files closed).
Does anyone have any clue how to influence this timeout period ?
Further testing : when creating the registry value
HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters\DormantDirectoryTimeout
on the client, after logoff, the sessions files close after the specified number of seconds, and the session disappears from cmd 'net sessions' a few seconds thereafter. This seems the solution.
- With the registry value, when **shutting down a client, **the user roaming profile files are now also being closed approx. within the specified time period.
However, the session remains listed in the 'net sessions' list, with the following files open (server: Computer Management-Shared Folders-Open files) :
ntuser.dat (Write+Read), ntuser.ini (Write+Read), NTUSER.INI (Read),
with Access By showing [Disconnected]
Shut downed sessions seem to disappear from 'net sessions' somewhere between 2 and 6 minutes after shutting down the client
But this can still cause roaming profile errors if the user starts up the pc within this time frame due to files still open/locked from the session before the shutdown.
To force a timely unlocking of files after a shutdown/restart, I wrote a Powershell script that runs continuously on the server, monitoring the 'net sessions' output every x seconds, pings each pc with an open session, and if no reply, closes the files located in the roaming profile share using close-smbopenfile.
Thursday, April 18, 2019 7:14 AM
Hi Pat,
We have the same issue. I cannot find a way to dm you, are you willing to share that script?