It might be a damaged profile, or a profile with a very long name, or a profile with security that's unexpected. Or, it may be a damaged disk (perhaps a bad sector?).
Sometimes just running with administrative privileges isn't sufficient (see unexpected security). Try running your script with backup and restore privileges.
FYI, the Remove-Item will also remove (or try to) any sub-directories it encounters. Is that your intention?
If you encounter a profile that's difficult to remove, you might want to try using delprof2 (https://helgeklein.com/free-tools/delprof2-user-profile-deletion-tool/).
I don't think this is a PowerShell problem, though.