Get-Unique not working?

Question

_{Thursday, July 12, 2012 12:13 PM}

While i admidt that the fact that WQL dose not have DISTINCT is a hassle, i approve of the breaking from the proverbial mold as it were and getting a command that is object oriented, but it has to WORK.

Let me know if im missing something here.

 $Dir = get-childitem \\directory\                                $Items = $Dir | Sort-Object                 for($i = 0; $i -le ($items.count*2 -3); $i += 2){                    $a = $a + (($Items -split "-n")[$i])                     }Get-Unique -InputObject $a

Basically i have a list of filenames, running this query gives me:

NTADDH0075M00NTADDH0075M00NTADDH0075M00NTADDH0075M00NTADDH0075M00NTADDH0075M00NTADDH0075M00NTAPPH3994M00NTAPPH3995M00NTAPTH3874M00NTAPTH3999M00NTAPTH4082M00NTAPTH4083M00NTAPTH4084M00NTAPTH4085M00...

Long story short, why isnt get unique sorting out the "Unique" items? My first thought was that its not treating the values as "Objects" however i do a $a.count and i get 82, which corresponds to the number of files.

Thansk in advance!

All replies (7)

_{Thursday, July 12, 2012 12:24 PM ✅Answered}

Try: 

 Remove-Variable a
 $a = @()
 $Dir = get-childitem \\directory\
 $Items = $Dir | Sort-Object 
    for($i = 0; $i -le ($items.count*2 -3); $i += 2) {
        $a += (($Items -split "-n")[$i])
    }
$a | select-object -unique

G. Samuel Hays

_{Thursday, July 12, 2012 2:53 PM ✅Answered | 2 votes}

I think there are some other thing to work through first.  What is this business with -split?  I can't figure out what you are trying to extract and assign to $items.  And calling "-split" in every iteration of the loop is not the most efficient approach, neither is building an array one element at a time.

If you just want a list of file names, I would suggest:

**$items = dir \directory |%{ $_.Name } **

Do you need the file name, but just the part before "-n" substring?  Then either of below could work:

**$items = dir \directory |%{ $_.Name -replace '-n.*$', '' } **
$items = dir \directory |%{ ($_.Name -split '-n')[0] } 

To get the unique items, you have a few options.  

Pipe to "sort" then to "Get-Unique":

$items = dir \directory |%{ ($_.Name -split '-n')[0] } | sort | Get-Unique

Use "sort -unique":

$items = dir \directory |%{ ($_.Name -split '-n')[0] } | sort -unique

Use "select -unique":

$items = dir \directory |%{ ($_.Name -split '-n')[0] } | select -unique

Apologies if I have misinterpreted your goal.

Thanks,
-Lincoln

_{Thursday, July 12, 2012 2:26 PM | 2 votes}

It is a known fact that extracting a set of unique values from an unsorted list requires more work than when dealing with a sorted list. That is why the help text for get-Unique says this:

DESCRIPTION
    The Get-Unique cmdlet compares each item in a sorted list to the next item, eliminates duplicates, and returns only
     one instance of each item. The list must be sorted for the cmdlet to work properly.

Al Dunbar

_{Thursday, July 12, 2012 3:00 PM}

Well, sort of.  If you have a list of items of length N, in general it takes time proportional to N * Log(N) to sort the list, then time proportional to N to extract the unique elements from that sorted list.  But even if you don't sort the list, you can still extract the unique elements in time proportional to N.

So if the goal is merely to get the unique elements, it's actually slower to sort the list first.  But if someone hands you a list which is already sorted, yes, the unique extraction will be faster than the unique extraction from an unsorted list.

That being said, you are totally right that Get-Unique requires pre-sorted input. :)

Thanks,
-Lincoln

_{Thursday, July 12, 2012 3:16 PM}

This worked flawlessly, thank you!

_{Thursday, July 12, 2012 3:17 PM}

This also works; however, there are folders in addition to files that show up when using this method. Good answer for another situation though!

_{Thursday, July 12, 2012 3:55 PM}

Ah, is that where the "- 3" in the loop bounds comes from?  You can filter out directories like this:

dir \directory |?{ -not $_.PSIsContainer } |%{ ($_.Name -split '-n')[0] } | select -unique