Making a perl array unique

Question

I am currently having a very simple problem with capturing the output from a backticked shell command. I apologize that the problem is rather simple.

I have some sorted array (@valid_runs) which I know contains consecutive, duplicate elements. I want to use backticks to echo this array to uniq. I want to capture the STDOUT in an array. I attempt to do so like this.

@unique_valids = `echo '@valid_runs' | uniq`;
print @unique_valids;

This print statement yields nothing. For that matter neither does this.

@unique_valids = `echo '@valid_runs'`;
print @unique_valids;

I know how to use uniq and echo. This seems rather odd to me. I would think this has more to do with perl arrays than proper use of those commands. I have searched around a bit elsewhere, so please don't pelt me with downvotes just because the solution might seem trivial. Thanks again for your time.

NOTE on solutions: TLP's solution is the most straightforward as far handling the uniq problem. I am being rather flexible, since all responses suggested not making a system call for this problem. If Perl's uniq function the same as Unix's uniq then the array ought to remain sorted.

John Corbett's solution works well if you don't care about a sorted result.

one potention problem: do the elements of `@valid_runs` end in a newline? otherwise `echo` will only produce one line of output as input to `uniq` — ErikR, Aug 03 '12 at 18:50
are you sure `@valid_runs` is not empty? this prints something for me: `@x=(3,2,1);@y = \`echo '@x'\`;print @y;` — ErikR, Aug 03 '12 at 18:54
@user5402 I am sure that @valid runs is not empty. I am sure that each entry has a newline at then end of the string. You example works. Which is nice; shows me that I am not proposing anything too crazy. Here is a snippet of @valid_runs: `/raid1/home/pharmacy/morguna/1experiments_copy/1experiments/test10-10/run36415 /raid1/home/pharmacy/morguna/1experiments_copy/1experiments/test10-10/run36415 /raid1/home/pharmacy/morguna/1experiments_copy/1experiments/test10-10/run36416 /raid1/home/pharmacy/morguna/1experiments_copy/1experiments/test10-10/run36416 ` — order, Aug 03 '12 at 19:31
For a discussion of all sorts of things you might want to do with arrays in Perl, such as getting the difference between two arrays, see the Perl FAQ: http://perldoc.perl.org/perlfaq4.html#Data:-Arrays — Andy Lester, Aug 03 '12 at 21:43

John Corbett · Answer 1 · 2012-08-03T20:14:03.163

8

you should just store the array into a hash, because hash keys are always unique. You can do that like this:

my %temp_hash = map { $_ => 1 } @valid_runs;
my @unique_valids = keys %temp_hash;

that's the perl way of doing it anyway. There's no need to use back tics here (I try to avoid those as much as I can).

edited Aug 03 '12 at 20:14

answered Aug 03 '12 at 18:55

John Corbett

1,595
10
19

This doesn't preserve the order of the array. – stark Aug 03 '12 at 19:02
he didn't say it needed to be preserved – John Corbett Aug 03 '12 at 19:03
In this context it is not essential, however I do wish to be able to do something so simple as throw an array into shell commands. @JohnCorbett Thanks for your solution. It achieves what needs to be done, but I still want to know how to do what I proposed. – order Aug 03 '12 at 19:12

score 8 · Accepted Answer · answered Aug 03 '12 at 20:21

8

Using system calls for something that can easily be accomplished with perl code is not a good idea. The module List::MoreUtils has a uniq function that does what you require:

use List::MoreUtils qw(uniq);

my @unique = uniq @runs;

The subroutine inside the module itself is very simple, though, exactly like theglauber's answer:

sub uniq (@) {
    my %seen = ();
    grep { not $seen{$_}++ } @_;
}

answered Aug 03 '12 at 20:21

TLP

66,756
10
92
149

Yeah, for the current solution I am just implementing something within Perl. Making unnecessary system calls is bad. I was just being lazy since I already knew of uniq. Thanks for your time. – order Aug 03 '12 at 21:22

score 5 · Answer 3 · answered Aug 03 '12 at 18:56

It's easy to do this in perl. Here's a rather obscure but fun way to dedup an array:

@dedup = grep !$seen{$_}++ @orig_array;

Figure out what this is doing by checking the documentation for the perl function grep.

If you have to use uniq, you probably need to put each array element in a separate line.

join("\n", @your_array)

should achieve that.

score 0 · Answer 4 · answered Aug 03 '12 at 20:07

#!/usr/bin/perl
use warnings;

@a = (1, 2, 3, 3, 4, 4, 5);

$cmd = "/usr/bin/uniq <<EOF\n";
$cmd .= $_."\n" foreach (@a);
$cmd .= "EOF\n";

$result = `$cmd`;
print "Cmd: $cmd\n";
print "Result is $result";

@u = split /\n/,$result;
print "After ",join " ",@u,"\n";

This does what you ask, but theglauber's answer is still better Perl.

Making a perl array unique

4 Answers4