Stack Overflow
Stack Overflow

Questions tagged [uniq]

uniq is a Unix/POSIX/Linux utility to remove or filter duplicate lines from a sorted file. It is also the name of a method to remove duplicates from an array in Ruby.

uniq is a Unix/POSIX/Linux utility to remove or filter duplicate lines from a sorted file. It is typically applied to the output of sort.

In Ruby , uniq is a method of the Array class to remove duplicates from an array. uniq creates a new array whereas uniq! modifies the array in place.

For questions about unique identifiers, keys, names, etc., see or more specific tags such as , , , , etc.

Documentation

454 questions
0
votes
3 answers

How do I count grep results for a string, but specify exclusions?

I have maillog file with below parameters relay=mx3.xyz.com relay=mx3.xyz.com relay=mx1.xyz.com relay=mx1.xyz.com relay=mx2.xyz.com relay=home.xyz.abc.com relay=127.0.0.1 I want to count all relay except 127.0.0.1 Output should like…
saggi
  • 1
  • 1
0
votes
1 answer

Extract records with duplicate field in Unix (without using uniq!)

I have a text file consisting of a variable-length key field in field 1, and other data in field 2. The fields are separated by the tab character ("\t"). The key fields contain spaces. The file is sorted on the key field. I'd like to filter this…
Chap
  • 3,649
  • 2
  • 46
  • 84
0
votes
1 answer

Unexpected difference with sort -u and uniq -u

Case: List all unique filenames in a directory. $ ls a.h a.i b.h b.i c.h d.i So c and d are unique in this case. For fixed width we can do: $ ls | uniq -w1 -u c.h d.i Not very helpful though as filenames won't be fixed width and uniq can…
Chris Seymour
  • 83,387
  • 30
  • 160
  • 202
0
votes
1 answer

qlikview syntax - uniq key

I've an excel file with 4 fields :a,b,c,key. I need to check in QV script that for each row a,b,c there is only on key. The rows that have diffrent keys should be the result. for example this is an uncorrect situation that I need to catch : key | …
user1613935
  • 11
  • 1
  • 3
-1
votes
3 answers

Uniq and counts

Have a file with 2 columns, need to use uniq on column 1 only and print both the columns in the results as well as the count of the occurrences (with -c). Example: 1 a 1 a 2 a 3 c 4 d 2 1 a 1 2 a 1 3 c 1 4 d
Kid_A
  • 47
  • 4
-1
votes
3 answers

Implementing custom uniq in linux shell development

I'm developing a custom shell. In this assignment, I need to implement uniq-like command. Given sorted lines, uniq should be able to print all unique values and (their number of occurences if the command is uniq -c). Example code is stated at the…
ihatec
  • 13
  • 3
-1
votes
1 answer

How i can find unique one one column(comma separated values) if we are applying group by two column?

I have data frame like this office salary | Column A | Column B | | -------- | -------- | | Montreal | a, b | | Cell 3 | b,c,d | | Montreal | b,c,d | | Montreal | c,d,e | | Toronto | n,c | Output: office salary 0 …
Rohit Laxman Bhojane
  • 11
  • 1
-1
votes
2 answers

sub uniq { my %seen; grep !$seen{$_}++, @_ } : No longer allowed. Fix?

Please read first the UPDATE below !!!! As can be seen elsewhere on stackoverflow, the following perl sub has been popular for some years for extracting the unique elements of a list sub uniq { my %seen; grep !$seen{$_}++, @_ } However, since some…
reikred
  • 151
  • 7
-1
votes
2 answers

I tried uniq function doesn’t work in Ruby

I have: article A with tags: tags: [chris, mark, scott] article B with tags: tags: [mark, scott, chris] I want to display a list of unique tags for all the articles. I tried using the .uniq function but it doesn’t work. Here’s the code I…
Alex
  • 1,576
  • 3
  • 17
  • 35
-1
votes
2 answers

Am I using the proper command?

I am trying to write a one-line command on terminal to count all the unique "gene-MIR" in a very large file. The "gene-MIR" are followed by a series of numbers ex. gene-MIR334223, gene-MIR633235, gene-MIR53453 ... etc, and there are multiples of the…
testtubeshawty12
  • 1
  • 3
-1
votes
2 answers

system verilog uniq on selected members of a struct

Can selected members of a struct be constrained to be uniq ? typedef struct { typeA a; typeB b; typeC c; typeD d; } config_t; rand config_t config[10]; For example can I constrain {a, c} to be uniq in config[10] instances ? If not is…
Mohan
  • 141
  • 3
  • 13
-1
votes
1 answer

How to delete all non-matched data on Linux using awk or grep

My data file is lines of network information, now I print all IDs. We need a generate ID and received ID, so if there is id: 4253, there should be another ID of 4253. How can I filter all data using awk or grep on Linux? I can use 'uniq' to filter…
Lin Dong
  • 1
-1
votes
1 answer

Is there a way to filter a file for lines that only occur once, based on a field?

I have a "diff like" file where the fields are separated by tabs. Is there a way to get only the lines that are present once, using only standard gnu tools, where two lines are equal if their second field is equal? Essentially, I want to filter…
Dave Fol
  • 515
  • 3
  • 11
-1
votes
3 answers

Is there a Linux command for string subtraction between columns?

I'm processing some SNP column into vcf format. the input columns are as following : ref ALT A A G A A T T C T G G T A A G C C G T G A G T C T T A G T expected output : ref …
Naibin Duan
  • 3
  • 2
-1
votes
2 answers

Filter specifik patern in a complex file

I have this logfile and im trying to avoid printing the text containing patern" APLHA". +++ skdfhahjsahsdjk > ** ALPHA EDFJDJFKLJDKFJKSDLFJL SDFSDGDRGRTG WEFETTFYRT #168113++- +++ skdfhahjsahsdjk > * KJENRFKES…
Dominik
  • 33
  • 4
1 2 3
…
30 31