Stack Overflow
Stack Overflow

Questions tagged [uniq]

uniq is a Unix/POSIX/Linux utility to remove or filter duplicate lines from a sorted file. It is also the name of a method to remove duplicates from an array in Ruby.

uniq is a Unix/POSIX/Linux utility to remove or filter duplicate lines from a sorted file. It is typically applied to the output of sort.

In Ruby , uniq is a method of the Array class to remove duplicates from an array. uniq creates a new array whereas uniq! modifies the array in place.

For questions about unique identifiers, keys, names, etc., see or more specific tags such as , , , , etc.

Documentation

454 questions
4
votes
1 answer

How to remove the leading spaces of the output from the command "uniq"?

[root@firstcentos scripts]# cat test1 00 00 01 01 00 00 02 02 03 aa aa aa [root@firstcentos scripts]# cat test1 | uniq -c 2 00 2 01 2 00 2 02 1 03 3 aa [root@firstcentos scripts]#
symb0l121cc00130
  • 69
  • 5
4
votes
1 answer

lodash uniq - choose which duplicate object to keep in array of objects

is there any way to specify which array item to keep based on a key being non-empty. it seems uniq just keeps the first occurrence. e.g: var fruits = [ {'fruit': 'apples', 'location': '', 'quality': 'bad'}, {'fruit': 'apples', 'location':…
John Paul Vaughan
  • 205
  • 1
  • 3
  • 9
4
votes
5 answers

sum occurrence output of uniq -c

I want to sum up occurrence output of "uniq -c" command. How can I do that on the command line? For example if I get the following in output, I would need 250. 45 a4 55 a3 1 a1 149 a5
user3144923
  • 71
  • 1
  • 6
4
votes
2 answers

Unique object instances in an array (Ruby)

I have a custom defined class Instruction. Instances are initialized and collected in an array. There are some duplicate (all instance variables identical) instances and I want to filter them out. class Instruction attr_accessor :date, :group,…
Adomas Zvirzdinas
  • 43
  • 1
  • 4
4
votes
4 answers

Making a perl array unique

I am currently having a very simple problem with capturing the output from a backticked shell command. I apologize that the problem is rather simple. I have some sorted array (@valid_runs) which I know contains consecutive, duplicate elements. I…
order
  • 335
  • 1
  • 7
  • 14
4
votes
1 answer

How to find max value grouped by multiple keys in array of hashes?

Have data that has this kind of structure. Will be in ascending order by 'c'. [ { 'a' => 1, 'b' => 1, 'c' => 1, 'd' => '?' }, { 'a' => 1, 'b' => 1, 'c' => 2, 'd' => '?' }, { 'a' => 1, 'b' => 1, 'c' => 3, 'd' => '?' }, { 'a' => 1, 'b' => 2,…
Douglas Mauch
  • 859
  • 2
  • 7
  • 18
3
votes
1 answer

Is there a way to use uniq to compare only the first n characters?

According to the documentation, http://www.computerhope.com/unix/uuniq.htm The command can be used to ignore the first n characters Suppose I wish to have the following 03/08/2002,2,HUST,RICC,53.0,,2.3,J S…
Hoa
  • 19,858
  • 28
  • 78
  • 107
3
votes
3 answers

How do you do custom formatting with the uniq -c option?

From wikipedia: uniq -c Generate an output report in default style except that each line is preceded by a count of the number of times it occurred. If this option is specified, the -u and -d options are ignored if either or both are also…
Jonah
  • 2,040
  • 7
  • 29
  • 32
3
votes
4 answers

Compare two lines in same file

Given a file like the following:- 01/09/2005 02/09/2005 03/09/2006 03/09/2006 I wish to compare if the last two lines are the same, and return a 1 if so or a 0 if they are not. I can get the last two using a cat tail -2
general exception
  • 4,202
  • 9
  • 54
  • 82
3
votes
2 answers

Output redirection of real-time stdout doesn't work after piping it into uniq or awk

I'm trying to run fswatch -tr /home/*/*/public_html | grep --line-buffered -E ".php|.xml" | awk '!seen[$0]++' >> log.txt or equivalently (by using uniq): stdbuf -i0 -o0 -e0 fswatch -tr /home/*/*/public_html | grep --line-buffered -E ".php|.xml" |…
EmmaJackson455
  • 31
  • 1
3
votes
1 answer

Get unique elements from HoA values and print

I have a HoA with certain values in it. I need to have only unique elements from the HoA. Expected Result: Key:1 Element:ABC#DEF Key:2 Element:XYZ#RST Key:3 Element:LMN Below is my script: #!/usr/bin/perl use strict; use warnings; use…
vkk05
  • 3,137
  • 11
  • 25
3
votes
3 answers

Sort a file broken into sections by (DOW MON DD YY) date in each section's header

I have a file that has repetitive entries .Entry entry starts with date , a blank line separates these two entries .How do I use sort command or uniq comand to sort the dates : * Mon Jan 29 2001 Bernhard Rosenkraenzer - Some fixes to…
Unixquest945
  • 103
  • 2
  • 10
3
votes
2 answers

Getting unique values from column in a csv file

I have the following input: no,zadrar,MENTOR,rossana@xt.com,AGRATE no,mittalsu,MENTOR,rossana@xt.com,GREATER…
kawther
  • 184
  • 2
  • 3
  • 16
3
votes
1 answer

Why do the groupby groupings change when you convert groupby to a list?

If you uncomment the commented line below, then the output will change (for all but the last key, the grouper object will be empty). Why is this? from itertools import groupby c = groupby(['goat', 'dog', 'cow', 1, 1, 2, 3, 11, 10, ('persons',…
James Ko
  • 32,215
  • 30
  • 128
  • 239
3
votes
4 answers

Is there any better solution to reverse uniq count

I want to reverse the uniq -c output from: 1 hi 2 test 3 try to: hi test test try try try My solution now is to use a loop: while read a b; do yes $b |head -n $a ;done
once
  • 1,369
  • 4
  • 22
  • 32
1 2 3
…
30 31