How can I emulate `uniq -d` in awk?

Question

I've got a busybox system which doesn't have uniq and I'd like to generate a unique list of duplicated lines.

A plain uniq emulated in awk would be:

sort <filename> | awk '!($0 in a){a[$0]; print}'

How can I use awk (or sed for that matter, not perl) to accomplish:

sort <filename> | uniq -d

Dennis Williamson · Answer 1 · 2012-06-04T15:17:45.313

6

On a busybox system, you might need to save bytes. ;-)

awk ++a[\$0]==2

edited Jun 04 '12 at 15:17

answered Jun 04 '12 at 13:36

Dennis Williamson

346,391
90
374
439

you can still get rid of two spaces. – Benoit Jun 04 '12 at 15:13
and if you put `alias a='awk ++a[\$0]==2'` into your bashrc, you can get rid of all characters but one. :) – Benoit Jun 04 '12 at 15:22

score 3 · Accepted Answer · answered Jun 04 '12 at 13:14

3

Could do this (needn't sort it):

awk '{++a[$0]; if(a[$0] == 2) print}'

answered Jun 04 '12 at 13:14

Benoit

76,634
23
210
236

didn't know you could do associative arrays. – Jamie Jun 04 '12 at 13:48
@Jamie: This is although exactly what you use in your first example, even if you thought it was actually a set. – Benoit Jun 04 '12 at 15:14

score 0 · Answer 3 · answered Jun 04 '12 at 15:33

This might work for you:

# make some test data
seq 25 >/tmp/a
seq 3 3 25 >>/tmp/a
seq 5 5 25 >>/tmp/a
# run old command
sort -n /tmp/a | uniq -d
3
5
6
9
10
12
15
18
20
21
24
25
# run sed command 
sort -n /tmp/a | 
sed ':a;$bb;N;/^\([^\n]*\)\(\n\1\)*$/ba;:b;/^\([^\n]*\)\(\n\1\)*/{s//\1/;P};D' 
3
5
6
9
10
12
15
18
20
21
24
25

How can I emulate `uniq -d` in awk?

3 Answers3