How can "squeeze-repeated" words?

Question

How can "squeeze-repeated" words? similar to "squeeze repeated characters" with tr -s ''

I would like to change for example:

hello.hello.hello.hello

to

hello

How do you justify `hellohellohellohello` ==> `hello`? So `hello` should be `helo`? — devnull, Oct 01 '13 at 14:08
First, you'll have to define what a word is. By most normal definitions, `hellohellohellohello` is a single word (that humans recognize as containing the same sub-word, `hello`, four times). If you've got to look for arbitrarily long repeats within a single contiguous block of non-space characters, you've got quite a problem on your hands — I'm not aware of any standard tools that will address the job. What will be the output for the input `banana hello hello abracadabra`? Is it `bana helo abracad`? If not, what, and why? — Jonathan Leffler, Oct 01 '13 at 14:13

score 1 · Accepted Answer · answered Oct 01 '13 at 14:10

1

This can be a way:

$ cat a
hello hello bye but bye yeah
hello yeah
$ awk 'BEGIN{OFS=FS=" "} 
  {  for (i=1; i<=NF; i++) {
       if (!($i in a)) {printf "%s%s",$i,OFS; a[$i]=$i}
     }; 
    delete a;
    print ""
  }' a
hello bye but yeah 
hello yeah

You can change the field separator:

$ cat a
hello|hello|bye|but|bye|yeah
hello|yeah
$ awk 'BEGIN{OFS=FS="|"} {for (i=1; i<=NF; i++) {if (!($i in a)) {printf "%s%s",$i,OFS; a[$i]=$i}}; delete a; print ""}' a
hello|bye|but|yeah|
hello|yeah|

answered Oct 01 '13 at 14:10

fedorqui

275,237
103
548
598

Did you see the sample input? `hellohellohellohello`! – devnull Oct 01 '13 at 14:12
1

Oh haha then the answer must be `echo "hello"`. – fedorqui Oct 01 '13 at 14:13
Very true. I too sought a [clarification](http://stackoverflow.com/questions/19118225/how-can-squeeze-repeated-words#comment28269864_19118225) but, alas, no response. – devnull Oct 01 '13 at 14:31

How can "squeeze-repeated" words?

1 Answers1