0

I have a big file (7 milion rows) where some of rows are repeated, I want delete a duplicate lines . For this reason, I used from:

sort {file-name} | uniq -u> data.txt

But I saw again in the new file, repeated data: for example:

14693 10167 184228271 184227954 184227954 2001 1 1 6 0 0 0 0 0.5 0 0 1 1700 0 0 0 -99
14694 10167 184228271 184227954 184227954 2001 1 1 6 0 0 52816 0 0.5 0 0 1 1700 41.91 1756.45 73612.74 -99

But 2 rows have different column , may be because of this subject?

Please help me about delete in repeate row in the my file.

Regards Yahya

Natalie Hedström
  • 2,607
  • 3
  • 25
  • 36
Y.Mohammadi
  • 1
  • 1

0 Answers0