0

There is a large file where the last symbols are \r\n. I need to remove them. It seems to be equivalent to removing the last line(?). UPD: no, it's not: a file have only one line, which ends with \r\n.

I know two ways, but both don't work for AIX:

sed 's/\r\n$//' file # I don't why it doesn't work
head -c-2 # head doesn't work with negative numbers

Is there any solution for AIX? A lot of large files must be processed, so performance is important.

Vikora
  • 174
  • 1
  • 6
  • 19
  • Program `split(1)` would be an option... Something like this: `export Fname=insertfilenamehere; Fsize="$(set -- `ls -l $Fname` ; echo $5)"; export NewSize=$(expr $Fsize - 2); split -b "$NewSize" -- "$Fname"; mv xaa ${Fname}_${NewSize}` – Zsigmond Lőrinczy Sep 04 '17 at 18:33
  • 1
    `dos2unix file` may be all you need. make a copy of your smallest problem file and test on that. With luck, it will load without issue or further messing around. Note that you can specify multiple files as arguments to one invocation of `dos2unix`. Good luck. – shellter Sep 05 '17 at 03:36
  • My last comment lost its formatting, next try: `export Fname=insertfilenamehere; Fsize="$(set -- $(ls -l $Fname); echo $5)"; export NewSize=$(expr $Fsize - 2); split -b "$NewSize" -- "$Fname"; mv xaa ${Fname}_${NewSize}` – Zsigmond Lőrinczy Sep 05 '17 at 04:06
  • Your `sed` command doesn't work, because the `\n` is not part of the string processed. Do you really want to remove that character? – Walter A Sep 05 '17 at 20:24

2 Answers2

0

Usually, if you need to edit a file via a script in place, I use ed due to historical reasons. For example:

ed - /tmp/foo.txt <<EOF
g/^$/d
w
q
EOF

ed is more than a bit cantankerous. Note also that you did not really remove the empty lines at the bottom of the file but rather all of the empty lines. With ed and some practice you can probably achieve deleting only the empty lines at the bottom of the file. e.g. go to the bottom of the file, search up for a non-empty line, then move down a line and delete from that point to the end of the file. ed command scripts act (pretty much) as you would expect.

Also, if they really do have \r\n, then those are not going to be considered empty lines but rather lines with a control-M (\r) in them. You may need to adjust your pattern if that is the case.

pedz
  • 2,271
  • 1
  • 17
  • 20
0

My answer https://stackoverflow.com/a/46083912/3220113 to the duplicate question should work here too. Another solution is using

awk ' (NR>1) { print s }
      {s=$0}
      END { printf("%s",substr($2, 1, length($2)-1) ) }
    ' inputfile
Walter A
  • 19,067
  • 2
  • 23
  • 43