How to remove a character at the end of each line in UNIX

Question

I would like to remove comma , at the end of each line in my file. How can I do it other than using substring function in awk?

Sample Input:

        SUPPLIER_PROC_ID BIGINT NOT NULL,
        BTCH_NBR INTEGER NOT NULL,
        RX_BTCH_SUPPLIER_SEQ_NBR INTEGER NOT NULL,
        CORRN_ID INTEGER NOT NULL,
        RX_CNT BYTEINT NOT NULL,
        DATA_TYP_CD BYTEINT NOT NULL,
        DATA_PD_CD BYTEINT NOT NULL,
        CYC_DT DATE NOT NULL,
        BASE_DT DATE NOT NULL,
        DATA_LOAD_DT DATE NOT NULL,
        DATA_DT DATE NOT NULL,
        SUPPLIER_DATA_SRC_CD BYTEINT NOT NULL,
        RX_CHNL_CD BYTEINT NOT NULL,
        MP_IMS_ID INTEGER NOT NULL,
        MP_LOC_ID NUMERIC(3,0),
        MP_IMS_ID_ACTN_CD BYTEINT NOT NULL,
        NPI_ID BIGINT,

Gilles Quénot · Answer 1 · 2023-04-13T11:39:38.603

87

Try doing this :

awk '{print substr($0, 1, length($0)-1)}' file.txt

This is more generic than just removing the final comma but any last character

If you'd want to only remove the last comma with awk :

awk '{sub(/,$/,""); print}' file.txt

edited Apr 13 '23 at 11:39

answered Feb 12 '13 at 20:14

Gilles Quénot

173,512
41
224
223

I was just trying something like this, and for some reason you need a `length($0)` without the `-1`... it doesn't seem to make a lot of sense to me either. But whatever, it's bash, as long as it works. – enriched Feb 20 '15 at 22:11

jlliagre · Accepted Answer · 2016-04-06T19:01:58.253

69

You can use sed:

sed 's/,$//' file > file.nocomma

and to remove whatever last character:

sed 's/.$//' file > file.nolast

edited Apr 06 '16 at 19:01

answered Feb 12 '13 at 20:14

jlliagre

29,783
6
61
72

Rather than sending the output to a newfile you can also append the -i switch to sed to write i to 'file' – bsmoo Aug 14 '14 at 07:27
2

@ubuntu101010101 Indeed but using this option might trigger a `sed` error instead. The question is tagged Unix. Using a non POSIX GNUism would make my suggestion not portable. – jlliagre Aug 14 '14 at 14:45

Juan Diego Godoy Robles · Answer 3 · 2016-07-29T14:32:58.907

An awk code based on RS.

awk '1' RS=',\n' file

or:

awk 'BEGIN{RS=",\n"}1' file

This last example will be valid for any char before newline:

awk '1' RS='.\n' file

Note: dot . matches any character except line breaks.

Explanation

awk allows us to use different record (line) regex separators, we just need to include the comma before the line break (or dot for any char) in the one used for the input, the RS.

Note: what that 1 means?

Short answer, It's just a shortcut to avoid using the print statement. In awk when a condition gets matched the default action is to print the input line, example:

$ echo "test" |awk '1'
test

That's because 1 will be always true, so this expression is equivalent to:

$ echo "test"|awk '1==1'
test
$ echo "test"|awk '{if (1==1){print}}'
test

Documentation

Check Record Splitting with Standard awk and Output Separators.

Upvoting this because I was able to pipe the output of `ls` into the second statement and strip a particular file extension. Some of the other answers might have worked for this as well, but this one was easier for me to read while still staying nice and short. — Doktor J, Jul 28 '16 at 21:30

score 0 · Answer 4 · answered Oct 07 '15 at 16:45

This Perl code removes commas at the end of the line:

perl -pe 's/,$//' file > file.nocomma

This variation still works if there is whitespace after the comma:

perl -lpe 's/,\s*$//' file > file.nocomma

This variation edits the file in-place:

perl -i -lpe 's/,\s*$//' file

This variation edits the file in-place, and makes a backup file.bak:

perl -i.bak -lpe 's/,\s*$//' file

How to remove a character at the end of each line in UNIX

4 Answers4

Linked

Related