Removing blank lines

Question

I have a csv file in which every other line is blank. I have tried everything, nothing removes the lines. What should make it easier is that the the digits 44 appear in each valid line. Things I have tried:

grep -ir 44 file.csv
sed '/^$/d' <file.csv
cat -A file.csv
sed 's/^ *//; s/ *$//; /^$/d' <file.csv
egrep -v "^$" file.csv
awk 'NF' file.csv
grep '\S' file.csv
sed 's/^ *//; s/ *$//; /^$/d; /^\s*$/d' <file.csv
cat file.csv | tr -s \n

Decided I was imagining the blank lines, but import into Google Sheets and there they are still! Starting to question my sanity! Can anyone help?

Answer 1

Use the -i option to replace the original file with the edited one.

sed -i '/^[ \t]*$/d' file.csv

Alternatively output to another file and rename it, which is doing the exactly what -i does.

sed '/^[[:blank:]]*$/d' file.csv > file.csv.out && mv file.csv.out file.csv

Answer 2

sed -n -i '/44/p' file

-n means skip printing
-i inplace (overwrite same file)
- /44/p print lines where '44' exists

without '44' present

sed -i '/^\s*$/d' file

\s is matching whitespace, ^startofline, $endofline, d delete line

Answer 3

Given:

$ cat bl.txt
Line 1 (next line has a tab)

Line 2 (next has several space)

Line 3

You can remove blank lines with Perl:

$ perl -lne 'print unless /^\s*$/' bl.txt
Line 1 (next line has a tab)
Line 2 (next has several space)
Line 3

awk:

$ awk 'NF>0' bl.txt
Line 1 (next line has a tab)
Line 2 (next has several space)
Line 3

sed + tr:

$ cat bl.txt | tr '\t' ' ' | sed '/^ *$/d'
Line 1 (next line has a tab)
Line 2 (next has several space)
Line 3

Just sed:

$ sed '/^[[:space:]]*$/d' bl.txt
Line 1 (next line has a tab)
Line 2 (next has several space)
Line 3

Answer 4

Aside from the fact that your commands do not show that you capture their output in a new file to be used in place of the original, there's nothing wrong with them, EXCEPT that:

cat file.csv | tr -s \n

should be:

cat file.csv | tr -s '\n'  # more efficient alternative: tr -s '\n' < file.csv

Otherwise, the shell eats the \ and all that tr sees is n.

Note, however, that the above only eliminates only truly empty lines, whereas some of your other commands also eliminate blank lines (empty or all-whitespace).

Also, the -i (for case-insensitive matching) in grep -ir 44 file.csv is pointless, and while using -r (for recursive searches) will not change the fact that only file.csv is searched, it will prepend the filename followed by : to each matching line.

---

If you have indeed captured the output in a new file and that file truly still has blank lines, the cat -A (cat -et on BSD-like platforms) you already mention in your question should show you if any unusual characters are present in the file, in the form of ^<char> sequences, such as ^M for \r chars.

Answer 5

If you like awk, this should do:

awk '/44/' file

It will only print lines that contains 44