How to use cat to lowercase large files in bash?

Question

I'm using the following command to lowercase all characters in a text file.

CORPUSLOWER=$(cat foobar.txt | tr '[:upper:]' '[:lower:]')
echo $CORPUSLOWER > foobar.txt

The problem, however, is that foobar.txt is about 20gig big and it only replaces a small chunk of the foobar.txt file.

My question: how can I use bash to convert a large file to lowercase?

Maybe using just tr '[:upper:]' '[:lower:]' < foobar.txt > newfile.txt, use less memory — Incrivel Monstro Verde, Nov 26 '18 at 15:55
([Useless use of cat.](https://en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat)) — Biffen, Nov 26 '18 at 15:59
Also, useless use of variable capture. Attempting to read the entire file into a shell variable just so you can `echo` it to standard output is pretty pointless, seeing as `tr` itself already is capable of printing its result to standard output (and not much else in fact). — tripleee, Nov 27 '18 at 04:56

score 2 · Accepted Answer · answered Nov 26 '18 at 15:59

2

It should be possible by

tr '[:upper:]' '[:lower:]' < foobar.txt | dd of=foobar.txt conv=notrunc

Or (which might be cleaner), use a temporary file

answered Nov 26 '18 at 15:59

ensc

score 1 · Answer 2 · answered Nov 26 '18 at 16:04

1

Assuming you have sponge from moreutils, you can do:

tr '[:upper:]' '[:lower:]' < foobar.txt | sponge foobar.txt

sponge "soaks up" all the contents and only opens the file when it saw the end of the file.

answered Nov 26 '18 at 16:04

L3viathan

2

If you dont have that `tail -n 10000000000` works well in general, though the corpus file might be huge. Is it only 1 line? – Gem Taylor Nov 26 '18 at 17:18
jep, that is helpful, thanks – Bob van Luijt Nov 26 '18 at 20:06

2 Answers2