6

I try to concatenate large files (some GB) in bash. I tried

    cat file1 file2 file3 > result

and it didn't work while 

    cat file1 file2 file3 >> result

worked. In both occasions the file result didn't exist before and my expectation would be, that both commands give the same result.

On the same system I tried the same thing with small files (just some bytes) and both commands produce the same output. I tried to find some explanation (for example here) but couldn't find any...

So, I know how to solve my problem, but I'm still puzzled. Is anyone able to produce a clue?

  • 4
    What happened when you run the first command? – lcd047 Jul 15 '15 at 12:04
  • Are those files all _regular files_? – Alepac Jul 15 '15 at 12:28
  • @lcd047 the individual files are .gz.parts and the result would not let me decompress it it gave an error –  Jul 15 '15 at 12:41
  • @Alepac yes no devices or anything virtual, just normal files –  Jul 15 '15 at 12:42
  • Are you using this command to write a conatenate file or you redirect through a pipe `|` to gunzip? – Alepac Jul 15 '15 at 12:47
  • @Alepac I tried playing with pipes and combining commands but when it didn't work I ended up creating a new concatenated file without any pipes, just the command as displayed above –  Jul 15 '15 at 13:04
  • What's the error that you have trying to write to file? – Alepac Jul 15 '15 at 13:08
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/83330/discussion-between-silentfury-and-alepac). –  Jul 15 '15 at 13:12
  • Which filesystem are you on? How big is each file? – knittl Jul 17 '15 at 05:36

1 Answers1

1

when I need to split file I use a trick that works very well:

tar --one-file-system -czv /home | split -b 4000m - /media/DRIVENAME/BACKUPNAME.tgz

then, to restore :

cat /media/DRIVENAME/BACKUPNAME.tgz.* | tar -x /

so cat do the job whatever the containt is. So if it doesn't work whether there is a bad production of your splited files, or a limitation with your filesystem. What filesystem are you using ?

dominix
  • 273
  • 1
  • 13