131

I need to concatenate some relatively large text files, and would prefer to do this via the command line. Unfortunately I only have Windows, and cannot install new software.

type file1.txt file2.txt > out.txt

allows me to almost get what I want, but I don't want the 1st line of file2.txt to be included in out.txt.

I have noticed that more has the +n option to specify a starting line, but I haven't managed to combine these to get the result I want. I'm aware that this may not be possible in Windows, and I can always edit out.txt by hand to get rid of the line, but is there a simple way of doing it from the command line?

James
  • 65,548
  • 14
  • 155
  • 193

13 Answers13

140
more +2 file2.txt > temp
type temp file1.txt > out.txt

or you can use copy. See copy /? for more.

copy /b temp+file1.txt  out.txt
ghostdog74
  • 327,991
  • 56
  • 259
  • 343
  • 4
    Of course! I would have preferred to have avoided the use of temporary files though. I tried to use parentheses, pipes and < to get it into one command, but couldn't get anywhere. The `copy` command is much faster, but it puts a SUB character at the end. Is there a way to avoid this? – James Mar 19 '10 at 13:13
  • 20
    I would add that if you want to concatenate ALL files you can do `copy /b *.txt combined.txt` without having to list the files individually. – Phlucious Jun 01 '15 at 18:13
  • 1
    more seemingly convert tab into spaces, pity! – Antonio Aug 18 '15 at 08:13
  • is there any command to retrieve original files from merged files? – swapnil gandhi Feb 25 '16 at 07:33
  • 3
    @ghostdog74: I think it has to be `type file1.txt temp > out.txt` to actually append the second file, without a header to the first one – Marius Feb 25 '16 at 09:20
62

I use this, and it works well for me:

TYPE \\Server\Share\Folder\*.csv >> C:\Folder\ConcatenatedFile.csv

Of course, before every run, you have to DELETE C:\Folder\ConcatenatedFile.csv

The only issue is that if all files have headers, then it will be repeated in all files.

Raj More
  • 47,048
  • 33
  • 131
  • 198
  • 2
    When I enter a filename for the concatenated file, which means it's listed at the end of the files in the loaction (alphabetical order), then windows seems to concatenate twice! I ended up using a filename of 1filename.csv to not have the problem. I guess concatting into a different folder should work also... – SebK Aug 06 '14 at 06:54
  • 1
    If you use > instead of >>, you do not have to delete the file beforehand. > redirects output and creates the file new every time. >> redirects output and appends. – Eddie Deyo Dec 05 '14 at 14:50
  • 1
    How does this skip the first line in file2, which the OP asked about? – Dan Dascalescu Aug 26 '15 at 22:03
  • 1
    It doesn't skip the first line in file2. I missed that part of the question. – Raj More Aug 28 '15 at 14:28
  • 1
    is there any command to retrieve original files from merged files? – swapnil gandhi Feb 25 '16 at 07:33
23

I don't have enough reputation points to comment on the recommendation to use *.csv >> ConcatenatedFile.csv, but I can add a warning:

If you create ConcatenatedFile.csv file in the same directory that you are using for concatenation it will be added to itself.

pmr
  • 58,701
  • 10
  • 113
  • 156
John Faughnan
  • 391
  • 4
  • 10
6

Use the FOR command to echo a file line by line, and with the 'skip' option to miss a number of starting lines...

FOR /F "skip=1" %i in (file2.txt) do @echo %i

You could redirect the output of a batch file, containing something like...

FOR /F %%i in (file1.txt) do @echo %%i
FOR /F "skip=1" %%i in (file2.txt) do @echo %%i

Note the double % when a FOR variable is used within a batch file.

Alberto Rossini
  • 149
  • 1
  • 2
5

Here's how to do this:

(type file1.txt && more +1 file2.txt) > out.txt
Waldo
  • 423
  • 4
  • 5
4

I would put this in a comment to ghostdog74, except my rep is too low, so here goes.

more +2 file2.txt > temp
This code will actually ignore rows 1 and 2 of the file. OP wants to keep all rows from the first file (to maintain the header row), and then exclude the first row (presumably the same header row) on the second file, so to exclude only the header row OP should use more +1.

type temp file1.txt > out.txt

It is unclear what order results from this code. Is temp appended to file1.txt (as desired), or is file1.txt appended to temp (undesired as the header row would be buried in the middle of the resulting file).

In addition, these operations take a REALLY LONG TIME with large files (e.g. 300MB)

Brian D
  • 2,570
  • 1
  • 24
  • 43
3

In powershell:

Get-Content file1.txt | Out-File out.txt
Get-Content file2.txt | Select-Object -Skip 1 | Out-File -Append out.txt
Eric Aya
  • 69,473
  • 35
  • 181
  • 253
JaR
  • 201
  • 2
  • 11
2

I know you said that you couldn't install any software, but I'm not sure how tight that restriction is. Anyway, I had the same issue (trying to concatenate two files with presumably the same headers) and I thought I'd provide an alternative answer for others who arrive at this page, since it worked just great for me.

After trying a whole bunch of commands in windows and being severely frustrated, and also trying all sorts of graphical editors that promised to be able to open large files, but then couldn't, I finally got back to my Linux roots and opened my Cygwin prompt. Two commands:

cp file1.csv out.csv
tail -n+2 file2.csv >> out.csv

For file1.csv 800MB and file2.csv 400MB, those two commands took under 5 seconds on my machine. In a Cygwin prompt, no less. I thought Linux commands were supposed to be slow in Cygwin but that approach took far less effort and was way easier than any windows approach I could find.

Andrew Mao
  • 35,740
  • 23
  • 143
  • 224
0

You can also simply try this

type file2.txt >> file1.txt

It will append the content of file2.txt at the end of file1.txt

If you need original file1.txt, take a backup beforehand. Or you can do this

type file1.txt > out.txt
type file2.txt >> out.txt

If you want to have a line break at the end of the first file, you can try the following command before appending.

type file1.txt > out.txt
printf "\n" >> out.txt
type file2.txt >> out.txt
Hamed
  • 1,175
  • 3
  • 20
  • 46
0

The help for copy explains that wildcards can be used to concatenate multiple files into one.

For example, to copy all .txt files in the current folder that start with "abc" into a single file named xyz.txt:

copy abc*.txt xyz.txt
GaTechThomas
  • 5,421
  • 5
  • 43
  • 69
0

In Windows 10, check out the FORFILES command. You'll need to rework my example below to append the original file while leaving that out of the mask or you'll lose the column headers. If you can arrange to have the column headers on the second line of the first file coming back from "*.csv", you'll be good anyway. Like, put it in the root directory over the subdirectories where the real .csv files are.

Be careful with the output path because it seems like the command likes to do a 'cd' to the file path parent directory before running the 'cmd'. So something like .\All.NotCSV will put one of those in each folder containing a .csv file and you'll be right back where you started. You are better off hardwiring the output file path.

forfiles /S /M *.csv /C "cmd /C more +1 @path >> ..\All.NotCSV"
B H
  • 1,730
  • 18
  • 24
-1
more +2 file1.txt > type > out.txt && type file2.txt > out.txt
laalto
  • 150,114
  • 66
  • 286
  • 303
Wujin
  • 11
-1

This takes Test.txt with headers and appends Test1.txt and Test2.txt and writes results to Testresult.txt file after stripping headers from second and third files respectively:

type C:\Test.txt > C:\Testresult.txt && more +1 C:\Test1.txt >> C:\Testresult.txt && more +1 C:\Test2.txt >> C:\Testresult.txt
Gibolt
  • 42,564
  • 15
  • 187
  • 127
user10301222
  • 1