Copy folders and store them in text file to exclude once copied folders on next run

Question

I try to write a script that can monitor a folder H:\Start and copy a new subfolder with its files in H:\Start to new location H:\Target. The script should store the copied folder and files in a text file.

Each time the script starts new and monitors H:\Start, it should check the text file and copy only those subfolders which are not yet included in the text file because copied already before.

I was searching the world wide web for examples, but could not really find a starting point. Any help would be nice.

I have so far not working good :)

@echo off
setlocal EnableDelayedExpansion
pushd %1
for /D %%d in (“H:\Start\*.*”) do set n=!n!;%%d
if defined n echo %n:~1% > C:\Desktop\list.txt
popd
endlocal
for /f %%i in (C:\Desktop\list.txt) do not (
    xcopy /d /s H:\Start H:\Target > C:\Desktop\list.txt >nul 2>&1
)

Mofi · Accepted Answer · 2015-08-01T18:33:38.693

I suggest using:

%SystemRoot%\System32\xcopy.exe H:\Start H:\Target /C /H /I /K /M /Q /R /S /Y >nul

For information about all those parameters of xcopy open a command prompt window and run there xcopy /? to get help of this command displayed which explains all those parameters.

The important one is /M which selects for copying process just files with archive attribute set and which removes the archive attribute on each file in H:\Start after copying the file. This avoids that a once copied file is copied once more as long as not modified in H:\Start since last copy.

If you want to log all copied files into a text file, I suggest using:

%SystemRoot%\System32\xcopy.exe H:\Start H:\Target /C /F /H /I /K /M /R /S /Y >>C:\Desktop\list.txt

The names of the copied files are with this command line appended to text file C:\Desktop\list.txt

The following commented batch code works with directory lists as asked for.

@echo off
rem Define source and destination directory as well as
rem the names of the used list files each with full path.
setlocal EnableExtensions
set "Source=H:\Start"
set "Destination=H:\Target"
set "MainDirList=C:\Desktop\list.txt"
set "CurrentList=%Temp%\CurrentList.tmp"
set "ExcludeList=%Temp%\ExcludeList.tmp"
set "FinalDoList=%Temp%\FinalDoList.tmp"

rem Write the names of all subdirectories found in source
rem directory into a list file in directory for temporary files.
dir /AD /B "%Source%">"%CurrentList%"

rem Check if list file is not empty because of no subdirectories.
call :CheckEmpty "%CurrentList%"
if %FileIsEmpty% == 1 (
    echo No directories in %Source%
    goto EndBatch
)

rem Start copying the directories if there is no main list file
rem from a previous execution of this batch file or the main list
rem file was deleted intentionally to force copying all again.
if not exist "%MainDirList%" goto CopyDirectories

rem Start copying also if main list file is an empty file.
call :CheckEmpty "%MainDirList%"
if %FileIsEmpty% == 1 del "%MainDirList%" & goto CopyDirectories

rem Search in main list for lines matching completely lines in current
rem list with ignoring case and write the found lines into an exclude
rem list file as those directories were copied already before.
%SystemRoot%\System32\findstr.exe /I /L /X /G:"%CurrentList%" "%MainDirList%" >"%ExcludeList%"

rem Copy all directories if no line in current list is found in main list.
if errorlevel 1 goto CopyDirectories

rem Get all lines from current list not listed also in exclude list.
%SystemRoot%\System32\findstr.exe /B /I /L /V /G:"%ExcludeList%" "%CurrentList%" >"%FinalDoList%"

rem Replace the current list with the reduced final list.
move /Y "%FinalDoList%" "%CurrentList%"

rem Check if remaining current list is not empty because
rem all subdirectories copied already before.
call :CheckEmpty "%CurrentList%"
if %FileIsEmpty% == 1 (
    echo Copied already before all directories in %Source%
    goto EndBatch
)

:CopyDirectories
rem Copy each directory in remaining current list file.
for /F "usebackq delims=" %%D in ("%CurrentList%") do (
    echo Coping %Source%\%%D
    %SystemRoot%\System32\xcopy.exe "%Source%\%%D" "%Destination%\%%D" /C /H /I /K /Q /R /S /Y >nul
)

rem Append the list of copied directories to main list file.
type "%CurrentList%">>"%MainDirList%"
goto EndBatch

:CheckEmpty
rem This little subroutine just checks if size of a list file is 0.
if %~z1 == 0 ( set "FileIsEmpty=1" ) else ( set "FileIsEmpty=0" )
goto:EOF

:EndBatch
rem Delete all not further needed listed files and environment variables.
del "%ExcludeList%" 2>nul
del "%CurrentList%"
endlocal

This batch file should work for the FTP folder mounted as drive on Windows. It does not depend on attributes or timestamps. It uses explicitly only the names of the directories in H:\Start. It also does not check which directories exist already in H:\Target. Therefore it is possible to delete a directory in H:\Target if not interested in and the deleted directory will be nevertheless not copied once again from H:\Start as long as the deleted directory is not also removed from main list file.

For details on parameters used on findstr run in a command prompt window findstr /? and read the entire help output into the window.

Thanks for this question as this was really an interesting batch coding task.

Thank you both, problem with xcopy is I do not have write permissions for `H:\Start` to removes the archive attribute on each file. I use a program (NetDrive2) that indicating an FTP server as virtual drive `h:\start`. I played a little bit with your information please have a look at my Edit above. — BASF, Jul 27 '15 at 11:56
Wanted to change this part here `for /F "usebackq delims=" %%D in ("%CurrentList%") do ( echo Coping %Source%\%%D` to this `for /F "usebackq delims=" %%D in ("%CurrentList%") do ( echo Coping %Source%\%%D if "!Source:[LABLE]=!" == "%%D" (` but that does not work, any idea about this? — BASF, Aug 14 '15 at 01:02
You either change the line `setlocal EnableExtensions` to `setlocal EnableExtensions EnableDelayedExpansion` to be able to use also delayed expansion or use `"%Source:[LABLE]=%"` in __IF__ condition. Well, I do not really understand why source directory should contain the string `[LABLE]` to be removed and then remaining string is compared with string from current list file. — Mofi, Aug 14 '15 at 05:57
Label was just an example. Folders that start with a particular word should not be loaded, that is all why I ask about. — BASF, Aug 14 '15 at 20:11
As this batch code already uses include and exclude lists, I suggest to add the directories to exclude manually to `C:\Desktop\list.txt` - the main directory list - as all directories in those lists are ignored by default. You could also replace __findstr__ parameter `/X` by `/B` to exclude directories just beginning with a string defined in main directory list instead of complete match. If you can't find the right method to exclude the directories you don't want copied, create a new question with explaining in detail what to copy and what to exclude on an example with a link to this question. — Mofi, Aug 15 '15 at 12:01

score 1 · Answer 2 · answered Jul 27 '15 at 05:21

You don't need to store anything, you can use

xcopy /d /s h:\start h:\target

/D:mm-dd-yyyy
Copy files changed on or after the specified date. If no date is given, copy only files whose source date/time is newer than the destination time.

but if you need a list of the files you can just use a redirection :

xcopy /d /s h:\start h:\target > logfile.txt

Copy folders and store them in text file to exclude once copied folders on next run

2 Answers2

Linked