2

I'm trying to use LFTP in my GitLab continuous integration setup so I can mirror JSON files with my destination. However, I'd like to only mirror on a set of folders using a wildcard, but I cannot seem to get this working.

I tried this mirror command configuration in LFTP, but this results in a "No such file or directory" error. I assume I'm parsing the wildcard wrong somehow.

What I tried: lftp -c "set sftp:auto-confirm true; open sftp://$DEVELOPMENT_DEPLOY_USER:$DEVELOPMENT_DEPLOY_PASSWORD@$DEVELOPMENT_DEPLOY_HOST:$DEVELOPMENT_DEPLOY_PORT; mirror -Rev ./somefolder_* $DEVELOPMENT_DESTINATION_FOLDER --ignore-time --parallel=10 --exclude .* --exclude .*/ --include ./*.json"

Results in: /home/gitlab-runner/builds/82ffc821/0/somegroup/someproject/somefolder_*: No such file or directory

I'm probably missing something obvious. Would appreciate any help.

Dennis van der Veeke
  • 775
  • 2
  • 9
  • 25
  • Try adding a `-v` to your command right after `lftp` to get more verbose output. – Stefan van Gastel Mar 11 '18 at 12:26
  • @StefanvanGastel it doesn't provide any more info in the console. The error remains; no such file or directory. – Dennis van der Veeke Mar 11 '18 at 13:52
  • Ok, try reducing the command to get the reason for this error. So test with uploading all files before adding wildcards. – Stefan van Gastel Mar 11 '18 at 13:53
  • The error is caused by the wildcard I'm trying to use for the source folder. What I'm trying to do is; I have two folders: somefolder_one and somefolder_two. I'm trying to mirror both by specifying somefolder_* in the command, as wildcard, but this doesn't seem to be parsed properly and it tries to load the exact folder somefolder_*, which does not exist. It should pick both folders as source and mirror those two. – Dennis van der Veeke Mar 11 '18 at 13:59

1 Answers1

1

Maybe old but find that in man :

Include and exclude options can be specified multiple times. It means that a file or directory would be mirrored if it matches an include and does not match to excludes after the include, or does not match anything and the first check is exclude. Directories are matched with a slash appended.

So first it will execute "include" argument, lastly "exclude" with is " --exclude .* --exclude .*/". After that glob, no file math to mirror. Use "--verbose" to check what files are touch by lftp

mwfearnley
  • 3,303
  • 2
  • 34
  • 35
Patryk
  • 329
  • 3
  • 12