2

I'm using LFTP to mirror some Git files to my server from a Docker image using a CI/CD Pipeline vendor. This works as expected but when using the mirror command from LFTP, I need to exclude any kind of file starting with a dot, like so:

lftp -u $USERNAME, -e "[...] mirror [...] --ignore-time -x '/(\.\w*.*)/$' --exclude README.md [...]; exit" sftp://$HOST

But none of those are working, it just jumps out that -x command and goes for the next one (e.g. --exclude README.md).

I guess is not about using -x or --exclude since, from the docs:

mirror [OPTS] [source [target]]

[...]

-x RX, --exclude=RX exclude matching files

This ones didn't work either:

-x '/(\.\w*.*)/$'

-x /(\.\w*.*)/$

-x /(\.\w*.*)/

-x (\.\w*.*)

-x \.\w*.*

What is wrong then? Isn't that a valid regex for Bash?

Community
  • 1
  • 1
Jimmy Adaro
  • 1,325
  • 15
  • 26
  • 3
    best I can tell from your code it's not bash that's trying to understand your regexp its `mirror`. idk about lftp specifically but very few tools will understand `\w` in a regexp. If you're trying to represent "word constituent characters" with `\w` then try replacing it with the POSIX `[[:alnum:]_]` instead, e.g. `-x '^\.[[:alnum:]_]*'`, but that's probably not necessary anyway since the regexp to match any string starting with a `.` is just `^\.`. – Ed Morton May 14 '19 at 14:22
  • This is your regex `\.` it matches the first dot it finds, doesn't matter where it is. If you want to only match at the beginning or spaces then dot, use `^\s*\.` or `^[ \t]*\.` –  May 14 '19 at 14:49
  • @EdMorton Almost. The regexp to match any string starting with a `.` is `^\..*`, because `^\.` would only match the dot itself. I hope this doesn't sound like nitpicking, but it is a difference. – bkis May 14 '19 at 16:46
  • @mumpitz in the context of strings, `.` is a string and that string starts with `.`. – Ed Morton May 14 '19 at 16:49
  • But that's not *any* string starting with `.`. It's just `.`. Anyway, I just wanted to be precise here - it's independent from the context of this question, so yeah, my bad. I just see many people getting confused because a regexp like `^\.` doesn't match their string `.foo`. – bkis May 14 '19 at 16:56
  • @mumpitz Oh I get where you're coming from now. To use `grep` as an example, I was thinking of `grep '\.'` vs `grep '\..*'` while you're thinking of `grep -o '\.' file` vs `grep -o '\..*'`. Understood. – Ed Morton May 14 '19 at 19:05
  • For some reason none of those regex could make it work using `ls -latr | grep '...'`, I need to exclude `.git` (and the content), `.editorconfig`, etc. – Jimmy Adaro May 14 '19 at 20:34
  • 1
    @EdMorton I'm happy we sorted that out - I started feeling like a pedantic pest. – bkis May 15 '19 at 08:50
  • @JimmyAdaro why not try what I suggested: `ls -latr | grep -v '^\.'`? Obviously if you were really using `ls` you just wouldn't include the `a` arg so `ls` wouldn't output the `.` files in the first place. – Ed Morton May 15 '19 at 12:54
  • 1
    @EdMorton You're totally right hahaha Anyway, `--exclude '^\..*$'` worked for me! – Jimmy Adaro May 15 '19 at 13:57

2 Answers2

2

For some reason the only way that worked for me is to use $ at the end of the --exclude regex, like so:

lftp -u $USERNAME, -e "[...] mirror [...] --exclude '^\..*$'[...]

Without the end-of-line delimiter it seems to not work at all, just ignores that --exclude or -x option and goes for the next one . Also, the regex must be wrapped in quotation marks (single or double).

Hope this helps someone :)

Jimmy Adaro
  • 1,325
  • 15
  • 26
1

It may be easier to use -X option like this: -X .* -X .*/

lav
  • 1,351
  • 9
  • 17