Python shlex split option that contains RegEx

Question

Lets say I want to give this command ./maryam -e crawl_pages -d domain.tld -r "a href=\".*" and split it.

When I run

>>>line = './maryam -e crawl_pages -d domain.tld -r "a href=\".*"'
>>>shlex.split(line)

I get the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/shlex.py", line 311, in split
    return list(lex)
  File "/usr/lib/python3.8/shlex.py", line 300, in __next__
    token = self.get_token()
  File "/usr/lib/python3.8/shlex.py", line 109, in get_token
    raw = self.read_token()
  File "/usr/lib/python3.8/shlex.py", line 191, in read_token
    raise ValueError("No closing quotation")
ValueError: No closing quotation

What I basically want is to have the -r option for the user to input a regular expression.
The result should look like this

['./maryam', '-e', 'crawl_pages', '-d', 'domain.tld', '-r', 'a href=\".*']

score 1 · Accepted Answer · answered Mar 18 '21 at 10:24

When you assign

# XXX BROKEN
line = './maryam -e crawl_pages -d domain.tld -r "a href=\".*"'

Python parses the string and parses any backslash escapes, so you end up with no actual backslash in the value of line. You can prevent one layer of backslash parsing by using a raw string instead;

line = r'./maryam -e crawl_pages -d domain.tld -r "a href=\".*"'

but if your expected value really should contain a literal backslash, your input is more fundamentally wrong.

I assume you just have an incorrect expectation, but if that's really really what you want, try

# XXX DUBIOUS
line = r'./maryam -e crawl_pages -d domain.tld -r "a href=\\\".*"'

which then however looks like this:

['./maryam', '-e', 'crawl_pages', '-d', 'domain.tld', '-r', 'a href=\\".*']

(because Python prints a double backslash to show a literal backslash).

Python shlex split option that contains RegEx

1 Answers1