-2

I am a backend dev at my current company. When the front-end team has an issue, they will sometimes use Chrome devtool's "copy request as curl" feature and send me the problematic curl request. I was thinking it would be cool if I could write a script to transform these curls in various ways -- removing, adding, or adjusting parameters, among other things.

The first step here I think would be parsing the command into something easier to work with. I was thinking about using something like click or docopt, but didn't see an easy command to do command line string -> intelligent python object or something like that. Ideally it would work not just in this case, but for any valid Unix command. Is there a lib that can do this easily? Or would I need to extend one of these libraries? Or am I better off just writing something from scratch?

Example of a curl I'm trying to parse:

curl http://myhost?myarg=ACTIVE&myotherarg=1 -H Pragma: no-cache -H Origin: http://localhost:5000 -H Accept-Encoding: gzip, deflate, sdch -H Accept-Language: en-US,en;q=0.8 -H Authorization: Token xxxxxxxx1234567 -H Accept: application/json, text/javascript, /; q=0.01 -H Cache-Control: no-cache -H User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36 -H Connection: keep-alive -H Referer: http://localhost:5000/projects/7/overview-all/active/ --compressed

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Jesse Aldridge
  • 7,991
  • 9
  • 48
  • 75
  • 1
    You might want to consider writing a python adaption to this: https://shibukawa.github.io/curl_as_dsl/ which is a generic curl-command-line-to-language-X converter project with support for many other languages already. – Daniel Stenberg Feb 09 '16 at 07:29
  • That's cool. Thanks for the link. – Jesse Aldridge Feb 09 '16 at 18:52

2 Answers2

0

The only way this could work in the general case is if all Unix commands (or at least the subset you're working with) had a consistent translation between their arguments and the Python counterparts to whatever arguments are being passed. This is highly unlikely.

For cURL specifically, the flags aren't too bad: Try using the Python argparse library (tutorial here) to write a simple parser that picks apart the command itself and checking out the requests library to establish a dictionary format that would correspond to its normal keyword arguments.

kungphu
  • 4,592
  • 3
  • 28
  • 37
  • It looks like with argparse (as with other similar libs) you need to manually tell the system what the arguments are ahead of time (e.g. parser.add_argument('-H', ...)). Not quite what I'm looking for. I'm looking for something that uses heuristics to infer what those arguments are by inspecting the command string. Imperfect results are ok. – Jesse Aldridge Feb 09 '16 at 18:55
  • Right, I understand, I just don't see how that could possibly be accomplished. Command line tools accept arbitrary arguments, flags, etc. in various combinations and configurations based on their own requirements and their authors' preferences; you could potentially just convert a command to a dictionary, but as far as having it be able to do anything useful without human intervention, the consistency is not there. – kungphu Feb 10 '16 at 01:49
0

Turns out the python module shlex can do most of the heavy lifting.

I wrote some code to handle my particular use case: https://github.com/JesseAldridge/chrome_to_local_curl

Jesse Aldridge
  • 7,991
  • 9
  • 48
  • 75