Bash: unexpected EOF while looking for matching

Question

I am trying to run a BitBucket pipeline on a remote AWS Windows Server 2016 Datacenter.

image: python:3.5.1

pipelines:
  custom: # Pipeline that only runs manually
    default:
      - step:
          caches:
            - pip
          script: # Modify the commands below to build your repository.
            - pip install awscli
            - <- 
              aws ssm send-command 
              --document-name "AWS-RunRemoteScript" 
              --instance-ids "i-000000000000000" 
              --parameters '{\"sourceType\":[\"S3\"],\"sourceInfo\":[\'{\"path\":\"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1\"}\'],\"executionTimeout\":[\"3600\"],\"commandLine\":[\"test.ps1\"],\"workingDirectory\":[\"C:\\\\TOM\"]}' 
              --timeout-seconds 600 
              --region us-east-2

When i try running the pipeline, i get the following error:

+ <- aws ssm send-command --document-name "AWS-RunRemoteScript" --instance-ids "i-000000000000000" --parameters '{\"sourceType\":[\"S3\"],\"sourceInfo\":[\'{\"path\":\"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1\"}\'],\"executionTimeout\":[\"3600\"],\"commandLine\":[\"test.ps1\"],\"workingDirectory\":[\"C:\\\\TOM\"]}' --timeout-seconds 600 --region us-east-2
bash: /opt/atlassian/pipelines/agent/tmp/bashScript5876925824407417765.sh: line 8: unexpected EOF while looking for matching `''

I've looked into the syntax 2-3 times but not sure what exactly the issue is. I don't see any extra single quote in the command. Any help will really be appreciated.

UPDATE 1:

@rici:

+ aws ssm send-command  --document-name "AWS-RunRemoteScript"  --instance-ids "i-0000000000000000"  --parameters '{"sourceType":["S3"],"sourceInfo":['\''{"path":"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1"}'\''],"executionTimeout":["3600"],"commandLine":["test.ps1"],"workingDirectory":["C:\\TOM"]}'

Error parsing parameter '--parameters': Invalid JSON: Expecting value: line 1 column 36 (char 35)
JSON received: {"sourceType":["S3"],"sourceInfo":['{"path":"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1"}'],"executionTimeout":["3600"],"commandLine":["test.ps1"],"workingDirectory":["C:\\TOM"]}

UPDATE 2:

+ aws ssm send-command --document-name "AWS-RunRemoteScript" --instance-ids "i-0000000000000000" --parameters '{\"sourceType\":[\"S3\"],\"sourceInfo\":["{\"path\":\"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1\"}"],\"executionTimeout\":[\"3600\"],\"commandLine\":[\"test.ps1\"],\"workingDirectory\":[\"C:\\\\TOM\"]}' --timeout-seconds 600 --region us-east-2
Error parsing parameter '--parameters': Invalid JSON: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
JSON received: {\"sourceType\":[\"S3\"],\"sourceInfo\":["{\"path\":\"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1\"}"],\"executionTimeout\":[\"3600\"],\"commandLine\":[\"test.ps1\"],\"workingDirectory\":[\"C:\\\\TOM\"]}

rici · Accepted Answer · 2018-08-12T14:56:21.327

In a shell, inside a single-quoted string backslashes don't escape anything. They are just ordinary characters. Also, you can concatenate multiple quoted or unquoted string segments by just writing them together, as long as whitespace is quoted.

So if you have:

'abc\'def\'ghi'

You get:

A single-quoted segment consisting of the four characters abc\. The second ' terminates this quoted segment.
The unquoted characters def
A backslash-escaped ' (because backslashes do escape quotes outside of single-quoted segments)
The unquoted characters ghi
And finally, an unmatched ', giving rise to the error message.

Since backslashes are ordinary characters inside single-quoted strings, you probably don't want the backslashes before the double quotes either. So it is possible that what you are aiming at is:

--parameters '{"sourceType":["S3"],"sourceInfo":['\''{"path":"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1"}'\''],"executionTimeout":["3600"],"commandLine":["test.ps1"],"workingDirectory":["C:\\\\TOM"]}'

Note the use of the sequence '\'' to:

close the single-quoted segment
insert a single-quote
open a new single-quoted segment

Since it is not possible to put a single quote inside a single-quoted string, this idiom is pretty common, although it is often written as '"'"'.

However, I'm not totally convinced that you really should be single-quoting the sourceInfo parameter. If that's supposed to be JSON, single quotes are not valid, so the sourceInfo parameter's value would need to be double-quoted and double-quotes inside that string value would need to be escaped:

--parameters '{"sourceType":["S3"],"sourceInfo":["{\"path\":\"https://s3.us-east-2.amazonaws.com/my-bucket-name/test.ps1\"}"],"executionTimeout":["3600"],"commandLine":["test.ps1"],"workingDirectory":["C:\\\\TOM"]}'

Thanks for the explanation and suggestion. Please check the UPDATE section in my post. — Technext, Aug 12 '18 at 06:52
@technext: yes, that is precisely what I was talking about in the last paragraph. Single quotes are not valid JSON, so you get a JSON parsing error. I explained the solution in that paragraph, too. — rici, Aug 12 '18 at 07:05
@rici: Nice explanation; It might be possible that a heredoc might work for the JSON, that would probably take care of the escaping issues... I haven't tried it though. — l'L'l, Aug 12 '18 at 07:17
@rici: Sorry for only trying the code block you first mentioned. Can you please check "UPDATE 2" to see whether i got your suggestion right? — Technext, Aug 12 '18 at 07:26
@Technext: I added an explicit example, also reintroducing the doubled backslashes into the filepath at the end, because I'm pretty sure that's what the JSON parser will require. — rici, Aug 12 '18 at 13:44
Why does it happen that after getting things working, error message becomes _little more evident_ or _quite evident_ than it earlier was? :( Thanks for explicitly mentioning the command @rici! Finally got to see a successful build message. Thanks a ton :) — Technext, Aug 12 '18 at 14:20

Bash: unexpected EOF while looking for matching

1 Answers1