0

I have a task in which I want to run below command on a shell on remote servers but whenever I run my playbook, it always gives me exception:

  - name: copy files
    shell: "machines=(machineA machineB machineC); for machine in $(shuf -e ${machines[@]}); do ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ threads }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'; [ $? -eq 0 ] && break; done"

Below is the error: I can't figure out what wrong I am doing here?

"stderr": "/bin/sh: 1: Syntax error: \"(\" unexpected", "stderr_lines": ["/bin/sh: 1: Syntax error: \"(\" unexpected"], "stdout": "", "stdout_lines": []}

I am able to run this command directly on shell on any remote servers but I am having issues while running through ansible.

This is my script that I have converted in one line and using in shell module:

machines=(machineA machineB machineC)
for machine in $(shuf -e ${machines[@]}); do 
    ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ threads }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'
   [ $? -eq 0 ] && break
done

Update:

I tried like this now and it is giving me different exception.

  - name: copy files
    shell: |
      set -x
      machines=(machineA machineB machineC)
      for machine in $(shuf -e ${machines[@]}); do
          ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ parallelism }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'
          [ $? -eq 0 ] && break
      done
    args:
      executable: /bin/bash

Full error: (I have shortened it down)

fatal: [some_machine]: FAILED! => {"changed": true, "cmd": "machines=(machineA machineB machineC)\n for machine in $(shuf -e ${machines[@]}); do\n ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/20180422/*' | parallel -j10 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'\n [ $? -eq 0 ] && break\n done", "delta": "0:00:37.546329", "end": "2018-04-29 23:27:44.003538", "msg": "non-zero return code", "rc": 1, "start": "2018-04-29 23:27:06.457209", "stderr": "ssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known\r\nssh: Could not resolve hostname : Name or service not known", "ssh: Could not resolve hostname : Name or service not known", "ssh: Could not resolve hostname : Name or service not known"], "stdout": "", "stdout_lines": []}

Also I tried this basic thing as well and this passes without any errors so I assume this works then?

  - name: copy files
    shell: |
      set -x
      machines=(machineA machineB machineC)
      for machine in $(shuf -e ${machines[@]}); do
          echo $machine
          [ $? -eq 0 ] && break
      done
    args:
      executable: /bin/bash

I believe it is some problem maybe with my scp statement? I am not sure just guessing?

Zoredache
  • 130,897
  • 41
  • 276
  • 420
user1950349
  • 223
  • 1
  • 3
  • 10
  • Why don't you just have ansible copy the files itself, rather than making some crazy shell script to do it? – Michael Hampton Apr 30 '18 at 03:28
  • I need to copy around 2000 files from either of those three boxes to all the remote servers (around 200 machines) that I have in my inventory. If ansible can copy in parallel then yes I can do that. Because in my above script, I am using gnu-parallel to do that. – user1950349 Apr 30 '18 at 03:35
  • @techraf sorry about that. I was not sure which one to use before so by mistake I posted twice. I will delete other one. – user1950349 Apr 30 '18 at 06:47

3 Answers3

2

Well I can't tell what your problem is exactly, but the error indicates that you have a syntax error in your command. Which is almost certainly related to something being escaped or not translated properly as it is executed.

A couple things I have to suggest. First use the yaml syntax to include a block of text which includes line endings.

- name: Run Script
  shell: |
    machines=(machineA machineB machineC)
    for machine in $(shuf -e ${machines[@]}); do
        ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' |
        parallel -j{{ threads }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'
      [ $? -eq 0 ] && break
    done

I think, that your fragment may have some bashisms. You may need to specify that your script be executed via bash instead of /bin/sh.

- name: Run command that requires bash
  shell: echo 'not a very good example.
  args:
    executable: /bin/bash

You might also want to add a set -x as the first line of your fragment. The more verbose output from the shell should help you see what exactly the error is.

Of course I would also suggest you try to use Ansible modules to do this instead. I suspect maybe the synchronize module combined with the async features.

HoldOffHunger
  • 141
  • 5
Zoredache
  • 130,897
  • 41
  • 276
  • 420
  • great suggestions. I tried them but unfortunately I am getting different error now. I have updated my question with what I tried and full error I am getting. It looks to me now I might be having issues with scp statement I have. Maybe this can give you some clue to help me figure out what wrong I am doing? – user1950349 Apr 30 '18 at 06:34
  • It is erroring out resolving the host name. So either your script variable with the host name isn't correct, or the remote machine you are on can't resolve the names machinea, machineb, machinec, or whatever your actual names are. Is name resolution working on the remote systems? Maybe add the `-v` option to your scp command. – Zoredache Apr 30 '18 at 06:43
  • Its very strange.. I just added echo infront of that whole one script line and stored the output in a variable which I printed out using debug module and I can see everything is fine in that one line without any issues. I even tried running that one line as it is and it copied all the files. So its very strange. Meaning I did this instead `echo "ssh -o StrictHostKeyChecking=no david@$machine 'ls -1 /process/snap/{{ folder }}/*' | parallel -j{{ parallelism }} 'scp -o StrictHostKeyChecking=no david@${machine}:{} /data/files/'"` and this printed out everything correctly. – user1950349 Apr 30 '18 at 06:46
  • On one of the remote server I did this `ps aux | grep scp` when my local box was running my ansible playbook and I found this there : `scp -o StrictHostKeyChecking=no david@:/process/snap/20180422/abc_944_log.data /data/files/`. You see it here after `david@` hostname is missing so somehow, its not able to interpret machine name correctly in my scp command. Any idea why it could be? – user1950349 Apr 30 '18 at 07:16
1

this work for me:

    - name: Generate new certificate
      shell: "keytool -genkey -keystore {{ keystore_key }} 
                -storepass {{ keystore_pass }} \
                -storetype {{ keystore_type }} \
                -alias {{ keystore_alias }} \
                -validity {{ keystore_validity }} \
                -keysize {{ keystore_keysize }} \
                -keyalg {{ keystore_keyalg }}"
Omar Odaini
  • 11
  • 1
0

Since you said, you are able to run this command directly on shell on any remote servers, try the script module with Ansible. 

- name: Run a script with arguments   
  script: /some/local/script.sh --some-argument 1234

Documentation is here: https://docs.ansible.com/ansible/latest/modules/script_module.html

cheers

CR7
  • 1