12

After setting up the salt-master and one minion, I am able to accept the key on the master. Running sudo salt-key -L shows that it is accepted. However, when I try the test.ping command, the master shows:

Minion did not return. [No response]

On the master, the log shows:

[ERROR ][1642] Authentication attempt from minion-01 failed, the public key in pending did not match. This may be an attempt to compromise the Salt cluster.

On the minion, the log shows:

[ERROR ][1113] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate

I have tried disconnecting and reconnecting, including rebooting both boxes in between.

tshepang
  • 12,111
  • 21
  • 91
  • 136
Toby
  • 12,743
  • 8
  • 43
  • 75
  • you tried re-registering the salt-minion? – tudoricc Feb 17 '16 at 14:57
  • Could you share the command you are executing on the master? Something like `salt '*' state.highstate`, I imagine. – Travv15 Feb 23 '16 at 16:26
  • At this point I'm unable to get past the command `sudo salt '*' test.ping`. I am assuming that if the test ping fails, no other commands will execute. – Toby Feb 23 '16 at 17:43
  • Possibly a [firewall issue](https://docs.saltstack.com/en/latest/topics/tutorials/firewall.html)? Or, maybe something like SELinux is blocking the communication with the minion? – Travv15 Feb 26 '16 at 19:19
  • Both boxes are debian-based (one Ubuntu, one Lubuntu) so I'm assuming that rules out selinux issues. I have no firewall running, and both boxes are on local networks. I will double-check that UFW or similar didn't somehow get activated though. – Toby Feb 28 '16 at 17:55
  • @Toby Did you find any solution? I have the same problem! – CPA Jul 25 '16 at 09:36

8 Answers8

6
Minion did not return. [No response]

Makes me think the salt-minion process is not running. (those other two errors are expected behavior until you accepted the key on the master)

Check if salt-minion is running with (depending on your OS) something like

$ systemctl status salt-minion

or

$ service salt-minion status

If it was not running, start it and try your test again.

$ sudo systemctl start salt-minion

or

$ sudo service salt-minion start

Depending on your installation method, the salt-minion may not have been registered to start upon system boot, and you may run into this issue again after a reboot.

Now, if your salt-minion was in fact running, and you are still getting No response, I would stop the process and restart the minion in debug so you can watch.

$ sudo systemctl stop salt-minion
$ sudo salt-minion -l debug

Another quick test you can run to test communication between your minion and master is to execute your test from the minion:

$ sudo salt-call test.ping

salt-call does not need the salt-minion process to be running to work. It fetches the states from the master and executes them ad hoc. So, if that works (returns

local:
    True

) you can eliminate the handshake between minion and master as the issue.

Travv15
  • 243
  • 1
  • 6
  • Thanks for this - the minion was definitely running, and I had restarted it and rebooted several times during troubleshooting. Running the `salt-call` command on the minion DOES return true though.The debug confirms that the config files and keys are the ones the minion is supposed to be using, and that it's trying to access the master on the correct IP and port. – Toby Feb 22 '16 at 14:29
  • in my case I have edited the /etc/salt/minion file and pointed to salt master but forgot to restart salt-minion service. so minion didnt get the change about salt master. After restarting minion all good. Thank you. – Raja G Sep 08 '17 at 07:43
3

I just hit this problem when moving the salt master to a new server, to fix it I had to do these things in this order (Debian 9):

root@minion:~# service salt-minion stop
root@master:~# salt-key -d minion
root@minion:~# rm /etc/salt/pki/minion/minion_master.pub
root@minion:~# service salt-minion start
root@master:~# salt-key -a minion

Please note the minion/master servers above.

complistic
  • 2,610
  • 1
  • 29
  • 37
2

If you are confident that you are connecting to a valid Salt Master, then remove the master public key and restart the Salt Minion. The master public key can be found at: /etc/salt/pki/minion/minion_master.pub

tudoricc
  • 709
  • 1
  • 12
  • 31
  • Thanks for the response - I tried this and the error remains. The logs show the same thing as before. – Toby Feb 17 '16 at 15:13
  • On the master: `[ERROR ][1642] Authentication attempt from minion-01 failed, the public key in pending did not match. This may be an attempt to compromise the Salt cluster.` And the minion: `[ERROR ][1113] The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate` – Toby Feb 17 '16 at 16:07
  • try deleting the minion key on the master and restarting the salt-minion service – tudoricc Feb 17 '16 at 16:08
  • I ran `sudo salt-key -d` and confirmed the key deletion. Then rebooted the minion, and running `sudo salt-key -L` on the master shows the unaccepted key. I ran `sudo salt-key -a minion-01` and confirmed the accepted key. Running `sudo salt-key -L` shows the key as accepted. However, running `sudo salt '*' test.ping` returns `Minion did not return. [No response]`. – Toby Feb 17 '16 at 16:14
  • The errors in the log are identical to the errors in the OP and the comment above. – Toby Feb 17 '16 at 16:22
  • have you tried updating the salt packages?I faced a similar issue at onepoint – tudoricc Feb 17 '16 at 16:25
  • and can you take a look at this issue http://comments.gmane.org/gmane.comp.sysutils.salt.user/13547 – tudoricc Feb 17 '16 at 16:29
  • there was an update available for the minion, I reset everything, rebooted both and the errors are the same. I checked the minion_id file mentioned in that support thread, it is correct. Thanks for the help, I'm going to kill both boxes and start from scratch. – Toby Feb 17 '16 at 16:42
1

Minion's public key position under master is /etc/salt/pki/master/minions

Just compare it with the minions own public key(under /etc/salt/pki/minion/minion.pub)

If it is not the same, excute salt-key -d * to delete the public key of minion from master, then excute service salt-minion restart to restart salt minion on the minion client. After this, Master can control the minion.

zhao
  • 1,094
  • 9
  • 9
0

I got the same error message.

I change user: root to user: ubuntu in /etc/salt/minion.

Stop salt-minion, and run salt-minion -l debug as ubuntu user. salt master could get the salt-minion response.

But, when I run salt-minion with systemctl start salt-minion, salt master got error. (no response)

I run salt-minion as root user and systemctl start salt-minion, it works.

I don't know if it is a bug.

Wanming Zhang
  • 323
  • 1
  • 8
0

I had this exact same problem when I was moving minions from one master to another. Here are the steps I took to resolve it.

  1. Remove the salt-master and salt-minon on the master server.
  2. rm -rf /etc/salt on the master hosts
  3. Save your minion config if it has any defined variables
  4. Save your minion config if it has any defined variables
  5. Remove salt-minion on the minion hosts
  6. rm -rf /etc/salt on all minion-hosts
  7. Reinstall salt-master and salt-minion
  8. Start salt-master then salt-minions

My problem was solved after doing this. I know it is not a solution really, but it's likely a conflict with keys that is causing this issue.

Robert Foster
  • 16
  • 1
0

Run salt-minion debug mode to see, if this is a handshake issue.

If so, Adding salt-master port(4506 or configured) to public zone with firewalld-cmd should help.

`firewall-cmd --permanent --zone=public --add-port=4506/tcp`
`firewall-cmd --reload`
ak1
  • 104
  • 4
0

Your salt-master keys on minion seems to be invalid (may be due to master ip or name update) Steps for troubleshoot:-

  1. From minion check if master is reachable (simple ping test )

  2. Delete the old master keys present on minion (/etc/salt/pki/minion/minion_master.pub)

  3. Again try from master to ping minion new valid keys will be auto populated

Ashwani Singh
  • 876
  • 11
  • 10