0

We have a cluster with a front node that admits normal users and LDAP users. Two days ago the ssh show a strange behavior:

  • The LDAP users can't login in the front node using password
  • but, The LDAP users can login if they setup ssh-key in authorized_keys
  • normal users (No LDAP, /etc/passwd ones) can login without problems
  • Other services using the same LDAP server works right

So, I think is a problem located in the front node. The LDAP appears to work, using getent [passwd|shadow] command we are getting all users.

At the same time, the ssh show a warning/error when we login from this node to other nodes. But it allow the ssh anyway:

[root@frontnode ~]# ssh othernode
/etc/ssh/ssh_config line 50: Unsupported option "GSSAPIAuthentication"
[root@othernode ~]#

Also, When I restart the ssh daemon, we have also errors related to GSSAPI:

[root@frontnode ~]# service sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd: /etc/ssh/sshd_config line 81: Unsupported option GSSAPIAuthentication
/etc/ssh/sshd_config line 83: Unsupported option GSSAPICleanupCredentials
/etc/ssh/sshd_config line 97: Unsupported option UsePAM
                                                           [  OK  ]

No one changed ssh configuration, neither sshd configuration. And was working until now. We don't know what the problem is.

The login node is a Scientific Linux release 6.2 (redhat based)

Zhen
  • 2,159
  • 4
  • 19
  • 31

1 Answers1

2

It looks like sshd or some libraries got changed on the front node.

The UsePAMoption is what would allow you to login with LDAP stored passwords.

Troubleshooting steps

  1. You should check your /var/log/yum.log for any package changes.
  2. Do you have a configuration management system running there? Make sure no one put in a new sshd binary.
  3. Check for rootkits using chrootkit or rkhunter.

Cleanup

Now that you have determined that there is a rootkit, you need to ensure that everyone changes their passwords and recommend that they change their passwords on other sites, too.

The safest option for cleanup is to reinstall and restore from a backup before this started. You can never really know if some seemingly innocuous file got modified and will reinfect after you clean up.

Jeff Strunk
  • 2,127
  • 1
  • 24
  • 29
  • Yes, sshd binary is change, but is the same yum version that the one in the rpm package. I reinstalled it with yum, and work again. But in no time some process change it again back to wrong binary. – Zhen Aug 21 '13 at 16:14
  • 1
    @Zhen If you're saying that reinstalling yum fixes the binary, but the binary gets overwritten again shortly afterwards, we cannot help you with this problem. Either a user on the system or some sort of automated job (configuration management) is overwriting it. – Andrew B Aug 21 '13 at 19:21
  • 1
    Do you have a configuration management system running there? Make sure no one put in a new sshd binary. Also, run a rootkit hunter like chrootkit or rkhunter. – Jeff Strunk Aug 21 '13 at 19:21
  • thank, was a rootkit, we are in process to remove it. I you put as an answer I can accept it. – Zhen Aug 22 '13 at 15:49
  • So sorry to hear that. I wish you the best in cleaning up. – Jeff Strunk Aug 22 '13 at 19:37