All servers support the ssh public key authentication only in my environment. When I read the ssh debug info, I notice the "Offering RSA public key" is different.
When I ssh server1 from host1
user@host1:~$ ssh -v server1
...
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/user/.ssh/online.pem
debug1: Authentications that can continue: publickey
debug1: Offering RSA public key: /home/user/.ssh/id_rsa
However, when I ssh to server2 from host2
user@host2:$ ssh -v server2
...
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: jumpbox <-- even not a file
Which ssh/sshd config item will affect the RSA public key offering method? Unlucky, the server1 and server2 are in isolated networks, so I can't ssh server2 from host1 or ssh server1 from host2 to eliminate some factors.