I have SSH key-based authentication setup to connect to a remote server from my Macbook Air. The private key was originally stored in ~/.ssh/id_rsa, but I have since moved that file to a secure external HD. I deleted the file from the ~/.ssh directory as well (in theory, no one would be able to connect without the private key on that external HD).
However, when I try to connect to my remote host via ssh (ssh user@12.33.539.295), it happily connects (without the external w/ the keyfile plugged in). It does say "last logged in at" at the prompt, so is my Mac caching the keyfile somewhere?