Disclaimer: I am not a lawyer.
According to https://en.gravatar.com/site/terms-of-service/:
Use of the Services by Third Party Websites. Third party websites may
enable the use of the Services on their respective websites as
expressly authorized by Automattic (e.g., API calls into Gravatar:
http://en.gravatar.com/site/implement); provided that they (i) do not
copy, store or modify User Submissions; (ii) do not distribute,
display, perform, use or otherwise exercise or exploit any User
Submission or other user information for any purpose other than as
necessary to perform the Services for the applicable user; (iii) have
a privacy policy in effect and posted on such website that complies
with all applicable laws and regulations and that notifies such user
that it will only use personally identifiable information as set forth
in (ii) above; and (iv) agree to be bound by and comply with all the
terms, conditions, and limitations set forth in the Terms of Service
(“Gravatar-enabled website”).
So, assuming you do not claim it as yours, do not store it, do not modify it and agree to comply with the other terms and conditions, you should be OK.