It's the same as adding any other external script or stylesheet. All you have to do is focus on giving the correct path and rel and type.
Note: When my favicon image was in the assets folder, it was not displaying the favicon. So I copied the image to the same folder as of my index.html and it worked perfectly as it should.
<head>
<link rel="shortcut icon" type="image/png/ico" href="/favicon.png" />
<title>SITE NAME</title>
</head>
It worked for me. I hope it works for you too.
You can also check some more answers here: Add Favicon with React and Webpack