How to make some markers on a BoxMap map?

Question

I am currently writing a paper and need to make some markers on a BoxMap map. I have used the folium in Python, the output in my map is as follows:

enter image description here

But what I'd like to have is:

enter image description here

I've tried other styles from the glyphicon and font-Awesome. But I cannot find the number icons.

I found the Leaflet maybe do this work, but I know too little about it.

Can someone help me out ? My codes is as follows:

#Python code
import folium
mymap = folium.Map(location=[43.51,4.75],
                    zoom_start=11,
                    tiles='https://api.mapbox.com/styles/v1/cycrazysnail/cj09lf34r002j2sqvmbuucfvp/tiles/256/{z}/{x}/{y}?access_token=pk.eyJ1IjoiY3ljcmF6eXNuYWlsIiwiYSI6ImNpemplYWMxYzAyNjkycW9kd29tMHo2dGYifQ.p1wiN_NbXH0axu44ollqgA',
                    attr='My Data Attribution'
                    )
folium.Marker([43.60333333,4.656666667],
              icon=folium.Icon(color='red')
              ).add_to(mymap)
folium.Marker([43.4299,4.9111],
              icon=folium.Icon(color='red')
              ).add_to(mymap)
folium.Marker([43.6408,4.8133],
              icon=folium.Icon(color='red')
              ).add_to(mymap)
folium.Marker([43.51,4.75],
              icon=folium.Icon(color='blue')
              ).add_to(mymap)

mymap.save(outfile='test.html')