I was struggling with a similar situation.
Slah's solution --- put a operational layer on top of the background image --- works if you don't need pinch to zoom the image, but in my case the positioned widgets get invalid when zooming in the image.
My solution is embedding the image into a webview, you may use webview_flutter because it can be easily plugged onto your widget tree, but I think in your case flutter_webview_plugin works too since you just need to listen to some click events. Add hyperlinks to the elements to be clicked, then play with the click event by webviewcontroller(webview_flutter) or onUrlChanged Stream(flutter_webview_plugin).
One benefit you can get immediately is the ability to use .svg without any other packages by Uri.dataFromString('<html><svg>some svg codes</svg></html>', mimeType: 'text/html').toString()
Then you have full control of the appearance(via css) and the behavior(via javascript). e.g.You may define irregular areas to be clickable, and responsive to clicks.