Maybe it is time to answer this question again. Contributions and edits are welcome.
Accessibility
SVG content is visual content. That in itself poses restrictions on accessibility. Nonetheless, the upcoming SVG2 spec has a paragraph on ARIA attributes that tries to give at least a bit of assistance. If you look more closely, you will find that for example a graphics primitive will get a default aria-role="graphics-symbol", which is not really helpful.
Whether screen readers implement any of this I don't know.
SVG gives you the opportunity to add <title> and <desc> tags anywhere. But the standard semantics that HTML5 offers through tags are lost. No header, no sections, no TOC or navigation. Try to imagine what the Firefox Reader View could make out of such a page: nothing.
Even linking to partial content of a page is difficult. While the spec is there, the semantic remains constricted: do you really want to highlight a link target by scaling it up to cover the whole viewport?
SEO in the end is just a specialized aspect of accessibility. Having an HTML <header> segment, a few things might be possible. Maybe someone else can comment on what Google implements.
Responsiveness
Problems start with the current inability of automatically line-wrapping text. Layout facilities are still a future project without working implementations. (While Firefox knows CSS inline-size, it is only implemented for HTML and does not work in SVG.)
CSS has put a lot of effort into providing layout mechanisms like flex and grid layout. In SVG, you lose all these techniques for reordering content in favor of simple automatic scaling.
Interaction
Text input is a solvable problem. You can use HTML inputs wrapped in a <foreignObject> tag, or you can build text input widgets from scratch by capturing keyboard input into a <text> element.
For most other forms of user input SVG might even be the superior platform. For example, clicking on/touching something to mark it and dragging it around/swiping it turn out to be only two aspects of basically the same interaction.
Especially Web components are the perfect complement to SVG for building interactive widgets. (Despite you still have to go through polyfills for compatibility with Firefox.)
Form submission
While HTML form submission for initiating HTTP requests might be absent, the advent of single-page apps has shown that it is possible to employ XHR requests instead.
Conclusion
If your content depends on or benefits from the standard semantics HTML5 offers, SVG is clearly not appropriate.
If you need to change the layout of your page in a responsive way, SVG will not be helpful.
SVG gives a lot of additional possibilities for creative, visually-oriented user interactions. As long as you find fallback solutions for visually-impaired users, your page will gain from it.
Overall, SVG gives you creative possibilities for specialized areas and widgets on your page, but falls short on the basic semantic questions a web page has to answer: How does your page relate to other resources on the web? What on your page is content, what is meta information, what is decoration?
The best of both worlds is to be gained by using a mix-and-match approach: structure your page with HTML, decorate it with SVG graphics, and build interactive/animated widgets with SVG and maybe Web Components.
