My background: a few years back, I worked on a cross-platform, cross-browser NPAPI plugin that used OpenGL to do image rendering. I've paid some attention to the space since then, but it's not what I do nowadays. I've written non-plugin extensions for Firefox but not Safari or Chrome.
A lot depends on exactly what you want your extension to do, and what sort of user interface you want to present. Different browsers have different ideas of what extensions may and may not do.
If you just wanted to display images or video in a custom format, NPAPI is probably your best bet. NPAPI is fundamentally a thin wrapper around native platform APIs. As such, cross-platform NPAPI code has to deal with individual platform quirks, as well as different levels of support for NPAPI from browsers. Google has a project called Pepper to fix some of NPAPI's issues, but Pepper doesn't yet have support from any browser besides Chrome.
Google's Native Client project provides an alternative to NPAPI for running native code, but it imposes restrictions on native code. For example, pthreads are allowed, but network access isn't. There is a NaCL subproject called c_salt which appears to address the sort of HTML/JS UI + native code integration you seek. Unfortunately it's still in the design phase, and thus isn't terribly helpful for writing extensions today.
My understanding is that, for now, a cross-browser extension which integrates with the browser UI and also integrates with native code is not feasible. Safari and Chrome both restrict extensions much more than Firefox does, and only allow extensions written in JavaScript. I don't know if it's possible to have an extension in Safari or Chrome interact with a browser plugin written in NPAPI. I'm guessing it's either impossible or incredibly painful.
Overall, you'll find life much, much easier if you carefully restrict which browsers and platforms to target. Exactly which browsers and platforms to choose is something that only you can decide.
