The gcc info page says:
Specifying soft
causes GCC to generate output containing library
calls for floating-point operations. softfp
allows the
generation of code using hardware floating-point instructions, but
still uses the soft-float calling conventions. hard
allows
generation of floating-point instructions and uses FPU-specific
calling conventions.
So using soft
isn't what you want for targets with an FPU.
Most currently-shipping devices have an ARMv7-A or better CPU, so unless you are targeting a specific device I'm not sure how much value there is in building two shared libs.
Update: There's actually a lot of devices shipping with older CPUs. All the high-end stuff is ARMv7-A, but there's a lot of volume in low-end devices.
ARMv6 + VFP is not a configuration supported by the Play Store -- the goal was to keep the set of configurations as small as possible. You'll need to use a fat binary, or (as it sounds like you're doing) configure your build carefully and list specific devices in Play Store.