No, Java JCA is a cryptography API. It doesn't support any higher level protocols, it just gives a relatively high level access to cryptographic algorithms. There is of course one noteworthy protocol that is implemented in Java: TLS within the (separate) JSSE provider. This requires support for using / verifying X.509v3 certificates and such, but that's about it (the JCA is very much geared towards supporting TLS as main purpose, for obvious reasons).
OpenPGP one the other hand is a protocol specification that isn't supported by Java. You'd need to build a protocol implementation on top of JCA to support OpenPGP. However, that's exactly what Bouncy Castle did; their implementation depends on the JCA interface rather than the Bouncy Castle "lightweight" API. So you can use Bouncy Castle's OpenPGP library and use the JCA, e.g. for support of AES-NI or hardware keys. Just make sure that the JCA provider you want to use has more priority than any third party ones, such as the Bouncy Castle provider (add it to the end of the list of providers!).
Of course, for support of some more esoteric options (notably support for specific EC curves) you may still want to add the Bouncy Castle provider to your runtime. The Maven module of the OpenPGP still lists the Bouncy Castle provider as compile requirement, but I'm not sure if that's not just for testing or that the library is directly required.
One issue that may be troublesome is that PGP uses its own version of the CFB encryption algorithm, which is not directly available from the default providers in the runtimes that I know about. So you may need to include the library for that functionality. This may also mean that AES acceleration is not available for the library.