J2SE is in itself supported by a specification (see JSR270) but J2SE does not include the defining rules for implementing a Java Virtual Machine (JVM). Although the JVM is defined by a specification, you probably are referring to the definition of the language itself which is defined in yet another specification (the JLS).
Regarding the JVM, there are several implementations (as with J2EE). This SO question lists several.
Back to your main question, though. The J2SE spec defines a collection of a basic set of support services that Java applications commonly use. If you look at it, the specification is basically an index for other specifications that then describe each of those technologies. Don't confuse specification (the JSR) with the reference implementation, which is the actual implementation of that specification (which is what you download from Oracle when getting the J2SE package).
Just to sum it up:
- Specification that defines the Java language: JSR 901
- Specification for the JVM: JSR 924
- J2SE: JSR 270