The utility class Collectors
contains a few static, preallocated unmodifiable sets of the various collector characteristics that are needed for the ones supplied by the JDK. There are only a few possible combinations of the collector flags, so this is quite easy to do.
On the other hand there are many spliterator sources. Encoding the many possible combinations of characteristics in sets might have been considered too costly. Especially if some implementers might choose to allocate them on the fly.
They also get modified a lot, adding or removing characteristics as the stream pipeline is built, which can be achieved cheaply with binary arithmetics.
int flags are simply cheaper.