Circular Dependency for multinode module (Xodus)

Question

After quite some trials, I find that it is not possible to use the multinode module at all. Since the multinode depends on entity-store module and vice versa. Thus including the multinode module into Gradle config of entity-store causes circular dependency.

Anyhow, I am still trying some hacks. Essentially the major issue I find is the creation of the S3BlobVault, since it is easy to (re)create the S3DataReaderWriterProvider from outside the Xodus project, the major issue is the S3BlobVault which needs an instance of the PersistentEntityStoreImpl which means it(the S3BlobVault) needs to be instantiated within/inside the PersistentEntityStoreImpl which is quite not possible due to circular dependency issue.

At the very least I did modify the PersistentEntityStoreImpl and added:

public void setBlobVault(BlobVault blobVault) { 
    this.blobVault = blobVault;
}

Then in my code(app), I added

final PersistentEntityStoreImpl store = PersistentEntityStores.newInstance(environment); 
S3BlobVault s3BlobVault = createS3BlobVault(store, environment.getLocation());
store.setBlobVault(s3BlobVault);

Creating the vault like this:

  private S3BlobVault createS3BlobVault(PersistentEntityStoreImpl store, String location) { 
    try {
      S3AsyncClient s3 = S3AsyncClient.builder()
              .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("", "")))
              .endpointOverride(new URI("https://s3.wasabisys.com"))
              .region(Region.US_EAST_1).build();
      S3BlobVault blobVault = null;
// Can't use code below (outside of package)

//      try {
//        final PersistentSequenceBlobHandleGenerator.PersistentSequenceGetter persistentSequenceGetter =
//                new PersistentSequenceBlobHandleGenerator.PersistentSequenceGetter() {
//                  @Override
//                  public PersistentSequence get() {
//                    return getSequence(getAndCheckCurrentTransaction(), BLOB_HANDLES_SEQUENCE);
//                  }
//                };
//        blobVault = new S3BlobVault(store,
//                new PersistentSequenceBlobHandleGenerator(persistentSequenceGetter), s3, "xodus", location, ".blobs", null);
//      } catch (UnexpectedBlobVaultVersionException e) {
//        blobVault = null;
//      }
      if(blobVault == null) {
        blobVault = new S3BlobVault(store,
                BlobHandleGenerator.IMMUTABLE, s3, "xodus", location, ".blobs", null);
      }
      return blobVault;
    } catch (Exception e) {
      throw ExodusException.toExodusException(e);
    }
  }

I still ended with the error:

Caused by: java.io.FileNotFoundException: s3:xodus\blobs\version (The filename, directory name, or volume label syntax is incorrect) 
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
at jetbrains.exodus.entitystore.FileSystemBlobVaultOld.<init>(FileSystemBlobVaultOld.java:106)
at jetbrains.exodus.entitystore.FileSystemBlobVaultOld.<init>(FileSystemBlobVaultOld.java:71)
at jetbrains.exodus.entitystore.PersistentEntityStoreImpl.createDefaultFSBlobVault(PersistentEntityStoreImpl.java:424)
... 95 more

Vyacheslav Lukianov · Answer 1 · 2019-07-01T12:08:17.703

In your project, you can add dependency on the multinode jar and create PersitentEntityStore in such a way:

final S3BlobVault blobVault = createBlobVault(...);
final Environment env = Environments.newInstance("location");
final PersistentEntityStoreImpl store = PersistentEntityStores.newInstance(PersistentEntityStoreConfig.DEFAULT, env, blobVault, "entityStore name");

Probably, this would work. At least, if you pass the blob vault for creation of PersistentEntityStore then you wouldn't require the circular dependency you mentioned. Dependency on the multinode module is enough to use functionality of the entity-store module.

Though, I have to emphasize that any functionality in the multinode module is incomplete, not announced, not documented, and is a subject to change. It may be removed completely in future versions.

You mean the S3 codes will be removed but the SPI would be retained, right? — quarks, Jul 01 '19 at 12:08
And about the BlobVault apparently it cannot be instantiated outside the PersistentEntityStoreImpl — quarks, Jul 01 '19 at 12:09

score 1 · Answer 2 · answered Jul 01 '19 at 09:05

1

Xodus build 1.3.91 shares S3 functionality as an experimental feature. There is no references in documentations for it also some tests failed for S3 file store. We do not recommend to use it in production code until there is no mentions in release notes and no section in documentation for it.

Result of using S3 as a store for Xodus at the moment is unpredictable.

answered Jul 01 '19 at 09:05

lehvolk

233
1
4

I am planning to create a external S3 vault and data reader/writer the problem is a Vault requires PersistentEntityStoreImpl instance thus it is not quite possible to have a data reader/writer outside the entity-store package – quarks Jul 01 '19 at 12:12

Circular Dependency for multinode module (Xodus)

2 Answers2