1

According to the comment from this post

"I have xodus directory I wan to share between two applications.. But one with read /write access and other with read-only access.. Is the any other way to create Environment apart from Environments.newInstance("xodusDir"); for read-only access to xodus database.. I need only read only access to xodus from other application.."

Answered with

"This would be possible with upcoming version 1.3.0."

Does it mean that it is possible to have multiple processes to create Environments / Persistent Storage pointing to the same directory:

  Environment environment = getEnvironment(xodusRoot, dir);
  final PersistentEntityStore store = PersistentEntityStores.newInstance(environment);

With this topology:

enter image description here

Is this possible or is there a workaround to do this?

From what I know, the xd.lck prevents two instance from accessing the same Xodus directory.

quarks
  • 33,478
  • 73
  • 290
  • 513
  • The motivation here is that NGINX is a load-balancer/traffic distributor in this scenario, and the two Nginx Unit apps are two apps for traffic distribution, also for zero-downtime topology. – quarks Jul 11 '19 at 19:02

1 Answers1

4

As of version 1.3.0, you can open a database in read-only mode ignoring the lock:

final EnvironmentConfig config = new EnvironmentConfig().
            setLogDataReaderWriterProvider("jetbrains.exodus.io.WatchingFileDataReaderWriterProvider").
            setLogCacheShared(false).
            setMemoryUsagePercentage(1);
final Environment env = Environments.newInstance(dir, config);
final PersistentEntityStore store = PersistentEntityStores.newInstance(env);

Not only it ignores the lock, but it also automatically fetches new data using java.nio.file.WatchService.

One instance Environment/EntityStore can be opened as usually (primary one) and several instances can be opened as above (secondary ones). Your proxy server should correctly distribute traffic so that secondary instances wouldn't get write requests. Obviously, your application have to resolve eventual consistency issues that are unavoidable with such architecture.

Secondary instances can be opened in any JVM. If you open secondary instances in the same JVM which is used to open primary instance, then keep an eye on memory usage since secondary instances don't use shared log cache. Primary instance (if you don't use setMemoryUsagePercentage) can consume 50% of maximum heap memory. It's good if overall memory usage percentage doesn't exceed 50-55% (though this depends on what GC and what GC settings you use).

If you open secondary instances in separate JVM(s) then skip setLogCacheShared and setMemoryUsagePercentage settings in the code above. The only thing you have to configure is watching DataReaderWriterProvider.

With the version 1.3.91, you can even run JVM(s) with secondary instances on different hosts. Each host should have a copy of the database on its filesystem, it can be opened as follows:

final EnvironmentConfig config = new EnvironmentConfig().
            setLogDataReaderWriterProvider("jetbrains.exodus.io.WatchingFileDataReaderWriterProvider");
final Environment env = Environments.newInstance(dir, config);
final PersistentEntityStore store = PersistentEntityStores.newInstance(env);

The only extra thing you have to do with your infrastructure is to provide periodical syncing of database changes. Version 1.3.91 is tested and works fine with rsync. Syncing each 5-10 seconds would be ok in most cases, though high writing workloads would result in some syncing delays.

Vyacheslav Lukianov
  • 1,913
  • 8
  • 12
  • Instead of having to copy the database on each JVM on a different host, what I did is to mount block storage into the Host operating system, what is the potential hazard with this approach? – quarks Nov 09 '19 at 02:22
  • is the rsync needed if we use network mount storage? Linode for example have block storage that we mount into the application that uses Xodus, if we have this topology then rsync is still needed or not? – quarks Jun 11 '20 at 00:46
  • The rsync is not need in that case. – Vyacheslav Lukianov Jun 11 '20 at 13:10
  • And there will be no issues? I mean, up to this point we haven't experienced an issue with block storage mount, but hey we haven't scaled big yet so I can't convince myself that there will be no issue with using Xodus in such environment, let alone two Xodus processes on two separate machine is mounted to that same block storage, quite scary in fact. Any ideas? – quarks Jun 11 '20 at 17:10
  • Btw, maybe you can share some ideas for this (related to Xodus) https://stackoverflow.com/questions/62330339/using-amazon-s3-as-a-database-storage – quarks Jun 11 '20 at 17:42
  • Nobody can guarantee that there will be no issues. Probably, `WatchService` would not work in your environment, who knows?. The answer to your question is: if several database environments share the same database location (in terms of file systems) then you don't have to use rsync. You can open one primary (writing) database environment and lots of secondary reading environments (as described in my answer) against single database location. – Vyacheslav Lukianov Jun 15 '20 at 14:34