0

There are two eXist-db servers. One is master and one is slave. I know we can replicate the content using ActiveMQ when they are running in the same time(I made this work). But I want to know is there a way to replicate the content manually? Like synchronizing content from A server when B server starts. Actually, they are the same folder in two servers. collection.xconf as below.

<collection xmlns="http://exist-db.org/collection-config/1.0">
    <triggers>
        <trigger class="org.exist.jms.replication.publish.ReplicationTrigger">
            <!-- 
                Class name of the initial context provider, default value 
                for ActiveMQ
                see javax.naming.Context#INITIAL_CONTEXT_FACTORY
            -->
            <parameter name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
            <!-- 
                URL of the message broker, default value for ActiveMQ 
                see javax.naming.Context#PROVIDER_URL
            --> 
            <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
            <!-- 
                Lookup connection factory
                see javax.naming.InitialContext#lookup(String) 
            -->
            <parameter name="connection-factory" value="ConnectionFactory"/>
            <!--
                Lookup destination (topic)
                see javax.naming.InitialContext#lookup(String) 
            -->
            <parameter name="destination" value="dynamicTopics/eXistdb-replication-example"/>
        </trigger>
        <trigger event="update" class="org.exist.collections.triggers.XQueryTrigger">
            <parameter name="url" value="xmldb:exist:///db/triggerUpdate.xql"/>
        </trigger>
    </triggers>
</collection>
xinpeng sun
  • 1
  • 1
  • I am really not clear what you are trying to do here. Is it the case that when a new server starts you want to pull content from an existing server? or something else? – adamretter Oct 02 '18 at 09:45
  • The goal is to synchronize contents between two servers. The use case would be: when A server is running, B can synchronize contents when B starts. – xinpeng sun Oct 02 '18 at 16:58
  • what about writes, say B updates some content would that need to be synced back to A? – duncdrum Oct 03 '18 at 14:19
  • no need. Actually, we have more than two servers. How to make sure each server has the same content? When they are running, we can fix it by replication. I wonder if one of them shut down, how to make sure it still has the latest content when it comes back. – xinpeng sun Oct 03 '18 at 17:11
  • i m afraid the answer is it depends. From what i can discern so far I would recommend configuring a kubernetes cluster or docker-swarm, with distinct data-volumes shared by multiple nodes – duncdrum Oct 04 '18 at 11:35
  • data volumes won't help you here. You can't sync based on the physical files on disk. You have to sync at the logical database level where you see all the XML documents etc. – adamretter Oct 11 '18 at 11:10
  • So you could write a startup trigger, so when server B starts it runs an XQuery or Java function of your devising. That function would need to scan the current server and Server A, come up with a diff, and then retrieve the missing files from server A, this could all be done over HTTP. – adamretter Oct 11 '18 at 11:12

1 Answers1

0

The extension actually has the sync($doc) function, you can use this function to send one resource into the broker. I will add documentation about this to the wiki soon.

That reminds that I need to release some minor fixes soon....

DiZzZz
  • 621
  • 3
  • 12
  • can we sync contents between two servers by using this function? – xinpeng sun Oct 15 '18 at 19:42
  • if you correctly setup the receiving servers and the transmitting server correctly, yes. It uses the same mechanism as when a document is stored/updated/renamed in the database. – DiZzZz Oct 15 '18 at 19:45
  • I am still a little confused. From documents about sync function, it tells one is source, one is target. How can we get access to another server through xQuery? Is it possible? By hostname, port number, account and password. – xinpeng sun Oct 16 '18 at 04:47
  • The configuration is per collection in the .xconf file. the `sync($doc)` function reads the configuration from that file, and will then send the document `$doc` using the activemq 'client' to the activemq broker (server), which will distribute the document to the subscribed "slaves" (receiving databases). – DiZzZz Oct 16 '18 at 09:11
  • I see. I think I got the configuration file correctly because replication works fine. If I want to synchronize one folder("replicated"). I wrote `let $x := system:as-user("admin","",file:sync("/db/system/config/db/replicated/collection.xconf","/db/replicated", dateTime(xs:date('2018-10-15'),xs:time('12:30:45-05:00')))) return $x`, but it failed. Error message is `Failed to create output directory: /db/replicated for collection /db/system/config/db/replicated/collection.xconf` – xinpeng sun Oct 16 '18 at 16:28
  • I paste my collection.xconf in question now – xinpeng sun Oct 16 '18 at 16:29
  • the `file:sync()` function is the wrong one, you need to use the `replication:sync()` – DiZzZz Oct 17 '18 at 19:31
  • Can you share a link with this function? I didn't find anything related to this one. Thanks~ – xinpeng sun Oct 17 '18 at 21:59
  • First back to the basics: what version of exist-db are you using? and the plugin? BTW I prefer to provide support via the github issue tracker, not via SO. I do not monitor this actively. – DiZzZz Oct 18 '18 at 08:20