1

I tried running infinispan in two machines and persisted the index data in one machine. When i try to run simultaneously in 2 machines with indexing and persisting(cache-store) into the database, i am getting the following exception,

Caused by: java.io.FileNotFoundException: Error loading metadata for index file: segments_2j|M|Course
 at org.infinispan.lucene.impl.DirectoryImplementor.openInput(DirectoryImplementor.java:134)
 at org.infinispan.lucene.impl.DirectoryLuceneV4.openInput(DirectoryLuceneV4.java:101)
 at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:113)
 at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:341)
 at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:57)
 at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:923)
 at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:53)
 at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:67)
 at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:131)
 at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:206)
 at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:108)
 ... 24 more

My infinispan config file is: infinispan-config.xml

<?xml version="1.0" encoding="UTF-8"?> <infinispan
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:infinispan:config:7.2 http://www.infinispan.org/schemas/infinispan-config-7.2.xsd
        urn:infinispan:config:store:jdbc:7.2 http://www.infinispan.org/schemas/infinispan-cachestore-jdbc-config-7.2.xsd"
        xmlns="urn:infinispan:config:7.2"
        xmlns:jdbc="urn:infinispan:config:store:jdbc:7.2">

    <!-- *************************** -->
    <!-- System-wide global settings -->
    <!-- *************************** -->
    <jgroups>
        <!-- Note that the JGroups transport uses sensible defaults if no configuration
            property is defined. See the JGroupsTransport javadocs for more flags.
            jgroups-udp.xml is the default stack bundled in the Infinispan core jar: integration
            and tuning are tested by Infinispan. -->
        <stack-file name="default-jgroups-tcp" path="my-jgroupstcp.xml"/>
    </jgroups>

    <cache-container name="HibernateSearch" default-cache="default" statistics="false" shutdown-hook="DONT_REGISTER">

        <transport stack="default-jgroups-tcp"/>

        <!-- Duplicate domains are allowed so that multiple deployments with default configuration
            of Hibernate Search applications work - if possible it would be better to use JNDI to share
            the CacheManager across applications -->
        <jmx duplicate-domains="true"/>

        <!-- *************************************** -->
        <!--  Cache to store Lucene's file metadata  -->
        <!-- *************************************** -->
        <replicated-cache name="LuceneIndexesMetadata" mode="ASYNC" async-marshalling="true">
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false"/>
            <transaction mode="NONE" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <persistence passivation="false">
                <jdbc:string-keyed-jdbc-store preload="true" fetch-state="true" read-only="false" purge="false">
                    <write-behind />
                    <property name="key2StringMapper">org.infinispan.lucene.LuceneKey2StringMapper</property>
                    <jdbc:connection-pool connection-url="jdbc:mysql://localhost:3306/hsearch"
                                          driver="com.mysql.jdbc.Driver" username="my-username"
                                          password="my-password"></jdbc:connection-pool>
                    <jdbc:string-keyed-table drop-on-exit="false" create-on-start="true" prefix="ISPN_STRING_TABLE">
                        <jdbc:id-column name="ID" type="VARCHAR(255)"/>
                        <jdbc:data-column name="DATA" type="MEDIUMBLOB"/>
                        <jdbc:timestamp-column name="TIMESTAMP" type="BIGINT"/>
                    </jdbc:string-keyed-table>
                </jdbc:string-keyed-jdbc-store>
            </persistence>
            <indexing index="ALL"/>
        <state-transfer enabled="true" timeout="480000" await-initial-transfer="true"/>
        </replicated-cache>




        <!-- **************************** -->
        <!--  Cache to store Lucene data  -->
        <!-- **************************** -->
        <distributed-cache name="LuceneIndexesData" mode="ASYNC" async-marshalling="true">
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false"/>
            <transaction mode="NONE"/>
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <persistence passivation="false">
                <jdbc:string-keyed-jdbc-store preload="true" fetch-state="true" read-only="false" purge="false">
                    <write-behind />
                    <property name="key2StringMapper">org.infinispan.lucene.LuceneKey2StringMapper</property>
                    <jdbc:connection-pool connection-url="jdbc:mysql://localhost:3306/hsearch"
                                          driver="com.mysql.jdbc.Driver" username="my-username"
                                          password="my-password"></jdbc:connection-pool>
                    <jdbc:string-keyed-table drop-on-exit="false" create-on-start="true" prefix="ISPN_STRING_TABLE">
                        <jdbc:id-column name="ID" type="VARCHAR(255)"/>
                        <jdbc:data-column name="DATA" type="MEDIUMBLOB"/>
                        <jdbc:timestamp-column name="TIMESTAMP" type="BIGINT"/>
                    </jdbc:string-keyed-table>
                </jdbc:string-keyed-jdbc-store>
            </persistence>
            <indexing index="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true"/>
        </distributed-cache>

        <!-- ***************************** -->
        <!--  Cache to store Lucene locks  -->
        <!-- ***************************** -->
        <replicated-cache name="LuceneIndexesLocking" mode="ASYNC" async-marshalling="true">
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false"/>
            <transaction mode="NONE"/>
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
            <indexing index="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true"/>
        </replicated-cache>
    </cache-container>

</infinispan>

My jgroup configuration file is, my-jgroup.xml

<config xmlns="urn:org:jgroups"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.6.xsd">
<TCP bind_addr="${jgroups.tcp.address:192.168.1.48}"
     bind_port="${jgroups.tcp.port:7800}"
     enable_diagnostics="false"
     thread_naming_pattern="pl"
     send_buf_size="640k"
     sock_conn_timeout="300"

     thread_pool.min_threads="${jgroups.thread_pool.min_threads:2}"
     thread_pool.max_threads="${jgroups.thread_pool.max_threads:30}"
     thread_pool.keep_alive_time="60000"
     thread_pool.queue_enabled="false"

     internal_thread_pool.min_threads="${jgroups.internal_thread_pool.min_threads:5}"
     internal_thread_pool.max_threads="${jgroups.internal_thread_pool.max_threads:20}"
     internal_thread_pool.keep_alive_time="60000"
     internal_thread_pool.queue_enabled="true"
     internal_thread_pool.queue_max_size="500"

     oob_thread_pool.min_threads="${jgroups.oob_thread_pool.min_threads:20}"
     oob_thread_pool.max_threads="${jgroups.oob_thread_pool.max_threads:200}"
     oob_thread_pool.keep_alive_time="60000"
     oob_thread_pool.queue_enabled="false"
/>
<MPING bind_addr="${jgroups.tcp.address:192.168.1.48}"
       mcast_addr="${jgroups.mping.mcast_addr:228.2.4.6}"
       mcast_port="${jgroups.mping.mcast_port:43366}"
       ip_ttl="${jgroups.udp.ip_ttl:2}"
/>
<MERGE3 min_interval="10000"
        max_interval="30000"
/>
<FD_SOCK />
<FD_ALL timeout="60000"
        interval="15000"
        timeout_check_interval="5000"
/>
<VERIFY_SUSPECT timeout="5000" />
<pbcast.NAKACK2 use_mcast_xmit="false"
                xmit_interval="1000"
                xmit_table_num_rows="50"
                xmit_table_msgs_per_row="1024"
                xmit_table_max_compaction_time="30000"
                max_msg_batch_size="100"
                resend_last_seqno="true"
/>
<UNICAST3 xmit_interval="500"
          xmit_table_num_rows="50"
          xmit_table_msgs_per_row="1024"
          xmit_table_max_compaction_time="30000"
          max_msg_batch_size="100"
          conn_expiry_timeout="0"
/>
<pbcast.STABLE stability_delay="500"
               desired_avg_gossip="5000"
               max_bytes="1M"
/>
<pbcast.GMS print_local_addr="false"
            join_timeout="15000"
/>
<MFC max_credits="2m"
     min_threshold="0.40"
/>
<FRAG2 />

My persistence.xml is,

<?xml version="1.0" encoding="UTF-8" ?>

<persistence-unit name="IC" transaction-type="RESOURCE_LOCAL">
    <class>com.csgsol.model.Course</class>
    <properties>
        <property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver"/>
        <property name="javax.persistence.jdbc.url" value="jdbc:mysql://192.168.1.99:3306/sampleDb"/>
        <property name="javax.persistence.jdbc.user" value="my-username"/>
        <property name="javax.persistence.jdbc.password" value="my-password"/>

        <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect"/>
        <property name="hibernate.hbm2ddl.auto" value="update"/>

        <property name="hibernate.search.default.directory_provider" value="infinispan"/>
        <property name="hibernate.search.default.indexmanager" value="org.infinispan.query.indexmanager.InfinispanIndexManager"/>

        <property name="hibernate.search.default.indexmanager" value="near-real-time"/>
        <property name="hibernate.search.default.exclusive_index_use" value="false"/>
        <property name="hibernate.search.default.worker.execution" value = "async"/>
        <property name="hibernate.search.lucene_version" value="LUCENE_4_10_4"/>
        <property name="hibernate.search.infinispan.configuration_resourcename" value="infinispan-config.xml"/>

                </properties>
            </persistence-unit>
        </persistence>

The versions i am using are Infinispan - 7.2.0.Final Lucene - 4.10.4

  • What version of Infinispan? Suggested configurations for Infinispan for storing Lucene indexes is indicated [here](http://infinispan.org/docs/stable/user_guide/user_guide.html#configuration_11) for the latest stable release. – Galder Zamarreño Jan 04 '18 at 14:30
  • Thanks for the reply, the versions are Infinispan - 7.2.0.Final and Lucene - 4.10.4 – Nadeem Pasha Jan 05 '18 at 04:40
  • I've just noticed [ISPN-6425](https://issues.jboss.org/browse/ISPN-6425) which looks similar to your case. Either using a SYNC or upgrading to latest stable release (9.1.x) should help. – Galder Zamarreño Jan 05 '18 at 11:20
  • I changed the indexes mode to SYNC, now i am not facing any exception. But the problem is when i run the same application/program in 2 machines at the same time, one of the inserted data is lost without any message or exception. – Nadeem Pasha Jan 08 '18 at 06:21
  • Have you verified that the cluster is forming? – Galder Zamarreño Jan 08 '18 at 15:00
  • yes the cluster is formed,i can see it in logs. And also the indexes is getting persisted in cachestore(database) when i run sequentially one after the other from different machines. The problem is when i run parallel. – Nadeem Pasha Jan 09 '18 at 04:47
  • Hmmm, I don't know TBH. I'd try a more recent Infinispan version. Latest is 9.1.4.Final. – Galder Zamarreño Jan 10 '18 at 14:00

1 Answers1

1

You should not use mode="ASYNC" for index caches. Since Infinispan 8.2.x (https://issues.jboss.org/browse/ISPN-4065) this configuration is forbidden

gnf
  • 483
  • 3
  • 8
  • Now i am not facing this exception and i changed the mode to sync for index caches(index data, metadata, locking). But when i execute on two machines simultaneously, only indexes from one of the machine(random) gets persisted to the database. I have added my persistence.xml to the question. – Nadeem Pasha Jan 05 '18 at 04:53
  • Also for index data cache, i changed it to replicated cache and added – Nadeem Pasha Jan 05 '18 at 05:09
  • Generally speaking, Lucene indexes can only be written to from 1 process at a time due to the index-wide lock constraint. How are you doing the writes? – gnf Jan 05 '18 at 12:40
  • I have added in order to persist indexes asynchronously. When i try to execute the application/program in 2 machines at the same time, 2 scenarios occurred, scenario-1: one process waits till the other process to get complete, and then the other one starts and it will get completed, and when i run search on both machines, i am able to see both the insertions. scenario-2: In this scenario both the process finishes at the same time, and when i search on both machines, i am able to get only one of the data inserted(which is random of 2) and other data is lost without exception. – Nadeem Pasha Jan 08 '18 at 04:57