ignite control tool command "reset_lost_partitions" not work

Question

After all server nodes restart and rejoin the baseline, the cache still failed to be read or written:

javax.cache.CacheException: class org.apache.ignite.internal.processors.cache.CacheInvalidStateException: Failed to execute the cache operation (all partition owners have left the grid, partition data has been lost) [cacheName=SQL_DEV_MONI_MONITOR, partition=840, key=UserKeyCacheObjectImpl [part=840, val=iii, hasValBytes=false]]
    at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1278)

Partitions owned by nodes is reblanced:

cacheGroups_public_LocalNodeOwningPartitionsCount{iin="fintech-test-grid-mcpl323", inci="7ffed004-2058-4045-875e-bca5ac37e12a", instance="10.16.23.47:9090", job="prometheus"}
544
cacheGroups_public_LocalNodeOwningPartitionsCount{iin="fintech-test-grid-mwpl037", inci="fcea873e-3822-4f1b-ab86-acf71d2e740b", instance="10.16.50.124:9090", job="prometheus"}
382
cacheGroups_public_LocalNodeOwningPartitionsCount{iin="fintech-test-grid-mwpl038", inci="d52c1baf-c34e-4c49-af93-89bff06f55f1", instance="10.16.50.123:9090", job="prometheus"}
210

The data region persistence was enabled, but reset_lost_partitions not works:

        <property name="dataStorageConfiguration">
            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                <!--
                    Default memory region that grows endlessly. A cache is bound to this memory region
                    unless it sets another one in its CacheConfiguration.
                -->
                <property name="defaultDataRegionConfiguration">
                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="Default_Region"/>
                        <property name="persistenceEnabled" value="true"/>
                        <!-- 1 GB memory region with disabled eviction -->
                        <property name="initialSize" value="#{100L * 1024 * 1024}"/>
                        <property name="maxSize" value="#{100L * 1024 * 1024 * 1024}"/>
                        <!-- Enabling SEGMENTED_LRU page replacement for this region.  -->
                        <property name="pageReplacementMode" value="SEGMENTED_LRU"/>
                        <property name="metricsEnabled" value="true"/>
                        <property name="warmUpConfiguration">
                            <bean class="org.apache.ignite.configuration.LoadAllWarmUpConfiguration"/>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>

After I stop all the nodes, and then start the nodes again, partition lost exception recovered. — Roc King, Feb 24 '22 at 01:21

score 0 · Answer 1 · answered Feb 22 '22 at 13:57

If a node has permanently left the grid, you need to update the baseline topology. There are a number of ways of doing that, but the simplest is the control script:

./control.sh --baseline

Then

./control.sh --baseline add [node]
./control.sh --baseline remove [node]

ignite control tool command "reset_lost_partitions" not work

1 Answers1